homelab | the broadcast

Business Case

My QNAP-NAS (TS-431X2) is slow

this must not be QNAPs fault, maybe the drives (4 disks, RAID5, between 147..167 MB/second for sequential read) are the root cause
adding a SSD-Cache didn’t improve performance when I had a 3-disk RAID 5
- many people around the world are troubleshooting this without success

so I decided to

use the QNAP-NAS as a Virtual-Tape-Library for Backup-Purposes etc
use my ESXi-Host as Storage-Server

„Storage Server“ VM

I regularily use „Nested ESXi“-Hosts for Lab Environments which should access the storage, so I decided to first give NFS a try – ESXi has no built in NFS-Server, so a small VM using Free-NAS, True-NAS, I didn’t decide yet, should provide „feature parity“ to the QNAP System.

RDM is no option

Adding my former „QNAP Cache SSD“ to the ESXi-Host showed that it was impossible (as expected) to passthrough this disk to a VM.

RDM is no option since it doesn’t allow direct hardware access, eg. SMART-Counters or other statistics.

PCIe SATA-Controller: Dell PERC H310

A PERC H310 can get used as a passthrough device for a VM which will get full hardware access for up to 8 disks.

as far I remember, this hardware has been released in 2011, but it might still be fast enough to be no bottleneck

IT Firmware

There’s nothing wrong with the original H310-RAID-Firmware, but since I won’t need those features i’d prefer the „IT“-Firmware, which offers much larger buffers to handle bursts easily.

This firmware has to be programmed, a lot of guides for this exist – I had to combine https://www.vladan.fr/flash-dell-perc-h310-with-it-firmware/ providing a link to a nice Dell-Firmware („6gbpsas.fw“) and a precise description what to do – but the „megarec.exe“ found in the archive didn’t work at my Asus mainboard („Error 8192“) so I found another megarec-release https://www.taste-of-it.de/flash-dell-perc-h310-mit-lsi-9211-8i-it-mode/ here.

[root@esx:~] esxcli hardware pci list | egrep -B8 -A26 "Device Name: Dell"
0000:06:00.0
   Address: 0000:06:00.0
   Segment: 0x0000
   Bus: 0x06
   Slot: 0x00
   Function: 0x0
   VMkernel Name: vmhba2
   Vendor Name: LSI Logic / Symbios Logic
   Device Name: Dell 6Gbps SAS HBA Adapter
   Configured Owner: VMkernel
   Current Owner: VMkernel
   Vendor ID: 0x1000
   Device ID: 0x0072
   SubVendor ID: 0x1028
   SubDevice ID: 0x1f1c
   Device Class: 0x0107
   Device Class Name: Serial Attached SCSI controller
   Programming Interface: 0x00
   Revision ID: 0x03
   Interrupt Line: 0x05
   IRQ: 5
   Interrupt Vector: 0x35
   PCI Pin: 0x00
   Spawned Bus: 0x00
   Flags: 0x3201
   Module ID: 4161
   Module Name: mpt2sas
   Chassis: 0
   Physical Slot: 1
   Slot Description: PCIE1
   Passthru Capable: true
   Parent Device: PCI 0:0:28:0
   Dependent Device: PCI 0:6:0:0
   Reset Method: Function reset
   FPT Sharable: true

[root@esx:~] esxcli device driver list
Device   Driver    Status  KB Article
-------  --------  ------  ----------
vmnic0   igbn      normal
vmhba2   mpt2sas   normal
vmnic2   ixgben    normal
vmhba1   vmw_ahci  normal
vmhba32  vmkusb    normal
vmnic1   igbn      normal
vmhba0   vmw_ahci  normal

[root@esx:~] esxtop
 4:48:13pm up 19 min, 852 worlds, 5 VMs, 17 vCPUs; CPU load average: 0.03, 0.03, 0.12

 ADAPTR PATH                 NPTH AQLEN   CMDS/s  READS/s WRITES/s MBREAD/s MBWRTN/s DAVG/cmd KAVG/cmd GAVG/cmd QAVG/cmd
 vmhba0 -                       1   992     0.77     0.19     0.58     0.00     0.00     0.71     0.03     0.74     0.01
 vmhba1 -                       1   992     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
 vmhba2 -                       1   600     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00
vmhba64 -                       0  1024     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00     0.00