Business Case
My QNAP-NAS (TS-431X2) is slow
- this must not be QNAPs fault, maybe the drives (4 disks, RAID5, between 147..167 MB/second for sequential read) are the root cause
- adding a SSD-Cache didn’t improve performance when I had a 3-disk RAID 5
- many people around the world are troubleshooting this without success
so I decided to
- use the QNAP-NAS as a Virtual-Tape-Library for Backup-Purposes etc
- use my ESXi-Host as Storage-Server
„Storage Server“ VM
I regularily use „Nested ESXi“-Hosts for Lab Environments which should access the storage, so I decided to first give NFS a try – ESXi has no built in NFS-Server, so a small VM using Free-NAS, True-NAS, I didn’t decide yet, should provide „feature parity“ to the QNAP System.
RDM is no option
Adding my former „QNAP Cache SSD“ to the ESXi-Host showed that it was impossible (as expected) to passthrough this disk to a VM.
RDM is no option since it doesn’t allow direct hardware access, eg. SMART-Counters or other statistics.
PCIe SATA-Controller: Dell PERC H310
A PERC H310 can get used as a passthrough device for a VM which will get full hardware access for up to 8 disks.
- as far I remember, this hardware has been released in 2011, but it might still be fast enough to be no bottleneck
IT Firmware
There’s nothing wrong with the original H310-RAID-Firmware, but since I won’t need those features i’d prefer the „IT“-Firmware, which offers much larger buffers to handle bursts easily.
This firmware has to be programmed, a lot of guides for this exist – I had to combine https://www.vladan.fr/flash-dell-perc-h310-with-it-firmware/ providing a link to a nice Dell-Firmware („6gbpsas.fw“) and a precise description what to do – but the „megarec.exe“ found in the archive didn’t work at my Asus mainboard („Error 8192“) so I found another megarec-release https://www.taste-of-it.de/flash-dell-perc-h310-mit-lsi-9211-8i-it-mode/ here.
[root@esx:~] esxcli hardware pci list | egrep -B8 -A26 "Device Name: Dell"
0000:06:00.0
Address: 0000:06:00.0
Segment: 0x0000
Bus: 0x06
Slot: 0x00
Function: 0x0
VMkernel Name: vmhba2
Vendor Name: LSI Logic / Symbios Logic
Device Name: Dell 6Gbps SAS HBA Adapter
Configured Owner: VMkernel
Current Owner: VMkernel
Vendor ID: 0x1000
Device ID: 0x0072
SubVendor ID: 0x1028
SubDevice ID: 0x1f1c
Device Class: 0x0107
Device Class Name: Serial Attached SCSI controller
Programming Interface: 0x00
Revision ID: 0x03
Interrupt Line: 0x05
IRQ: 5
Interrupt Vector: 0x35
PCI Pin: 0x00
Spawned Bus: 0x00
Flags: 0x3201
Module ID: 4161
Module Name: mpt2sas
Chassis: 0
Physical Slot: 1
Slot Description: PCIE1
Passthru Capable: true
Parent Device: PCI 0:0:28:0
Dependent Device: PCI 0:6:0:0
Reset Method: Function reset
FPT Sharable: true
[root@esx:~] esxcli device driver list
Device Driver Status KB Article
------- -------- ------ ----------
vmnic0 igbn normal
vmhba2 mpt2sas normal
vmnic2 ixgben normal
vmhba1 vmw_ahci normal
vmhba32 vmkusb normal
vmnic1 igbn normal
vmhba0 vmw_ahci normal
[root@esx:~] esxtop
4:48:13pm up 19 min, 852 worlds, 5 VMs, 17 vCPUs; CPU load average: 0.03, 0.03, 0.12
ADAPTR PATH NPTH AQLEN CMDS/s READS/s WRITES/s MBREAD/s MBWRTN/s DAVG/cmd KAVG/cmd GAVG/cmd QAVG/cmd
vmhba0 - 1 992 0.77 0.19 0.58 0.00 0.00 0.71 0.03 0.74 0.01
vmhba1 - 1 992 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
vmhba2 - 1 600 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
vmhba64 - 0 1024 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
PCIe-Passthrough
Passthrough has to be enabled manually, reboot the host and voila:

Now this device can get added to _one_ VM:


The (Ubuntu 20.04 Server-)VM seems to be fine with this setup:
