One of the improvements that came along with Nutanix AOS 5.5 was IO path optimization feature called “AHV Turbo”.
While the marketing department at Nutanix might have fallen asleep and woken up in 1990s with god-awful name like this, it is actually a GOOD feature. Every time I hear word “Turbo” in IT related matters, it just reminds me of my first ever PC, a 286 with a “Turbo” button, which supposedly boosted performance by increasing frequency from 8 Mhz to 12 Mhz, but did actually do diddly-squat 🙂
What does AHV Turbo do?
In short the old IO path based on Qemu SCSI Data Path contained parts that were serialized, i.e some functions could be only executed one operation at a time. In AHV Turbo those parts are now upgraded to version where multiple operations can be issued at the same time, operations can now run in parallel. For more in-depth description of AHV Turbo, please check out: What’s .NEXT 2017 – AHV Turbo Mode.
What are the benefits of AHV Turbo?
By being capable of doing multiple things at the same time, you will get more work done and get better aggregate throughput (more IOPS) and response times (lower latency). This is where computers are different from human beings, with computers multitasking actually helps, most often with humans not so…
Is single Virtual Disk per VM enough to maximize performance?
According to Josh Odgers’ post above: “Another benefit of AHV and Turbo mode is that it eliminates the requirement for administrators to configure multiple PVSCSI adapters and spread virtual disks across those controllers.”
I think that above comment has generated some misunderstanding and I’ve heard from several sources that you only need one vdisk with AHV turbo to max out VM IO performance. Well that is not quite true, the post mentions that the need of multiple PVSCSI adapters is eliminated with AHV turbo, but does not say that you wont need multiple disks.
Testing single vDisk VM vs multi vDisk VM
To clear out confusion, I wanted to test if number of virtual disks had any relevance to max IO performance with AOS 5.5 (with Turbo mode). I created two Windows 2016 Server Virtual Machines with identical setup, except one point: E: drive that was holding test data was configured differently between Server1 and Server2. Server1 had only on vdisk. While Server2 had five vdisks, which were combined together as striped dynamic volume to form a single E: drive under windows.
While Nutanix EULA prevents me to show actual numbers, I can tell you that with multiple vDisks Server2 was outperforming single vDisk Server1 by a lot, Server2 was doing easily 4-5 times more IO operations than Server1 while maintaining similar level of latency. For both servers throughput and latency were better than with the same tests run on previous AOS 5.1 version.
“Multiple vDisks required” point is also verified in Michael Webster’s post 1 Million IOPS in 1 VM – World First for HCI with Nutanix. Where he states “The VM used is configured with locally attached disk volumes, 33 in total (1 for OS, plus the rest for the IO load).”
To get the absolute maximum performance
Both Windows and Linux workloads benefit from AHV Turbo without making any tuning or changes at the operating system layer.
However to squeeze out the absolute maximum out of Windows workload, a new multiqueue driver is required. Current VirtIO-SCSI driver package does not contain this driver yet, but the future versions will do.
Linux kernels 4.14 or later have already built-in multiqueue driver. You need to set SCSI MQ parameter to get the maximum out of multiqueue features of AHV Turbo. More info: Enabling AHV Turbo on AOS 5.5
You will still need multiple virtual disks to drive maximum IO performance out of single virtual machine, even with AOS 5.5 AHV Turbo. In future multiqueue drivers will boost performance even more.
Call to action
Update your Nutanix cluster to AOS 5.5 to enjoy improved IO performance. This demonstrates the beauty of software defined architecture: Even the existing customers will get the benefits of improved architecture. If your architecture is heavily hardware dependant, you will have to upgrade your hardware (and spend some money) to enjoy similar benefits.
5 thoughts on “First experiments with Nutanix AHV Turbo.”
Windows by default actually won’t benefit from one of the significant changes in AHV Turbo and that is multiqueue. You need the Windows Multiqueue driver and that isn’t yet GA. However when it is GA it will boost performance significantly. Single vDisk performance will benefit as well as multiple vDisks. The default VirtIO-SCSI driver won’t cut it to max out the performance in Windows. However the driver is built into Linux and that is what the additional parameter is needed.
Hi Michael. Thanks for the comment, I will update my post with this in mind
Do we have an idea when GA will be for Windows Systems?
You are referring to GA of multiqueue driver for Windows, right? I did not find anything about this from VirtIo1.1.2 release notes, so my guess is that is still under constuction. Maybe somebody from Nutanix can chime in…
That is correct, yes I am looking forward to testing this in my full windows environment.