SR-IOV technology on POWER – my few words

Finally the announcement day arrived. POWER8 and SR-IOV have been introduced to the world, and I can share my experience from my last residency.

Single Root I/O Virtualization are not new adapters, but this is a new technology, standard, specification. In fact, it uses adapters which are on the market for some time already. Specification allows to get a slice of a physical adapter and assign it an LPAR. This feature requires a proper hardware, firmware, and all elements around it.

What is all about? Basically it allows to virtualize a physical adapter (in the SR-IOV world its called ‘physical functions’) to multiple logical adapters (called ‘virtual functions’) which might be used by independent clients (LPARs).

For Ethernet it’s very similar to the Logical Host Ethernet Adapter (HEA), the technology present on POWER6, and early POWER7 machines. The SR-IOV has all benefits of predecessor plus brings few extra.

• scalability – you can have up to 48 LPARs per adapter
• you can have multiple SR-IOV cards in the machine, PCIe or Integrated multifunction adapters
• you don’t have to restart the entire server while you increasing number of logical ports – that was the requirement in the HEA, when you changed the MCS value
• all configuration is thru the HMC, user doesn’t have to bother about any commands
• it doesn’t consume main CPU cycles, while the Shared Ethernet Adapter does
• it offers Quality of Service
• it offers Link Aggregation support

Unfortunately, Live Partition Mobility, suspend and resume features are not supported if the SR-IOV logical ports directly being used. The interface is treated as regular physical adapter from an OS perspective. If you want to use above features in conjunction with the SR-IOV, you need to use the SEA in between.

This is the biggest inconvenience, otherwise the SR-IOV would be almost perfect. I think, it would be possible to redesign the SR-IOV, hence to support the LPM, but it would require much more effort from different development teams: hardware team, Hypervisor team, and several Operating systems teams. That would significantly increase the development cost. But this is my private humble opinion.

What do you need to use it?

• compatible iron (MMD,MHD, EAD)
• compatible adapter (EN10,EN11,EN0H, EN0K)
• platform firmware (780)
• VIOS (2.2.3.3)
• HMC (7.7.9)
• AIX 6.1 TL9 SP2, AIX 7.1 TL3 SP2
• IBM i 7.1 TR8 or 7.2
• Linux SLES 11 SP3
• Red Hat 6.5
• PowerVM Standard or Enterprise license

At the present day, it’s supported only on POWER7 platform, but IBM has statement of direction to get it running on POWER8.

I think for several IBM i shops its a good deal, and the good product. LPM is still not popular. Configuration is very easy, and the scalability plus simplified implementation is a big advantage.

Personally, I’m not so sure how the market will behave, when IBM brings simplification for PowerVM SEA management with the newest HMC release. I saw, and tried it. It’s absolutely amazing stuff for managing a VIOS by people who doesn’t know UNIX. Everything is done via a GUI interface. Not need to touch CLI at all.

The fact is that the SEA or the SR-IOV have pros and cons – with a VIOS scenario – you get an extra layer, which you have to manage, back up, secure. While the SR-IOV its basically just enhancement of the hardware.

One important feature which I’d like to highlight for IBM i, is an extra performance metric (file) added into the Collection Services. The file called QAPMETH. Basically it allows to collect/track all data, pockets which are transfered thru a physical port, where you IBM i machine is connected to. This is possible when the performance information collection has been enabled on the partition properties.

The next step for IBM is prepare the SR-IOV for fiber channel adapters. Thus, would make much easier NPIV configuration. Many IBM i shops might finally start to use PowerVM virtualization in the way as it is designed to. But for this, we must to wait few more months.