Older generation SAS 6Gbps ports use QSFP form factor connector while newer generation SAS 12Gbps ports use smaller Mini-SAS HD form factor connector (at least with NetApp gear). These form factors are not compatible, i.e you cannot cable QSFP connector to Mini-SAS HD port or vice-versa.
To make things more confusing, there are also 12Gbps capable ports that are using QSFP form factor connector. In case of NetApp, X2069-R6 adapter is example of such combination.
In order to connect ports using different form factor connectors, a special transitional cable is required:
- One cable end with QSFP form factor connector
- With NetApp onboard ports speed is typically limited to 6 Gbps
- With NetApp adapter cards speed can be up to 12 Gbps
- One cable end with Mini-SAS HD form factor connector
- Typically speed is 12 Gbps
- But can also operate at 6 Gbps
There are few use cases where transitional SAS cables are required.
6Gbps QSFP controller ports – to – 12 GBps Mini-SAS HD disk shelf ports
QSFP Controller ports
- Onboard SAS ports on previous generation FAS/AFF controller
- Operates at 6 Gbps
- 4-port 6 Gbps SAS adapter card
- X2065 Operates at 6 Gbps
Mini-SAS HD Disk shelf ports
- DS212C and DS224C disk shelf
- Equipped with IOM12 module
- Operates at 12 Gbps
Typical use case
- Older generation FAS/AFF controller using new generation disk shelves
- NetApp P/N for 1 m cable:
- X66020A-R6
Visio shapes
- Controller side operates at 6Gbps
- Disk shelf side operates at 12 Gbps
- hence name SAS6-SAS12
- Since “destination” ports are labeled with numbers instead of shapes, the cable shapes are named accordingly.
- Variants
- Node1 cables: dark blue
- A or C ports to “1” port
- B or D ports to “3” port
- Node2 cables: light blue
- A or C ports to “1” port
- B or D ports to “3” port
- Node1 cables: dark blue
- Cable ends
- Square and Circle shapes with either 6G or 12G text indicating speed
Screenshot: SAS6-to-SAS12 transitional cable shape thumbnails in Visio Shapes Window
Example: Older generation FAS2552 with new generation SAS DS224C shelves
- SAS cable shapes have QSFP (“6G”) style cable ends at controller side
- SAS cable shapes have Mini-SAS HD (“12G”) style cable ends at disk shelf side
- No ACP cabling since new disk shelves do not have separate ACP ports
- Effective speed determined by the slowest component
- Controller ports 6Gbps
- IOM12 ports 12Gbps
- Shelf back-plane 12Gbps
- = Effective speed 6Gbps
- Instead In-Band Alternate Control Path (IB ACP) is used instead (requires Ontap9)
- Can operate without ACP with older Ontap versions (with some limitations on admin operations)
12Gbps Mini-SAS HD controller ports – to – 6 GBps QSFP disk shelf ports
Controller ports Mini-SAS HD
- Onboard ports on new generation controllers
- FAS26xx / AFF A200
- FAS8200 / AFF A300
- FAS9000 / AFF A700
- 4-port SAS adapter
- X92071A for FAS9000 / AFF A700
- No adapter with Mini-SAS HD ports for FAS8200
QSFP Disk shelf ports
- DS2246 / DS4246 / DS4486 disk shelf
- Equipped with IOM6 module
- Operates at 6 Gbps
Typical use case
- New generation platform connecting to older generation disk shelf
- Since there is no new 24-bay disk shelf for 3.5″ inch drives (ie no DS424C shelf)
- I would expect that most common example would be new FAS26xx using older generation DS4246 shelf
- NetApp P/N for 1 m cable
- X66020A-R6
Visio shapes
- Controller side operates at 12Gbps
- Disk shelf side operates at 6 Gbps
- hence name SAS12-SAS6
- Since “destination” ports are labeled with shapes, the cable shapes are named accordingly.
- Variants
- Node1 (N1) cables: dark blue
- A or C ports to Square port
- B or D ports to Circle port
- Node2 (N2) cables: light blue
- A or C ports to Square port
- B or D ports to Circle port
- Node1 (N1) cables: dark blue
- Cable ends
- Square and Circle shapes with either 6G or 12G text indicating speed
Screenshot: SAS12-to-SAS6 transitional cable shape thumbnails in Visio Shapes Window
Example: New generation FAS2650 with Older generation SAS DS4246 shelves
- SAS cables shapes have Mini-SAS HD (“12G”) style cable ends at controller side
- SAS cables shapes have QSFP (“6G”) style cable ends at disk shelf side
- Effective speed determined by the slowest component
- Controller ports 12Gbps
- Shelf back-plane 6 Gbps
- IOM6 ports 6Gbps
- = Effective speed 6Gbps
- No ACP cabling since new FAS platforms do not have “Locked Wrench” port used for ACP cabling – No out-of-band Alternate Control Path (OOB ACP)
- Instead In-Band Alternate Control Path (IB ACP) is used instead (requires compatible shelf firmware for IOM6 shelves & Ontap9)
- Minimum Ontap for FAS2650 is 9.1RC1
12Gbps QSFP controller ports – to – 12 GBps Mini-SAS HD disk shelf ports
QSFP Controller ports
- 4-port SAS Adapter
- X2069-R6
- For FAS8200 / AFF A300
- And older FAS generations
- Is cabable of 12 Gbps
Mini-SAS HD Disk shelf ports
- DS224C / DS212C disk shelf
- Equipped with IOM12 module
- Operates at 12 Gbps
Typical use case
- FAS8200 with more than two SAS stacks
- or
- FAS8200 with two SAS stacks and quad-path HA cabling
- Both require additional SAS ports in form of PCI SAS-adapter
- Four onboard controller ports are using Mini-SAS HD form factor
- Mini-SAS HD – to – Mini-SAS HD cable
- NetApp P/N for 1 m cable:
- X66031A
- Four SAS adapter (controller) ports are using QSFP form factor
- QSFP – to – Mini-SAS HD cable
- NetApp P/N for 1 m cable:
- X66020A-R6
- I expect that most FAS8200 / AFF A300 systems will only have max two SAS stacks and no SAS adapter in PCI slot
- Platform maximum spindle count: 480
- Can be achieved with only two SAS stacks (when using only spinning disks)
- Quad-Path HA will most likely be fairly rarely used with FAS8200
- most likely bottle neck will not be SAS throughput
- BUT when using more than four SAS ports with FAS8200
- This is when things can get messy
- expect to see configurations ordered with wrong cables
- I hope that NetApp will release 4-port Mini-SAS HD adapter for FAS8200 series, so only one type of SAS cable can be used
- There is already one available for FAS9000 platform
- This is when things can get messy
Visio shapes
- Not available yet, still figuring out what do
- With other SAS cable shapes “6G” = QSFP and “12G” = Mini-SAS HD
- Problem is that we have now QSFP operating at 12 Gbps speed
- Maybe create a special “Q12G” or similar cable end
- Very limited use case, might just confuse more
- Or use existing transitional cable shapes and foot note that it operates at 12 Gbps despite cable label
Example: Quad-Path HA cabled SAS Stack with FAS8200 using both QSFP and Mini-SAS HD ports on controller side
- Four “Mini-SAS HD -to- Mini-SAS HD” cable shapes
- Both cable end shapes with “12G” text
- Four “QSFP -to- Mini-SAS HD” cable shapes
- QSFP cable end with “6G” text
- Mini-SAS HD cable end with “12G” text
- Effective speed determined by slowest component
- Controller ports 12Gbps
- Shelf back-plane 12 Gbps
- IOM12 ports 12Gbps
- = Effective speed 12 Gbps
- No ACP cabling since no ACP ports on controller or IOM12 module, No out-of-band Alternate Control Path (OOB ACP)
- Instead In-Band Alternate Control Path (IB ACP) is used instead
DontPokeThePolarBear 
Thanks for the excellent Visio templates and examples posted on your site.
Not directly shown in this article, but I believe the NetApp-Examples-FAS2650-one-DS224C-shelf drawing available from your general downloads page is in contradiction with NetApp’s best practice guide when combining the FAS2650 with DS224C (IOM12) shelves.
According to Page 29 of:
https://library.netapp.com/ecm/ecm_get_file/ECMLP2485106
For FAS2600 series configurations, you must have same-domain connectivity between controller port 0b (the internal storage) and the stack of disk shelves it is connected to.
When a controller is located in slot A of the chassis (controller 1), its internal storage port (0b) is in domain A (IOM A); therefore, port 0b must connect to domain A (IOM A) in the stack.
When a controller is located in slot B of the chassis (controller 2), its internal storage port (0b) is in domain B (IOM B); therefore, port 0b must connect to domain B (IOM B) in the stack.
LikeLike
Hi Rob,
Thanks for kind words and comments.
I’ve fixed the example drawing you mentioned, there are few more iterations of the same drawing to be fixed. I will fix these later.
This is probably the reason why SAS ports on older FAS25xx platform are in “weird” order, counting from left, first 0b port(with square label) and then 0a port (with circle label).
With the new FAS2600 platform NetApp fixed the “weirdness” with port naming/order and SAS ports are in “normal” order, first 0a port and then 0b port. Since NetApp has removed shape based SAS port labels “Square / Circle”, some of the visual cues are missing and it is easier to make mistakes, like the one I did. I wish that they would have continued using shape based labeling of SAS ports, maybe just adding numbers behind the shapes.
LikeLike
Update: All FAS2650 example drawings with SAS cabling on downloads page have now been fixed. Similar error was also in one of the drawings in this post “Example: New generation FAS2650 with Older generation SAS DS4246 shelves”, fixed too
LikeLike
Hi Sami,
Thanks for the fast response.
With the recommendation for the SAS 0b ports on the FAS2600 series controllers to connect to an IOM port on the same respective domain (i.e., ctrl-A, 0b connects to IOM-A / ctrl-B, 0b connects to IOM-B), this brings up another consideration:
Should these 0b connections actually go to port 3 rather than port 1 on the IOM12 modules? This would keep consistency with the general rules of:
– A and C ports always connect to the first disk shelf in a stack
– A and C ports always connect to disk shelf IOM ports 1 and 2.
– B and D ports always connect to the last disk shelf in a stack
– B and D ports always connect to disk shelf IOM ports 3 and 4
The FAS2600 cabling worksheet on page 30 from my link above appears to indicate this is the case. Hopefully I’m not becoming a pest, appreciate your input.
LikeLike
Hi Rob,
No worries, questions like this was one of the reasons why I started to make Visio drawings, which eventually led to tweaking the original NetApp stencils 🙂
You are correct with your findings.
It seems that NetApp has decided to fix some inconsistencies that were present with FAS2500 platform, where 0b port was labeled also with “Square” shape. This indicated that the counter part port on the shelf side would be a “Square” port. To follow other cabling rules that port would reside on IOMA (for controller A). This led to situation where FAS2500 was not following generic SAS cabling rules, because a “B” port was connecting to a “Square” port.
Even with these modifications FAS2600 platform is still little bit odd-ball. With other platforms Controller A 0a port would connect to IOMA (port 1), instead of IOMB (port 1). But I think that this is “cleaner” cabling sequence than with FAS2500 platform.
Dropbox link to a drawing using this cabling sequence:
https://www.dropbox.com/s/s38cqp9szjmas5c/DontPokeThePolarBear-NetApp-Examples-FAS2650-two-DS224C-shelves_v2.pdf?dl=0
I will fix the other examples later, got to run now
BR,
Sami
LikeLike
FAS2600 example drawings updated, once more, hopefully I got it right this time 🙂
LikeLike
Very awesome Visios. I use these all the time…can’t figure out how to make the square/circles at the endpoint on your cables smaller, but still very cool.
I believe the 25xx system is backwards on the connections. The 0a port is actually on the right and 0b on the left for each controller. The way you have it above it will probably spit out an error on Config Advisor. Also, there is normally a mixing rule violation when adding IOM12 to an existing stack of IOM6 shelves. Is this ignored for FAS25xx systems since they come with IOM6 (or is there an upgrade module for this like there is for the DS2246s? Config Advisor will complain about the cabling and will even tell you that the shelf has no path to it.
https://docs.netapp.com/platstor/index.jsp?topic=%2Fcom.netapp.doc.hw-ds-sas3-icg%2FGUID-C20AC326-96F5-4F58-9C18-6AD278A69C81.html
“You can hot-add a stack of disk shelves with IOM12 modules to a system that has existing stacks of disk shelves with IOM6 and IOM3 modules.
Disk shelves with IOM12 modules must be in their own unique stack”
Thanks,
Chris
LikeLike
Hi Chris,
Thanks for kind words.
It has been almost three years since I’ve worked with NetApp, so my recollection why I did something the way I did are bit hazy 🙂
The endpoints were designed to be fixed size in order to make them fit over various ports and “Begin Arrow Size” / “End Arrow Size” settings found on “Format Shape” / “Line” menu do not change the size of the endpoint. Also changing the size might cause problems the text within the shape.
However the custom line endpoints can be edited and their properties can be changed. Master copy of the shape can be found from “Drawing Explorer” / “Line Ends”. Select any of the shapes – right click – “edit pattern shape”. This opens an editor window for the shape, if I recall correctly, it is somewhat limited and doesn’t always function as other Visio editing windows. And the changes you make won’t always behave the way you expect.
There is also right-click menu item “Pattern Properties” where you can change some settings for the endpoint shape.
It might safer to test with copy-pasted version of a line-end, make required changes and then change line end for one of your existing cable shapes to see how it behaves. If you edit directly existing line end shape, it might affect all your lines using that line end shape. To test this you probably need Visio Professional version and you need to run Visio in Developer mode.
My brain cells containing any FAS25xx information have been wiped clean, all I can remember that SAS ports were somehow illogical with FAS25xx and caused much headache. Same goes with IOM6 and IOM12 transitions, I have a vague recollection that either with FAS26xx or FAS25xx one transition between IOM6 and IOM12 was allowed, but I might be wrong.
BR,
Sami
LikeLike