NetApp MetroCluster cabling

General Information:

These cable shapes are NetApp MetroCluster specific and have next to no use in documenting other environments. Following information is heavily NetApp MetroCluster focused, if you have no interest in NetApp MetroCluster, you can stop reading now :). MetroCluster specific cables might have limited value in documenting other Metro capable solutions, Nutanix MetroAvailability comes to mind as one example.

Since NetApp Metrocluster solution spans over two sites, some additional cable shapes are required. This article if focused on NetApp Fabric Metrocluster.

There are also other flavors of NetApp MetroCluster than Fabric based Metrocluster, few words about these before we dive into Fabric MetroCluster

Stretched ATTO / FC based NetApp MetroCluster

• This one can be documented with existing cables
• Instead of using Back-End SAN to carry traffic between sites, each connection is carried over dedicated fiber link
• Typically implementing this over distance is cost prohibitive
• Normally used over short distances, like between rooms, rather than over larger geographical distance

Stretched SAS cable based NetApp MetroCluster

• I have experimented with cabling for this one
• But haven’t seen any to be sold
• Cost of required fiber pairs steers typically to either ATTO based streched MetroCluster or Fabric MetroCluster
• So I haven’t put much effort into developing cabling
• One major hurdle is required special cable
  • The other end of cable has just one connection point
    • MPO cable carrying multiple fiber pairs
    • Connects to QSFP connector on the controller side
  • The other end of the cable is the difficult one
    • In order to carry four fiber pairs over normal lc-lc connected fiber, the cable has to split into four lc-lc connector pairs
    • This would require a line shape that splits into four cable ends in the other end
  • I’ve reserved colors for these cables for future development
    • Bright Yellow for primary connection
    • Purple for alternate connection
  • Don’t hold your breath while waiting for these shapes to complete
    • Now that I know how to make custom cable ends, there might be a solution, but this is on the very-bottom-of-my-to-do-barrel

Fabric MetroCluster

Required Cables

• FCVI cabling for NVRAM mirroring across sites
• ISL cabling / links for spanning networks across sites
  • For Fabric MetroCluster at least Back-End SAN has to span both sites
    • Uses FC ISL cable shape between sites
  • Normally at least 10GbE or 8/16G Front-END SAN spans both sites
    • Uses 10GbE ISL or FC ISL cable shapes between sites
  • Quite often if you have both 10GbE and 8/16G SAN networks, both span sites
    • Uses 10GbE ISL and FC ISL cable shapes between sites
  • The cable between sites can be a “dark fiber” or there might be additional networking equipment in between, such as CWDM or DWDM
    • There are no xDWDM shapes in my stencils, so ISL cable can represent either “dark fiber” or xDWDM connection

FCVI cable shapes

I am slowly running out of primary colors and have to revert into using shades. FCVI cable has only one usage, it is only used with MetroCluster. Further more this cable has quite distinctive way of using, so I didn’t spend too much time thinking about this one and just picked a color. The color in question happened to be “dark”  or “dirty” red.

• FCVI cable specs
  • Color: “Dirty Red”
  • Cable End: “Square” with “16G” speed
  • Variants:
    • Primary (solid line)
    • Alternate (dotted line)
  • Speed:
    • 16G
      • All Clustered Ontap Metroclusters use 16G FCVI connections
    • If you need to document older / slower Metroclusters
      • Use old FCVI cable with no speed indicated, just “Open Square” cable end
      • or
      • Pull one “normal” 8G FC cable into the drawing
      • This will import custom 8G FC cable end into the drawing
      • Use right-click menu “format” to change line end for FCVI cable from 16G to 8G

Screenshot: FCVI cables in Visio Shapes Window

• FCVI cable usage
  • Running between FCVI adapter or UTA2 ports (with FAS8200 /  AFF A300)
  • and
  • Back-End SAN switch FC ports
  • Primary FCVI cable variant is connected to first Back-End SAN switch
  • Alternate FCVI cable variant is connected to second Back-End SAN switch
  • Normally there should not be more than two FCVI cables per controller (at least for now, might change in future)
  • Otherwise follow general instructions for running cables, like ones found in FC general cabling instructions

ISL cable shapes

For ISL cable I chose to use “dark” or “dirty” orange. Since ISL cable in real world is the same type of cable as normal fiber cable used with 10GbE or FC (at least the first leg of the cable run), so it would be nice if there is some relation between the cables also in colors.

• ISL cable specs
  • Color: “Dirty Orange”
  • Cable End:
    • For FC “Square” with speed
    • For 10GbE “Round” with speed
  • Speed:
    • For FC: “8G” and “16G”
      • 4G is too slow for Clustered Ontap ISL
      • If you need 4G ISL
        • Use old FCVI cable (no speed indication)
        • use same tricks as with 8G FCVI to change the speed to 4G
    • For 10GbE: well 10G 🙂

Screenshot: ISL cables in Visio Shapes Window

• ISL cable usage
  • For cables running between sites
  • For FC two fabrics are formed with one member per site
  • For 10GbE one layer 2 networks is spanned over two sites
  • Connects to FC / 10GbE switch ports on both sites / sides
  • No connections to host or storage controller ports
  • Shape used in ready-made switch shapes for MetroCluster
    • Only needed if not using ready-made MetroCluster switch shapes
    • If using follow general instructions for running cables, like ones found in FC general cabling instructions

Screenshot: ISL Cables with MetroCluster Switch Shape

Sorry about the tiny size, the shape is just too wide:)