Disk shelves as stacks

Background:

Older DS2246 / DS4246 / DS4486 based shelves contain lots of intra shelf cabling and running cables every time when drawing rear view was quite boring task. So I decided to organize rear view disks shelves into complete SAS stacks instead.

With New DS224c and DS212C shelves there are less cables as there are no ACP cables to run anymore, but you still have to run SAS cabling, so I’ve organized these as SAS stacks as well.

Stencils:

• DontPokeThePolarBear-NetApp-Stacks-2246-Stencil-<DD-MM-YYYY>
• DontPokeThePolarBear-NetApp-Stacks-4246-Stencil-<DD-MM-YYYY>
• DontPokeThePolarBear-NetApp-Stacks-4486-Stencil-<DD-MM-YYYY>
• DontPokeThePolarBear-NetApp-Stacks-DS2xxC-Stencil-<DD-MM-YYYY>

Original source of disk shelf shapes:

• NetApp

General Information:

• Shelves are organized into complete SAS stack shapes
  • DS224C: 1-10 shelves / stack
  • DS212C: 1-10 shelves / stack
  • DS2246: 1-10 shelves / stack
  • DS4246: 1-10 shelves / stack
  • DS4486: 1-5 shelves / stack
    • Cannot share DS4486 shelves between nodes, min 2 shelves / HA-pair
• Two variants of stacks
  • Stacks running up from controller and stacks running down from controller
• MetroCluster stacks
  • Disk shelves are deployed in pairs
  • With 2-node Metrocluster it is best practice add shelves in increments of four to maintain shelf-based ownership
    • 4-shelf and 8-shelf shapes for 2-node metroclusters
  • With 4-node Metrocluster it is best practice add shelves in increments of eight to maintain shelf-based ownership
    • 8-shelf shapes for 4-node metroclusters
• Shelves are grouped within a Grouping Box
  • Data Fields for Grouping Box:
    • Stack Number: (0-9)
    • Start with Stack 1 (ID:10-19) to match stickers that come with the unit
    • Stack 0 (ID: 0-9) is reserved for 25xx systems and exceptions
      • In very rare case that system comes with 10 stacks, Stack 10 = ID:0-9
    • Stack Owner: (Mix-Disk, Mix-Shelf,node1,node2..node24)
  • Grouping Box contains empty box for SAS Stack Port Assignment shape, located next to the first shelf of the stack
    • Port assignments vary based on FAS model and number of SAS adapters/SAS stacks
    • Choose appropriate port assignment shape (more about this later)

Shelf Data Fields:

• ShelfID
• Number of Disks (HDD)
• Disk Size (HDD)
• Number of Disks (SSD)
• Disk Size (SSD)

Notes on Shelf Data Fields:

• Early versions of the stacks contained lists of available disks within shelf shape
  • This proved to be a major source of frustration when NetApp released a new disk size. I had to individually update hundreds of separate shelf shapes to contain a new disk size
  • Current version stores such lists in master shape sheet for the document, instead of storing data individually within shapes or their shapesheets. This way if I need to add new disk model I only have to modify one list
  • These lists are referenced by individual shapes
  • Same mechanism is used widely in my stencils, if there is a list of possible values for a data field, most likely it can be found from the document master shapesheet
• If stack contains multiple shelves, changing Shelf ID for the first shelf of the stack will change all the other IDs accordingly (incrementing shelf ID by one for each additional shelf)
• Additionally for the new DS224C and DS212C shelves disk configuration of the first shelf is mimiced to other shelves in the stack
  • You can override this behaviour, by changing “Unique shelf config” from “No” to “Yes”.
  • Turn setting back to “No” to will make the shelf mimic the disk configuration of the first shelf
• DS224C and DS212C stacks are actually using the same stack shapes since there is no visible difference between the shelves in rear view.
  • Change shelf type in shape data window
  • Available disks lists are populated based on the model chosen

Update 22.11.2016

• DS2246/DS4246/DS4486 shelves have now same functionality as DS212C/DS224C shelves, disk configuration of first shelf of stack will be displayed in the label for all the shelves in the stack (unless overidden)
• And are using new SAS6 / ACP cabling

Update 22.11.2016 Quad-Path HA cabling

• There is now a separate stencil for quad-path ha cabled DS212C/DS224C stacks
  • I tried to combine dual-path and quad-path into one stencil, but unfortunately using Visio layers only works with visio, when exported to PDF quad-path cabling was always visible dispite layer settings in Visio
• quad-path HA cables in stacks are on different Visio layer than rest of the shapes.
• You can toggle quad-path cable visibility by right-clicking first shelf of stack
• By default quad-path cables are not visible
• One caveat is that quad-path visibility setting is done at page level, so all DS2xxC shelves will have quad-path cables visible or none of the DS2xxC shelves will have quad-path cables visible
• if you have both dual-path and quad-path cabled shelves, separate them on different pages or run quad-path cabling manually

Screenshot: right-click menu of first shelf of a stack, quad-path cabling hidden (only blue SAS cables)

Screenshot: right-click menu of first shelf of a stack, quad-path cabling shown (blue & green SAS cables)

Update 16.11.2016 New cable shapes for DS4246 stacks

• Since there is no new 12Gbps 4U shelf (yet?), I could see that some customers still want to order old DS4246 shelves even with the new FAS2600/FAS8200 platforms
• I have updated all cable shapes, also old SAS6 and ACP cables
• Replaced cabling for DS4246 non-metrocluster shapes to use new cables

 Screenshot: DS4246 stack with new ACP & SAS cables (updated cable ends with ACP / 6G text)

Example of first DS224C/DS212C shelf shape data window:

First shelf of the stack is chosen (balloons visible on either side of the disk shelf). There is no “Unique shelf config?” field in shape data as this is the first shelf of the stack. Any changes made to disk configuration will be mirrored to the label/text shown for the other shelves in the stack (unless overridden). First shelf of the stack contains “ShelfId” field, if you need to change ShelfIDs for the stack, change the ID for the first shelf and all the other shelves will change accordingly.

Example of subsequent DS224C/DS212C shelf shape data window:

The second shelf in the stack is selected (little balloons on either side of the shelf). Shelf ID is not pull-down menu as you should NOT manually edit it. You will break automatic shelf id assignment,  if you decide to manually edit the Shelf ID for any other shelf than the first shelf of the stack. Only way to fix this is to remove the stack shape and pull a fresh one from stencil.

“Unique shelf config?” is set to “No”, so this shelf will have same disk config as the first shelf. Any changes made for disk configuration of this shape will not be shown in the drawing, like shown in the example. Even though in Shape Data window Disk Size (HDD) is set to “SSD 1.6TB”, in the drawing disk size is shown as “SSD 3.8TB”, because the shelf is mimicking the disk configuration of the first shelf.

Example of subsequent DS224C/DS212C shelf shape data window where shelf has unique configuration:

The second shelf in the stack is selected (little balloons on either side of the shelf). “Unique shelf config?” is set to “Yes”. This enables this shelf to have unique disk configuration.

Unlike other shelves in the stack ShelfType is set to “DS212C”.

Disk shelf shape has some additional coding, if you change ShelfType it also has impact on other data related to disk shelf. Different shelf types can have different maximum number of disks and can be populated with different types of disks.

With DS224C shelf = Max 24 disks, so both “Number of Disks (HDD)” and “Number of Disks (SSD)” pull-down menus have a range of values from “1x” to “24x”. Additionally “DiskSize” pull-down menus for both HDD and SSD are populated with disk types compatible with the DS224C disk shelf

DS212C can have maximum of 12 disks, so both “Number of Disks (HDD)” and “Number of Disks (SSD)” pull-down menus have a range of values from “1x” to “12x”. Additionally “DiskSize” pull-down menus for both HDD and SSD are populated with disk types compatible with the DS212C disk shelf

By using “Unique shelf config” setting you can build mixed DS224c/DS212C stacks if required or just have same type of shelves but different disk configuration as needed.

Cautions when working with stacks:

• Do not edit shape text fields manually as it breaks the connection with data
• You can mix and match SSD/HDD as you like
  • Every shelf can have different mix of disk sizes and types
  • There is no checking
  • Disk shelf can contain maximum of 12 disks for DS212C
  • Disk shelf can contain maximum of 24 disks for DS2246/DS224C/DS4246
  • Disk shelf can contain maximum of 48 disks for DS4486
• So Number of Disks (HDD) + Number of Disks (SSD) should be = Max number of disk per shelf type
• DS4486 only HDD, no SSD allowed
• Do not change shelf ID for any other shelf than the first shelf of the stack
  • It will break automatic shelf ID assignment
• When adding shelves to stack and updating drawing
  • Remove old stack shape
  • Use new stack shape with correct amount of shelves and reconnect cables
  • Might have to move other objects in order to make room for the new shape with more shelves
  • Might be a good idea to leave some free space around disk stack shapes
• Manually adding shelves to stack shape will break automatic shelf id assignment
• Only if there is no shape with correct amount and/or type of shelves edit stacks manually
• Most likely case requiring manual editing is mixed stacks with both DS2246 and DS4246 shelves
• Like always there are always exceptions that you can’t prepare for

Examples:

SAS stack with eight DS224C shelves:

Running up from controller, first shelf of the stack on the bottom. There is no ACP cabling as DS224C (or DS212C) shelves do not use it. Shelves are connected with 12gb SAS cables.

SAS stack with three DS224C shelves:

Running down from controller, first shelf of the stack on top. No ACP cabling. Shelves are connected with 12gb SAS cables

SAS stack with six DS2246 shelves:

Running down from controller, first shelf of the stack on top. It is recommended to use ACP cabling (red cables) with DS2246 shelves, so there are more cables. SAS cabling is using 6 gb cabling (blue cables)

SAS (DS2246) stacks for 2-node MetroCluster

Four is recommended minimum number of shelves for 2-node MetroCluster. Each node has one shelf per site (pool 0 and pool 1) and data is mirrored between shelves ( and sites). Each shelf is owned by a single node and you can use shelf based ownership and replaced disks are automatically assigned to correct node / pool.

If number of shelves is not divisible by four, you will end up  using disk based ownership. When replacing disks you have to assign disk ownership manually, as there are four possible pools, so you will have 25% chance of assigning the disk to correct pool. I highly recommend against disk-based ownership with Metroclusters, as you might accidentally place the disk in wrong pool on wrong site and end up in a situation where both copies of your syncmirror reside on same site and should your site go down, you will lose or corrupt some data.

SAS (DS2246) stacks for 4-node MetroCluster

Eight is recommended minimum number of shelves for 4-node Metrocluster. With four nodes you have eight pools for the disks to reside and in case of disk based ownership 12.5% chance of getting right pool when assigning replaced disks. With 4-Node Metrocluster I wouldn’t recommend using disk based ownership. Add shelves in increments of eight and use shelf based ownership.

stack of ten DS4246 shelves

ACP / SAS cables have different routing than with DS224x shelves.

Stack of five DS4486 shelves

DS4486 is a rare “bird” in my neck of the woods. I haven’t seen one live :), so I don’t have much to say about this creature. Some additional restrictions when implementing:

• no disk based ownership
• no SSD drives allowed
• Max five shelves per stack