Saturday, January 2, 2010

VMware vSphere Cluster Size

How many ESX/ESXi hosts do you put in a Cluster?

If there is a need for >8 hosts, then I'd go for 8 hosts per cluster.

Reasons why 8, not 4 or 12 or 16 or 32 hosts:
  • Best practice for cluster is to give is same hardware spec with same CPU frequency.
    • Eliminates risk of incompatibility
    • Complies with Fault Tolerant best practices
    • So more than 8 means it’s more difficult/costly to keep them all the same. You need to buy 8 hosts a time, which can be expensive.
  • Too few hosts result in overhead (the “spare” host)
  • Too many hosts are harder to manage (patch, performance troubleshooting, too many VMs per cluster, HW upgrade)
  • Some cluster changes in the Advanced Attributes requires cluster to be disable and enable.
    • It is harder/longer to do this when there are many hosts
  • DRS: 
    • 8 hosts give DRS sufficient space to “maneuver”
  • #VM per host decrease by 4x in >8 host.
    • 160 VMs per host if <= 8 hosts in cluster
    • 40 VMs per host if > 8 hosts in cluster
    • We should avoid being near the limit.
    • 40 VM/host is easily reached in Lab Manager or View environment.
  • Availability: Able to withstand 1-2 host failures
  • A balance between too small (4 hosts) and too large (>12 hosts)
  • Allow us to isolate 1 host for VM-troubleshooting purpose
  • Upgrading >8 servers at a time is expensive ($$) and complex
  • Storage: 8 hosts/LUN à a safe value of 16 paths to a LUN
  • Consistent with VMware View
    • View 4 cluster is limited to 8 hosts if we use View Composer
  • 8 is easy number to remember. And a lucky one. And we all know that production needs luck, not just experience :-)

That's for my first posting. I sincerely hope it's useful.

7 comments:

  1. Hi Iwan,

    Thanks for your post, good information. Looking forward to other posts from you as well.

    Best wishes on your new blog.

    Ken

    ReplyDelete
  2. Found this while justifying our cluster sizing. Very useful, thanks.

    ReplyDelete
  3. Found this while justifying our cluster sizing. Useful summary, thanks.

    ReplyDelete
  4. Thanks Ed. Glad to know it has been useful. We are all learning together in this cutting edge journey of virtualisation, so sharing is definitely good.

    ReplyDelete
  5. Could you provide some resources for the following comments?

    "Some cluster changes in the Advanced Attributes requires cluster to be disable and enable."
    Which advanced attributes would those be?

    "# 160 VMs per host if <= 8 hosts in cluster
    # 40 VMs per host if > 8 hosts in cluster"

    I've been looking for this kind of direction for some time. I've heard the 8 host recommendation many times but cannot seem to find documentation to back it up with technical reasons.

    Thanks,

    Bill Griffith

    ReplyDelete
  6. Hi Bill,
    Sorry that I can't find that info... I saw it before but unable to recall where it was... Generally speaking, cluster to me is a unit of operation. It is a building block. DRS, EVC, DPM, HA, FT operates within the cluster boundary. The cluster hierarchy is also recognised by some management/addon products like View, Chargeback, CapacityIQ, etc.

    And in some unfortunate situation, problems may impact hosts within the same cluster.

    As per the 40 VM per host if >8, that is in vSphere maximum config guide. The good thing is, this is no longer the case in 4.1.

    ReplyDelete