Ever since VMware has been around it’s been difficult to troubleshoot and isolate issues to do with storage. In the early days you had a RAID set and a volume, present the volume to VMware and then from then you would carve up virtual machines. VMware is like an Ogre, it has layers. However, if there ever was a performance issue it was difficult to know where in the stack the problem resided, was it the storage, memory balloon driver, network, vmotion.  I remember when Navisphere was first able to show a LUN and its corresponding VMware volume name. It was like finding that last ice cream at the bottom of the freezer. From there we’ve had plug-ins, tools, different tools, SRM etc. but still if there was an issue you had to go through each layer in the stack looking for issues. Enter vCenter™ Operations Manager (vCOps), an analytics VM that is constantly watching and learning about your virtual environment and providing dashboards to help resolve current issues, prevent future issues and look at ways the environment can be better optimized.

vCops Overview

vCops Overview

Unlike traditional monitoring tools that work on thresholds, like space full, latency, etc; vCops learns how your environment typically operates day to day and displays events that would be above or below normal. Over time it knows when your backups run, when the Monday morning report is run, when everyone takes off to the pub for Friday afternoon drinks. It looks for patterns and then triggers when events or metrics fall outside that normal pattern. It takes about 30 days of collecting data before vCOps data becomes useful. For example: a virtual machine that typically does 300 IOPS sustained and that changes to 700 IOPS sustained, the virtual machine health status will change and alert in vCOPs. Likewise, every Thursday a full backup is run and the storage is smashed, vCOps knows that’s the expected workload for a Thursday night so its all ok. There is a lot more to vCOps and if you want more info I suggest checking out @mwpreston’s site as he loves the stuff. http://blog.mwpreston.net/tag/vcops/ As a product vCOps has been around for a few years but with the latest version (5.7.1 when I wrote this) it now supports external data sources and custom dashboards, in particular around external storage metrics.

vCOps Storage Adapters

Heads up, it must be pointed out you need either advanced or enterprise versions of vCenter Operations Manager for the storage adapters to work.  Also the Dell Storage Adapter is a licensed product, but not too expensive in the scheme of things. Finally, most of this content is taken from a deck by Jason Boche and David Glynn at Dell User Forum in Miami this year, from a session called Monitoring a vSphere Environment with Storage Center and EqualLogic vCenter Operations Manager Adapters … say that three times with marbles in your mouth. vCOps Key FeaturesThe Dell storage solution pack adds value in a number of ways:

  • It delivers a servers to storage view of the health of your VMware infrastructure
  • Provides increased visibility, metrics, and rich storage analytics
  • Pre-packaged dell storage dashboards for alerts, health, performance, capacity, and top-utilization metrics
  • Performs analysis from VM to storage volume to speed root cause analysis

The storage adapter collects data through enterprise manager on Compellent (SC) and San HQ with EqualLogic (PS).  The data collection isn’t real time and can be up to 15 minutes behind depending on the values you choose to use (default is 5 mins for SC and 2 mins for PS).  This information goes into vCOps and it does its analytic magic from there. There are different UIs for VCOps, standard out of the box, Admin, and Custom.  The storage UI will have its own custom page. Standard UI

  • https://vcops UI FDQN
  • https://vcops UI FDQN/vcops-vsphere
  • Native VMware objects

Admin UI

  • https://vcops UI FQDN/admin
  • Configure/update vC Ops and install adapters

Custom UI

  • https://vcops UI FQDN/vcops-custom
  • 3rd party adapter configuration
  • 3rd party adapter objects
  • Custom dashboards

Loads of metrics are gathered by the adapter, arrays stats, controller, volume, front end and back end port stats.  By default there are a number of dashboards included with the adaptor but that doesn’t stop you creating your own custom dashboards that suit your business.  One bright feature is that shows you a visualization of the volumes presented to vCenter.  Volumes with issues will come up highlighted which you can then click and get information about the specific volume. There are little differences between the Compellent and EqualLogic metrics and I have included a screen shot of the differences below.

There is a great video by Josh Raw who is a Dell Storage Product Manager on Youtube that goes through the Solutions Pack, how to install it and demos a lot of the different dashboards (demo starts about the 7 min mark). He goes through the process of discovering an issue with a Virtual machine and drilling down to the volume that is the source of the problem, then eventually working out another VM on that volume is running a rouge process.  https://www.youtube.com/watch?v=_6ZkNOolQnY 

Requirements

As of July 2014, best to check the release notes:

  • vCenter Operations Manager 5.7.1 or higher (Advanced or Enterprise)
  • Storage Center 5.5 or newer, 6.3 or newer
  • Enterprise Manager 2014 R1 or newer
  • Enterprise Manager listens on TCP 3033
  • Credentials for EM or SAN HQ
  • SAN HQ version 3.0 or greater
  • EqualLogic Statistics Agent installed on SAN HQ Server
  • EqualLogic Statistics Agent listens on TCP 5040
  • EqualLogic firmware 6.0.7 or newer

 Screenshots of Storage Dashboards