HealthITGuy's Blog

The evolving world of healthcare IT!

Cisco Unified Computing System: User Impressions . . .

Yesterday we finished our 3 week implementation with our Cisco Advanced Service’s engineer.  We had started the process with a good size list of tasks we wanted to accomplish and test items.  I believe we covered all of the configuration items and now have working service profiles, policies, pools, etc. the way we want.  We also completed all of the hardware related testing that we could come up with.  Things like turn off the primary 6120 and watch it failover, took away an Ethernet path, Fiber Channel path, powered off a chassis and powered off a blade.  All things tended to react and be handle as expected.  The errors and warnings were logged in the manager, generally made sense and cleared up once the “failure” was resolved.


We have a server policy for boot sequence to use boot from SAN and then go to PXE LAN boot if the SAN LUN does not exist.  Our build process for a new ESX host on a UCS blade calls for it to not see a SAN LUN (this is to be expected since there is no OS yet), so it goes to the next boot order, in this case, PXE boot.  The PXE boot’s to our UDA (ultimate deployment appliance) and loads the ESX OS image.  After the ESX is loaded it reboots and does not fully see the SAN LUN due to some remaining steps.  Our work around is to associate the “almost” built ESX host to another blade where it completes the process successfully.  Strange steps to figure out at first. 

 vCenter Host Profiles:  We discovered an issue with using Host Profiles in vCenter when trying to apply them to an ESX host on a UCS blade with the 10 G CNA adapter.  The problem turned out to be the port speed option was set to auto negotiate and the CNA does not have an auto option.  This was resolved by hard setting the NICs in the Host Profile to 10 G and now it functions as expected.

vCenter Distributed Switch:  Our 2U servers have 6 to 8 – 1 GB NICs that we use for connectivity for each ESX host.  On the UCS blades we go to only 2 – 10 G NICs.  This means we need to have 1 distributed switch for our 2U servers and 1 for our UCS servers.  Not a real big deal, but something we had to work out.  I expect with the Palo CNA you could logical configure your blade I/O to appear to be just like your 2U server configuration, however, I am not sure you would want to do it that way.

Is UCS a more or less complex way to provide compute power? 

Early in our implementation process two of our engineers were debating this question.  Yes, there are more “things” to set/define for a service profile (server) which makes it seem more complex.  On the other hand, I would say the additional “things” are not really new, they are things you had to deal with in traditional blades or servers, it was just done in a non-central manner.  In addition, once you have defined your configurations and standards using the UCS tools such as, pools, policies and service profiles you end up with a more consistent and easier to manage environment.  So I think the complexity is really in the initial process of thinking through how to best utilize the pools, policies and service profiles.  This takes some time to define with the correct people in your organization.  Meaning your storage person needs to understand how defining your own WWPN can streamline your zoning and registration process.  Once these types of functions are laid out, the complexity is minimal.

How long did implementation really take? 

I would say racking, stacking, cabling and powering up the system was accomplished over a 2 day period. 

Once the Cisco implementation engineer arrived we spent 1 day going over concepts, scenarios and some white boarding. 

We jumped into about 2-3 days of “playing around” in the system creating pools, messing with policies, creating service profiles, etc.  This was our hands-on training process with about 3 or 4 team members.  During this time is when we configured the “northbound” LAN connections to our 6500 switch which took a little extra time due to the confusion with different terms used for “native” vlan.  We also took care of the SAN connectivity to the Cisco MDS switches, very straight forward.

The next 2 to 3 days was used to clean up our sandbox configurations and build our production configuration for pools, policies, templates, etc.  We also determined we would move to boot from SAN for all UCS blades to take full advantage of the stateless functionality.  For organizations already familiar with boot from SAN on a Clariion for an ESX enviroment this would eliminate 1 day of work. 

1 day was used to test all of our hardware failure scenarios that we had come up with; pull out all 8 fans, pull out 3 of 4 power supplies, kill the power to a chassis, etc.

By the end of the second week our focus shifted from how to build and use UCS to actually building our production ESX hosts with our new processes.  I would say 2 of the days were heavy on trial and error and determining the best process within UCS Manager, vCenter and Navisphere.  The remainder of the week was normal vCenter work building ESX hosts and VMware guests.

Time Summary:

—  2 days spent before implementation kickoff
—  5 days used for concepts, training, sandbox/hardware testing and architecting the environment
—  3-4 days configuring, building and working out processes for the production environment
—  4 days of performing our vCenter VMware related build processes for the guest servers (we are also doing Microsoft clustering with guests under VMware which has its own restrictions and requirements).


The UCS system met or exceeded my expectations.  Terms that come to mind that describe UCS are simplified management, elegant design, paradigm shift, future of computing, time and cost saver, etc.  Cisco got it right; I look forward to the changes brought by UCS.

Implementation of VMware ESX for server virtualization in our environment changed the way we function in a very positive way. I see the Cisco UCS having a very similar positive impact on our organization.  It expands on our VMware environment by reducing physical complexity, streamlining the SAN and LAN configurations, reduces cabling and required I/O connectivity infrastructure.  It adds additional flexibility with further abstraction of the server hardware.  I see the use of UCS outside of VMware ESX hosts as a big step forward as well.  By using boot from SAN and service profiles, we now have the ability to easily move non-VMware workloads to different hardware blades in the same UCS or to a secondary site (using future functionality in BMC software, etc.).  This functionality will be huge and will be a UCS growth area for us.

Short-term plans are to grow UCS to additional chassis’s to support immediate projects, add a blade to run my vCenter server (W2K8 directly on the blade), and move all existing UCS blades running ESX to the new Palo card when it becomes available. 

Long term, using UCS for a secondary datacenter would make a lot of sense by reducing the complexity and increasing our flexibility.  This can be accomplish by using VMware SRM as well as the functionality of the UCS Service Profiles.

Check out Cisco UCS if you get a chance it is well worth the time.


November 21, 2009 - Posted by | Cisco UCS | ,


  1. Michael:

    Good blogging–its been interesting reading to follow you experiences with UCS. With regards to your comment about complexity in the initial process, in retrospect, is there anything we could do to make that process easier/simpler?


    Omar Sultan

    Comment by Omar Sultan | November 23, 2009 | Reply

    • Omar,
      I have been thinking about your question . . . the process we went through was good and fit our workflow and organization well. The paradigm shift that UCS brings with it requires a period of time to start grasping the new concepts and ways of accomplishing tasks. When my engineer was telling me he thought it was more complex, I was thinking the opposite. Once I understood the basic foundational aspects of UCS (pools, templates, policies and service profiles) it clicked for me. This understanding came from seeing these things being used in UCS as well as white boarding out each item and how they all related together to build a successful service profile.
      Another thing that I think added to the other’s thoughts of complexity was they became exposed to the LAN and SAN details of getting the I/O connectivity working. Previously, they had no understanding or need to know that side of things and I think that added to their thoughts of complexity.
      So what could help make the process easier/simpler?
      Get your customers to an understanding of the basic foundational aspects of UCS early on in the process by hands on activity and presentations.
      Limit the exposure of the various aspects of the system to only the staff who really need to know that piece to minimize confusion. This would have its pros and cons. The less they are exposed the less independent they may be overall, but maybe that is ok based on their role. My approach was to get as many staff members involved to gain the greatest benefit overall. I think that was accomplished.
      Thanks for your comment.

      Comment by healthitguy | November 25, 2009 | Reply

      • Michael:

        Thanks for the comments–I’ll pass them along to the team.


        Comment by Omar Sultan | November 30, 2009

  2. Have you tried to play with orgs, policy overrides, hierarchical pool extensions? This stuff is mad!

    Comment by enemes | November 26, 2009 | Reply

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: