Cisco B230 Blade: Got to love the memory!

Today was a cool day.  We setup a fiber channel port-channel for the connection between our 6120 and the MDS 9148 switches.  Then I recieved my first B230 blade all tricked out!

It took me all but a minute to rip over the box and pull the cover off.  Man, 32 DIMM slots on a half blade, what a pretty sight.

Yes, those are 32 – 8 GB memory DIMMS for a total of 256 GB of memory.  Also, we have a pair of Intel Xeon L7555 CPUs each with 8 cores!

So our plan next week is to zone the blade to the new VMAX, load VMware ESXi 4.1 and start messing around with it.

This stuff is exciting.

Advertisement

UCS 1.4 Code: What is cool so far.

Here are a few things that have jumped out so far that I like with the 1.4 code:

— Support for the new B230 blades; half size blade with 2 sockets and 32 DIMM slots (16 cores and 256 GB memory!!)

— Able to manage the C-Series servers via UCS Manager (the rack servers)

—  Power capping to cover groups of chassis; this is very powerful now.  Think about it, you can have 4 chassis in 1 rack all sharing 2 or 4 – 220 circuits.  Now you can cap, monitor and manage the amount of power by groups of chassis not just per blade or chassis.

—  Software packaging for new server hardware that does NOT require the IO fabric upgrade to use the new servers.  Nice!

—  Bigger and Badder!  Support for up to 1024 VLANS and up to 20 UCS chassis (160 servers from 1 management point!!).

—  Fiber Channel connectivity options, now can do port channeling and FC trucking as well as some limited direct connection of FC based storage (no zoning ability . .  yet).

OK the list goes on and on, they have packed a lot into this release.

Checking out the new items in UCSM, I had to grab a few screen shots of the following:

Power!  You ever wonder how much power a server or chassis is using?  Now you know, check this out!  I am loving this.

This is cool stuff!

 

For those UCS users out there, it has not always been very clear what the impact of making various changes to a Service Profile might do to the workload.  They have improved with each release, but this is some great detail now:

Cool stuff in Cisco UCS 1.4 code, I hope to have more time to share with everyone as we continue to maximize our investment.  Time to go home . . .

Cisco UCS 1.4 Code: Upgrade is done, this is cool stuff

I am geeked! We just completed the code
upgrade on our production Cisco UCS environment and it was
awesome!

We have been in production on Cisco UCS for 1
year and 22 days now and have ran on 1.1, 1.2, 1.3 and now 1.4
code. So today was our 3rd code upgrade process on a
production environment and each time things have gotten better and
cleaner. Why am I so excited? Think about it . . . with UCS
there are 2 Fabric Interconnects with some chassis hanging off of
them with a bunch of servers all using a single point of
management. Everything is connected with redundancy and if
all of that redundancy is operational and live you truly can reboot
half your IO fabric and not drop a ping or storage connection.
In a storage world this is standard and expected but in a
server blade world you would think to accomplish the same level of
high availability and uptime provided by a SAN, there
would have to be a lot of complexity. Enter Cisco UCS! An
hour ago we upgraded and rebooted half our IO infrastructure that
serves over 208 production VM Guest Servers running on 21 VMware
ESX hosts and another 8 Windows server (all running active SQL or
Oracle databases) blades without dropping a packet. Then I
did the same thing to the other IO infrastructure path with NO
ISSUES. This is just badass. I suspect in a year this
type of redundancy and HA level in a x86 server
environment will be an expectation and not an exception.

UCS Code Upgrade Experiences:

In March 2010 we
performed the first upgrade while in production to 1.2 code (you
can check out my blog post for all the details). The major
impact we experienced with this one was due to a human issue; we
forgot to enable spanning tree port fast for the EtherChannels
connecting our Fabric Interconnects. Our fault, issue fixed,
move on. In December 2010 we implemented 1.3 code for a few reasons
mainly related to ESX 4.1 and Nexus 1K. Our only issue here
was with 1 Windows 2003 64-bit server running on a B200 blade with
OS NIC Teaming which failed to work correctly. Again,
not a UCS code issue but a server OS teaming issue. We had 3
servers using NIC Teaming in the OS, so we decided to change these
servers to hardware failover mode provided in UCS instead of in the
OS. Changes made, ready to move on. It just so happened on
the same day we did the 1.3 upgrade Cisco released 1.4 code just in
time for Christmas (thanks SAVBU). This time we had all our
bases covered and each step worked as expected; no spanning tree
issues, no OS NIC Teaming problems, it was smooth! There was
some risk with moving to the new code so fast, but we have several
projects that are needing the new B230 blades ASAP. There are
several UCS users and partners that have already been going through
1.4 testing and things have been looking very good. Thanks to
all who provided me with feedback over the last week.

New
Features and Functions:

Now we get to dig into all the
new cool and functional features in the new code. I am
impressed already. I will put together a separate posts with
my first impressions. I do want to point out one key thing that I
referenced above; the need to upgrade the infrastructure to use new
hardware (B230 blades). Now that I am on 1.4 code this
requirement is gone. Yep, with 1.4 code, they have made
changes that will NOT require a upgrade of the IO infrastructure
(Fabric Interconnects and UCS Manager) to use new hardware like a
B230. So yes, things are sweet with Cisco UCS and it just got
even better.

Cisco UCS: Current ESX Config

Wow, we are at the 1 year mark of having Cisco UCS in our environment which is now our standard, what a year it has been. I was fortunate enough to present to a lot of people at a few conferences, sit on some expert panel discussions as well as the Unified Compute Advisory Board and talk to a lot of new customers during their investigation periods into UCS. It has been satisfying to hear the majority of reference customers I have talked with decided to go with Cisco UCS. There are even a few that are blogging about it!

I figured it is time to update everyone on the current configuration build we are using. I think back to when we started with VMware on 3.5 and how much more complex it all seems now but with that complexity we all have gained greater control, cost savings and agility.

Current Build:

Cisco UCS B200 M1 or M2 blade with 2 CPU, 96 GB memory, Cisco VIC (Palo) interface card.
Cisco Nexus 1000v distributed virtual switch.
EMC PowerPath/VE for fiber channel multi-pathing.
VMware 4.1 Enterprise Plus (not yet using 4.1i, but soon) with Enhanced vMotion enabled.

So what has changed for us since last December when we went into production on UCS? Well the new technology that Cisco, VMware and EMC keep creating and it all still fits and works together.

Hardware:

The release of the M2 blade brought with it the Intel Westmere CPU and 6 cores. For about the same price point we were able to add 50% more processing power in a half size blade. Then by enabling Enhanced vMotion in our VMware cluster these new M2 blades work seamlessly with the M1 blades.

Cisco VIC (Palo) mezzanine interface card was released and has provided us with a more flexible means to implement converged networking at the host level. There are a few things you can do with the Virtual Interface Card but one of the main advantages we have incorporated is carving out additional Ethernet interfaces on our ESX hosts. So what, you might say? For example, we created 2 NICs for ESX management which reside outside of the virtual distributed switch that services our guest traffic. This simplifies our build process and allows us to control and manage the host if there is an issue with the vDistributed Switch.

Cisco Nexus 1000v has been around for a little while but we have now implemented it to really bring the networking side of the virtual environment out to the control of our network engineers. As our environment has grown the desire and need to have visibility into our guest server traffic has increased. The N1KV has already been helpful in troubleshooting a few issues and we are likely to be implementing some QoS in the near future. Note, when you pair the QoS functions within UCS, the VIC and Nexus 1000v you have a very powerful tool for implementing your key compute functions in a virtual environment. You can garentee your service levels and have confidence in the implementation.

EMC PowerPath/VE development has continued and our driver issue with the VIC has been resolved for a while. The coolest new thing here is on the ESXi front, PP/VE is now supported with boot from SAN (that will be are next step moving forward).

VMware ESX 4.1 & 4.1i keeps us current on the newest tools and optimizations.

As you know IT and technology is very dynamic and we are already planning on changing things within 60 days by going to an ESXi build with boot from SAN, implementing new UCS 1.4 code so we can implement may of the new UCS features which include the new B230 blades with 16 cores and 256 GB memory all in a half size blade. Yes all this within 60 days. I can’t wait to see the workloads the B230 will handle. Oh, and we will also throw in a new EMC VMAX to push the performance level even higher.

IT in healthcare has a growing demand for performance, agility and uptime and the above technologies are what will allow organizations to handle the changes. Hang on tight it is going to be a fun ride.

Cisco UCS C210 M2 2U Rack Server: First Thoughts

Here are a few pictures for you to see at the below link of our new Cisco rack mount server, a 2U C210 M2 Server.  It has 2 Westmere 6 core CPUs, 48 GB memory, ISL MegaRaid 9261 controller, 2 – 146 GB 10K Raid 1, 4 – 146 GB 10K Raid 5 and 1 hot spare disk.

http://tinyurl.com/32e6ypb

CIMC:  Cisco Integrated Management Controller

We are in the process of installing Windows 2008 64 bit R2 for the OS.  I personally have not be doing the install but I have seen the CIMC:  Cisco Integrated Management Controller (version 1.1.1).  You can think of this like the iLO from HP or DRAC from Dell.

The CIMC for a C-Series Rack Server

I wish there was more information in this tool, more like what I am use to in the full UCS Manager.  For example, we needed to load the W2K8 drivers for the Raid controller and could not remember the model number.  We looked in the inventory of the CIMC and it only listed the disks in the storage section.  I had to go back to my order information to determine we purchased it with the ISL MegaRaid 9261.

Checking out more of the CIMC, there is a good amount of information in and function in here.  You can mount virtual media, remote to the console, configure LDAP for authentication, define alerts, update firmware, etc.  So there is what you would expect to find in a component like this for a rack server.

CIMC view of Admin - Event Managment

Thinking down the road, I can see value in having the ability to manage my C-Series servers via the UCS Manager tool.  I would hope that will bring with it greater info on hardware details, service profiles and hardware abstraction as well?

Cisco UCS: B230 M1 Blade is Sweet!

I have been waiting for this product announcement and I am very excited to see it is now on the Cisco web site. Once you start to check out the spec on this new server I think you too will think that it is amazingly cool and a sweet addition to the Cisco unified computing product line.

Our current production “state of the art” blade configuration has been a B200 M2 blade with 96 GB memory with 12 cores or in 1 -6U chassis with 768 GB of memory and 96 cores.

The new B230 blows the current capacity of a 6U chassis out of the water.  Check this out, with the initial release (they will add support for the 16 GB DIMM later) you can get 2048 GB of memory and 128 cores in one chassis! That is badass in my book.

Cisco UCS B230 M1 Blade Front Veiw

Sean McGee, Cisco UCS engineer, has posted a blog with a great table breakdown comparison with 3 competitor blade products, you should check it out.

http://tinyurl.com/234p9dm

The link to the official Cisco page on the product:

http://tinyurl.com/28lfd6d

A few things that standout to me:

—  32 DIMM slots in a half size blade (more memory footprint)
—  Intel Xeon 6500 or 7500 series processors (more cores available in a 2 socket box)
—  Optional 2 SSD disk drives (the size of the SSD allows for more room for DIMM slots, nice)
—  Many options for Interface cards (including the Cisco VIC for virtual interfaces)

This thing was built for virtualization and large workloads in mind.   I will definitely use the B230 M1 for new ESX hosts and I can see purchasing a few for large SQL workloads that require a physical server (of course I will use the hardware abstraction functionality in the Service Profile to maximize the flexibility).

Front of blade is to the left, notice no HDD space needed.

The physical design of this blade is impressive as well.  They have used SSD drives to reduce the physical space needed for local disk which gives you more room for DIMM slots.  Personally, I like this move mainly because of more memory but also because I think it will push more people away from local disk and to boot from SAN.  It is when you are able to utilize boot from SAN the power of Cisco UCS can shine, using the hardware abstraction in service profiles.

I have seen a few twitter questions about what code version will be required for this new blade. Based on a few things listed in the Cisco datasheet, I believe it will be code version 1.4 which should be available next quarter.

Another question I have is when can it be ordered and shipped.  Oh yeah, pricing will be important too.

Cool stuff!

Good YouTube Videos: UCS Whiteboard Series

I have been busy lately, but this weekend I was logged into YouTube and was suggested some new videos that had been posted called Cisco UCS Whiteboard Series.  I looked closer and realized Jeremiah from Varrow has put together, what turns out to be a great series, five videos describing the pros and cons of 4 common approaches to implementing compute in a data center.

What I like about the videos is Jeremiah’s common sense approach to breaking down the evolution that has occurred moving from the stand alone rack server, to traditional blade systems, unified fabric/converged network to Cisco UCS.  I recommend this series to anyone looking for more information about this topic.

Part 1: Intro: http://youtu.be/KDpwnojbBwY
Part 2:  Traditional Srvs: http://youtu.be/3i8rcjBFRgE
Part 3:  Traditional Blades: http://youtu.be/5yWKMQwr99s
Part 4:  Traditional Srv w/ Unified Fab: http://youtu.be/LhEZl_PwDFM
Part 5: Cisco UCS: http://youtu.be/amLXLWn2qOQ

CiscoLive 2010: Day Two

The second day started with my favorite session so far and I did not expect it. I am not a programmer but I thought the UCS XML API session was great. I have heard a few times that all functionality to manage UCS via the UCS Manager, the XML API or the CLI is full featured but never dug deeper. Catherine Liao gave us 15 minute primer on XML, XML Schema, etc. to get us started. Then we got into a few ways you can code to the XML API and see API response and error handing. What brought out the geek in me was talking about different use cases. Here are the notes from that section:

1. Display System: You could use this type of example to generate a report of service profiles, etc. and you could include firmware version.

2. Mobile System Monitor, they have a beta iPhone app that uses the API. Took him about 80 hours to develop it. Check it out at http://www.timeline.com/simu

3. Auto provisioning of systems like what Cisco IT is using to deploy workloads.

The other highlight of the day was the keynote with John Chambers. I always like to hear him speak, he knows how to communicate his ideas very well to a crowd. The big WOW from the session was the announcement of the new Cisco Cius “See-Us” tablet device. The first reaction is that it is a competitor to the iPad but it really is focused as a collaboration tool using unified communication (with “Video as the new Voice”) being a key piece of it. Plus it is focused as a business, education or communication device. Afterward when everyone was leaving the event I noticed John Chambers still hanging out and talking with people. So I headed down and had a few minutes to chat with him and get a picture.

It was a pretty cool day that ended on the patio at the top of Mandalay Bay looking out on the strip listening to great live music from Phat Strad. Thanks to the Cisco datacenter group for organizing this fun event.

CiscoLive 2010: Day One

I am a little late on my day one blog but here it is . . . The focus of my first day was around the core network in the datacenter. We have Cisco UCS so I attended a few sessions on FCoE and the Nexus 7000 hardware.

FCoE covered basic concepts for the first half and then they dug into the details of how FCoE is really FC but contained in an Ethernet packet. So it was a good refresher on FC concepts and how things map under FCoE. We also got an understanding of how the FCoE protocol has been built to provide the lossless nature needed to support fiber channel traffic by using flow control. During this session I wanted to login to one of my FC Cisco MDS switch to check out some show information as we covered them in the session. Since I am only using my iPad this week I used the Citrix Receiver app to remote to my desktop. It worked very well, pretty cool.

For the Nexus 7000 hardware session I now know the basics of how it is structured. Since my experience is with the 6500 what stood out for me was the fabric for the switch is not on the supervisor but on it’s own fabric modules that go in the back of the chassis. This allows for up to 5 fabric modules and up to 230 Gbps of bandwidth. Based on your needs you can start small (3 is the base recommendation for redundancy and adequate capacity) and add to it over time. Today Cisco announced a new module line called the F Series that will allow you to add IO modules that will be able to start to reach that 230 gbps of bandwidth. Another interesting thing from today is next year there will be a 9 slot Nexus 7000 as well as new fabric modules that will get the bandwidth up to 560 gbps (I am going on memory for that number).

I shifted gears in the afternoon and attended an IT Management session from Cisco’s Paul McNab focused on IT Aligning to Business Strategy. This was pretty cool discussion that gave insight into how Cisco has approached disrupters as opportunities and how using collaboration to increase speed to respond and broaden your scope. An example that stood out for me was that the value of the technology we purchase today may only have 18 months of life, this means IT needs to get value out of that investment very fast or you will loss your competitive footing. By using new tools and technologies such as collaboration tools, unified computing, etc. to maximize your time and resources are ways IT can bring value to the business. This session is one I would not have expected to see at a tech conference, however, it fits well into my interests.

Finished out the day with the show floor, World of Solutions. It has been a few years since attending a Cisco conference and it reminded me how diverse Cisco is. You see cabling, storage, data center, server components and voice vendors all in one room. We had a great first day and look forward to the rest of the week.

Cisco Live 2010 . . .

I am on a plane headed to Las Vegas for this year’s Cisco Live conference. It has been a few years since I last attended Cisco Networkers back before it evolved and expanded just like Cisco and the rest of IT has changed. Back then I was interested in wireless, network security and just getting an understanding of storage area networks and the MDS line of gear.

Today I continue to have a heavy focus on the importance of the datacenter but there is a diverse and wide variety of technologies that all have to come together for a health system to deliver exceptional patient care.

It is time again for our organization to refresh our wireless to support greater bandwidth by using 802.11n, provide for a substantial increase in the number of wireless devices and simplify the management of it.

How we look at our LAN moving forward will need to expand the support of Quality of Service (QoS) to handle more video and voice. How it will all come together will be key as we move forward. Just like we are building in high availability and redundancy in the datacenter we have to focus similar attention on the network. This is the northbound traffic heading out of the datacenter to all of the end node devices no matter what, where and how they are connected. All of this done in a secure manner, of course.

Back in the datacenter, key technologies that are interesting to me are the new things coming in the Nexus product line and how it will tie in with UCS. So as we are expanding and scaling out our compute capacity on UCS what is going to be the most efficient and cost effective way to deliver this high level of service.

I am looking forward to this week and it should be a good time as well. I always like to hear John Chamber’s keynote, I wonder what his focus will be this year. Cloud Computing?