The Virtual Data Center

DISCLAIMER: This is long and the opinions are mine.I’ve written a good bit here about the various ways Microsoft and Citrix overlap in the hypervisor space, ranging from topics like shared code base through competition for the desktop space. To me, these two players have always been the underdogs battling for the right to go head-to-head against VMware in the main enterprise (and now cloud) virtual data center event. I’ve long said here that I think Microsoft is in the best position to make that move, but to be honest, Citrix currently has better technology. In other words, Microsoft has a better strategic play, Citrix a better tactical play. The announcements that came of out Synergy last week prove that. Citrix knows what it’s doing and they know how to build virtualization products to compete with VMware.As has been asked many times before, here and elsewhere: What would happen…what would be the benefit to the market…if Microsoft were to acquire Citrix and merge the best strategy and tactical solutions into one? The idea and rumor has been around for a while, so why am I revisiting it today? Since these rumors first started to really circulate in September of 2008 (around VMworld) there’s been very little advancement from the Microsoft camp on Hyper-V, and a tremendous amount of advancements from Citrix and the Xen products. We’re also seeing a few cases where the two have opted to work together. Case in point: the Essentials family for managing XenServer and Hyper-V VMs and storage. Citrix has made some excellent headway in the VDC with product announcements this year; that’s the real reason to take another look at this idea.For better or for worse, Microsoft and Citrix are already collaborating, both individually and to an extent togeter, to go after VMware. In the grand scheme of things why continue to do that on their own when they can do it together, mount one single offensive with one single goal, and bring enough technology to actually make a dent in VMware’s VDC footprint? Join forces and all that 2 against 1 stuff. Let’s look at a few categories where this makes sense, where Microsoft acquiring Citrix technology would go head to head against VMware and actually have a chance of winning:

• Networking and Application Delivery: To me recent movement from Citrix in this space is the paramount camel’s straw/tipping point for why Microsoft should finally take the leap. Citrix’s application delivery product line, NetScaler, has been a good appliance-based product for Citrix. Not a market leader, but they’ve held their own against F5 and Cisco. They manage application delivery well enough. With the announcement last week of NetScaler VPX, their virtual appliance version of MPX, NetScaler has made the leap into software-based application delivery, ala Zeus. This is huge for the acquisition discussion. First of all it could bring networking and application delivery into Microsoft’s world, something they’ve avoided with Hyper-V to date. Customers use virtualization for applications and they need to deliver those applications outside their data center. Couple VPX with the new software switch Citrix announced to compete against Cisco’s Nexus 1000v and you have the critical missing pieces for application deliver via Hyper-V (as well as another angle for Microsoft to compete against Citrix). And then add in the Citrix desktop and access-related apps for the non-MS platforms, like the iPhone, and Microsoft makes a huge push owning the application delivery stack from the VDC to the client, any client.
• VDI: Citrix has done an amazing job on virtualization geared towards the client. Going back to Metaframe and Presentation Server and then today with the work they’re doing with Xen on client virtualization, Citrix has always been focused on the client. Ironically, even though Microsoft is the de facto enterprise desktop client (in a sense), it hasn’t addressed the client virtualization markets too well. App-V is a step forward, but MED-V (with desktop virtualization code based on Virtual PC rather than Hyper-V) is a step back. VMware is making a huge push in this market with VMware View; if any player is going to win the VDC space completely they have to include a VDI solution, one that works locally and remotely, in their portfolio. Citrix could help Microsoft make that push by combining their respective solutions for hypervisor and application virtualization technologies. Many of the enterprise desktops and apps are Microsoft; the underlying technology running those desktops and apps in the data center and over the network are Citrix.
• Cloud Platforms/Providers: Xen owns a good bit of real estate in cloud and service provider data centers. Although Microsoft has good presence with customers running Windows operating systems, it doesn’t have the same exposure for Hyper-V as a platform that VMware and Xen have. I think MS is looking to change this with Azure but it will still be limited to the MS-only solution (for the short term anyway). Acquiring Citrix would give Microsoft that cloud provider mindshare by name alone. They could then take that business and technology model that Xen has built and create a best of breed service provider platform between Xen and Hyper-V for customers that want to run non-Windows apps on Xen and .Net-based apps on Hyper-V. This could drastically help Microsoft’s Oslo application lifecycle plan moving forward with cloud providers while not alienating non-.Net apps.
• Application Virtualization: As you know, I’m a huge fan of a true application virtualization model, something that I believe App-V will ultimately be able to deliver. However it will most likely be focused on .Net and Microsoft apps only and is still a few years away from full delivery and even more from adoption. In the mean time we have this bridging technology between VDI, client virtualization, and streaming apps. VMware is getting there with tools like View and ThinApp, but Citrix is staying in lockstep. Microsoft could use a Citrix acquisition to springboard App-V into a multi-focused application delivery platform, taking what’s good today with streaming apps and client virtualization and continue to work on true application virtualization for all apps.
• Customer/Device Support: And as a roll-up benefit of the above categories, we have application delivery to devices. I don’t want to place too much emphasis on supporting remote access via the iPhone, but when you look at Microsoft’s historic relationships with Apple and Linux (as a whole), of which Citrix has obvious ties into both now, that’s an appealing way for Microsoft to jump right into those groups. That doesn’t mean they’ll keep the momentum alive, but at least it would give them more opportunity than they have today. The overlap between VDI, XenApp, secure remote access, and the iPhone is an extremely appealing proposition for mobile users; a turn-key solution for Microsoft to cover a huge gap in their overall cloud and virtualization offerings.

And let’s be honest: Microsoft has had some challenges with their virtualization solutions and their overall direction. Client virtualization based on Virtual PC and no enterprise VDI solution? Hyper-V management hiccups through SCVMM/SCOM and delaying live migration for so long? Azure wanting to change the way applications run and are written on-premise? These raise questions in my mind, a lot of “Why?” questions. Citrix, on the other hand, is heading squarely in the right directly for virtualization solutions.  Citrix continues to plow ahead against VMware at a good pace, whereas Hyper-V isn’t quite at that same pace. The virtual switch announcement from Synergy last week is an excellent example; we haven’t seen any movement or advancements on virtual switching or networking for Hyper-V at all. Sophisticated virtual networking and switching management is an absolute critical component for virtual and cloud-based platforms, IMO. Moving internal roles and tasks to VMs running on the platforms is something we’ve seen for a while with VMware, even going so far as to running the full version of ESX 4.0 in a VM on top of ESXi 4.0. Citrix is doing the same with their Dazzle product. In other words both VMware and Citrix are finding optimized ways to use their own technology for their own benefit. We’re not seeing this today from Hyper-V. Again, there’s nothing to say that Microsoft acquiring Citrix would change that, but at least it might help grease the skids a bit towards internal product unification. Citrix knows how to do it well.To be clear, I am not being critical of Microsoft technologies or business practices (as any long-time readers of my blog will undoubtedly know). I am suggesting that when compared on a chart, Citrix is closer today to where the market and VMware are going for virtual platforms, and if the goal is to compete with VMware for both enterprise and cloud virtual platforms then Microsoft could benefit in leaps and bounds by acquiring Citrix for both Xen and their networking products. Microsoft would get virtual platform, application, and networking tools that they don’t have today.I’ll leave you with one final thought on how compelling a Microsoft/Citrix acquisition could be: Imagine a year from now if Azure launched out of beta running on both Xen and Hyper-V. This would be the best of both worlds: Microsoft could continue to push it’s current developer-based approach to Azure, SaaS, and application cloud computing, focusing on .Net and helping to push users to re-write their current and new apps. They could also support non-.Net customers by allowing them to run their services on Xen in Microsoft’s cloud. Customers wouldn’t have to choose based on their app needs. That would be the ultimate competitor to both Google and Amazon for cloud mindshare, bridging the two cloud models together and backed by the Microsoft brand.  Awesome. Will we ever see it? I hope so for market and customer needs.“Wish You Were Here” Image © 1975 EMI, Storm Thorgerson

Frequent readers will understand my love of lists, my affinity for the 4D attack plan methodology (Define, Design, Develop, Deploy), and my need to break things into small addressable (bite-sized) chunks. Over the past week I’ve been laying the groundwork for securing the cloud; not the technical “Use this VM, configure these VLANs, tether the clients this way” stuff but the larger macro business planning for techies on securing the cloud. Today follows suit in the Define category, going straight to the hardest problem first in cloud security: securing shared data plane resources: CPU, RAM, and bus.

Like it or not, we’re going to have to address and solve security of physical computing resources in the cloud sooner rather than later. And by sooner I mean now. First thing. Put down your VM security appliance and step away from your network and packets. This morning. Stop what you’re doing because I’m about ruin the image and the plan that you’re used to*. We need to figure out how to secure VM computing traffic over shared resources like CPU, RAM, and bus – the data plane implemented by virtual platforms and thus the backbone of the dynamic cloud. We’re going to deal with near-limitless attack vectors across all parts of the cloud but if we don’t secure the running environment first then we’ll be asking for someone to find an open door and take our virtual CPUs, our virtual networks, our virtual I/O.

Tools like VMware’s vShield Zones are good starts but they don’t go deep enough (at least from what about Zones today; I’ll know more after the Partner Exchange in April), managing policies and trust levels in the zones down to the bit level, not just the packet level. Exploits against the physical and virtual data planes will make network and application attacks looks like child’s play because the data plane owns the transport and storage of the targets of those attack. It’s going straight to the source. It will allow attacks from inside the cloud out through all those oh-so-useful networking and framework tools that have built up the cloud. It’s a like a microwave: attack the molecules from the inside.

Saying we’re going to do it is one thing, actually doing it is something different. Many moons ago I wrote a three-part piece about the hypervisor/platform vendors taking responsibility for their own virtual space. Their virtual CPUs, their virtual switches, their virtual IPC between host and guests; these items can all easily be secured by the vendors. But what about securing data in the hardware and the step that moves data from virtual software to the hardware? That, too, is mostly the platform vendors, but not all. With virtualization now happening in the CPU, securing that shared data in transit will require the platform vendors to work with the hardware manufacturers to address the problem and establish trust. How can the CPU trust bits from the hypervisor are safe and vise versa?

So how are the manufacturers and vendors going to do that? Easy: the platform vendors will need to create dedicated virtualization security teams that include working with hardware vendors, and start talking about this today. Get the word out that this is a critical issue and concern. Sound familiar? See, it’s all coming together to form an easily manageable plan of attack and execution for securing the cloud from the inside-out. But we have to start somewhere, and I prefer to start with the most difficult task first, the core issue and technologies at the center of the cloud, and then move out from that.

You’re reading this Microsoft and VMware, right? I thought so, just wanted to make sure.

*Yes, I did just borrow and paraphrase the opening line from The Humpty Dance, thank you very much.

As reported by Practical Technology yesterday, Citrix has announced that it will be giving away XenServer (the hypervisor portion of their acquired Xen solution) for free. Very similar to the VMware model, they will continue to charge for management and advanced features as add-ons to the XenServer platform but the core virtualization technology will be available for free. Will this have any impact in their market share or their stake in the virtual data center? I’m skeptical for a number of reasons:

1. Free doesn’t always mean better, and usually it’s the exact opposite. I’m reluctant to think that enterprise customers and mission-critical cloud providers will take the free route solely to save money. Maybe that’s more likely this year than last but I’m still not sold it will make that much of an impact, especially since this won’t be an OSS offering ala XenSource. They’re also competing up-stream against VMware’s ESXi and Hyper-V, both of which have value-add above and beyond the hypervisor. For Citrix to make this move work they’re going to have to prove that they have the same type of value-add with XenServer that the other two big players offer. And free doesn’t count as value where your mission critical apps are concerned.
2. They’re relying heavily on management solutions to bring in the money for XenServer customers. We all know you can’t deploy a truly reliable virtualization solution without having management. In practice free deployments of XenServer will be few and far between, instead they will be bundled with some management solution. But which one? If MS System Center will manage both Hyper-V and XenServer, then we’re back to that value-add question for MS shops. If I’ve already deployed MS SCOM/SCVMM then what’s the value in deploying a Citrix solution instead of Hyper-V — which is also integrated into my server platform? VMware shops will stick with VMware because they’re the only company that has an end-to-end solution today. So is management the golden ticket for Citrix?
3. They’re announcing a new partnership with Microsoft. Excluding their existing partnership (whether that partnership has benefitted Citrix is a decently debatable topic so let’s gloss over those particulars for now) — Has a partnership with Microsoft between competing companies ever worked out for the non-Microsoft partner? Yes, MS is an excellent partner resource for technologies that don’t compete, but really, this partnership (and the ability to manage XenServer) will drastically benefit Microsoft by extending their heterogeneous virtual management solutions with Systems Center. I don’t see it going the other way for Citrix. Again, we’re going back to the value for a complete solution and who offers the most value here: Microsoft or Citrix? My vote is very strongly printed in the Microsoft column.

Maybe this will work in Citrix’s favor by at least getting XenServer in more hands to play with, and maybe their technology value will bubble to the top. But how many times have we watched a better technology (assuming XenServer is better) go by the wayside because they didn’t offer enough business value? In this case, I question whether a free XenServer offers any business value above and beyond what Hyper-V or VMware ESXi already offer.

I’ve hinted in the past on my ultimate application virtualization scenario – where I want the market to be for deploying and supporting remote applications for clients in the future. I’m still working on that giant whiteboard architecture map in my basement on what AppVirt looks like from the DC computing side, but today I want to write about the client side of that architecture. And while brevity has eluded me in all parts of my lingual life recently, I’m going to try to be succinct here (expect failure).

I attended the first of a four-part series on Cloud Computing last night held by the WTIA, which included an excellent presentation by Aaron Kimball from Cloudera on the basics of the cloud from the data point of view. Having retired the engineer title for marketing a few years ago, it always makes me happy to see someone who spends their career designing complex systems stand up and give an intro presentation that also includes the business benefit. So often engineers address the How rather than the What and the Why, and Aaron did an excellent job with the latter.

His presentation, along with others last night, got me thinking about what application virtualization in the cloud would look like to the client (and I’m not talking about GMail here).  Let’s look at a real example:

I bought a Mac a few months ago primarily to run Lightroom, so I spec’d out the Mac to go high-end because it would be running a very beefy photo application (along with Photoshop in the future I’m sure). The machine also had to run VMware Fusion in parallel (no pun intended; sorry to my Parallel friends) - I have a photo stitching application that’s currently Windows-only. Standard operation keeps me stable at 75% RAM and 40% CPU on average.

But what if I didn’t need to buy local computing resources and everything was processed remotely? Let’s jump ahead 10 years (a big leap, I know) and look at how this could be different if client apps were in the cloud.

I buy a local processing machine that’s drastically stripped down from my current Mac. I boot this machine to a web browser, where I head over to Adobe and say “This is Alan; I need to run Lightroom.” Adobe says “No problem. Let me push down the secure Lightroom App Shell. Ok, now you’re ready. Here’s a list of your albums pulled from Amazon S3.” I say “I need to process the latest batch of Mt. Baker HDR images.”  HDR images take a substantial amount of computing power to process, so Adobe comes back and says “No problem. I’m going to need 2gig RAM and a dedicated CPU core for this, but your monthly subscription only covers 1gig and .5 core. I’ll charge you $0.021/minute if you’d like to burst.”  I say “Great, let’s do it.”

Amazon then pulls its own resources from AWS and start distributing my HDR processing over thousands of machines/cores/RAM, all controlled from my local Lightroom App Shell. To me it’s displayed as one 8ghz core (remember, 10 years from now:) ), but in practice that value is an aggregate of distributed resources pooled from thousands of machines.I process my HDRs, they’re stored back in my Lightroom library which is managed by Adobe.com, I take my tiny laptop over to the client’s office, rinse, repeat.

But what if I don’t want the processing done exclusively in the cloud? I mean after all the above scenario is very similar to what I can do today with a web browser. What if instead my local computing platform is moderately beefy and I process everything locally. But as my machine starts to bog down from stitching together HDR panoramas it uses the Lightroom App Shell to request raw computing resources from AWS, and I become part of the distributed cluster? I scale and process background tasks remotely and only use my local resources for rendering the images to my display. My machine is part of the cluster and the line between local and remote processing becomes blurred. That’s some cool client-based application virtualization. The application running state is spread across elastic resources, of which my local resources are a part of.

True application virtualization will be a huge undertaking and this is simply one part and one idea. But why not think big? Go for the gusto I say. Oh, and there’s more here, but I’ve long since crossed the brevity line. Maybe more later.

I’m going to start this morning with a question that I’ve been pondering lately: Should we work out all our security issues with virtualization before we start looking at cloud security? Would it take an army of security researches to work on platform and cloud threats in tandem?

My gut response is “Yes, these are linear and we should address them in a very deliberate order.” A good bit of the cloud is based on virtualization technologies that we’re all using yet no one has thoroughly addressed the security threats associated with mass adoption of these platforms. The truth is we don’t have the luxury of time to ponder all the threat vectors of virtual platforms and deliberately control our roll-outs for security: these technologies are already live and, fortunately or unfortunately, supporting a vast number of our applications and data centers today.  That shipped has sailed and it’s full of ESX and Hyper-V boxes.

Of course this scares the <you know what> out of me, and yes, it keeps me awake at night wondering when this is all going to collapse due to one silly little worm. How long before we see a virtual Sasser that has real business reproductions and sinks an entire ship?

In addition to the security concerns we have on virtual platforms, we also have different security concerns, arguably more frightening, with cloud computing. Like the three categories of virtual security I wrote about way back during RSA last year, cloud security falls into two primary categories in my head: securing the data in transit and securing access (yep, I categorize everything and make lists like you wouldn’t believe).  These differ from the virtual platform threats in that cloud security is about process rather than code. We have to implement security checks and border patrol on the Cloud Highway: everything is inspected as it comes in and out and only certain workloads are allowed on the highway. We have to look inside every freight package that traverses the Highway, and that’s going to be one mighty endeavor.

I was having a conversation with a VDC reader last week and the topic of adoption vs risk came up: If the risk is so great (as I think it ultimately is) then why isn’t it getting more attention? I believe that no matter how great the risk, the rewards associated with implementing virtualization, and eventually moving outside the data center to the cloud,  are too great to be stalled by theoreticals, and if you can’t see risk then there is no risk. The mobility, the consolidation, the new levels of management and granularity, all of these are so much more important to business process and IT agility than security threats that frankly don’t exist yet. Business moves ahead.

So what do we do? We address these security risks in parallel. We start making virtual and cloud security a more visible issue. We need more virtual security researchers — dare I say it, because I’m not a fan of the branding — more white hats out there focusing on both sides. We need PhDs looking at hypervisor and resource-based security and we need ethical hackers focusing on the cloud. We need new Cloud PenTesters.

We need giant posters in every college depicting Max Headroom (the only virtual icon I can think of) smiling, pointing, and saying “I Wa-wa-wa-wantttt You-ou-ou!” The virtual security army needs new recruits, and we need them now.

I’ll close with another question to ponder: Will these recruits even know who Max Headroom is? Were they even born yet? 

There’s a nice write-up at the New York Times site today about HP using their ProCurve line to go after Cisco in the data center. It’s a nice follow-up to the piece posted last week on Cisco’s play in the virtual server space.  This is definitely an interesting tug of war to watch, I’ve got my popcorn ready.

What’s most interesting to me is the way each of these companies is coming at this based on where they’re coming from. Here’s what I mean, and where I think each stands in this virtual fisticuffs:

• Cisco: They obviously bring the networking heavyweight to this rumble. No question they know L2/L4 backbone gear – speeds and feeds – better than anyone. And they have a vested monetary interest in VMware and are driven to bring these two technologies together; everybody wins in that scenario. The concern I have is that today, networking and virtualization (in the scope of virtual platforms and VMs) really are night and day. In fact, they’re not even the same ballpark, game, or sport. So this is a leap into new territory for Cisco. I’m not saying that can’t make it work or be successful, it’s just going to take a lot of work in both becoming proficient in virtual technologies and then delivering compelling solutions that integrate that newly created expertise into their networking arm. Cisco is also extremely divested already from a product stand-point, so will the market accept yet another non-core solution from them? What will it take for the Cisco name to be synonymous with application servers in the data center?
• HP: On the flipside, HP knows servers, and they know hardware management for virtual platforms extremely well. They’ve been working closely with VMware in the data center for years supplying solid, stable, and beefy hardware for products such as ESX. I think HP is a trusted name in the hardware server space for virtual platforms. But beyond that they also have ProCurve and a solid history of networking. They have resource virtualization experience with blades and chassis, they have WAN experience, they have management experience, and they have it (mostly) under one umbrella product line. So they bring a bit more to the ring than Cisco in that they have good experience and a name in both networking and virtualization.

So to sum, this challenge feels like a topic for the debate team on the colonial formation of the United States in the 18th century: both sides know their history, but one team also has the war buff who knows how war impacted the formation of the US. It feels like one side is stacked.

Now how this plays out in 12 months when the economy (hopefully) opens up and IT departments start looking at how all these siloed technologies in the data center can work together to provide new services while saving money…well, that remains to be seen. But in my view, the winner will be the company that figures out how map virtual platforms and virtual networking into a solution that makes sense, not a solution where these two disparate pieces are cobbled together. You can’t do virtualization in the data center without networking, but you can do networking without virtualization. Which giant walks out a winner depends on which one can marry those two into one solution that makes sense, that works, and that tells IT departments everywhere that they can’t use the cloud until they deploy this gear.

Citrix has announced that they’ll be offering an alternative to traditional on-line VDI deployments (ala [Xen|Presentation]Server) in the form of a client hypervisor. In essences, a client hypervisor allows a user to run a full-blown virtual machine on their desktop — basically a throw-back to running VMware Workstation/Player in the enterprise before we had all these fancy virtual platforms like Virtual Infrastructure and Hyper-V.

Functionally, I’m a fan of this model: it allows users to work anywhere without concern about having an uplink. On planes, in coffee shops where there’s no Wifi or the connection is too small to reliably delivery a VDI experience, or even in the woods (solor panels for power optional). It also paves the way to true application virtualization, where the CPU-based hypervisor can be trimmed down to support running applications directly without the bloated guest operating system.

But this un-tethered model also brings up concerns, specifically around management. IT departments are going to have to face tough questions before deploying this type of technology. For example:

• How to deliver client VMs to users, especially remote users? Off-line via media or online?
• Change control: Will users be allowed to do anything with these images and run them like their own laptops? Will there be any type of reconciliation between the remote VM and a central policy source? How often?
• Mobility: Will users be able to transport these VMs between machines? Or will the VMs be bound to a single piece of hardware? If the latter, how will this be implemented and regulated?
• How are these VMs backed up and kept in state? What if the user drops the host (ie laptop) off the side of a ferry while finalizing her critical presentation just minutes before she’s scheduled to present (something a true VDI model would address)?
• In the case of remote users, what happens when the user “checks” their VM back into the corporate network when they’re at HQ?
• Remote access: Will these VMs be tied to a VPN solution? If so, how will that differ from how the host laptop/desktop connects?
• Policy management: How will tools like GPO apply to the host and guest simultaneously?
• Support: What happens when a user has a hypervisor-related issue? Will the support staff be trained to troubleshoot the host, the guest, and the hypervisor?

Many of these questions apply to any IT desktop policy, whether it’s a physical machine or a virtual machine. And to me, many of these issues beg the question “Are off-line client hypervisors worth the cost and effort?”  How does offering client VMs differ than just issuing everyone in the field a “normal” laptop?  In practice I’m inclined to agree with Kane Edupuganti on this issue:

We haven’t explored the idea of putting a [hypervisor] kernel on the devices, because you’d still need a regular PC.”

VDI solves many of the management problems listed above, but not all of them and also brings its own list of issues to the party (network, storage, advanced virtual platform management in the data center). There are security concerns for both models as well, but before we can address how to identify and solve security concerns, we need to be able to manage the environment first. If the goal is to make managing desktops easier and more streamlined, then I do think VDI is looking like a better alternative.

But I do think that the client hypervisor model is an important step towards pushing the computational tasks of running applications down to the client. That application hypervisor is getting closer, and steps to figure out how to manage local computing resources through a client hypervisor are all right by me. 

I was reading Alex’s post on SearchServerVirtualization this morning and really focused in on this comment from John Humphreys at Citrix:

If you think about what the operating system does, it only does three things. It provides an interface to the hardware [and] to the software and provides middleware,” Humphreys said. “If the hypervisor is doing one-third of that job, that’s a potential price cut for [Microsoft].”

This comment has had me thinking all morning, and now as I sit down to enjoy some lunch, I’m really starting to noodle a this idea (the boucing between past and present is intentionl - this has been sitting with me all morning, seriously).  First I started thinking about the OS vs. hypervisor idea. Do these two ideas compete? Is it really the goal of the hypervisor to replace the OS, or at least take on 1/3rd of the duties as John suggests? I think the answer is “No” to both of these.  I think the hypervisor’s role is to manage access to and, to a certain extent, emulate hardware, for operating systems, not instead and not to replace. Long term I think this will allow the OS to function more on resource management for its own applications rather than having to also worry about where those resources come from.  Seperation of duties: let the OS manage the apps so they all play nice together and then let the hypervisor manage the hardware for the OS, while the OS still owns allocating the resources. Even better when the OS’ and hypervisors start working together ala paravirtualization, which is an interesting side note to all of this and may have answered my question, but before I get to that…

So then I latched on to this comment from Gordon Haff:

Haff believes Microsoft’s attitude is misguided. Rather than “stress out” about delivering a hypervisor, it “could have done more in the partnering arena and saved the world 75% of the ‘Microsoft doesn’t get virtualization’ stories’ and never lost a penny.”

…and that got me thinking about the “Why didn’t MS partner for the hypervisor?” idea. And here, I’m stumped. We all know MS isn’t a hardware company, and minus the XBox they admit this as well. So if the hypervisor truly is meant to emulate hardware rather than software (ie the OS), why would MS want to build a virtual hardware platform? As far as the OS is concerned, the hypervisor is the hardware platform. Why buy then build rather than partner? Why not work with a company that’s 100% focused on the hypervisor?  I think we all also agree that MS is a software company; their revenue comes from software licenses. They make apps and operating systems to run those apps.

This begs the question: What was the original benefit evaluation for MS to start building Hyper-V (knowing that Hyper-V grew out of Virtual Server which was acquired by their puchase of Connectix in 2003)?  Why did they head down that path rather than working with VMware (remembering that until recently, Xen wouldn’t have been an option due to its *nix lineage)? MS makes money every time an enterprise installs VMware to run a Windows operating system. With issues like sprawl, MS could stand to make a good amount of money on both OS and app licenses.  And as we saw on Mike D’s blog (although take it with a grain of salt b/c Mike does work for VMware and has vested interest in propogating the MS vs. VMware price war), the MS licensing model does require a finite number of virtual instances of their OS for every license except with a full Data Center license, which makes sense.

Now please don’t misunderstand me: I am in no way suggesting that MS shouldn’t be in the hypervisor business or kill Hyper-V.  I’m simply thinking out loud. Long time readers of this blog will know that I actually think MS is in the perfect position (long term) to completely change how we think about using discrete hypervisors, operating systems, and apps. The day I can run Sharepoint on a bare metal, bootstrapped hypervisor (ie hardware -> Hyper-V/WinKernel -> Sharepoint) is the day I jump for joy. Now that we have Hyper-V, I think MS does have the ability to own the virtual stack and deploy true application virtualization better than anyone. And I also disagree with Haff’s comments about MS not getting virtualization. I think they get it extremely well, they’re just up against a technology originator (in the x86 space anyway) and have to defend every thing they do. Which is funny given the nature of this post I guess. 

But before we had Hyper-V…To me it seems they would have had nothing to lose from partnering with a VMware hypervisor company and jointly developing a “WinOptimized Hypervisor” or something of the like, in a similar manner to working directly with Intel for CPUs. Was their goal to own the stack for app virtualization without an OS? To create a paravirtualized running environment where Server 2008 and Hyper-V, for example, separate duties (which they’ve done on the hypervisor-only front)? Even if that comes at a huge cost from competing with VMware and having to become hardware experts for things like virtual switches and infrastructure? What’s in it for MS?

Anyone know the history (more recent than Connectix) and/or want to chime in? I’ll be the first to admit if I’ve got this wrong, so let the opinions fly! What am I missing?

Ok, we have to talk about Hoff’s comments on VM sprawl, and it’s lack of security severity, for a second.  Now I’m guessing that he’s more focused on picking at the wound to spark conversation, and driving an excellent discussion on terminology, than he is in actually suggesting VM sprawl isn’t a security threat. But I’ll bite for a minute, and expand on my comments to his original post.

Yes, I do believe VM sprawl is a very serious threat, specifically due to what he calls out: lack of management. But it’s more than just a lack to me; I believe it’s an inability to even try to manage all VMs that is the problem. This same problem exists with physical servers, too. If someone can get access to the data center, bring in a box, find a switch port and know how that network is configured, then she can install a rogue physical server. But the barrier is so, so much lower with VMs because everything is “flat” once it’s virtualized. Our management choices today, such as vCenter, are doing a good job at creating virtual security barriers analogous to a keycard on the data center door, but we’re not quite there yet.

This is like managing termites. You can do everything in your power to protect your house from those little invaders by managing your perimeter and trying your best to find where they’re coming in, but if your builder “accidentally” left an exposed beam in your root cellar then built concrete blocks around it, it’s going to be very difficult to find that point of entry. Just like rogue VMs. It’s going to be extremely difficult to stop new VMs from spinning up at every point in your network.  There are ways to do this on the network level in the data center with IP registration, passive packet sniffing the virtual switches, 100% control over application traffic routing, etc.  But they’re not full-proof, and it becomes much more difficult in the desktop world.

Example: I use VMs for work on a daily basis. When I travel for work, those VMs come with me on my laptop. Inevitably, when I fire up my laptop and VMs on the hotel network and go through the “pay to play” registration process, they want to charge me for both my laptop and my VM. On a basic level, this makes sense: they charge and auth based on MAC address, I typically run my VMs in bridge mode, they ARP their own MAC over the NIC attached to the hotel network, cha-ching!.  But it’s an easy fix that I forget about until I see the 2nd $19.95 charge for the same day appear on my bill: NAT mode. Hiding the VM behind the MAC that the hotel is already aware of saves me from calling the front desk.

This is a classic example of where VM sprawl is such a threat. I could easily set up 100s of VMs on their hotel network and only pay for one access connection, and that would have no insight into what I was doing on their network with my VM farm (assuming I’m doing something malicious outside of their network and an encapsulating in SSL). Sure, it would be slow as molasses, but I’m not concerned with speed, I want power. 

This is the same threat that enterprises are currently facing with VM sprawl.  The specific threat is the ease at which users can spin up VMs: they don’t even need IP addresses, just a terse understanding of NAT. And for the most part, I don’t believe the threat of rogue VMs comes from sending traffic TO the rogue VM anyway. It comes from what that rogue VM is sending OUT on the network.  What is it spewing across a corporate or production network?  How hard is it to track and shut down if it’s NAT’d? Even if it’s only up for 5 minutes, that’s enough. VDI Bombs only take seconds.

So as always, I thank Hoff for posing an excellent question and for the lively discussion on Twitter between him, myself, and Lori. I do agree that a proplery managed VM infrastructure, with tools like vCenter and 3rd party VM network monitoring tools drastically help stave off full-blown VM sprawl. But it’s the inability of these solutions to monitor everything that scares me the most.  But if nothing, Hoff knows how to pick at the market’s shared security wounds. 

For those who follow the music business, or for anyone that just reads and/or watches the news, you’ll know that 80’s metal icons Guns n’ Roses finally released the much anticipated “Chinese Democracy” album on Sunday, Nov. 23rd, 2008. I say finally because we’ve been waiting on it for years. Every year just before Christmas rumors began to fly about a possible release date that year. And this year was no different than other years; those of us that were actually looking forward to this release hedged our bets once again and said “Sure, I’ll believe it when I see it,” including Dr. Pepper. Who even knew the Dr. was a Guns fan?

See way back in March of ‘08, Dr. Pepper ran a campaign – if Chinese Democracy came out this year, anyone who wanted one would get a free 20oz bottle of Dr. Pepper. I’m assuming Axl took that as a throwdown challenge becuase he delivered, leaving the Dr. to start giving out free bottles. The Dr. came through on their end of the bargain and opened up their website for the free 20oz requests for 24 hours, only valid on the day Chinese Democracy hit shelves, Nov. 23rd. The official word came down about 2 months ago that indeed, Democracy would actually drop as expected (with distribution and promo details to back it up), leaving the great Dr. ~2 months to plan for the onslaught of freebie requests. If we know anything about humanity it’s that no one can say no to “Free.”

I’m not sure what Dr. Pepper was expecting with their offer, but it certainly didn’t plan for the flood of internet traffic that would be directed their way beginning at 12:01 AM EST on the 23rd. Within hours their site became unresponsive, going through phases of availability: first, the intro Flash got stuck with no way to skip past it; then they finally caught on and started redirecting all users to a text-only portion of the site where the “Free Dr. Pepper” link was buried way down at the bottom which directed to a page that wouldn’t load; and finally, the coup de grace, the entire site went down and was returning nothing more than a 503 Service Unavailable error. As of this writing, we’re still at 503, and I’m still sans a Dr.

This very temporary unplanned (well, they had two months to plan, but let’s stick with un- for a second) outage is a textbook case for elastic provisioning: spinning up services infinitely to address temporary need and then spinning those services back down when traffic returns to normal levels. It’s like cable or satellite Video on Demand requests spiking during a snow storm on a Friday night. Services that normally handle X amount of traffic are suddenly forced to handle exponentially larger volumes of traffic, typically unexpectedly, but they still need to deliver that service.

To give them the benefit of the doubt, maybe Dr. Pepper didn’t anticipate the massive traffic volumes brought on by every major news source in the country picking up on the Gn’R bet, but that’s the point of elastic provisioning: accommodating service need without anticipating fixed levels. All they needed was the ability to scale for 24 hours, then things could have been returned to normal. So what went wrong? How much is this free offer (which has now been extended 18 hours, although it doesn’t matter if the site is still down) going to ultimately cost them in marketing, name, and not so good press? I’m guessing less than it would have cost to spin up a temporary site with the free offer that could scale for 24 hours.

And this one hits home: I rarely drink soda today but do have fond memories of the Pepper; I was looking forward to throwing on some 80’s hair metal and kicking back like I was in 7th grade again. But the Dr. teased me; they offered and didn’t deliver. At least Axl finally did, but I’m not waiting 17 years for a free Dr. Pepper; I’ll stick with coffee.