The Virtual Data Center

I’ve hinted in the past on my ultimate application virtualization scenario – where I want the market to be for deploying and supporting remote applications for clients in the future. I’m still working on that giant whiteboard architecture map in my basement on what AppVirt looks like from the DC computing side, but today I want to write about the client side of that architecture. And while brevity has eluded me in all parts of my lingual life recently, I’m going to try to be succinct here (expect failure).

I attended the first of a four-part series on Cloud Computing last night held by the WTIA, which included an excellent presentation by Aaron Kimball from Cloudera on the basics of the cloud from the data point of view. Having retired the engineer title for marketing a few years ago, it always makes me happy to see someone who spends their career designing complex systems stand up and give an intro presentation that also includes the business benefit. So often engineers address the How rather than the What and the Why, and Aaron did an excellent job with the latter.

His presentation, along with others last night, got me thinking about what application virtualization in the cloud would look like to the client (and I’m not talking about GMail here).  Let’s look at a real example:

I bought a Mac a few months ago primarily to run Lightroom, so I spec’d out the Mac to go high-end because it would be running a very beefy photo application (along with Photoshop in the future I’m sure). The machine also had to run VMware Fusion in parallel (no pun intended; sorry to my Parallel friends) - I have a photo stitching application that’s currently Windows-only. Standard operation keeps me stable at 75% RAM and 40% CPU on average.

But what if I didn’t need to buy local computing resources and everything was processed remotely? Let’s jump ahead 10 years (a big leap, I know) and look at how this could be different if client apps were in the cloud.

I buy a local processing machine that’s drastically stripped down from my current Mac. I boot this machine to a web browser, where I head over to Adobe and say “This is Alan; I need to run Lightroom.” Adobe says “No problem. Let me push down the secure Lightroom App Shell. Ok, now you’re ready. Here’s a list of your albums pulled from Amazon S3.” I say “I need to process the latest batch of Mt. Baker HDR images.”  HDR images take a substantial amount of computing power to process, so Adobe comes back and says “No problem. I’m going to need 2gig RAM and a dedicated CPU core for this, but your monthly subscription only covers 1gig and .5 core. I’ll charge you $0.021/minute if you’d like to burst.”  I say “Great, let’s do it.”

Amazon then pulls its own resources from AWS and start distributing my HDR processing over thousands of machines/cores/RAM, all controlled from my local Lightroom App Shell. To me it’s displayed as one 8ghz core (remember, 10 years from now:) ), but in practice that value is an aggregate of distributed resources pooled from thousands of machines.I process my HDRs, they’re stored back in my Lightroom library which is managed by Adobe.com, I take my tiny laptop over to the client’s office, rinse, repeat.

But what if I don’t want the processing done exclusively in the cloud? I mean after all the above scenario is very similar to what I can do today with a web browser. What if instead my local computing platform is moderately beefy and I process everything locally. But as my machine starts to bog down from stitching together HDR panoramas it uses the Lightroom App Shell to request raw computing resources from AWS, and I become part of the distributed cluster? I scale and process background tasks remotely and only use my local resources for rendering the images to my display. My machine is part of the cluster and the line between local and remote processing becomes blurred. That’s some cool client-based application virtualization. The application running state is spread across elastic resources, of which my local resources are a part of.

True application virtualization will be a huge undertaking and this is simply one part and one idea. But why not think big? Go for the gusto I say. Oh, and there’s more here, but I’ve long since crossed the brevity line. Maybe more later.

I’m going to start this morning with a question that I’ve been pondering lately: Should we work out all our security issues with virtualization before we start looking at cloud security? Would it take an army of security researches to work on platform and cloud threats in tandem?

My gut response is “Yes, these are linear and we should address them in a very deliberate order.” A good bit of the cloud is based on virtualization technologies that we’re all using yet no one has thoroughly addressed the security threats associated with mass adoption of these platforms. The truth is we don’t have the luxury of time to ponder all the threat vectors of virtual platforms and deliberately control our roll-outs for security: these technologies are already live and, fortunately or unfortunately, supporting a vast number of our applications and data centers today.  That shipped has sailed and it’s full of ESX and Hyper-V boxes.

Of course this scares the <you know what> out of me, and yes, it keeps me awake at night wondering when this is all going to collapse due to one silly little worm. How long before we see a virtual Sasser that has real business reproductions and sinks an entire ship?

In addition to the security concerns we have on virtual platforms, we also have different security concerns, arguably more frightening, with cloud computing. Like the three categories of virtual security I wrote about way back during RSA last year, cloud security falls into two primary categories in my head: securing the data in transit and securing access (yep, I categorize everything and make lists like you wouldn’t believe).  These differ from the virtual platform threats in that cloud security is about process rather than code. We have to implement security checks and border patrol on the Cloud Highway: everything is inspected as it comes in and out and only certain workloads are allowed on the highway. We have to look inside every freight package that traverses the Highway, and that’s going to be one mighty endeavor.

I was having a conversation with a VDC reader last week and the topic of adoption vs risk came up: If the risk is so great (as I think it ultimately is) then why isn’t it getting more attention? I believe that no matter how great the risk, the rewards associated with implementing virtualization, and eventually moving outside the data center to the cloud,  are too great to be stalled by theoreticals, and if you can’t see risk then there is no risk. The mobility, the consolidation, the new levels of management and granularity, all of these are so much more important to business process and IT agility than security threats that frankly don’t exist yet. Business moves ahead.

So what do we do? We address these security risks in parallel. We start making virtual and cloud security a more visible issue. We need more virtual security researchers — dare I say it, because I’m not a fan of the branding — more white hats out there focusing on both sides. We need PhDs looking at hypervisor and resource-based security and we need ethical hackers focusing on the cloud. We need new Cloud PenTesters.

We need giant posters in every college depicting Max Headroom (the only virtual icon I can think of) smiling, pointing, and saying “I Wa-wa-wa-wantttt You-ou-ou!” The virtual security army needs new recruits, and we need them now.

I’ll close with another question to ponder: Will these recruits even know who Max Headroom is? Were they even born yet? 

For the most part, I’ve avoided talking about the current state of security in the cloud.  For one, there are many, many smart people and great bloggers already talking about it and I love reading what they write (and almost always agree with it, almost ).  And two, I spend a good bit of my day job (and have for the past 3 years) focused on the ideas around contextual-based access and security: security has different meanings and requirements depending on the context of how a particular service is accessed or invoked.  Now that’s some cool stuff, and the cloud access model is a natural extension of contextual-based access. You’ll see much more on this idea from me over the next year-ish or so.

Over the holiday break, I spent a good bit of time thinking about how and when cloud security was really going to become a watercooler issue (while playing Super Mario World on the Wii Virtual Console, ironically enough). We’re nowhere near there yet - heck, we’re still discussing the nuances of the cloud itself; we have to build that before we can worry about who or what gets in. But there a few anecdotal things that kept coming back to me:

1. In a previous life, I used to audit the security of data centers, from the physical rack and cage installations (even testing the man gates, now that was fun) all the way down to the network and applications.  I even conducted interviews of all DC staff, both enterprise and hosting, on what they knew, how they responded in the event of a breach, where root passwords were kept, all that meaty security goodness. My goal was to verify that a particular application was installed in the most secure running environment available, primarily to limit legal exposure should there be a breach against this application.  What I took from that multi-year experience: It’s extremely expensive to conduct these types of audits, and at some point the liability baton is passed to the people actually implementing the technology, away from those who designed it.  I could interview people all day, and spend weeks walking through their network, but once I left the premises and filed my report, it was up to them to stick to those procedures. We had to trust (in our case legally) that what I saw remained in place.

2. As I’ve mentioned here before, I own a small photography company, so small in fact, I use a cloud provider (for lack of a better term) for all our business documents, including balance sheets (very basic stuff: sold 10 photos, bought a new lens, etc). Before the end of the year, I was spending a nice, cold, snowy Sunday in beautiful downtown Bellingham at a local coffee shop where, over their WEP-enabled wifi network, I was working on said balance sheet, getting ready for the fun that is tax season. Me being extremely security conscious, I immediately noticed that the cloud App provider doesn’t use SSL to access my docs, and in fact, says they don’t even offer SSL unless I pay a much higher premium. Now lucky for me simply changing the login URL manually to HTTPS worked fine and I was happily encrypted. But I’m guessing I’m in the minority and most of their users won’t notice that they’re accessing what could be sensitive documents in a completely insensitive manner, or that it could easily be forced to work. Most users who got to step two would read the Help and see they had to pay for that service and then do their own value assessment: is the risk worth paying extra?

3. Amazon’s lack of security notification in their AWS mid-December, well documented by Craig on CloudSecurity.  A weakness in Amazon’s signing algorithm opened up the possibility for more-frequent-than-normal cryptographic collisions (yep, the same core issue that’s currently plaguing SSL CA providers who use MD5). Amazon did fix the problem, but it took them 7.5 months and they kept it under wraps, a very common response in the security world.

So noodling on these three things while on vacation led me to this realization: Cloud services are going to create a new level of trust between end-users, providers, and enterprises. There’s no way around it, simply because there are so many moving pieces to a cloud service that are outside our control.  There’s nothing in the world I could have done during the 7.5 months that Amazon was fixing their signing flaw except change my transport to Amazon, and I probably wouldn’t have done that because the issue wasn’t public and I didn’t know about it. Once I start sending requests into the AWS, I have to trust that the systems in place are going to work and work securely.

Now that’s a big leap: one that I wouldn’t make with a more standard computing model. But in the cloud model, we have to trust so many new components in the stack. Of course we can have safeguards (SSL) and checks and balances (pen-tests, people who responsibly disclose security flaws) but at a minimum, those require near unfettered access to systems that are no longer in our control and require knowledgeable people to address them. In my auditing days I had unfettered access, during a specific window of time. Once I was done my access went away, so even then there was no continual checking of those systems.

So moving forward, as the security people tear apart the (in)security of cloud computing, the rest of the world will just need to take that leap of trust. A lowering of our standards for what we can control in the cloud’s outsourced data model. As an end user it kills me, but I know I have to make those sacrifices if I want to use those services. So I have to modify my level of trust, and apply new and stronger safeguards to the rest of my workflow processes (personal and professional) to make sure I’m able to recover if/when there is a massive breach that’s beyond my control.  My recovery is something I can control, and I definitely trust myself.

One of the best parts of my job is getting out in the field and talking to people about what’s possible tomorrow: the future of the data center, the future of virtualization, the future of security, the future of Second Life and Holodecks…all that way out there stuff, although none of it is really that far out there anymore. Sometimes I can sit down with other’s who think like me and we just jump straight past the what-ifs to the whens and hows and start charting a course. But most of the time, especially in regards to public speaking, I start from the beginning, a “First there was light!” kind of thing, and then build up through three acts to the dramatic peak. I love talking to people in each group, but would probably lean towards the “Light Group” group because it always challenges me to help them understand.

During this steady ascent up with the Light Group, I rely heavily on analogies. We use analogies in every day conversations with our friends, and they’re an excellent vehicle for carrying new and foreign ideas. In my presentations, there’s always a baseline starting point, a referential section relating these new concepts to ideas that are already well understood, and then the theorem “thus” moment, bringing it all together. It’s just how I think, and how I believe people listen.

• Want to talk about VLANs? Ok, think of your network as a train, and certain people have tickets for the 1st Class coach and others have tickets for the Economy coach. You know what coach you’re in before you board the train, and you stay in your coach. The coaches are VLANs and you’re a packet…
• Curious about IP Reputation spam blocking? If your brother or sister walks up to you in an airport and hands you their luggage while they find the closet Starbucks, you’ll gladly hold their luggage and watch after it. You know them, you trust them, and you trust that they’re not handing you malicious luggage. If a complete stranger does the same thing, you’ll run to the closest TSA agent. The person giving you the luggage has a known and accepted reputation based on your history with that person. Same is true for MTAs and spam.

Relation is the key to successful informative communication. If you want or need someone to understand what you’re saying, you have to phrase and present it in a way that they’re expecting and, most importantly, so they can project themselves into that situation. People want to understand you and feel comfortable; if they can’t relate to what you’re saying, then they aren’t going to listen, digest, and understand.

Now you’re probably thinking “Great Alan, but what in the world does this diatribe have to do with the Virtual Data Center?” Excellent question! Before we can get to the advanced VDC discussions, we need to find a relate-able starting point and “baby step” our way to the meaty goodness. This is why I focus so heavily on terminology and definitions.

To paraphrase Neil Peart, we need to start with a waltz; the most fundamental component of what we’re doing that everyone understands, and then we need to build on that concept and take it to new levels. As we progress, we will get more complicated, but we’ll do it one baby step at a time. Even though I’m a musician, it’s very difficult for me to relate to a Neil Peart drum solo, but I can absolutely relate to a waltz and providing a basic backing rhythm. And then I can relate to adding patterns over the basic foundation of the waltz. And once I reach that point of relation, I’m able to sit back and appreciate Neil taking me from the baseline waltz, through the relate-able sticking motions, well into the “thus, I’ll never be this accomplished as a drummer” moment.

I’m a stickler for terms and definitions. If we’re all not using the same lexicon, how can we communicate? Did Enemy Mine teach us nothing?

There are two virtualization terms that are being used in a very loose manner lately, and me personally, I think they’re being used incorrectly. I’m not knocking the sentiment of their usage, just their assigned and (becoming) generally accepted usage. At first it just appeared to be analysts using these terms in a non-standard manner, but lately I’ve started to see vendors use them incorrectly as well. Yes, “incorrectly” is subjective, but let me explain by addressing these two terms:

• Virtual Appliances: When you live and breath in a world made up of hardware appliances, you tend to know what the word “appliance” refers to in a very black-and-white manner - It’s a black box that you buy, plug in, IP, then ssh/https into. That’s it. So virtual appliances are, quite literally, virtual software versions of their hardware big brothers. Zeus builds and distributes a perfect example of a pure virtual appliance. All too often lately the term has been used to describe full-blow virtual machines that have pre-installed OS’ and applications. So basically this is being used to represent a virtual version of a Ghost image, which is a disservice to true virtual appliances. Me, I prefer to still call these virtual machines or virtual images; they just happen to be pre-built to run on a specific hypervisor. You still use an off-the-shelf OS, like Windows 2003, and still use off the shelf software, like IIS. There’s nothing stopping you from downloading one of these virtual machines and re-purposing it for another use, something you can’t do with a true virtual appliance. And that’s basically the deciding factor: If you can use it for something beyond it’s original intent, it ain’t a virtual appliance.
• OS Virtualization: Same idea as above, “OS Virtualization” already has a definition: virtualizing an entire operating system, from hardware all the way up to apps. Now this term is being used for a completely different type of virtualization technology, something I call “kernel virtualization.” Kernel virtualization is, as the name suggests, virtualizing the actual kernel of the host operating system. It’s like paravirtualization on speed. The same kernel that is hosting the bare metal OS is also hosting the virtual guests running on top of it. OpenVZ is a good example of kernel virtualization. It’s really cool technology, but the specific term “OS Virtualization” already means one thing, so why is it being co-opted for something else? Kernel virtualization can be classified as a sub-category of OS virtualization; that, I’m fine with. But leave OS virtualization alone; it didn’t do anything to you for you to start picking on its given name.

To me, the naming terms of a particular technology should describe the technology. There are (according to some estimates) 250,000+ words in the English language. Do we really need to use the same words to describe virtualization technologies over and over again? We don’t need to be creative here, let’s just be literal.

WARNING: I’m all riled up today and in a mood, so this is going to get long. Read at your own peril, and grab some coffee.

In an alternate life, I was a Linux bigot. No computer was coming into my house or sitting on my desk that didn’t run Linux. My WiFi AP was a Linux laptop that bridged between two PCMCIA cards; I tried to convince my wife that Gnome had everything she needed; I spent all of my free time compiling new graphic drivers to make sure I had fluid transparency in my SSH connection windows. But Linux ultimately got it wrong: there are far fewer people that want to spend their weekends tweaking CLI arguments than want their computers to “just work,” and ultimately I realized I was wasting my life trying to figure out why my ethernet driver didn’t come back up after I resumed from hibernate. I wanted my computing (personal and professional) environment to “just work.”

So one day, relatively out of the blue, I sucked up my pride, dropped a new hard drive in the Dell Latitude, and installed XP. And oh my friends, how the sun shone that day! Within an hour, I had a fully functioning portable computer: the sound worked and didn’t phase in and out when I as accessing the network; I could access the full resolution of my graphics card and move beyond 800×600; I didn’t have to manually edit a text file when I moved from one wireless network to another; presentations and projectors miraculously starting working (Andre: I’ve been there many, many times). In fact, everything “just worked.”

And most importantly, I became productive. After tooling around with XP on the removable hard drive for a few weeks, I realized that with Linux I was wasting so much time fussing with my working environment that I was actually becoming counter-productive. And suddenly, my laptop became a tool to get my job done rather than a machine that always needed TLC. Now granted I’ve had to defend my “Windows is just better” decision to my close circle of Linux supporters (Tux tattoos and all), but it’s been so worth the inner-circle humiliation and ridicule. But it didn’t have to be that way: There was a time when Linux was <this close> to focusing on the why people use computers instead of how computers work. SuSE and Mandrake where the closest to building a plug-and-play Linux distro for the masses, but ultimately they forgot about the normal users. And don’t me started on Red Hat, who I personally blame for the Linux downfall (Full Disclosure: I am an RHCE, so I’m not just ranting and raving on this one).

And likewise, data centers are just larger tools. I’ve talked on here before about the VDC as a service, and it’s all the same thing: use a tool designed to solve a goal for that goal and then move on. So you can imagine how torqued up I get when I get in conversations where people say “Oh, well this solution would work better if it ran on Linux instead of Microsoft…” Maybe for some people, yes, but for others it would run better on Solaris, or in Java, or on z/OS. Who freakin’ cares what it runs on as long as it accomplishes the goal?! Imagine how productive we could all be with full VDCs that didn’t require us to spend all day trying to get one API to talk to another, only to find out that we have to do it again 3 times for 3 different APIs to support everyone’s virtual OS infrastructure.

If the VDC is going to emerge as the disruptive powerhouse I think it will, we all have to put our biases aside and focus on the end goal: A DC that sucks in requests and spits out responses. Sure, we’ll have the Linux team and the Windows team, the VMware team and the Hyper-V team, the Cisco team and the F5 team, the network tap team and the SPAN port team, the plenum cable team and the non-plenum cable team…you get where I’m going with this. The VDC can not become a reality if we’re all fighting religious wars. So check your biases at the door and choose a tool that solves the individual “baby step” problem you’re trying to solve in your VDC today, move on to the next problem, rinse, repeat.

Now back to my regularly schedule Apple virtualization research project: Words can not describe the size of the smile on my face when I found the Terminal in Finder and opened it in Pro mode. A 50% transparent command line with bash! ls -alFrt works! Oh sweet *nix, why did I ever leave you?

I know, it’s been quiet around here lately. What can I say? Pitching the VDC message and the future of the data center has kept me very active lately (it’s a great time to be focusing on what I focus on). Now if people would just listening to me when I rant and rave about the problems with silo’ing virtualization technologies in the data center and the problems with software switches today, I’d be such a happy person.

Speaking of happy, the MacBook Pro arrived a few days ago. Man, does Apple know how to package or what? I’ve always wanted to live in a retro-future house; you know, like the one that you used to ride through in Spaceship Earth at Epcot Center at DisneyWorld in the 80’s (long before the 90’s remodel that included the futuristic Internet). Every childhood summer included a trip to Disney, and as a budding technologist, Epcot’s vision of the future was always my destination of choice. Forget Magic Kingdom; I wanted to play in the Imagination science lab and eat exotic Japanese food. Some 70’s children grew up obsessed with Star Wars, I grew up obsessed with the lifestyle of the future.

So my first thought when unpacking the MacBook Pro was something like “It’s silver, it’s white, and it glows when I turn it on!” You guessed it, they had me at “Designed in California.” Now this is old news for the millions of people who already have a MacBook, but coming from the world of Linux, Dell, and Vista (btw I love Vista as a powerhouse production Operating System), just opening the box instantly took me back to the retro-future, and I immediately ran upstairs and looked for my ultra-sonic dishwasher. But alas, it’s only 2008 and we’re still using water and chemicals to clean our dishes.

And before we even talk about the tech details, I have to take my hat off to Apple for combining lifestyle with technology, something sorely missing from other technology companies. We all have computers, we all have HDTVs and 6.1 surround systems, and yet there is so little work in the personal technology sector (IMO) on form, it’s all function with very little thought about How it’s used vs. What it does (more on that here).

As I write this, the MacBook Pro is sitting across from me on a chrome and glass table in my very modern-looking office, lid closed, with the pulsating “I’m sleeping” white light phasing in and out on the front, sitting immediately next to a replica antique soapstone and slate Chinese chess set. I’m struck by the beauty and juxtaposition of each of these tools, each centuries apart, yet both do exactly what they’re designed to do and both look amazing.

And beyond looks, the damn thing actually works like a champ. But no time to write about that now, maybe later. Right now, I have to go do that cool magic trick where you can lower your hands above the MacBook speakers and magically turn on the keyboard light…