The Virtual Data Center

I’ve been heads-down for the past few weeks in product land so I haven’t had much time to poke my head up into the clouds. I’m now standing up straight again, stretching, and getting back to thinking about the larger macro issues that cloud presents. Today I’m going to talk about one very specific issue: overlapping IP addresses and spaces by users of a common cloud.

This topic was spawned by a comment to an earlier post on the cloud. Here’s the comment:

In the Virtual Data center deployments/cloud, do you see a chance of IP address clash with the subscribers? Subscribers would want to have freedom to select the IP addresses. In such cases, how the network setup will be?

Ravi, the commenter, brings up a great point. The cloud is shared computing space so that’s got to include the network, and unlike computing resources and other parts of the network, such as MAC addresses, which are always unique until exhausted, IP addresses are such a small, finite pool of unique identifiers they have to overlapp at some point. We’ve seen this for years with companies that acquire other companies and attempt to combine and merge two different networks that share overlapping private IP addresses. And on a much smaller scale with home router users that end up using default IP addresses, typically either 192.168.0/24 or 192.168.1/24; basically any of them if they don’t change the default Linksys/Netgear/D-Link setting.

Like home users, the concern with share IP addresses in the cloud isn’t with public IP addresses; those are governed and for the most part rigid. The problem is inside the cloud, where IPv4 cloud providers are limited to one of three banks of private, non-routable IP addresses: 10/8, 172.16/12, and 192.168/16. On their own, this range encompasses almost 18 million IPv4 addresses, which will be plenty for most cloud providers. However there are two issues for pause, even with a pool of 18 million IPs to choose from:

1. Extremely large cloud providers, such as AWS. It’s not unrealistic to imagine that Amazon could support at some point more than 18 million unique workloads at once (in this case workloads meaning applications tethered to a unique IP address). Even as they were to get close to that number, management of re-usable IP addresses would be a challenge. They would have to know by the second which workload was using which IP address, when that workload relenqueshes that IP address, re-assigning the IP, and then the technical challenge of ARP management with such high-volume IP turnover.
2. Cloud providers that accept pre-configured workloads, say from internal private clouds. Company A may have a series of applications, bundled together in something like VMware’s vApp, that all use IP addresses in 192.168.22/24. Company B may have an vApp bundle with an overlapping range, such as 192.168.20/22. Given a large enough customer pool using vApp in an external cloud provider, there will be customers that use overlapping IP addresses on the same cloud network, and these customer IP ranges most likely won’t fall on easily defined subnet borders.

So what’s the solution (ruling out IPv6 since Ravi is probably asking about IPv4)? Thankfully there are a number of tools available that will virtualize networks above layer 2, virtualizing the IP space in the same way that VLANs virtualize frames. To address this issue, a cloud provider can use some network management device that will virtualize layer 3 before or at the same time customer traffic in the cloud is translated (via NAT) from a public IP address to a private IP address. As a request for a service in the cloud comes into the cloud edge, the network or application management tool will recognize the request for a specific customer and move that request over to the segmented layer 3 network isolated for that specific customer. Very much in the same way we tag frames and ports with VLAN data, a segmented layer 3 network can be created with “tagged” IP addresses that are destined for different parts of the network.

Ravi hit a core requirement for the cloud: in a true multi-tenant environment, overlapping IP addresses must be supported and cloud customers must be able to select any IP address they’d like, either during service creation or as part of a pre-configured virtual application bundle. Customers need to be able to say “This application must use this IP address” and not have to worry about the cloud provider returning a “Sorry, IP address already in use” error. So to answer his question, yes, I absolutely see a need for overlapping IP addresses inside the cloud, but as long as the cloud is built to support that architecture, overlapping IP addresses shouldn’t be a problem for cloud customers.