All for the Want of a Global Load Balancer

Christmas Eve and the AWS/Netflix outage to me aren't so much about whether or not the Cloud is viable or scary or dangerous. Rather, the event resonated with users across the United States because the Cloud delivers so much utility to each of us. And regardless of who was at fault--Netflix or Amazon Web Services, the event made it clear that there's no going back and that the Cloud has quickly become a part of our culture and our everyday lives. This is significant because while the Internet itself is a technology consumers have grown to love, Cloud is a way of delivering service that makes a service like Netflix streaming possible, and at a measly $8 bucks a month. The Netflix business model of delivering outsize utility for a low price point makes the business of streaming video all the more difficult. HBO, Cinemax, and the networks for me are unusable. I sense something beyond just making money remains in play at Netflix. Somewhere in that organization seems to beat a heart that quickens for humanity.

[Dear reader, I apologize for the rough editing of this blog post. For me it works best to publish, then edit. I will be working on this over the next days andyou are welcome to read it now and I welcome comments and ideas and all that good stuff]

While much is made of the sparsity of the Netflix streaming catalog, keep in mind that streaming is the disruptor's second act. The Wicked Witch, aka Blockbuster is dead, I think in part because each of us desired a little bit of payback for all the times we returned videos late and were charged outrageous fees. Rather than punish Netflix Streaming for innovating, and for challenging the iron grip of the legacy media houses, I praise Netflix and personally admire the generous open source contributions Netflix has made to the Cloud Community.

I once tweeted that the value of the code and tools Netflix has shared on GitHub may well be larger than the value of the streaming business as a whole. That may be true for about five minutes. We're at a tipping point in streaming video media similar to that of the music business just before the iPod. Yet each evolution carries forward a little bit of spin from the last disruption. In this iteration all the players know the iPod playbook and so they are trying very very hard to fight the gravity of disruption. Having lost Blockbuster and effective control over the distribution of their content, the old guard hold onto to their catalog stubbornly hoping Netflix will disappear. But we all know how this movie ends. I see a much larger Netflix catalog of content on the horizon and substantial value for consumers. Once the old guard have been disintermediated and a more open, consumer friendly market for content prevails ever Christmas will be a little brighter (that's kind of a stretch?). In my opinion what's in play today with respect to old media firms is not so different from the agency model that kept eBook prices artificially high for many consumers.

So what about the Global Load Balancer Already?
Most Cloud Architect and Solution Architects of high availability systems are familiar with Load Balancers. AWS designed a mostly well-intended and useful service around this technology and it's often referred to as the (infamous) ELB (Elastic Load Balancer). Functionally a "classic" load balancer distributes service requests across hosts grouped into a pool of largely identical servers. Spreading requests across multiple servers is a key capability required for the horizontal scale-out favored by Cloud and to a lesser degree Enterprise Architecture.

A Global Load Balancer operates one or more levels above the Local Load Balancer--much like DNS, the Global Load Bancer returns an ip address. Often the IP address is the address of a Local Load Balancer, but it could be any ip address either public or private, based on the specific solution requirements. Given that Global Load Balancer technology stands on the shoulders of a fundamental and highly robust and ubiquitous technology like DNS the service can be very resilient if configured that way. If LLB (Local Load Balancer) high-availability solution is designed correctly the Load Balancer monitors the health and responsiveness of a pool of servers and directs requests to nodes of an application. A global load balancer performs a similar function, except at a data center level. A Global Load Balancer detects when a Local Load Balancer is no longer available, and when this happens can return the ip address of a load balancer or endpoint in another data center anywhere in the world. The Global Load Balancer in part is based on DNS technology, and is this way is not coupled to a specific network. Further, Global Load Balancers can determine the nearest endpoint for a specific user request, and can return an endpoint closest to a user. In times of network failure, the endpoint that is normally "closest" or "lowest latency" frequently becomes a very high latency endpoint. So the ability to monitor latency and to adaptively respond to requests enables a very fine grained capability to route users around system and network anomalies such as those that continue to plague US-EAST-1.

My observation is simply this: why don't Cloud or Web firms take a lesson from classic availability architecture and use a load balancer to enable endpoints to fail across data centers. For enterprise, why make Cloud an either/or question. And why gamble with the availability of important systems when it's not so difficult (especially if you've figured out the persistence layer) to balance systems across multiple Clouds. Using such an architecture to enable rapid switching of endpoints to other data centers could enable an enterprise to run an application both in the Cloud and in the data center and to balance traffic across them not in the sense of the much hyped "Cloud Bursting" which seems to focus mostly on "bursty capacity" and somehow seems rooted in Private Cloud but rather simply as a strategy for mitigating risk. Hurricane Sandy in New York City would have been far more disruptive that it was if not for architectures built using Global Load Balancers for high availability. While there's no comparison to the complexity of failing over smaller scale apps to Netflix, several non-web facing internal applications for which I designed the architecture in 2011-2012 failed over to alternate datacenters simply by  changing the endpoint returned by the Global Load Balancer.  In the case of Netflix, the option to migrate all the data was not an option, yet it puzzles me to no end why given the extensive focus on Cloud failure modes, the AWS ELB remains a single point of failure for Netflix and many other applications running in AWS whether at modest or extreme scale.

I wonder if Netflix will select an additional Cloud in 2013 and in doing so create some real competition in the Cloud Service Provider space. After such a high profile failure (Christmas Eve, for Christ's Sake), I feel certain the decision has already been made. The impact of such a move would legitimize another Cloud. People who know these things tell me that 80% of Amazon Web Services capacity is in US-EAST-1. To me this suggest Amazon Web Services has fallen victim to it's own success. The nature of Clouds is such that as they grow bigger the network effect driven by the efficiency of locating data and compute capacity as close together as possible becomes overwhelming and the penalty for locating in another region, such as the West Coast, or another Cloud in another data center, becomes higher. While recently AWS has announced some capabilities to make it easier to migrate to the US-WEST-2 (Oregon region, which is priced about the same as US-EAST-1) such capabilities don't really seem to matter.

I spend a great deal of my time learning from web scale best practices such as those co-developed by firms like Netflix, Heroku, Pinterest, Google and redefine what people think they know about distributed computing..  Yet based on the theme of AWS re:invent, and my personal experience in Fortune 500 Cloud, 2012 may not only "not be the end of time and ancient calendars," but 2012 may mark the year when this thinking makes the leap and infects the DNA of Fortune 500 technology. The itchy little problem of Cloud as ready for big business remains the persistent failures, yet the risk of failures in hindsight can be minimized--and using technology commonly used by Enterprise that unlike Oracle RAC clusters and other High Availability technology works just fine in both Cloud and traditional data centers.

Listen closely to what Cloud and web scale practitioners have to teach. Architecture of Cloud applications deviates in ways that will make your database and application engineers pull-out their hair, scream, and storm out meetings (based on what I've seen, except for the hair pulling). For example the application server architecture and scale-out strategy for scaling applications in the Cloud is very different from how most of Fortune 500s do Enterprise Application Server clusters today. And after you fail in the Cloud trying to kick it old school, the enlightenment comes quickly. Yet at the same time, don't get too caught up in the hype and forget everything. The Global Load Balancer commonly used in large enterprise, when deployed effectively, could very well help you build applications which balance the new with the familiar.

Further Reading / Cultural Reception of Cloud

Netflix Outage From Amazon AWS Failure Shows Cloud Service Can Fail Any Company - The CIO Report - WSJ 

Amazon Studios taps the Onion, Daily Show writers, others for comedy pilots 

 The data center and mobile web drive Lightower Fiber’s and Sidera’s $2B merger

Christmas Eve AWS outage stings Netflix but not Amazon Prime

Summary of the December 24, 2012 Amazon ELB Service Event in the US-East Region


Some of the media reception of the events. I find the coverage to be grossly inaccurate, yet it's part of the cultural reception of the Cloud, so I've included some links:



The New York Times story on the Outage
‘The Cloud’ Challenges Amazon

Heroku, if you were a character on South Park, I would call you Kenny.Every time AWS has an episode you're killed. bit.ly/U7pu9w
— Brian McCallion (@BrianMcCallion) December 25, 2012