Innovation17 March 2016
Tomorrow we'll release Season 2 of Marvel's Daredevil to 190 countries simultaneously. Netflix members all over the planet will instantly be able to stream the show on any internet-connected device. Even though millions of people around the world will be watching, there will be very little additional traffic on the “internet” because of a decision we made in 2011 to build our own content delivery network, or CDN.
Since we went global in January, we’ve had increased interest in how we deliver a great Netflix viewing experience to 190 countries simultaneously. We achieve that with Netflix Open Connect, our globally distributed CDN. This map of our network gives you a sense for how much this effort has scaled in the last five years.
ISP Locations Internet Exchange Point (circles are sized by volume)
Netflix Open Connect delivers 100% of our video traffic, currently over 125 million hours of viewing per day. This amounts to tens of terabits per second of simultaneous peak traffic, making Netflix Open Connect one of the highest-volume networks in the world.
Globally, close to 90% of our traffic is delivered via direct connections between Open Connect and the residential Internet Service Providers (ISPs) our members use to access the internet. Most of these connections are localized to the regional point of interconnection that’s geographically closest to the member who’s watching. Because connections to the Netflix Open Connect network are always free and our traffic delivery is highly localized, thousands of ISPs around the world enthusiastically participate.
We also give qualifying ISPs the same Open Connect Appliances (OCAs) that we use in our internet interconnection locations. After these appliances are installed in an ISP’s data center, almost all Netflix content is served from the local OCAs rather than “upstream” from the internet. Many ISPs take advantage of this option, in addition to local network interconnection, because it reduces the amount of capacity they need to build to the rest of the internet since Netflix is no longer a significant factor in that capacity. This has the dual benefit of reducing the ISP’s cost of operation and ensuring the best possible Netflix experience for their subscribers.
We now have Open Connect Appliances in close to 1,000 separate locations around the world. In big cities like New York, Paris, London, Hong Kong, and Tokyo, as well as more remote locations — as far north as Greenland and Tromsø, Norway and as far south as Puerto Montt, Chile, and Hobart, Tasmania. ISPs have even placed OCAs in Macapá and Manaus in the Amazon rainforest — on every continent, except Antarctica and on many islands such as Jamaica, Malta, Guam, and Okinawa. This means that most of our members are getting their Netflix audio and video bits from a server that’s either inside of, or directly connected to, their ISP’s network within their local region.
As our service continues to grow in all of the new global locations we’re reaching, so will our Netflix Open Connect footprint, as ISPs take advantage of the costs savings available to them by participating in our Netflix Open Connect program. That means Netflix quality in places like India, the Middle East, Africa and Asia will continue to see improvements.
How Does Open Connect Work?
We shared in a recent blog post that Netflix uses Amazon’s AWS “cloud” for generic, scalable computing. Essentially everything before you hit “play” happens in AWS, including all of the logic of the application interface, the content discovery and selection experience, recommendation algorithms, transcoding, etc.; we use AWS for these applications because the need for this type of computing is not unique to Netflix and we can take advantage of the ease of use and growing commoditization of the “cloud” market.
Everything after you hit “play” is unique to Netflix, and our growing need for scale in this area presented the opportunity to create greater efficiency for our content delivery and for the internet in general.
To understand how all of this happens, let’s look a little more deeply at how Open Connect came about, and how it works:
Netflix Open Connect was originally developed in 2011 (and announced in 2012) as a response to the ever-increasing scale of Netflix streaming. Since the launch of the streaming service in 2007, Netflix had proved to be a significant and increasingly large share of internet traffic in every market in which we operated. Although third-party content delivery networks were doing a great job delivering Netflix content (as well as all kinds of other content on the internet), we realized we could be much more efficient based on our knowledge of how our members use Netflix. Although the number and size of the files that make up our content library can be staggering, we are able to use sophisticated popularity models to make sure the right file is on the right server at the right time. These advanced algorithms share some common approaches, and sometimes common inputs, with our industry-leading content recommendation systems.
As we touched on above, pre-positioning content in this way allows us to avoid any significant utilization of internet “backbone” capacity. Take the continent of Australia, for example. All access to internet content that does not originate in Australia comes via a number of undersea cables. Rather than using this expensive undersea capacity to serve Netflix traffic, we copy each file once from our US-based transcoding repository to the storage locations within Australia. This is done during off-peak hours, when we’re not competing with other internet traffic. After each file is on the continent, it is then replicated to dozens of Open Connect servers within each ISP network.
Beyond the basic concept of pre-positioning content, we were also able to focus on creating a highly efficient combination of hardware and software for our Open Connect Appliances. This specialization and focus on optimization has allowed us to improve OCA efficiency by an order of magnitude since the start of the program. We went from delivering 8 Gbps of throughput from a single server in 2012 to over 90 Gbps from a single server in 2016.
At the same time, Open Connect Appliances have become smaller and more power efficient. This means each TV show or movie that is watched by a Netflix subscriber requires less energy to power and cool a server that fits into a smaller space. In fact, our entire content serving footprint is carbon neutral, as we recently pointed out in this blog.
This year, we’ve extended our service everywhere in the world, with the exception of China. We’re excited about the role Netflix Open Connect can play in bringing enjoyment to people all over the planet. It feels like the adventure is just beginning!
Ken Florance is Vice President, Content Delivery at Netflix