How to build a Netflix-like multiscreen OTT service (part 1)

05/31/15 update : this article has been translated in Chinese by Mariah Feng from Conax. Thanks for this effort ! Read it here

With the fall of Megaupload, legal VOD sites are quickly gaining back popularity as consumers are eager to watch fresh video contents on all their connected devices. If you are a content owner, a TV channel or a telco, it may be the right time to (re)launch you Multiscreen OTT VOD offer. This post intends to start from the reference tech choices in this game – Netflix’s ones – explain the major challenges of such type of service and the associated DRM issues, and finally drive you throughout the different market options you have to setup your own service on a close basis (this part is covered in part 2 of the post). Everything would have been easier if Netflix did sell its solution as a white label platform, but it’s not (yet) the case, so this leaves fun territories to explore !

Learning from Netflix

To make a long success story short, Netflix’s one relies on several key technological choices :

Netflix API Device Endpoint principle

– API reign : their API, launched in 2008, is the foundation of all their platform, the alpha & the omega of service reusability and agile development on a maximum of heterogenous client platforms. They got clever API architecture with dedicated endpoints by devices (read latest API architecture redesign story here) to optimize the data exchanges, API degradability strategies [Watch presentation by Daniel Jacobson, Director of Engineering, API]… This is definitely the major key of Netflix success and the reason why their API generates the most traffic in the US for a single service. API development shall definitely be on your top priority list…

– Wide cross-platform client support : Netflix is now available on hundreds of different client platforms (PCs, smartphones and tablets, game consoles, connected TVs and BluRay players, connected media players…) allowing to start and finish playback everywhere the consumer wishes. If you don’t provide multiscreen support, your service is bound to fail, so you must invest heavily on this topic and hire a crew of bleeding-edge frontend dev ninjas.

Netflix PS3 UI

– HTML5 mutualization : while not (yet) all of their client application rely on HTML5 as a the UI medium, Netflix made a substantial investment on this technology as it conveys a definitive advantage as regards developments mutualization and flexibility of UI repackaging. They proved that on controlled environments, the HTML5 user experience has nothing to envy to native apps. Given the fact that consumers are increasingly facing HTML5 frontends, their tolerance to less-shiny UI experiences tends to raise, and that’s a good point for you. It gets complicated currently if you can’t deploy your own engine and have to rely on various HTML5/video/DRM combos but still it’s a good target for keeping the dev budgets low.

– Custom Webkit engine :  to achieve the cross-platform reusability of their custom developments and optimizations, they went for building their own flavor of Webkit+QT engine and use it as basis for the SDK they propose device manufacturers to integrate. This may be the impossible target to reach for you, as this requires a leveraging power that few brands apart from Netflix do have on the market today.

– Streaming secret sauce : Netflix relies on a custom DASH subset + PlayReady DRM (with a confirmed pinch of Widevine DRM for the Wii) bundle that they embed in their SDK and use on (almost of) their client deployments. It allows them to produce the strict minimum of packaged streams variants and therefore to minlimize their storage needs. While this combo might not be available as a standard on your various target platforms SDKs, it’s a good objective for your project on the long run. Additional DRMs might be required to cover the whole clients range, though.

– Cloud hosting & transcoding : by migrating their platform to the cloud during the past year, Netflix has achieved an incredible jump towards scalability and failover. While this might be overkill for a medium size OTT project, it gives a good clue of how you can achieve dimensioning for both content preparation/service and massive API service. This is also one of the most rocket science topics in the panorama, so you’ll need sharp backend/workflow talents to achieve a similar deployment…

OTT service components and decision criterias

To build your Netflix-like service you will need to develop or integrate a wide range of service components (see following diagram for required blocks). Here is an insights list on key points to evaluate when choosing or building a solution.

Multiscreen OTT Service Reference Architecture

General points

• Time to market : this will be the key difference between technology options. Building everything from scratch results obviously in the longest TTM and highest challenge. With the most packaged market solutions, 6 months is typically the minimum delay to get deployed on the first devices – and the delay gets higher if the starting point for client apps is only the service API and not full-fledge reference apps.
• DRM coverage : choosing the right technical partners or recruiting experts in this domain is a key success point as DRMs are mandatory and there is a good chance that several of them will be used to cover all the target devices. This implies making sure from the beginning that adding new DRMs on the platform will not highly impact backend architecture – that’s the role of DRM umbrella servers (Netflix licensed Irdeto ActiveCloak) who provide a unified backend interface to all DRM servers flavors and mutualized business rules : see specific DRM chapter below.
• Deployment model : whether your API platform will be deployed fully in the cloud, on a SaaS mode or on-premises/self-hosting, this will impact the planning and change the level of difficulty. While cloud deployment seems to be the most straightforward, it’s also a source of concern over redundancy. Not-in-the cloud SaaS brings concerns about SLA/scalability and self-hosting requires time to reach the required failover and dimensioning levels.
• Automation : it may sound trivial but no one should underestimate the need to perform automated tasks in OTT platforms, ranging from sending targeted emails upon very specific customer situations to triggering backup plan when media ingests fail or increasing platform availability dynamically when load gets higher.
• UltraViolet compliance : while it may not yet be a topic today, there is a chance that UltraViolet finally manages to become a new standard in digital media business as BluRay did for physical media, which means that your platform will have to integrate with other service providers using this interoperability method. Being compliant since the beginning thus means being right on time when the market  confirms its choices.
• Second-Screen support : this subject being a highly popular feature nowadays for live (and soon for on-demand contents), your chosen solution must allow linking of complex and various metadata to the programs (the actual datas being hosted somewhere else) as well as their synchronized display on smartphones and tablets. This means integration with the right watermarking or fingerprinting technologies on the backend and support for the synchronization inside the frontends.

Content preparation (transcoding/DRM)

• Deployment model : for transcoding and DRMization, you can meet several types of deployment models. The on-premises model is the old-fashioned one where you transcode and DRMize contents with your own processing farm : good thing about this one is that you know how it performs and that you can adjust workflows more easily. The bad thing resides in the fact that it’s not quick to scale and that it can require tricky engineering to integrate all needed DRMs or produce exotic stream format+DRM combinations, depending on which transcoding farm you are using. The SaaS model is interesting because it offloads the engineering charge and the scalability issue on the service provider you choose, but this probably won’t be a dedicated platform with instant scalability, so your contents will be processed alongside or after other clients’ ones, meaning long hours of wait until the packaged contents gets available in the catalogue. Finally, the Cloud model allows to benefit from scalability and recent evolutions towards GPU acceleration of EC2 instances, but few providers are already offering the right combination of speed, security and DRMization flexibility you’ll need. It might be more suited to use the Cloud to offload jobs when on-premises capacity has been reached.
• Assets mutualization : here the game is to minimize the number of assets that you will have to produce in order to deliver to all your target devices. That’s not an easy task for the moment as PC and mobiles/tablets are a bit ahead of connected TVs as regards ABR streaming support and DRM support but situation is evolving quickly and, given the fact that PlayReady for HLS has not been normalized as expected by Microsoft (instead, they did put all their efforts behind DASH support), the HLS+PlayReady combination can just be considered as a quick-win before a general migration to a DASH-based combination in 2013. As of now, the panorama is a bit puzzled, and you have to provide monoblock and fragmented formats with different DRMs in order to support the current devices generations and brands. So the platform you choose must support current chaotic requirements and allow smooth&quick migration to more federating standards like DASH combined with PlayReady or Marlin.
• ABR support : HLS/SMOOTH/HDS/DASH. That’s what you need to support to deliver to current and upcoming devices which support ABR formats, if you don’t develop your own video player stack based on each target platform low-level SDK. You need to support HLS for all iOS and Android devices (>3.1), you need to support Smooth for Xbox and Windows Phone 7 smartphones, you need to support HDS for the widest PC/MAC/LINUX coverage – and you’ll need soon to support DASH in both M2TS and fMP4 profiles to support all these devices plus all connected TVs when everyone will have migrated towards the new format. That’s a heavy duty today, that will be easier tomorrow. Stay cool!
• Content prep chain flexibility : one thing important to take into account is that not all CDNs will support all the flavor combinations of stream formats and types (live/vod) so you must require multi-CDN support from your chosen platform (this can also be useful to handle load-balancing under heavy audiences), as well as multiple workflows support (like usual transcoding/DRMization production on premises, combined with exotic production in SaaS mode). In all cases, you shall require flexibility in ingest and CDN output.
• Multilingual contents and subtitling support : there is a small chance that you will have to deal with English-only content, so be prepared to mux or burn subtitles with your videos, be prepared to mux several audio tracks or produce several separate files per language as not all devices do support non-burnt subtitles and multiple audio tracks. This also impacts the catalogue part of the platform which must be able to deal with several variants of the same program.

Subscribers & devices management

• Subscribers management : it’s not a good idea to manage your subscribers right inside your OTT platform, that’s better to do it in an external Subscriber Management System (SMS), more specialized and powerful, in case you happen to add different services types like IPTV, and for features reasons. If you don’t have time, try to get insurance for a smooth migration afterwards and require minimalistic flexibility to build custom emailings for various customer cases.
• Devices management : your chosen solution shall allow various types of devices use limitations, from global number of devices per account to devices families limitations, devices monthly renewal limits and simultaneous streams limits per account. Here the key is to provide to rights-owners the best level of guarantee that their contents will be consumed accordingly to your hardly-negotiated contracts…
• Payment gateway integration : if you’re not a telco, you’ll need to grab clients/subscribers’ money from their credit card or Paypal account, depending on your various TVOD/SVOD scenarios (the latter implying recurring payments). Your chosen solution must therefore provide the maximum flexibility and the shortest TTM for implementing these scenarios with your local payment gateway service provider. This is a very sensitive part of the project – so don’t underestimate the difficulty of this integration!

Offer/catalog management & promotion

• Commerce engine flexibility : your chosen solution must support various commercial packages/bundles, with all kinds or promotions/discounts means, in order to fulfill the needs for various SVOD/TVOD scenarios. Apart from VOD, your offer may include live and catchup services, so they should also be supported – which means at least ad-insertion capability.
• Personalization capability : even if the catalogue is the same for all your consumers/subscribers, it’s good to include personalized recommendations for each user, in order to maximize video playback rates. Therefore, your solution must provide at least a minimalistic recommendation engine capability, or better – integrate with a specialized market solution for recommendations.

Frontends

• Social media integration : as you will want to maximize your communication impact on social networks, you will at least require Facebook & Twitter “like” comments publishing in the frontend apps, and why not getting the data streams back from the networks inside the frontends. You shall also require users voting feature in order to facilitate fine recommendations issued from social buddies tastes.
• Devices coverage : frontend development from scratch (pure API) is a huge task – so you’d better get a grip on pre-engineered frontend bootstrap apps if you want to lower your TTM. Connected devices generally do not behave like standard browsers and PCs or propose smooth dev tools, so get prepared for some headaches and painful debug experiences… A specific effort on exploring the devices streaming requirements and capabilities is also needed prior to getting deeply in this business.
• Native/HTML5 : it’s up to you to determine whether you want to offer the ultimate native UI experience to users of a given platform – if not you can reuse the same HTML5 frontend with specific animations optimizations for each target platform’s browser engine. This is supposed to provide you a better TTM and an easiest evolution path for your apps.
• Documentation : short TTM for your apps relies on extensive API docs and good bootstrap tutorials/reference apps. Hunt for these elements !

Logging/analytics

• Logging : extensive logging is the basis of a rich analysis capability afterwards, so this must be a hard requirement for your service API.
• Analytics : solution providers offer analytics features ranging from API on which you must develop your custom reporting tool, to OLAP cubes and advanced reporting generators. Second option is obviously better if you don’t need to reintegrate your OTT analytics inside a wider reporting system.

Taking the best DRM path

Things would be too easy if you could cover all of your target devices with a single DRM. Yes, PlayReady expansion outside of the PC world has been remarkable in the CE world but there are still holes in the gruyere, as you can see hereafter…

Listed DRMs have all been endorsed by the DECE consortium as UltraViolet accepted DRMs – but there are other non-UltraViolet DRMs which are accepted by studios like Azuki or Verimatrix ones. However, due to presumed traction effect of UltraViolet and awaited convergence around Common File Format and Common Encryption, the matrix shortlisted only UltraViolet compliant DRMs.

NB : this table can certainly be more accurate. Please contribute with missing informations via comments or contact form.

The first fact to point out is that the PlayReady+Marlin combination covers almost all the scope (apart from Wii for which seems to be supported only by Widevine, which Netflix licensed in 2010). PlayReady is by far the most trending OTT DRM, as : support for DASH will finalize soon, new connected TVs generations generally integrate it and GoogleTV chose it as primary DRM. However, few CE devices are as of now using PlayReady in combination with an ABR streaming method (but rather with monoblock MP4 over progressive-download HTTP). While Xbox 360 and Windows Phone 7 will certainly go on with Smooth Streaming as implemented now, we can expect a quick growing base of DASH+PlayReady support later in 2012 & 2013. Current connected TVs implementations of DASH players do not use PlayReady yet (rather Marlin) or do support AES-encryption only (this is the case for Samsung 2012 models) but there is a also a very high chance that they’ll all support PlayReady before end of Q1 2013.

The worst case as of now is the one of HTML5 on desktops : there is no specified DRM to go alongside H.264 or WebM streams, and the situation will likely be fragmented for a long time, leaving space for “legacy” Silverlight and Flash apps which cover more or less all the platforms in this category. As of now, you can compile special Chromium versions with Marlin support but it’s more a POC than a deployable solution (as you will prefer to benefit from browser natively implemented DRM support or standalone cross-browser DRM component which lacks today).

For your multi-DRM support needs, here is a quick matrix that details the known DRM integration inside DRM umbrella servers. Some of them are PlayReady focused and will probably extend their coverage to Marlin soon (Authentec, Conax). A bit apart in this list with Flash Access support, Irdeto has already claimed support for GoogleTV and Boxee. First my theory was that Irdeto had integrated Widevine DRM in order to support Netflix Wii app version, but Glenn Morten (Widevine CTO) precised  me : “We did not use Cloakware in any of the work we did for Netflix. Arxan is our preferred whitebox crypto and obfuscation provider …but we have used others. We have never used Cloakware”. Sounds pretty clear, but hereby results in a lack of (known) integration of Widevine DRM in the usual DRM umbrellas on the market – we’ll see if this position is sustainable in the long run while UltraViolet sharpens competition between DRMs. I would like to finish with a comment on a precision from Glenn Morten : “Widevine formats are Widevine ABR (aka .wvm), HLS and .mp4. We are watching Dash but remain undecided.” <= Many of us are thinking that the standardization effort around DASH is the key to optimized video production workflows and mature HTML5 streaming, so don’t let this opportunity go !
[NB : since this article was originally written, Widevine has confirmed on its developer mailing-list its DASH orientation, without details about its potential of the proprietary WVM format] 

OK, I guess this closes the (verbose) introduction to the wide topic of Multiscreen OTT services – may it be helpful to jumpstart your project !

Now, if you want to go deeper in technical considerations, you can read the second & final part of this post :
“Choosing your bricks, mortar and trowels…”
where I cover the various technical options (apart from already covered DRM topic)
that you can use to build your service, including DIY options and ready-to-go platform offerings from :
Azuki, Endavo, KIT Digital, SyncTV, The Platform and Tvinci