EOD Part 2 outlines how much bandwidth and storage are required to support all of U.S. video consumption. This is an expanded version of material I presented during a panel discussion on Internet video, March 18 2009 [1].
Table 1 shows how much delivery bandwidth is needed to support total U.S. video viewing.
Total bandwidth is calculated by multiplying the total number of viewers, 285M [4] times 70% peak simultaneous viewers times the bandwidth of the encoded video being viewed.
Three encoding rates are presented assuming an advanced video codec such as H.264: 1.2 Mbs for a standard definition video which is comparable in quality to broadcast television, 2.5 Mbs for an extended definition video which is comparable to DVD quality, 4.0 Mbs for high definition video quality. [A lower bandwidth encoding for mobile devices is intentionally not included at this time. This will be discussed at a later time.]
Table 1 — U.S. Video Bandwidth
| Total U.S. | 1 of 25 Regions | ||||
| Tbs | 10G servers | Tbs | 10G servers | ||
| 1.2 Mbs | SDTV | 240 | 23,966 | 10 | 959 |
| 2.5 Mbs | EDTV | 499 | 49,930 | 20 | 1,997 |
| 4.0 Mbs | HDTV | 799 | 79,888 | 32 | 3,196 |
The total U.S. delivery bandwidth needed to allow 70% of the viewing population to watch a single dedicated video stream is ~500Tbs, assuming that viewers are watching a mix of SD/ED/HDTV. This also assumes that all content, even live events, are delivered as a dedicated on-demand video stream. Higher simultaneous usage, multiple streams per person, and a shift toward higher bandwidth videos will increase the total needed bandwidth.
[Note, Table 1's bandwidth values assume optimal bandwidth provisioning by a single provider or broker. In practice, significantly more bandwidth will actually need to be provisioned.]
Assuming that a single server can deliver video at an aggregate of 10Gbs, ~50K servers would be needed to support 500Tbs for the U.S. 50K servers is not an outlandish number. A single large modern datacenter can house 50K servers. However, it would make more sense to distribute servers geographically to lower network transport costs and to isolate potential local system outages. A typical large metropolitan area would require ~2K servers. [Note that currently deployed servers tend to perform at more like 1Gbs. However with current technology, it is eminently feasible for a single server to deliver 10Gbs of video.]
The average household has 2.5 viewers (= 285M viewers / 114 households). Thus, the average household would need at least 6.25Mbs of broadband access bandwidth for video. This also is not an outlandish number. Interestingly, a modern cable plant already could provide ~7-10Mbs of dedicated bandwidth per household if the cable spectrum were used for dedicated service instead of mostly being used for broadcast service. Of course, to dedicate bandwidth in this manner would require a major change and upgrade to the cable headend infrastructure.
Table 2 shows how much storage would be needed to support the U.S. video market.
Total storage is calculated by multiplying the 2.5Mbs average video bandwidth times the nominal video duration times the approximate number of titles.
To figure out how much content is really available, here are a few significant reference points.
The total number of commercially available DVDs is ~160K [7]. The historical trend is that the number of DVDs increases by ~10K titles per year. This addition represents both newly produced content for the year and previously produced content from library archives that has become cost effective to produce on DVD.
The equivalent of an additional ~650K DVDs worth [8] of professionally produced content exists in various media libraries, but the titles have insufficient market appeal to justify the expense of a full production DVD release. The lower cost of provisioning online delivery compared to DVD production can justify making these titles available.
The upper bound of available content is all of user generated content (UGC) made by individuals for use by friends and relatives and onlookers. Much of this content is only of value to a small group of friends and relatives, but some of it does become viral and interesting to a broader niche audience or occasionally a mass audience. In 2003, ~100M mini-DV video camcorder tapes were produced worldwide [9]. The majority of these tapes are used for home videos, and this serves as a proxy for UGC. Today, of course, most on-line accessible UGC is generated using webcams and flash- or disk-based camcorders, which, without the use physical recording tapes, complicates any direct measure for total UGC. We can assume that the amount of UGC is at least what it was in 2003.
[Note, Table 2 represents the storage requirements for a single title that is encoded at a single data rate. In practice, a single title will be encoded at multiple data rates, and a single title may be stored multiple times in slightly different forms. Both of these effects multiply the storage requirements beyond those listed here. Further note, the table represents the storage requirements for a single instance of the data. In practice, the data will be replicated to multiple instances across datacenters and within datacenters. These multiple instances further multiply the storage requirements.]
Table 2 — How Much Video Storage?
| Mbs | hours | titles | storage | racks | |
| DVD | 2.5 | 2 | 160 K | 360 TB | 2 |
| DVD per year | 2.5 | 2 | 10 K | 23 TB | 0.1 |
| DVD potential | 2.5 | 2 | 650 K | 1.5 PB | 7 |
| UGC per year | 1.2 | 1 | 100 M | 54 PB | 270 |
To hold all existing DVD quality content, that exists or could exist on DVD, requires ~2 PB of storage. Today, ~200TB could be packed into a single modern rack cabinet. Thus, 2PB could be housed in ~10 racks, which could be considered trivial for a modern datacenter. By late 2009 or early 2010, a single rack should be able to house 400-500TB, and using very dense disk packaging, a single rack might hold 1-2PB.
The storage required to house the whole of UGC is more challenging but still feasible for a modern datacenter. However, UGC that is typically housed at existing video and social networking sites is a very small fraction of all UGC that exists. Detailed statistics are not available publically, however, informal reports generally ascribe only a few petabytes to major websites. [10]
In summary, by most measures, wide scale use of video requires TBs of access bandwidth and multiple PB of storage.
[The back of the envelop analysis present here is an update of related analysis performed many years ago. [5] [6]]
References
[1] The State of Video on the Internet, Boston, March 18 2009.
http://www.vilnashul.org/index.php/events/event/the-state-of-video-on-the-internet
http://herot.typepad.com/cherot/2009/03/video-in-transition.html
http://www.newsoftwarepathways.com/blog/?p=116
[2] Comscore, U.S. Online Video Viewing Surges 13 Percent in Record-Setting December, February 4 2009.
http://www.comscore.com/Press_Events/Press_Releases/2009/2/US_Online_Video_Viewing_Sets_Record
[3] Nielsen, Primetime Broadcast Ratings, March 10 2009
http://blog.nielsen.com/nielsenwire/media_entertainment/primetime-broadcast-ratings-march-10-2009/
[4] Nielsen, A2/M2 Three Screen Report 4th Quarter 2008, February 2009.
http://blog.nielsen.com/nielsenwire/wp-content/uploads/2009/02/3_screens_4q08_final.pdf
[5] Gerovac, Branko J. Implications of interactive long-format video on the Internet. Multimedia Systems and Applications II. Proceedings of SPIE, Volume: 3845, 20-22 September 1999 Boston, MA. ISBN: 9780819434388. DOI: 10.1117/12.371201.
[6] Gerovac, Branko J.. Local Access to the Information Infrastructure, Tutorial, Telecommunications Policy Round Table, September 1995.
[7] BIG DVD List — Michael’s Movie Mayhem, http://dvdlist.kazart.com/.
[8] Amazon, IMDB Statistics, http://www.imdb.com/database_statistics.
[9] Peter Lyman, Hal R. Varian, et al, (UCB). How Much Information 2003?
http://www2.sims.berkeley.edu/research/projects/how-much-info-2003/index.htm
http://hmi.ucsd.edu/howmuchinfo.php
[10] Various mentions of storage at UGC video and social networking web sites.
http://www.symfony-project.org/blog/2009/02/18/dailymotion-powered-by-symfony