COM 597Streaming Media

Lecture 3

July 5, 2006

Codecs

• Which is best? There really is no one-stop solution.

• All of the major standards do a very good job of compressing most media

• The differences are subtle

There are real advantages to choosing one standard as your default distribution method

• Users will only need to rely on one playback solution

• Hardware will be optimized to that solution (servers and production tools)

• Licensing of technology is easier• Staff is trained with emphasis on only one

platform• Support is easier

There are real disadvantages to choosing one standard as your default distribution method

• Not a large user base for the technology• Format of files is not competitive with

changes in the marketplace• High licensing fees• Flaky software• Challenges with using the files on all types of

hardware (WM on a Mac)• You are tied to the fortunes of the company

that created your codec

Format compatibilities and incompatibilities

• QuickTime Mac and PC are OK, no Linux support

• Windows Media PC is great, Mac support is limited to older versions, no Linux

• Real Networks Supports most every operating system

• Flash Supports most every operating system because it runs in a browser

• MPEG will play in many of the proprietary systems H.264 shows promise as a common standard.

How might you examine a codec to see if it will work for your needs?

You might choose one compression solution for your production and another for delivery to your users.

Property Production Codec Delivery Codec

Compression Ratio Low usually between 1:1 and 4:1

High typically 50:1 or more

Quality Lossless (perfect) or close Good to acceptable, will denigrate to sub-VHS quality

Lossy Zero for some codecs, for others very minimal

Very lossy

Bit rate 25-50 and up to 150 Mbps As low as 300 Kbps for video and 24Kbps for audio

File Size :20 seconds per GB (180 GB per hour)

As low as .4 GB per hour

Editable Yes Not really, although many editors will accept a compressed file as a source. The quality will be dubious at best

Compositable Yes Only if file has alpha channel

Conversion to other formats Easy Additional degradation of quality is likely

General rule is that as compression schemas improve the files get smaller and the images either improve or stay the same.

Let’s look at how we compress a file to get an understanding of all the variables we can manage

Prior to compression you need to get your media in the best shape possible. Often it is easier in production to prep the media than it is to heal bad video during compression.

Video Preparation

• Deinterlacing the footage

• Frame rate agreement (2-3 cadence, 15fps, PAL v. NTSC, HD)

• Noise filters (dubious benefits at best)

• Grain removal

Video Preparation

• Color correction (gamma adjustments)

• Scaling (size your canvas can speed up compression significantly)

• Aspect ratio adjustment including anamorphic adjustments

Compression Example

• What the heck do those letters mean? And why do I care?

• I-frame Intra frame or key frame• P-frame predicted difference frame• B-frame bi-directional difference frame

• Compression Demo

Streaming v. Streaming

• As we have discussed, the term “Streaming Media” has become somewhat of a catch-all phrase for digital media distributed over an IP network

• This is technically inaccurate

Streaming v. Streaming• IBM’s definition to delineate the different types of digital

media works well• Digital media is unstructured content – audio, video, and

images – that cannot be stored in a traditional database.

The two major segments of digital media are:• Media that has “intrinsic value” or in other words value that is

inherent to the media itself (e.g. movies, documentaries, and music videos)

• Media that has “business process value” where the media becomes valuable due to the context in which it is used (e.g. training and corporate communications.)

• So for the purpose of this class we will be exploring “Digital Media” over the course of the term.

• Tonight we will be studying… Streaming Media.

What is a webcast?

• Webcasts can include audio, video and other data like PowerPoint, animation, and even whiteboard applications

• Many webcasts also include some form of communication between the audience and the talent or host.

Some advantages of webcasting over traditional broadcasting

• Lower cost of entry• Unlimited spectrum – don’t need a license

from the FCC• Not limited by geography• Can be targeted to a specific audience

Some of the disadvantages of webcasting

Smaller audience

Incremental cost per viewer

Quality

• Many business and education enterprises are deploying webcasting as a cost-cutting enterprise for both internal and external communication

What is streaming media and how does it work

• Webcasting is a specialized application of streaming media technology

• Streaming enables real-time (or close to it) delivery of media across a network

• This technology is similar to how the web delivers web pages, with one big difference:

1. Web servers download files, streaming media servers stream files.2. Downloaded files are displayed after the entire file has been

delivered

Streaming files play as they are delivered (this is not progressive download)

For large files the advantage of streaming are:

• Real time• Control/interactivity (real-time presentations)• Security (not stored on the hard drive)• Live broadcasting

Bandwidth Capacity

Connection Type Bandwidth / Bitrate

Dial-up (56K modem) 34-37 Kbps

Broadband (Cable / DSL) 200-1000 Kbps

Frame Relay, Fractional T1 128Kbps – 1.5 Mbps

T1 1.5 Mbps

Ethernet 10 Mbps

T3 45 Mbps

YMMV

• Bandwidth of individual audience members determines how the webcast should be encoded

• The bandwidth at the site determines how much data can be sent to the distribution servers

• The aggregated bandwidth used determines the bandwidth charge

Load balancing

• Streaming media servers receive the incoming stream from an encoder (or encoders) and then may distribute the media to the audience.

• A server may also send the stream to other servers for load balancing and redundancy

• Load balancing is where a number of servers are used to distribute a webcast to improve performance and reliability

Protocols• To facilitate communication there are standardized

protocols used so the servers, players and encoders work properly

• For example “http” stands for hypertext transfer

protocol. • “ftp” stands for file transfer protocol• “rtsp” stands for real time streaming protocol This

is the most common protocol used, but not the only one.

Expected streaming media quality at different bit rates

Bit rate Audio Quality Video Quality

32 Kbps (56K modem) FM Radio Quality Very small screen (160x120) Acceptable for talking heads

64 Kbps (ISDN) Very good quality. CD Quality at approximately 96 Kbps, depending on codec used

Small screen (196x144, 248x180) Acceptable for talking head content, dreadful for high-motion contnent

256 Kbps (typical low end broadband sustainable speed)

Excellent Quarter Screen (320x240) Good quality for talking heads, acceptable quality for most content except very high motion content like sporting events and music videos

700 Kbps Excellent Excellent quality for all content at quarter screen size (320x240); good quality for full screen size (640x480)

1.5 Mbps (Ethernet and High Speed Internet)

Excellent Excellent full screen quality for almost all content

Business Considerations

• Who is the intended audience for this webcast?• What is the purpose of the webcast? What is the

need? Is a webcast the most efficient way to distribute the message?

• Where is the webcast taking place?• When is the event?• Why? Is there really a need to do this or is it just

“cool”? Will the webcast pay for itself with new revenue or through cost savings?

If you have good answers to the previous questions then ask these:

• Does it have to be live?• What are the cost considerations?• What is the ROI?• Are there legal considerations?

Live is always more expensive than recording and archiving it for later use.

What are possible events with time sensitive content?

• Breaking News• Distance Learning• Stock Market activities• Music Events• Sporting Events• Community Event

Where are you going to spend your money?

• Budget will determine both the quality and reliability of a webcast

• Single or three camera shoot? – Your most exorbitant line item will be your production costs.

• Bandwidth costs

Where are you going to spend your money?

Unlike radio or broadcast television, you must pay for the additional bandwidth required each time someone tunes into your webcast. The prices are astoundingly random based upon what carrier you use. Figure somewhere between $75 and $35 per megabit from network carriers. Shop around before you buy.

How do you know your bandwidth needs?

• Measured in bits per second (bps) of the information sent concurrently

• Because these numbers get big fast we use Kbps and Mbps

• A kilobit is 1024 bits • Megabit is 1024 Kilobits (1024x1024) or

1,048,576 bits

How do you know your bandwidth needs?

• Here is the rub…

• Data is measured in bytes, not bits.• There are 8 bits to a byte • A gigabit is 1024 Megabits

So…

Let’s say you have a show

• There are 100 viewers for your webcast• They will all watch a 300Kbps stream • 300Kbps x 100 = 30000Kbps This number is too unwieldy, so let’s figure out

how many Megabits per second

30000 / 1024 = 29.3 Mbps • Add about 20% for overhead and you land at:• 35Mbps sustained bandwidth

But how long is your show?

Let’s say it is 1 hour • 35Mbps x 60 seconds per minute x 60 minutes

per hour 35x60x60 = 126000 Megabits

But wait, we need bytes, not bits.

126000 / 8 = 15750 Megabytes • The number is still too unwieldy so divide by

1024 to get Gigabytes 15750 / 1024 = 15.38 gigabytes

How do you know your bandwidth needs?

So when you talk with a content delivery network about prices you can confidently state that you need approximately 35 megabits of concurrent capacity, and you expect to transfer about 16 gigabytes of data.

Usage Logs and Metrics

• They will vary based on the CDN you chose• How many watched• How long they watched• Where they were from

Rights

• Do you have rights to the images and music used in the webcast?

• Do you have rights to the graphics and video used?• How likely are you going to be sued?• Do you have releases from all people and locations

used in the webcast?• Do you have a labor union to deal with either on

site or licensed material?

So how are webcasts produced?

• The process is essentially the same as producing a program for broadcast, except the signal is sent to a streaming encoding solution instead of a broadcast tower or cable system.

The same approach of good audio and video

engineering applies to both. • Crap in equals crap out.

The difference between an on demand streaming file and a webcast is there is no room for error. It is live so you need to have it tested and working prior to the event. Because it is a real-time event, it will affect every phase of production.

This includes:• Planning – justifying costs, location, tools, crew• Production• Encoding• Authoring – connecting the audience to the

webcast via a link on a web page• Distribution – securing an infrastructure that is

robust for your needs Remember: If you have not planned it right your webcast

can grind to a halt. This can be a career-ending move. Be certain you have not only created a plan, but you have redundancy in place in case of failure

Where do we start?

• Step one: Plan – 95% of the work happens before the event

• Step two: Plan some more• Step three: Check out your location – do you

have power? How do you connect to the web? What are the acoustics like? Who are your points of contact?

• Step four: Change your plans

Where do we start?• Spend a significant amount of time planning• Do a site check• Acquire and test all the equipment necessary for the webcast• Acquire and test all the encoding hardware necessary, bring

extra• Examine and test your streaming server architecture – any

license issues with the number of streams you expect to distribute simultaneously?

• Do a body count – do you have enough of the right people in the right jobs?

• Test connectivity on site BEFORE the webcast This includes testing the broadcast equipment, load-testing the server, test the links on the website

Where do we start?

Bottom line:• Take the time to plan• Keep it simple• Bring two of everything• Be kind• Start early

Business considerations

As discussed earlier:

• Who is your audience?• Do you really NEED to do a webcast?

Is there a better solution?• How much might it cost?• What is the return?

Production Considerations

Do it yourself or outsourcing

• Which parts?• How do you find a partner?

– Production Partner– Distribution partner– Marketing partner

• Ask to see examples of their work

Some nasty questions to ask a potential encoding partner

• Do they know how to set up redundant encoders?• Have they had experience bonding ISDN lines and

knowing how to force them to dial long distance?• Do they know how to remix the audio and switch

between multiple video inputs?• Do they know how to work with the camera

technicians to have the video shot in a way that works best for a webcast?

• Will they advise you on what backgrounds and colors work best for streaming media?

From “Hands-On Guide to Webcasting”

Pricing and Providers

Entering into an agreement with a CDN (Content Deleivery Network) is not unlike buying a car

There is lots of smoke and mirrors and no one

can agree on what to call the same service from provider to provider

Two primary charge models for webcasting distribution:

• RSVP Model – charged based on the number of simultaneous streams used at any one given time. Usually a reserved number of streams

• Throughput model – charged based on the total amount of data delivered over the

network during the course of the event.

The factors common to any charging model would be:

o Length of the evento Time of the evento Number of formatso Number and size of bit rateso Geographic distribution

Additional fees may include:

• Some sort of registration to authenticate a user

• Pay Per View• Late notice or last minute event

Who to talk to about a CDN Solution

• Limelight Networks (www.limelightnetworks.com)• Akamai (www.akamai.com)• Mirror Image (www.mirror-image.com)• SAVVIS (www.savvis.com)• VitalStream (www.vitalstream.com)

For Small Business• PlayStream (www.playstream.com)

For Financial Business• Thompson (www.thompson.com)• ON24 (www.on24.com)

OK, you are on your own

Equipment and crew

• What is your video source (type and fps of video feed)?

• What is your audio source?• Are you renting the gear? Purchasing it?• Are you hiring a 3rd party company to

produce the video feed?

Some of the gear you will need:• Microphones (Lav, handheld, shotgun, spares)• Cables of many types• Camera• Audio mixer• Video switcher if multi-camera• Distribution Amps (so you can split your feeds)• Isolating transformers and humbuckers• Batteries• Encoding computers• Monitors• Communication system (for crew and back to

broadcast operations center)

What Crew might you need?

• Camera Operators• Audio Engineers• Video Engineer / Technical Director• Encoding Technician• Production Assistants• Director• Producer

Computer hardware• How much horsepower will you need?• Do you need a capture card?• Do you need an external capture unit?

What software do you need?• If you are encoding via software do you have

the proper encoder?• Will you be generating multiple platforms?

(Real & Windows Media for example)

What do you need to deliver?• What is your desired bit rate?• What is your desired canvas size?• Do you need to deliver multiple bit rates?

When you stream you also archive

• Oddly, most viewers will see the webcast as archived media

• You will need to archive the webcast somewhere• If not on tape or from the distribution point then

you will archive locally

Make sure you have room on your hard drive

Bit Rate One Hour File Hours of storage per Gigabyte

37 kbps 16 megabytes 64

128 kbps 56 Megabytes 18

300 kbps 132 Megabytes 7 ¾

700 kbps 308 Megabytes 3 ¾

These files are not that huge so space should not be that much of a problem unless you are archiving uncompressed media

Suggested minimum Audio bitrates for streaming

Bit Rate Minimum Audio Bit Rate Quality

Dial-up (34-37 kbps) 8 kbps AM Radio

Low Broadband (64-128 kbps) 16 kbps Nearly FM Radio

Standard Broadband (200-300 kbps) 32 kbps FM Radio

Suggested Screen resolutions

Bit Rate Low Motion Programming High Motion Programming

Dial-up (34-37 kbps) 176x132 160x120

Low Broadband (64-128 kbps) 240x180 192x144

Standard Broadband (200-300 kbps) 320x240 240x180

For the most part, these sizes are glorified postage stamps

Some Hardware solutions for streaming media

Communitek (www.communitekvideo.com)Digital Rapids (www.digital-rapids.com)Sunnyside Software (www.sunnysidesoft.com)Viewcast (www.viewcast.com)Winnov (www.xstreamengine.com)

Distribution Planning

Audience size will directly impact the amount of distribution you require

Small may only need a single local server, large may require geographically distributed server farms. Discuss your needs with your network administrator

How will you author the web site?

• Will the stream be incorporated into a dynamic site?

• Have you already built and tested the shell?

• Will this be a pop-up player or embedded into a web page.

Authoring considerations:

• How is the website going to be featured on the website?

• Link on the homepage?• Will that link go to an intermediate page or directly

to the stream?• Will a user have to fill out forms or enter

passwords?• Will the media be embedded, use a pop-up

window or a stand-alone application?

Authoring considerations:

• Always put a link to your webcast on your home page. There is no reason someone should have to search for the feed.

• A complex home page combined with a streaming

media event can overload your server and bring the whole thing down. Think about perhaps simplifying your home page on the day of a big event

Emedded or stand-alone• No clear answer.• Some embedded players are seen as a security risk

by the browser or operating system on a user’s computer. While it only requires enabling something as simple as a java script to run, some users may be scared off by the warning.

• On the other hand, a player like Flash is installed on over 98% of the web browsers world-wide.

• Consider offering the option of a link to an embedded player and one to a stand-alone player

Linking to a stand-alone player• You can’t link directly to the streaming media. • Browsers use HTML and streams use RTSP• Browsers have no clue what that data means • You create a metafile that the browser links to• These are small text files that are placed on web

servers• They use special file extensions such as .ram

and .asx that are uniquely associated with media players. Because it is text, the browsers understand it

Windows Media Metafile example• Windows Media uses an XML-based format

(you can recognize it by the angled brackets)• Similar to HTML <asx version=”3.0”>

<entry><ref href=”rtsp://drewke.wmserver.com/streamingclass_live” />

</entry> </asx> • Windows Media uses three different file extensions for streaming

media, all ending in x– .wax for Windows Media Audio– .wvx for Windows Media Video– .avx for generic Windows Media

Real Media Metafile example• Real Media metafiles are plain text and take either .ram

or .rpm file extension– .ram is used when you want the standalone RealPlayer to play the

stream– .rpm is used when you are authoring an embedded player

• A Real metafile would contain:

rtsp://drweke.realserver.com/streamingclass_live.rm

Type this into a text editor, save it with the proper file extension and link to it on your page

QuickTime Metafile example

• A QuickTIme metafile uses the .qtl (QuickTime Link) file extension and is also XML based

<?xml version 1.0”><?quicktime type+”application/x-quicktime-media-link”?><Embed src=”rtsp://drewke.qtserver.com/streamingclass_live.mov” />

An example of embedding a Quicktime player into a web page using ActiveX

• IE uses ActiveX Control to embed functionality into web pages

• ActiveX controls are recognizable by the use of the <object> tag

<object classid=clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B”

codebase=http://wwwapple.com/qtactivex/qtplugin.cabwidth=”240” height=”180”><param name=”src” value=”rtsp://drewke.seanet.com/streamingclass_live.mov”>

</object>

The problem with this solution is it is unique for Iinternet Explorer

You would have to sniff out the user's browser and write the code so it will work across multiple browsers with unique entries for each.