Upload
jesse-chien-chen-chen
View
875
Download
2
Embed Size (px)
Citation preview
INTRODUCE YAPI.JS
(AN ADAPTIVE STREAMING WEB PLAYER)
PART2
KKBOXVideo陳建辰 jessechen
AGENDA
• yapi.js, video element and MSE
• Introduce yapi sample control panel
• Test cases
HOW TO USE YAPI.JS
var yapi = new yapi.MediaPlayer().initialize();
yapi.attachView(<video></video>);
yapi.load(MANIFEST_URL);
YAPI.JS / VIDEO ELEMENT / MSE
VIDEO ELEMENT
var videoNode = document.createElement(‘video’);
videoNode.src = VIDEO_URL;
With html5 video element, you can play single video source easily
more info here: https://goo.gl/Fyi3Z3
MSE
“ MSE (Media Source Extension) extends HTMLMediaElement to allow JavaScript to generate media streams for playback.
Allowing JavaScript to generate streams facilitates a variety of use cases like adaptive streaming and time shifting live streams. “
var video = document.createElement(‘video’);
video.src = VIDEO_URL
MEDIASOURCE IS A ‘SOURCE’
set ‘src’ attribute of video element to an url pointed to media source
new window.MediaSource();window.URL.createObjectURL(ms);
SOURCE BUFFER
sourceBufferVideo = ms.addSourceBuffer(VIDEO_CODEC); sourceBufferAudio = ms.addSourceBuffer(AUDIO_CODEC);
// get stream buffer via network
sourceBufferVideo.appendBuffer(buffer);
// sourcebuffer provides buffer info after append complete
BUFFER INFO
var buffered = sourceBuffer.buffered; buffered.length; // how many discontinuous buffered time range
buffered.start(0); // first buffer start time
buffered.end(0); // first buffer end time
get buffer information from buffered attribute
ADAPTIVE STREAMING// assume segment of time 0s~5s is needed // yapi would decide which bitrate to download loadSegment(segment_1_240p_url); // segment file doesn’t have meta data, we have to append it first sourceBuffer.appendBuffer(init_240p_buffer); // then loaded segment sourceBuffer.appendBuffer(segment_1_240p_buffer); // if yapi decide to load 480p for 5~10s (adaptive algorithm) // then yapi repeat same process, but with 480p // then playback would be 240p during 0~5s and 480p for 5~10s
GRAPH
yapi append stream buffer to sourcebuffer ‘adaptively’
MediaSource
Media Element media element plays
while available
MSE EXTENDS MEDIA ELEMENT
• MSE focus on providing stream buffer to media element
• playback behavior still hold by media element
MEDIA EVENTS
http://www.w3.org/2010/05/video/mediaevents.html
Dispatched by HTML5 media element, notifying playback process
USEFUL EVENTS FOR YAPI.JS
loadstart: Indicate loading media begins, this fires when setting src attribute
loadedmetadata: while element has basic info of playback, e.g duration
timeupdate: while current time of playback is ticking
seeking/seeked: while conducting seek
ended: playback ends
play/playing: playback resume from other status to playing
yapi
YAPI PROXY EVENTS
Media Elementeventevent with same name
ADDITIONAL EVENTSloadedmanifest: after manifest is loaded/parsed
bitratechanged: when bitrate is changed
enableabr: when adaptive activation changed
buffering: when playback pending
cuechanged: webvtt subtitle cue changed (in and out)
BEHAVIOR WITH MEDIA SOURCE
SEEK BEHAVIOR 1
• mediaElement.currentTime = TIME_SEEK_TOwill conduct seek
• use yapi.seek(TIME_SEEK_TO) instead
SEEK BEHAVIOR 2
• seek is totally done by mediaElement, so yapi.js listen to seeking and seeked event to know seek starts and ends
• seeking event dispatched by yapi would have seekFrom and seekTo as parameters
PENDING DETECTION
• No useful event to indicate pending situation waiting stalled emptied
• pending detection is done on every timeupdate event, yapi.js would check media source buffered length
PLAYBACK END
• mediaElement dispatch ended event natively when playback goes to end
• but not while attaching media source to it
• mediaSource.endOfStream() must be invoked beforehand
yapi
YAPI ENCAPSULATE MEDIA ELEMENT
Media Elementexternal api
SUMMARY
• media source provide stream buffer for mediaElementmediaElement.src = createObjectURL(ms)
• listen to events dispatched by yapi, instead of mediaElement
• use api exposed by yapi (call api of mediaElement not recommended) e.g seek
M3U8 ON SAFARI
• attach m3u8 to mediaElement on safari by simplymediaElement.src = URL_TO_M3U8
INTRODUCE YAPI SAMPLE UI
MediaCtrlPanel… and its children
Volume UI
HOW’D YOU ASSEMBLE?
ProgressBar slider
slider
client
yapi
SLIDER CLASS
• slider constructor takes a node and orientation as mandatory parameters
• it focus on 1. locate pointer with given param (event or percentage)2. return location with given param3. notify listeners events occurred
TAKE A LOOK AT PROGRESS BAR
• played bar
• buffered bar
• seek time indicator
• preview
AND VOLUME UI
• value bar
BEHAVIOR FLOWscenario 1: progress changed by app. e.g yapi.seek(TIME) invoked
MediaCtrlPanel ProgressBar slideryapi
scenario 2: progress changed by user interaction, e.g click on slider
MediaCtrlPanel ProgressBar slideryapi
SCENARIO 1: FROM APP
MediaCtrlPanel.onSeeking(event)yapi.seek(time)
MediaCtrlPanel listens to ‘seeking’ event emitted from yapi, then calculates percentage of current progress
(currentTime / totalDuration)
SCENARIO 1: FROM APP
MediaCtrlPanel. ProgressBar.locatePointer(percentage)yapi.seek(time)
SCENARIO 1: FROM APP
MediaCtrlPanel. ProgressBar.locatePointer(percentage)
slider.locatePointer(percentage)
yapi.seek(time)
progressBar update played bar
SCENARIO 1: FROM APP
MediaCtrlPanel. ProgressBar.locatePointer(percentage)
slider.locatePointer(percentage)
yapi.seek(time)
slider.locatePointer would invoke method locatePointerInternal,with1. it’s argument 2. notToDispatchEvent as true (I know it’s tricky)
SCENARIO 2: FROM SLIDER
assume user conduct seek by click on slider, b/c slider.addEventListener(‘click', locatePointerInternal);
so it will dispatch POINTER_LOCATED event
slider
(Remember notToDispatchEvent param?)
SCENARIO 2: FROM SLIDER
progressBar listens to ‘POINTER_LOCATED’ event andupdate played bar
ProgressBar slider
SCENARIO 2: FROM SLIDER
MediaCtrlPanel also listens to ‘POINTER_LOCATED’ eventand it will invoke yapi.seek(calculatedTime)
(percentage of time is within POINTER_LOCATED event obj)
MediaCtrlPanel ProgressBarslideryapi
A LESSON
“ take care of flow direction, or it might be an infinite loop “
MediaCtrlPanel ProgressBar slideryapi
yapi MediaCtrlPanel ProgressBar slider
LET’S TALK ABOUT EVENT
LISTEN TO EVENT
consider on these 2 phrases
“ progressBar listens to ‘POINTER_LOCATED’ event “
“ MediaCtrlPanel also listens to ‘POINTER_LOCATED’ event “
BAD IMPLEMENTATIONbut it works!
progressBar.addEventListener();mediaCtrlPanel.addEventListener();
slider.addEventListener(POINTER_LOCATED, callback);
ADD LISTENERit would call method with same name of eventBus
slider eventBus.addEventListener(POINTER_LOCATED, callback)
while invoking this, eventBus will manipulate an object holding map of event name and callback.
in the case, POINTER_LOCATED would be key, and an array as value with callback pushed into it.
DISPATCH EVENTPOINTER_LOCATED is an event from slider
it will find array of key: POINTER_LOCATED, and invoke every element (callback) of it.
slider eventBus.dispatchEvent(POINTER_LOCATED)
WHAT’S THE BAD PART?• mediaCtrlPanl needs a reference of slider
MediaCtrlPanel ProgressBar sliderconflicts with
mediaCtrlPanel.addEventListener();
slider.addEventListener();
• for every dispatcher, an eventBus instance is neededslider eventBus.addEventListener() eventBus.dispatchEvent()
assume progressBar would dispatch a ‘PROGRESS_BAR_HIDE’ event, then every module which listens this event needs progressBar reference,
and progressBar needs a private eventBus for mapping
progressBar? another eventBus
TAKE A LOOK AT DOM EVENT
while click on <td> <table> would also get it
“by default”
http://www.w3.org/TR/DOM-Level-3-Events/
HOW COME?
<td>.addEventListener(‘click’, callback)<table>.addEventListener(‘click’, callback)
TAKE A LOOK AT DOM EVENT
http://www.w3.org/TR/DOM-Level-3-Events/
a singleton eventBus holds by every node
while add listener
it knows which node listens
while dispatchingit invoke callback with node as context
THEREFORE
MediaCtrlPanel… and its children
Volume UI
ProgressBar slider
slider
eventBus
that’s why DI needed
SUMMARY
• find common part and reuse it
• remember: behavior has direction
• event indicates states of application, handle it well
client
INITIALIZE CTRL PANEL
MediaCtrlPanelProgressBar
Volume UI
• in client app, invoke new MediaCtrlPanel()
• call setup method, it loads template html
• then apply component functions and initialize progress-bar and volume-ui
INITIALIZATION DETAIL• in client app, invoke new MediaCtrlPanel()
• in panel.setup(), loads template html
• after html loaded, apply component functions
also initialize progress-bar and volume-ui
IF A MODULE HANDLING THIS• that module called ‘yapiDOM’ with a method ‘render’
• takes 2 parameters:1. constructor of a component, 2. selected node this component would mount on
• while `render()` invoked, yapiDOM will instantiate given constructor and run setup method of component(load template and apply component functions )
IT WOULD BE LIKE…
very similar to React !
ATTACH VARIABLE
• attach yapi player to media ctrl panel, and listens to player event
IF YAPIDOM ALSO HANDLES THIS
• yapiDOM has another method ‘createElement’
• takes 2 parameters:1. constructor of a component, 2. object with map of key/variable
• while `createElement()` invoked, yapiDOM will instantiate given constructor and run setup method of component
SETUP COMPONENT
1. load html (js sync / html async)
2. apply component functions
3. hold reference of given map object
4. return created component node
IT WOULD BE LIKE…
just like React !
SUB COMPONENT
• create target component inside setup method of media ctrl panel
SUMMARYComponent
1. have a conventional ‘setup’ method with yapiDOM
2. setting these while setup:• template• behavior• reference
yapiDOM
1. createElement method:run instance method ‘setup’ of given component class
2. render method:append node to target node
attach singleton eventBus here
UNIT TEST JASMINE / KARMA
JASMINE =MOCHA +
CHAI + SINON
framework
assertion
spy
COMPARISONsetup of running on browser
jasmine
mocha
COMPARISON
https://github.com/ddhp/test-framework-comparison
RUN ON COMMAND LINE
• can be run on phantomJS, a headless browser
• easier integrate with CI
TASK RUNNER
• grunt-mocha takes html file as source
• for grunt-jasmine, set up source, vendor, spec javascript files in task config
WHICH I PREFER
• flexibility comes with complexity
• mocha can do everything, so it’s important that you make choice and being consistent
• jasmine is capable enough for client-side unit test
TDD / BDD
• mocha.setup() can decide which strategy to use (link)
• it’s just the difference between syntax
TIPS OF WRITING TEST• it doesn’t matter which strategy to adopt, just get your hands
dirty
• get code coverage 100%
• add relevant test case while adding feature (b/c there must be a reason you do this)
• also while fixing bug (prevent from occurs again)
INTRODUCE KARMA
CONSIDER THESE FEATURES
• watch on modification
• preprocess
• run on different browsers simultaneously
• code coverage report
KARMA IS…
• a test runner
• would spawn a server by default on port 9876 which serves test cases
CONFIG KARMA• list needed plugins
• decide which framework and browsers to run on
• setup preprocessors
• list reporters
• list required helpers, vendors, sources and specs in files
TIPS
• setup different config for developing and CIdeveloping: runs on different browsers CI: runs on phantomjs with coverage report
• if grunt-watch is exploited, it’s better to disable watch feature of karma i.e each karma run is a single run, and re-run it when file changed
THANK YOU
w3 mse spec, media event sampleMDN media element doc, HTMLMediaElement docdash.jsAngular.js Directive docReact.js Getting Start tutorial
ref: