Upload
ivan-gracia
View
82
Download
8
Embed Size (px)
Citation preview
Creating rich WebRTC Applications with Kurento
IETF 96 Hackathon17 July 2016, Berlin
Real-Time media Communications
Capture Encode Cipher Transport Transport Decipher Decode RenderNetwork
The media plane
The signaling planeI wan to call you, do you accept?
Yes, do it in this way
WebRTC to the rescue.
Before WebRTCFirst wave of
WebRTC technologies
Begin End• APIs• Standards• FOSS
Development experience
when working with real-time media
Common WebRTC application (p2p communications)
WebRTC video stream
4
WebRTC infrastructures
Peer-to-Peer WebRTC Application (no media infrastructure)WebRTC video stream
WebRTC Application based on media infrastructuremedia infrastructure
5
WWW VS (Web)RTC apps
Control
Multimedia Application logic
(developers’ code)
Multimedia CapabilitiesRecording
TranscodingRoutingMixing
AnalyzingEtc.
MediaTraffic
RTC Media APIs
Multimedia Clients
Events
Application Signaling
Control
WWW Application logic
(developers’ code)
Database CapabilitiesData storageData recoveryData querying
Data processingEtc.
DD.BB. API
WWW Clients
Events
Application Signaling
6
7
Control
Multimedia Application logic
(developers’ code)
Multimedia CapabilitiesRecording
TranscodingRoutingMixing
AnalyzingEtc.
RTC Media APIs
Events
8
Kurento: a WebRTC infrastructure and its APIs
Control
Multimedia Application logic
(developers’ code)
RecordingTranscoding
RoutingMixing
AnalyzingAdapting
MediaTraffic
RTC Media APIs
Multimedia Clients
Events
Application Signaling
Computervision
Augmentedreality
BlendingEtc.
Multimedia Capabilities
Kurento Media Server(KMS)
Kurento Client APIKurento Room APIKurento Tree API
9
Cooking Kurento
9
10
The Kurento FOSS Community
Community support WebRTC worldwide reference
International awards More than 300 companies
Very active mailing list
First result in Google
11
The Kurento FOSS project− http://www.kurento.org
• Main web site
− https://www.twitter.com/kurentoms • Main social channel
− https://groups.google.com/forum/#!forum/kurento• Main mailing list
− https://github.com/kurento • Main repository
− https://www.youtube.com/channel/UCFtGhWYqahVlzMgGNtEmKug• Kurento Youtube channel
12
Developing with KMS:The Kurento Client API
Media Element• Provides a specific media
functionality
› Send/receive media
› Process media
› Transform media• Exchange media through› Sources
› Sinks
Media pipeline• Chain of media elements
implementing the desired media logic.
• The Media API provides the capability of creating media pipelines by joining media elements of the toolbox
Media Element
Sin
k
SR
C
13
A modular API− Modular in the sense of modularity*
• Isolationo Internal states of a module don’t affect the rest
• Abstraciono Internal states are hidden behind an interface
• Composabilityo Interface must enable module recombination and assembly
• Reusabilityo If a module has it, you can use it
• Extensibilityo If a module does not have it, you can add it.
* Baldwin, Carliss Young, and Kim B. Clark. Design rules: The power of modularity. Vol. 1. MIT press, 2000
14
The paradigm of modularity
15
The Kurento Client API in one word
sourceElement.connect(sinkElement)
connect
16
Kurento hello world
17
//MediaPipeline is a holder of media elementsMediaPipeline pipeline = kurento.createMediaPipeline();
//Create your media elementsWebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build();
//Connect your media elementswebRtcEndpoint.connect(webRtcEndpoint);
Control
Multimedia Application logic
(developers’ code)
RecordingTranscoding
RoutingMixing
AnalyzingAdapting
Kurento Media APIs
Multimedia Clients
Events
Application Signaling
Computervision
Augmentedreality
BlendingEtc.
Multimedia Capabilities
//Where to render the local and remote streamsvar options = {
localVideo : videoInput, remoteVideo : videoOutput, onicecandidate : onIceCandidate }
//Start media capture and communicationswebRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(
options, function(error) { if (error) return console.error(error); webRtcPeer.generateOffer(onOffer); });
Application Server Code (Java)
Client Server Code
Application source for Spring (Java EE)Application source for Node.js (JavaScript)
18
Recording
Sin
kS
RC
Sin
k
//MediaPipeline is a holder of media elementsMediaPipeline pipeline = kurento.createMediaPipeline();
//Create your media elementsWebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); RecorderEndpoint recorderEndpoint = new RecorderEndpoint.Builder(pipeline).build(“file:///myfolder/myfile.mp4”);
//Connect your media elementswebRtcEndpoint.connect(webRtcEndpoint);webRtcEndpoint.connect(recorderEndpoint);
Application source for Spring (Java EE)Application source for Node.js (JavaScript)
19
Playing
Sin
kS
RC SR
C
//MediaPipeline is a holder of media elementsMediaPipeline pipeline = kurento.createMediaPipeline();
//Create your media elementsWebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); PlayerEndpoint playerEndpoint = new PlayerEndpoint.Buildr(pipeline).build(“file:///myfolder/myfile.mp4”); //RSTP and HTTP URIs also supported
//Connect your media elementsplayerEndpoint.connect(webRtcEndpoint);
Application source for Spring (Java EE)Application source for Node.js (JavaScript)
Interoperating
20
Media Pipeline
WebRTC audiostreaming
Sin
kS
RC
SR
CS
ink
Media PipelineS
ink
SR
C
SR
CS
ink
WebRTC videostreaming
RTP audiostreaming
RTP videostreaming
Transparent transcoding!!
21
Some Media processing modules− Augmented reality
• MarkerArModule• MarkerlessArModule
− Computer vision• FaceDetector• NoseDetector• EyeDetector• CrowdDetector• PointerDetector• MotionDetector• VirtualFence• Etc.
Sin
k
SR
C
Sin
k
SR
C
Sin
k
SR
C
Sin
k
SR
C
Sin
k
SR
C
22
Using media processing modules
Sin
kS
RC S
ink
SR
C
CrowdEventsS
RC
Sin
k
SR
C
CrowdDetector
RtspPlayer
Sin
kS
RC
See demo
See demo
23
Implementing group communications: just connect
Media Pipeline
WebRTCstreaming
Sin
kS
RC
WebRTCstreaming
SR
CS
ink
SR
CS
ink
WebRTCstreaming
WebRtcEndpoint 1
WebRtcEndpoint 3
WebRtcEndpoint 2
SR
CS
ink
WebRTCstreaming
WebRtcEndpoint 4
User 1 User 2
User 3
User 4
24
Group communications: RTP topologies
− Media Mixing Mixer• Composite
− Media Switching Mixer• Connect primitive• Natural topology
− SFU• WebRtcSfu
o Simulcast
Sin
kS
RC
SRC Sink
SinkSRC
SR
CS
ink
25
Creating your own modules
KurentoIDL Compiler
JSON
Module IDL Description
Java Java-Script
cpphpp
Media API code Toolchain
Auto-generated code
ModuleImplementation
cmake
Interface
Developer
(1)
(2)
(3)
26
Going up the API stack: the room API
createRoomdeleteRoom
joinRoomleaveRoom
publisMediaunpublishMedia
onParticipantJoinedonParticipantLeft
onMediaPublishedonMediaUnpublished
onMessage
See demo
27
Going up the API stack: the tree API
createTreedeleteTree
joinAsPresenterjoinAsViewer
leave
28
What’s next
29
Where to start
− http://www.kurento.org/documentation • Installation and administration guide• Java tutorials for Spring and Java EE
developers• JavaScript tutorials for Node.js developers• JSON-RPC protocol documentation
− https://github.com/kurento • Main repository
30
Thanks