TFX platform

So I haven’t written anything worthy in a while , just published some posts that were lying around in my drafts . Here I write about the main thing . some thing awesome that I was trying to accomplish in the last quarter .

<< TFX is now live in chrome store , open and free for public use . No signin or account required , no advertisements   : https://chrome.google.com/webstore/detail/tfx-sessions/aochimdcllmgleokpnlabijehdlmkdga >>

TFX Sessions is a plug and play platform for VoIP ( voice over IP ) scenarios.  Intrinsically it  is a very lightweight API package and shipped in form of a Chrome Extension . It is a turn-key solution when parties want instant audio/audio communication without any sign-in ,plugin installation or additional downloads  . Additionally TFX Sessions is packaged with some interesting plugins which enable the communicating parties to get the interactive and immersive experience as in a face to face meeting.

There is a market requirement of making a utterly simple WebRTC API  that has everything needed to build bigger aggregate projects but the available solutions are either just to basic or much too complex . So I initially started writing my own getuserMedia APIs, but left it midway and picked up simplewebrtc API instead for want of time .Then I focused on the main crux  of the project which was the widget API and ease of integration.

How Does TFX Sessions Work ?

  • Signalling channel establishes the session using Offer- Answer Model
  • Browser’s  media API’s , like getUserMedia and Peerconnection are used for media flow
  • Media only flows peer to peer
TFX WebRTC platform architecture . socket io signalling
TFX WebRTC platform architecture . socket io signalling

A Widget is essentially any web project that wants communication over webrtc channel . Once the platform is ready I have core APIs , widgets and signalling server. Then came up the subject of enterprise internet blocking my communication stream . Time for TURN ( Coturn in my case ).

TFX create / join room
TFX create / join room
TFX startup screen
TFX startup screen

Components of TFX

Client Side Components of tangoFX :

broplug API
Inhouse master library for TangoFX. Makes the TFX sessions platform .Masks the low level webrtc and socketio functions .
Provides simple to use handles for interesting plugins development in platform .
simplewebrtc
performs webrtc support , peer configuration , wildemitter , utils , event emitters , JSON formatter , websocket , socket namespace , transport , XHR more like so
socket.io.js
exports and listener for real time event based bidirectional communication
Jquerry
JS library for client side scripting
Bootstrap
HTML and CSS-based design templates for typography, forms, buttons, navigation and other interface components.

Server Side components

Signalling Server -signal master
socket.io server for webrtc signalling
TURN -Coturn
TURN protocol based media traversal for connecting media across restricting domains ie firewalls, network policies etc .
redis Data structure to maintain current and lost sessions

TFXSessions Components

Architecture

So here is the final architecture of TFX chrome extension widget based platform .

  • The client side contains widgets , chore extension APIs , chrome’s WebRTC API’s , socket.io client for signalling , HTML5 , Jquerry , Javascript , and CSS for styling.
  • The Server Side of the solution contains socket.io server for signalling , manuals and other help/support materials , HTTPS certificate and TURN server implementation for NAT .
TFX whitepaper v2.0
TFX platform Server client components . WebRTC media and socketio communication . Build as chrome Extension

Salient Features

  • The underlying technology of TangoFX is  webrtc with socket based signalling  . Also it adheres to the latest standards of W3C , IETF and ITU on internet telephony .
  • TangoFX sessions is extremely scalable and flexible due to the abstraction between communication and service development. This make it a piece of  cake for any web developer using  TangoFX interface to add his/ her own service easily and quickly without diving into the nitty gritties .
  • TangoFX is currently packaged in a chrome extension supported on chrome browser on desktop operating system like window , mac , linux etc .
  • The call is private to both the parties as it is peer-to-peer meaning that the media / information exchanged by the parties over TangoFX does not pass through an intervening server as in other existing internet calling solutions.
  • TangoFX is very adaptive to slow internet and can be used across all kinds of networks such as corporate to public without being affected by firewall or restricting policies  .

TFX Widget Screens

Alright so that’s there . Tada the platform is alive and kicking . Right now in beta stage however . Intensive testing going on here . However here are some screenshots that are from my own developer version .

TFX recording widget
TFX recording widget
TFX face detection and overlay widget
TFX face detection and overlay widget
TFX multilingual communication
TFX multilingual communication
TFX screen-sharing
TFX screen-sharing
TFX video Filters
TFX video Filters
TFX audio visualizer
TFX audio visualizer
TFX text messaging widget
TFX text messaging widget
TFX cross domian access . flicker here
TFX cross domain access . flicker here
TFX draw widget
TFX draw widget
TFX code widget supportes many programming languages
TFX code widget supportes many programming languages
TFX  webrtc dynamic stats
TFX webrtc dynamic stats
TFX  introduction widget
TFX introduction widget

Note that the widgets described above have been made with the help of third party APIs.

TFX Sessions Summary

We saw that TFX is WebRTC based communication and collaboration solution .It is build on Open standards from w3c , IETF , Google etc.
Scalable and customizable. Immersive and interactive experience .
Easy to build widgets framework using TangoFX APIs.

TFX User Manual : https://tfxserver.above-inc.com/static/manuals/material/TangoFXv0.1Usersmanual.pdf

TFX Developer’s Manual : https://tfxserver.above-inc.com/static/manuals/material/TangoFXv0.1Developersmanual.pdf


TangoFX v1 demo on youtube
TangoFX reseracg paper on academia.edu
TangoFX article on Linkedin

Steps for building and deploying WebRTC solution

Step 1  : USE Local machine to test the client server WebRTC funcationality

Pick any WebRTC API and run its demos . It works kool . download and run in local-machine with nodejs server . Awesome . Everything is Awesome !!

You can learn more about some WS based WebRTC API here:  https://altanaitelecom.wordpress.com/2014/12/02/current-state-of-webrtc/.

If you are a diehard telecom engineer and only want SIP based WebRTC solutions go here : https://altanaitelecom.wordpress.com/2014/07/16/interoperability-between-webrtc-sip-phones-and-others/

Steps for building and deploying WebRTC solution Step 1 : Pick a WebRTC API and run locally ( ie open 2 browsers and run on local machine )
Steps for building and deploying WebRTC solution
Step 1 : Pick a WebRTC API and run locally ( ie open 2 browsers and run on local machine )

Step 2 : Use cloud Server and different client Browsers  

Now what good is it doing to anyone if its running locally on my machine with addresses like localhost and 127.0.0.1  . Let us put it on the cloud and at-least let my colleague / friends enjoy it .  Cloud Web Server and Nodejs signalling server . That is okay use amazon’s Ec2. works for most of the people most of the time .

Steps for building and deploying WebRTC solution Step 2 : Put Server on cloud and WebRTC clients on different machine
Steps for building and deploying WebRTC solution
Step 2 : Put Server on cloud and WebRTC clients on different machine

Here is when we discover the issues of ICE ( Interactive Connectivity Establishment ) I have mentioned this in detail on the post NAT Traversal using STUN and TURN .  Briefly ICE helps us in coping up with NAT ( Network Address Traversal and Firewalls ) .

Note that this step only works if everyone you want to connect to is either on same intranet or on public internet without and UDP blocks / firewalls / restriction .

As we try to connect 2 WebRTC clients from different machine and different networks we find that network address from client’s OS and network card fails to connect to Signalling Server due to either Firewalls issues or other Network policies . We therefore use a STUN server to map the private IP to a publicly accessible IP that will help in completing the signalling

The Signalling is establishes using a STUN server for address mapping and NAT . One can use google’s default STUN server stun.l.google.com:19302. Easy and free .

Steps for building and deploying WebRTC solution Step 2.1 : Put Server on cloud and WebRTC clients on different machine + STUN for address discovery ( NAT traversal )
Steps for building and deploying WebRTC solution
Step 2.1 : Put Server on cloud and WebRTC clients on different machine + STUN for address discovery ( NAT traversal )

There you go everything is looking good from here now , both peers join the session successfully  , but the video may appear black . This is so because the media under most inter network conditions fails to flow between private and public network .

This is where step 3 comes into picture ie using a TURN ( media relay ) server .

Step 3: TURN server to Call people in a inter-network fashion 

Sure the architecture I have setup is bound to work everywhere where the network is open and public . However error in connectivity , errors in console , blank video are the problems that might appear when one tries to connect from private to public connections.

To bypass network firewalls , corporate net policies , UDP blocks and filters we require a TURN server which help in media traversal across different networks in a relay mechanism.

Now we have 3 options to choose from

1.  Use a wildly popular http://numb.viagenie.ca/

2. Build your own TURN server with RFC 5766 ( COTURN )  , or rather easier would be to use any open source TURN server code available in Github.

3. Pay and use a commercial TURN service provider or you can even use their trail version to see if things work out for you ( example Xirsys)  .

Remember you can use any TURN service it does not affect your WebRTC API functionality . All we need to do is add it to Peerconnection configuration like

&lt;/address&gt;&lt;address&gt;peerConnectionConfig: {<br>
iceServers:[&lt;/address&gt;&lt;address&gt;{"url": &lt; stunserver address &gt;},&lt;/address&gt;&lt;address&gt;{"username":"xx","url":&lt; turn server address&nbsp;transport=udp&gt;,"credential":"yy"},&lt;/address&gt;&lt;address&gt;{"username":"xx","url":&lt; turn server address transport=tcp&gt; , "credential":"yy"}]&lt;/address&gt;&lt;address&gt;},&lt;/address&gt;&lt;address&gt;

There we go , now anyone from anywhere should be able to use our WebRTC setup for making audio , video calls or just exchanging data via DataChannel ( like screen-sharing , file transfer , messages , playing games , collaborative office work etc )  .

Steps for building and deploying WebRTC solution TURN based media Relay for WebRTC traffic
Steps for building and deploying WebRTC solution
TURN based media Relay for WebRTC traffic

The setups covers scenarios wherein user is on office corporate network , home network , mobile network , no problem as long as he / she has a webrtc enables browser ( read Chrome , Mozilla , Opera ) .

It is noteworthy that ideally voice should be traversing on TCP while video and data can go around in UDP however unless restrained the WebRTC API’s self determine the best protocol to route the packets / stream .

Debug helper

Common issues around media playback

  • DOMException: The play() request was interrupted by a new load request
  • webrtcdevelopment_min.js:1 [Violation] Only request notification permission in response to a user gesture.

Read more about best WebRTC frameworks and code in this book

WebRTC SDKs Analysis

In the last few months, I have been keenly tracking how the course for WebRTC is turning out. In my opinion, it is an incredible game-changer and a market disrupter for the telco industry plagued by licensed codecs and heavy call session control software.

Contrary to my expectations the fundamental holes in WebRTC specification are still the same with less being done to fulfil them – Desktop sharing on Chrome Extension, media compatibility with desktop popular H264 being prominent obstacles to adoption. Of course, now there exists an abundance of interactive use-cases for WebRTC APIs from gaming to telemedicine. However, none of the applications is complete and standalone since each uses a new gateway to connect to their existing platform or service.

As new webrtc SDKs and open-sourcing platforms surface, many seem to be wrapping around the same old WebRTC functions ( getusermedia data-channel and peer-connection ) with few or no addons. Few of I am listing down some popular working ones in this blog but there still exists no concrete stable reliable guide to set up the backbone network ( yes I am referring to Media interconversion, relay, TURN, STUN servers ). It is left to a telecom software engineer/developer to find and figure out the best integrations to configure session handling and PSTN or desktop application compatibility.

Some commercial off the shelf service providers have begin to extend interconnecting gateways ( SBC’s) for their backend for Web-Javascript based WebRTC implementations but there are concerns on the end to end encryption and media management as it passes via transcoding media server and many points of relay. This in my opinion completely defeats the objective of WebRTC’s peer-to-peer communication which by design is supposed to be independent of centralised server setup. WebRTC was meant to *everything you can’t do with proprietary communication tools and networks*.

Well moving on , here are some nice API implementations of WebRTC ( only for Websockets no SIP/WebSockets ) which can be quickly used by web developers to create peer to peer media session on web endpoints via a WebRTC supported browser web page.

1.appRTC

apprtc
apprtc2

Neat process of setting up offer-answer and SDP . Notice the Relay candidate gathering 

apprtc3

Session Description ( SDP) for the WebRTC peer with audio / video codecs and other session specificatiosn such as bitrate , framerate , codec profile , RTP specs etc.

apprtc4
apprtc5

No over the top media control which is good as media flows end to end here without any centralized media server .

2. talky.io

Also related to SimpleWebRTC which is a lightweight MIT licensed library providing wrappers around core WebRTC API to support application building while easing the lower level peerconnection and session management from developer.

talky1

talky2

talky3

Simmilar offer-answer handshake and SDP excahnge.

talky4 talky5 talky6

There seems to be better noise control management which could be my browser acting on my fluctuating network bandwidth as well . 

3. tokbox

tokbox2
tokbox1

More control from UI on Media settings which is provided as part of getusermedia in WebRTC specification. Read more about the webrtc APIs in my other writeup 

tokbox7
tokbox6
tokbox5

Simmilar to previous WebRTC session ( totally dependant on my own network and CPU ) , independant of any thord party control . 

As my peers network degrated , my webrtc session automatically adjust based on RTCP feedback to send lower resolutions and framerate.

tokbox4
tokbox3

4. webrtcdevelopment

Open source MIT licensed libray to spin up Webrtc calls quickly and easily from a chrome supported browser . It is a fork of best features from multiple libraries such as apprtc , webrtcexperiments , simplewebrtc and is maintained by n open community of users including me.


WebRTC Media Stack is explained in following articles

Call Continuity from Mobile GSM network to WebRTC

In  the present age of IP telephony when telecom convergence is the big thing all around the world , need of the hours is to enable fixed and mobile Service Providers ( SP )  to monetize the subscriber’s phone number by extending it to new web based services.SPs can offer a WebRTC Communicator endpoint that uses the same phone number as the subscriber’s fixed or mobile phone.

Advanced features enable calls to be transferred between fixed-line, mobile and WebRTC endpoints.

Find the diagram depicting this below :

Transfer mobile callto WebRTC session
Transfer mobile callto WebRTC session

SPs can offer 3rd Party WebRTC endpoints to access the user’s phone number and subscription . E.g. enable web applications such as Facebook, Amazon or Netflix to allow their users to make/receive calls or messages directly from the web applications

Revenue Streams :

  • monthly fee for access to WebRTC endpoints and for receiving calls from by 3rd Party WebRTC endpoints
  • One time upgrade fees for Accessing the Web service integration with telecom network like a plan upgrade

Brownie points

  • No software is required to be downloaded on the subscriber’s computer, tablet or mobile phone
  • No desktop support required for the service provider

Plans For Consumer Customers:

  • Subscribers can use the WebRTC endpoints on their computers, tablets or mobile phones as a fixed-line device at home, as a desktop solution when away from home and to avoid international tolls when traveling
  • Subscribers can connect their web services (e.g. Websites , Facebook, Amazon, Netflix) to their fixed or mobile services subscriptions using their SP-provided phone number

Plans For SP Enterprise Customers:

  • Enterprises can deploy a WebRTC endpoint for their employees that provides a single corporate communications endpoint that can be connected to any of the corporation’s UC/PBX and Call Recording systems
  • Employees can use the WebRTC endpoint as their office phone at work, home or when traveling
  • Connects to all leading UC/PBX and Recording platforms simultaneously
  • Enterprises can deploy a single WebRTC endpoint across all their UC/PBX and Recording platforms – current and future
  • Easy for IT departments to deploy – no software is required to be downloaded to employees’ computers, tablets or mobile phones
  • Enables corporate policies and features from the WebRTC endpoint including
  • Displaying the corporate identity
  • Routing calls via corporate networks
  • Tracking and Recording calls and messages

WebRTC communication over Web Services

This post is about communication from any application to WebRTC using Web Services.

HTTP and XML is the basis for Web services

Advertisement Engine with WebRTC

WSDL
  • WSDL stands for Web Services Description Language
  • It specifies the location of the service and the operations (or methods) the service exposes.
  •  XML-based language for describing Web services.

SOAP
  • SOAP stands for Simple Object Access Protocol
  • SOAP is an XML based protocol for accessing Web Services.
  • SOAP is based on XML

UDDI
  • UDDI stands for Universal Description, Discovery and Integration
  • UDDI is a directory service where companies can search for Web services.
  • UDDI is described in WSDL
  • UDDI communicates via SOAP

RDF
  • RDF stands for Resource Description Framework
  • RDF is a framework for describing resources on the web
  • RDF is written in XML
uses :Web services can offer application-components like: currency conversion, weather reports, or even language translation as services.
…………..

WebRTC Media Streams

SDP signaling and negotiation for media plane

Read more on SDP and its attributes : https://telecom.altanai.com/2014/01/03/sip-in-depth/(opens in a new tab)

Media plane adaptation is done at the SBC for network carried media, it should be done for all network hosted media services which face peer-to-peer media.

The high-level architecture elements of WebRTC media streams can be divided into two areas:

Adaptation of WebRTC Media Plane to IMS Media Plane

Encryption, RTP Multiplexing, Support for ICE —

Audio – Interworking of differing WebRTC and codec sets —

Video – Use of VP8 , Support for H.264 —

Data – Support of MSRP ( RCS standard for messaging over DataChannel API)

—Peer-to-Peer Media

Direct connection to media servers and media gateways .—

Use common codec set wherever possible to eliminate transcoding —Use regionalized transcoding where common codec not available Note: Real-time video transcoding is expensive and performance impacting

On-going standards/device/network work needs to be done to expand common codec set. WebRTC codec standards have not been finalized yet. WebRTC target is to support royalty free codecs within its standards. —

MediaWebRTClegacy
AudioG.711, OpusG.711, AMR, AMR-WB (G.722.2)
Audio – ExtendedG.729a[b], G.726
VideoVP8H.264/AVC

Supporting common codecs between VoLTE devices and WebRTC endpoints requires one or more of the following: 1.Support of WebRTC codecs on 3GPP/GSMA 2.Support of 3GPP/GSMA codecs on WebRTC 3.WebRTC browser support of codecs native to the device

References :

https://tools.ietf.org/id/draft-ietf-rtcweb-sdp-08.html#rfc.section.5.2.8

Regulatory/Legal Considerations and CALEA with WebRTC development

This post is deals with some less known real world implication of developing and integrating WebRTC with telecom service providers network and bring the solution in action . The  regulatory and legal constrains are bought to light after the product is in action and are mostly result of short sightedness .  The following is a list of factors that must be kept in mind while webRTC solution is in development stages

  • WebRTC services from telecom provider depend on the access technology, which may differ if the user accessing the network through a third party Wi-Fi hotspot.
  • User/network type may also dictate if decryption of the media is possible/required.
  • For Peer-to-Peer paths, media could be extracted through the use of network probes or other methodology

Then there are Other Considerations such as specific services, for example if WebRTC is used to create softphones software permitting users to receive or originate calls to the PSTN, the current view is to treat this as a fully interconnected VoIP service subject to all the rules that apply to the PSTN – regardless of technologies employed.

CALEA

Communications Assistance for Law Enforcement Act (CALEA) , a  United States wiretapping law passed in 1994, during the presidency of Bill Clinton.

  • CALEA requirement for an LTE user may be very different than the CALEA requirements for a user accessing the network through a third party Wi-Fi hotspot.
  • For media going through the SBC, CALEA may use a design similar to existing CALEA designs.
calea intercept infrstructure
calea intercept infrastructure

Read more on WebRTC Security here which discusses SOP (single origin policy ) , CORs ( cross origin requests) , JSONP , ICE , location sharing , scerensharing , Long term access to camera and microphone , SRTP DTLS as well as best practises for secure communication

VoIP and WebRTC platform security largely depend on the underlying protocols such as SIP . SIP is an robuts and time tested VoIP proctol to facilitate VoIP calls . To learn more about SIP security against atacks like

  • Registration Hijacking
  • Impersonating a Server
  • Temparing Message bodies
  • mid-session threats like tearing down session
  • Denial of Service and Amplification

Also security mechnisms like

  • Full encryption vs hop by hop encrption
  • Transport and Network Layer Security
  • SIP over TLS
  • SRTP

Read more about Certificates , compliances and Security in VoIP which summarized

  • HIPAA (Health Insurance Portability and Accountability Act) ,
  • SOX( Sarbanes Oxley Act of 2002) ,
  • Privacy Related Compliance certificates like COPPA (Children’s Online Privacy Protection Act ) of 1998  ,
  • CPNI (Customer Proprietary Network Information) 2007 ,
  • GDPR (General Data Protection Regulation)  in European Union 2018,
  • California Consumer Privacy Act (CCPA) 2019 ,
  • Personal Data Protection Bill (PDP) – India 2018 and
  • also specificatiosn against Robocalls and SPIT ( SPAM over Internet Telephony) among others

Read about General Data Protection Regulation (GDPR) in VoIP

STIR/SHAKEN – Secure Telephony Identity Revisited / Signature-based Handling of Asserted information using toKENs

WebRTC compatible android client

This post describes the requirement of creating a SIP phone application on android over the same codecs as WebRTC ( PCMA , PCMU , VP8) . In my project concerning the demonstration of WebRTC inter operability ( presence , audio / video call , message )  with a native android client , I had to develop a lightweight Android SIP application , customized for the look and feel of the webrtc web application . This also enables the added services to WebRTC client such as geolocation , visual voice mail , phonebook , call control options be set from android application as well .

Aim :

Android webrtc- sip client development , using sipml5 stack implemented through web services and native android programming .  

Software Used:

⦁ Eclipse IDE
⦁ Java SE Development Kit 7.0
⦁ Android SDK

Tasks :

⦁ Authorization of a user, based on his/her credentials (Database local to the application).

webrtc_android_2
⦁ Navigation Drawer on the home page which shows a menu giving the user various options like:
⦁ View Home Page
⦁ View Contact List
⦁ View/Edit My Profile
⦁ View My Location
⦁ Sign Out

⦁ Phonebook sync : Importing contact list of the Android Phone into the application. Editing user profile with values like  User Name ,  Password ,  Domain. 

webrtc_android_1
⦁ Inclusion of a Web View in the application which currently opens the desired webpage(http://sipml5.org/call.htm).

⦁ Geolocation: Showing marker for the current location of user in Google Maps.Displaying the address of the user in a Toast Message.

webrtc_android_4

⦁ Audio / Video call capability 

android_webrtc

figure 1 : Login page , figure 2 : Call page , Figure 3 : Menu bar 

Future Roadmap:

⦁ Connecting the application to a database which sits on the cloud.
⦁ Based on the entries in the database the user will be able to:
⦁ Login to the application.
⦁ View or edit his/her details in the My Profile Section.
⦁ Understanding codes of sample applications for making SIP calls from Android OS like:
⦁ SipDroid
⦁ SipDemo
⦁ IMSDroid
⦁ Modifying the existing application to be able to make SIP calls like one of the apps listed above.

Modules :

Development Done:
  1. Development of an authorization page connecting the application to a local database from where values are inserted and retrieved.
  2. Development of navigation drawer where additional options for the application will be displayed making it a user friendly application.
Development Planned:

1.Connectivity to a cloud database.  

2. App engine on cloud.

3. Importing contacts from phone address book .

4. Offine storage of profile details and few call logs .  

Architecture:

webrtc_android_enviornment

……………………………………………………………………………………..

Difference between WebRTC and plugin based communication

A lot of service providers ie telecom operators had deduced their own ways to provide Web based communication even before WebRTC was born . With time , as WebRTC has become stronger , more secure , resilient to failure they have come around to migrate their existing system from previous closed box native APIs to opensource WebRTC APIs.

The first figure ( given below ) depicts a communication platform build over plugins and proprietary APIs using HTTP REST based signaling .

2014-07-22_1212
Web Communication Service Architecture over HTTP/ REST API

As the migration took place the proprietary API components were replaced by Open standard based entities such as plugins were replaced by WebRTC APIs, HTTP REST based signalling was replaced by SIP ( Session Initiation Protocol ) .

Web Communication Service Architecture over WebRTC SIP
Web Communication Service Architecture over WebRTC SIP

Note telecom operator network did not had to face transformation by integration of WebRTC elements .

E-Learning

True that the number of teacher today are not enough to teach the number of kids . For example even in India there is often 1 teacher for a class of 60 students in one subject. Also the experience and output of learning from a human teacher cannot be ever replaced by a software or ebook or application no matter how user-friendly or informative it is .

In this post I am going to describe an e-learning platform which harness the power of Internet for the purpose of distance education and where students around the world volunteer to teach each other any subject they wish to. This will be made possible through a combination of real time communication technologies like WebRTC and plethora of knowledge repositories.

Aim :

Platform to connect volunteers ( children ) teach each other a subject in a stipulated time through Web based Real Time Communication.

Working :

A student enrolls himself for a subject or a course it could be anything from arithmetic to french language . Another student who know French language for example find this in portal and sign up to be teacher for that child. They can anytime connect with each other in audio , video , message , file sharing , screen sharing session through WebRTC and learn the subject. The students earn reward points .

e-leaning service on WebRTC
e-leaning service on WebRTC

Technologies  :

  • WebRTC for communication
  • MySQL for data storage
  • Apache Tomcat as Webserver
  • HTML5
  • CSS
  • JavaScript

Conclusion :

By encouraging a child to take responsibility and teach another child , we will not only encourage friendship between them but also give them a sense of accomplishment .


WebRTC SIP / IMS solution

We started in winters on 2012 with Webrtc . At time time it just looked like a new tech jargon that might fade away when new ones comes . In many many WebRTC’s buzz has died down since its massive adoption. But i nevertheless still see a lot of potential and development around it.

What really is WebRTC ? I made an entry on it  here .

Around nov – dec 2012 , team and I spend the time learning the nitty-grities of HTML5 based media operation and Javascript sip stack of SIPML. I remember toward the end of the year ie before Christmas , We were done with the explanation and education aspects of WebRTC , a technology that will revolutionise communication in ages to come , at-least so says the numerous other blogs ,  and documents i read so far .


Usecases for WebRTC range across a wide variety , of them the most revenue generating ones are around video conferencing with realtime HD audio-video-data streams ,

To bridge the flow between a webrtc client to a PSTN endpoint via IMS , interworking between webrtc media standards and codecs with that of gateways in IMS is critical . For instance WebRTC mandates secure RTP ( SRTP) the media engine / gateway should be able to support and connect with RTP from PSTN endpoints.

client BOB -> webrtc2sip Gateway -> SIP server -> client Alice

can be  understood with the callflow of a simple SIP Invite initiated from one html page towards another which passes through the configuration of gateway to IMS world ,  SIP Telecom Application server , Database , nodes of IMS environment etc.

For the purpose of a simple Explanation a simplified call flow ca be depicted as ,

webrtccallflow

A very high level architecture of solution deployment in IMS world could be

solution arch2

As the solution matures into a full fleshed project . The alpha version has been released with the following feature set . The WebRTC platform Suite offers a easily deploy-able solution to enable communication

Alpha Release WebRTC platform Suite

  • Single Sign On
  • Login with id and password to access all services
  • Audio / Video Call
    • Call Hold / Call Transfer
  • Messaging:
    • SIP Instant Messaging
    • Message to Facebook Messenger
    • Message delivered as Email
  • Chatroom
    • group chat between multiple users . Room is created for set of users .
  • Video Conferencing
    • video chat between multiple parties . Room is created for set of users .
  • File Transfer
    • Sharing of files from local to remote , in peer-to-peer and broadcasting fashion .
  • Third party Webservices
    • Widgets like calendar , weather , stocks , twitter are embedded.
  • Visual Voice Mail
    • Record and deliver voice message to recipients voice mail inbox which can be accessed/ played from web client .
  • Phonebook
    • cloud integration
    • add new entries
    • add photos to contacts identity
    • import contacts from google account
  • Click to Call :
    • Drop down list of contacts form mail call console
    • 2 step Click to call from Phonebook
  • Presence :
    • Publish online / offline status
    • Use Subscribe / notify requests of SIP
  • Web Ssocket to SIP Gateway
    • Conversion between the signal coming from the WebRTC and SIP client to the IMS core
    • Conversion of “voice/video ” media between sRTP and RTP
    • Conversion of other media (data channel) towards MSRP and Transcoding.
    • Support of ICE procedure
    • Implementation of a STUN server
  • QoS Support

Beta Release WEBRTC PLATFORM SUITE

  • Logs
    • calls logs
    • Message logs
  • User Profile
    • user details like address , email and social networking accounts
    • Phonenumber for GSM integration through SMS
    • User’s Media storage like Pictures , profile picture , Audio , video
    • File sharing documents storage for future access in the same format
  • Real Time and Offline Analytics
  • service usage with graphical and tabular history trends
  • Session Management
    • Single Sign-on
    • Forgot password regeneration using secure question
    • Registration of new user account
    • Logout and clearance of session parameters
  • Security
    • No redirection to any page through url entry without valid session
    • No going back to home page after logout by back button on browser
    • No data vulnerability
    • Multiple login through different devices handled
  • OAuth
    • Login via IMAP / token through facebook and Google
  • Phonebook with Presence functionality inbuilt
  • Directory Service based on country / region
  • Geolocation of approximate location detection of device logged in and visibility to others
webrtc solution
WebRTC client deployment view , accessible devices , network elements
WebRTC deploymenet overview and inetraction with other network elemets such as gateway , cloud storage ,  sipserver , IMS
WebRTC deploymenet overview and inetraction with other network elemets such as gateway , cloud storage , sipserver , IMS

Commercial release features specs for WebRTC over IMS

  • Integration with new age CSP deployments like VoLTE, ViLTE, VoWiFi
  • Multi vendor support
  • Interactive webrtc services
  • Media Services
    • Automated Natural language Speech recognition
    • Semantic processing via ML
    • Enhanced incall services replacing IVR ( touch -tone)
    • VQE (voice Quality Enhancements)
    • Encoding and Decoding – Multiple Codec Support
    • Transcoding
    • Silence Suppression
  • Security via TLS, encryption and AAA
  • Http, NFS caching
  • NAT using Xirsys TURN
  • Recording, playback and media file compression
  • active frame selection
  • DTMF (Dual Tone Multi Frequency)
    • SIP info messages (out-of-band)
    • SIP notify messages (out-of-band)
    • Inband DTMF not supported yet
  • Audio
    • mixing
    • announcements ( VXML, MSML )
    • filters
    • gain control ( AGC using webrtc stack)
    • noise suppresesion ( webrtc stack)
    • speakers notification
    • Narrowband, Wideband, and Super Wideband
    • dynamic sample rate
  • Video
    • continuous presence ( Face detetion )
    • floor control
    • video lipsync (sync)
    • speaker tile selection
  • VQE (Voice Quality Enhancement )
    • Acoustic Echo Cancelation
    • noise reduction
    • noise line detection
    • noise gating
    • Packet Loss concealment
  • Call analyics
    • progress analysis
    • MOS , R-factor ( derived from latency , jitter , packet loss )
  • CDR (Call detail records ) and accounting
  • Lawful interception

Updating this article 2019

There was a long journey from traditional telecom architectures to NFV cloud based architectures ( like openstack). supported over web , 4G , LTE or other upcoming networks. Many OTT providers prefer using the public cloud over a NFV data centre.

Multinode / Multiedge computing platforms like Media Resource Function are expected to meet the need for quick delivery with additional features like hardware accelerated media , algorithms for optimised data flow (packetization, decongesting , security ) etc . With th decomposed architecture they can better utilise the

  • CPU – contains couple of cores optimised for sequential serial processing such as   graphics or video processing
  • GPU – contains many smaller cores to accelerate creation of images for computer display . Can include texture mapping, image rotation, translation, shading or more enhanced features like motion compensation, calculation of inverse DCT, etc. for accelerated video decoding.
  • DSP- processing data representing analog signals

Although IMS based solutions are more suited to telephony applications and CSPs ( Communication service providers like telecom companies ) but similar or same architectures are widely finding their into newer developed cloud communications solutions supporting tens of millions of subscribers and hyper scale deployment . It could be around applications such as

  • HD (High Definition ) calls
  • UCC ( conf , draw-board, speech recognition , realtime streaming)
  • immersive experiences ( Augmented reality , virtual reality , face recognition , tracking )
  • contextual communication ( transcription etc)
  • video content delivery with deep media analytics

Demand these says is for a decentralised system of pool of servers ( media and signalling ) that can scale independently to match up to peak traffic at any moment , with ofcourse carrier class performance . Not only these flexible solutions reduce complexity but also OpEX .

Ref:

Unified Communicator and Collaborator for Enterprise

Modular enterprise communicator solution for enterprise based communication and collaboration . Use sipml5 client side library to provide webRTC based media stream capture and propagation from client side without external plugins.

Github Repo – https://github.com/altanai/unifiedCommunicator

Unified Communications and Collaborations ( UC&C ) – https://telecom.altanai.com/2013/07/12/unified-communication/

WebRTC business benefits to OTT and telecom carriers

Historically, RTC has been corporate and complex, requiring expensive audio and video technologies to be licensed or developed in house. Integrating RTC technology with existing content, data and services has been difficult and time consuming, particularly on the web.
Now with WebRTC the operator finally gets a chance to take the shift the focus from OTT ( Over The Top service providers like SKype , Google chat WebEx etc that were otherwise eating away the Operators revenue ) to its very own WebRTC client Server solution , hence making the VOIP calls chargeable , while at the same time being available from any client ( web or softphone based )

To know more about what webrtc is read : https://telecom.altanai.com/2013/08/02/what-is-webrtc/

To read about how webrtc integrates with the SIP/IMS systems read https://telecom.altanai.com/2013/10/02/webrtc-solution/

OTT
OTT ( Over The Top ) Applications

Where are we Now ?

WebRTC has now implemented open standards for real-time, plugin-free video, audio and data communication.

Many web services already use RTC, but need downloads, native apps or plugins. These includes Skype, Facebook (which uses Skype Flash ) and Google Hangouts (which use the Google Talk plugin).
Downloading, installing and updating plugins can be complex, error prone and annoying , such as Flash , Java .,etc

Plugins can be difficult to deploy, debug, troubleshoot, test and maintain—and may require licensing and integration with complex, expensive technology. It’s often difficult to persuade people to install plugins in the first place/ bookmark it or keep it activated at all times.

WebRTC support across various browsers
WebRTC support across various browsers , pic source : caniuse.com

API support from browser

  • PeerConnection API
  • getUserMedia
  • WebAudio Integration
  • dataChannels
  • TURN support
  • Echo cancellation
  • MediaStream API
  • Multiple Streams
  • Simulcast
  • Screen Sharing
  • mediaConstraints
  • Stream re-broadcasting
  • getStats API
  • ORTC API
  • H.264 video
  • VP8 video
  • Solid interoperability
  • srcObject in media element
  • Promise based getUserMedia
  • Promise based PeerConnection API

WebRTC trends

disruptive graph
Biz users
ic source : Disruptiveanalysis

The APIs and standards of WebRTC can democratize and decentralize tools for content creation and communication—for telephony, gaming, video production, music making, news gathering and many other applications.

pic source: iswebrtcreadyyet.com

What is WebRTC?

webrtc draft
 

WebRTC 1.0: Real-time Communication Between Browsers – W3C Candidate Recommendation 13 December 2019 https://www.w3.org/TR/webrtc/

Read more in the layers of webrtc  and their functionalities here :  WebRTC layers

webrtc_development_logowebrtcdevelopment
Open Source WebRTC SDK and its implementation steps https://github.com/altanai/webrtc

What is WebRTC ?

WebRTC (Web Real-Time Communication) is an API definition drafted by the World Wide Web Consortium (W3C) that supports browser-to-browser applications for voice calling, video chat, and P2P file sharing without the need of either internal or external plugins.

  • Enables browser to browser media streaming over secure RTP profile
  • Standardization , on a API level at the W3C and at the protocol level at the IETF.
  • Enables web browsers with Real-Time Communications (RTC) capabilities
  • written in c++ and javascript
  • BSDD style license
  • free, open project avaiable in all major borwsers 

As of the 2019 update the W3C defines it as

a set of ECMAScript APIs in WebIDL to allow media to be sent to and received from another browser or device implementing the appropriate set of real-time protocols. The specification being developed in conjunction with a protocol specification developed by the IETF RTCWEB group and an API specification to get access to local media devices.

 The following is the browser side stack for webrtc media .  

WebRTC media stack Solution Architecture
WebRTC Media Stack

Open and Free Codecs

Codecs signifies the media stream’s compession and decompression. For peers to have suceesfull excchange of media, they need a common set of codecs to agree upon for the session . The list codecs are sent  between each other as part of offeer and answer or SDP in SIP.

WebRTC uses bare MediaStreamTrack objects for each track being shared from one peer to another. Codecs associated in those tracks is not mandated by webrtc soecification.

For video as per RFC 7742 WebRTC Video Processing and Codec Requirements , the manadatory codesc to be supported by webrtc clients are : VP8 and H.264‘s Constrained Baseline profile

For Audio as per RFC 7874 WebRTC Audio Codec and Processing Requirements ,browser must support  Opus codec as well as G.711‘s PCMA and PCMU formats.

Video Resolution handling

Unless the SDP specifically signals otherwise, the web browser receiving a WebRTC video stream must be able to handle video at at least 20 FPS at a minimum resolution of 320 pixels wide by 240 pixels tall.

In the best scenarios ( avaible bandwidth and media devices ) VP8 had no upper mark set on resolution of vdieo stream hence the stream can even go asfar as  maximum resolution of 16384×16384 pixels.

Independant of Signalling 

Webrtc does not specify any signalling / telecommunication protocl and it is upto the adoptor to perform ofeer/answer exchaneg in any way deemed fit for the usecase . For ex maple for a web only application on may use only plain websockets, whereas for a teelcom endpoints compatible app one should SIP as the protocol . 

Read more about WebRTC handshakes :

NAT-traversal technologies such as ICE, STUN, and TURN

Have written in detail about TURN based WebRTC flow diagrams .

https://telecom.altanai.com/2015/03/11/nat-traversal-using-stun-and-turn/. The post describe ICE  (Interactive Connectivity Establishment )  framework which is  mandatory by WebRTC standards.  It is find network interfaces and ports in Offer / Answer Model to exchange network based information with participating communication clients. ICE makes use of the Session Traversal Utilities for NAT (STUN) protocol and its extension, Traversal Using Relay NAT (TURN) 

NAT and TURN Relay

Learn about hosting / integrating different TURN servers for WebRTC

TURN server for WebRTC – RFC5766-TURN-Server , Coturn , Xirsys – https://telecom.altanai.com/2015/03/28/turn-server-for-webrtc-rfc5766-turn-server-coturn-xirsys/

Why is WebRTC importatnt ?

Significantly better video qualityWebRTC video quality is noticeably better than Flash.
Up to 6x faster connection timesUsing JavaScript WebSockets, also an HTML5 standard, improves session connection times and accelerates delivery of other OpenTok events.
Reduced audio/video latencyWebRTC offers significant improvements in latency through WebRTC, enabling more natural and effortless conversations.
Freedom from FlashWith WebRTC and JavaScript WebSockets, you no longer need to rely on Flash for browser-based RTC.
Native HTML5 elementsCustomize the look and feel and work with video like you would any other element on a web page with the new video tag in HTML5.

The major players behind conception and advancement of WebRTC standards and libraries are  :

IETF , W3C , Java community , GSMA .   The idea is to develop a Light -weight browser based call console , to make SIP calls from Web page .This was successfully achieved using fundamental technologies as Javascript , html5 , web-sockts  and TCP /UDP , open source sip server.It is good to note that there is no extra extension, plugin or gateway required , such as flash support  .Also it bears cross platform support ,  including Mozilla , chrome so on .

 Peer to peer Communication

 WebRTC forms a p2p communication channel between all the peers . that means as the participant count grows  , it converts to  a mesh networking topology with incoming and outgoing stream towards direction of each of its peers .

Two party call p2p

Peer to peer calling

two party call
p2p call

Multiparty Call and mesh network

Mesh based arrangement .

Multiparty party call
Mesh based webrtc video confeerncing

 In special case of broadcasting or  large number of viewers ( without outgoing media stream ) it is recommended to setup a Media Control Unit ( MCU) which will replay the incoming stream to large number of users without putting traffic load on the clients from where the stream is actually originating .   Important note :     1.It should be notes that these diagrams do not depict the ICE and NAT traversal and have been simplifies for better understanding. In real world scenarios there is almost all the time a STUN and TURN server involved .  

More on TURN Servers is given here : NAT traversal using STUN and TURN

2.Also the webrtc mandates the use of secure origin ( https ) on the webpage which invoke getusermedia to capture user media devices like audio , video and location .

Browser Adoption

As of March 2020 , webrtc is supported on following client’s browsers

  • Desktop PC
    Microsoft Edge 12+[25]
    Google Chrome 28+
    Mozilla Firefox 22+[26]
    Safari 11+[27]
    Opera 18+[28]
    Vivaldi 1.9+
  • Android
    Google Chrome 28+ (enabled by default since 29)
    Mozilla Firefox 24+[29]
    Opera Mobile 12+
  • Chrome OS
  • Firefox OS
  • BlackBerry 10
  • iOS
    MobileSafari/WebKit (iOS 11+)
  • Tizen 3.0

Furthermore , read about the Steps for building and deploying WebRTC solution – https://telecom.altanai.com/2014/12/04/steps-for-building-and-deploying-webrtc-solution/

TURN based media Relay

WebRTC APIs

Javascript functions  to access and process the browser media stack

getUserMedia

acquires the audio and video media (e.g., by accessing a device’s camera and microphone)

Properties

ondevicechange

Methods

enumerateDevices()
getDisplayMedia()
getSupportedConstraints()
getUserMedia()

navigator.mediaDevices.getUserMedia({ audio: true, video: true })
.then(function(stream) {
  var video = document.querySelector('video');
  // Older browsers may not have srcObject
  if ("srcObject" in video) {
    video.srcObject = stream;
  } else {
    // Avoid using this in new browsers, as it is going away.
    video.src = window.URL.createObjectURL(stream);
  }
  video.onloadedmetadata = function(e) {
    video.play();
  };
})
.catch(function(err) {
  console.log(err.name + ": " + err.message);
});

DOMException Error on getusermedia

Rejections of the returned promise are made by passing a DOMException error object to the promise’s failure handler. Possible errors are:

AbortError
Although the user and operating system both granted access to the hardware device, problem occurred which prevented the device from being used.

NotAllowedError
One or more of the requested source devices cannot be used at this time. This will happen if the browsing context is insecure( http instead of https) or if the user has specified that the current browsing instance /sessionis not permitted access to the device or has denied all access to user media devices globally.

NotFoundError
No media tracks of the type specified were found that satisfy the given constraints.

NotReadableError
Although the user granted permission to use the matching devices, a hardware error occurred at the operating system, browser, or Web page level which prevented access to the device.

OverconstrainedError
no candidate devices which met the criteria requested. string value is the name of a constraint which was not meet, and a message property containing a human-readable string explaining the problem.

exmaple conatraints :

var constraints = { video: { facingMode: (front? "user" : "environment") } };

SecurityError
User media support is disabled on the Document on which getUserMedia() was called.

TypeError
The list of constraints specified is empty, or has all constraints set to false.

RTCPeerConnection

enables audio and video communication between peers. It performs signal processing, codec handling, peer-to-peer communication, security, and bandwidth management.

Properties

canTrickleIceCandidates
connectionState
currentLocalDescription
currentRemoteDescription
getDefaultIceServers()
iceConnectionState
iceGatheringState
localDescription
onaddstream
onconnectionstatechange
ondatachannel
onicecandidate
oniceconnectionstatechange
onicegatheringstatechange
onidentityresult
onnegotiationneeded
onremovestream
onsignalingstatechange
ontrack
peerIdentity
pendingLocalDescription
pendingRemoteDescription
remoteDescription
sctp
signalingState

Methods

addIceCandidate()
addStream()
addTrack()
close()
createAnswer()
createDataChannel()
createOffer()
generateCertificate()
getConfiguration()
getIdentityAssertion()
getReceivers()
getSenders()
getStats()
getStreamById()
getTransceivers()
removeStream()
removeTrack()
restartIce()
setConfiguration()
setIdentityProvider()
setLocalDescription()
setRemoteDescription()

 signalling state transitions diagram , source W3C

RTC Signalling states

stable
There is no offer/answer exchange in progress. This is also the initial state, in which case the local and remote descriptions are empty.

have-local-offer
Local description, of type “offer”, has been successfully applied.

have-remote-offer
Remote description, of type “offer”, has been successfully applied.

have-local-pranswer
Remote description of type “offer” has been successfully applied and a local description of type “pranswer” has been successfully applied.

have-remote-pranswer
Local description of type “offer” has been successfully applied and a remote description of type “pranswer” has been successfully applied.
closed The RTCPeerConnection has been closed; its [[IsClosed]] slot is true.

RTCSDPType

offer
SDP offer.

pranswer
An RTCSdpType of pranswer indicates that a description MUST be treated as an [SDP] answer, but not a final answer.

answer
treated as an [SDP] final answer, and the offer-answer exchange MUST be considered complete. A description used as an SDP answer may be applied as a response to an SDP offer or as an update to a previously sent SDP pranswer.

rollback
canceling the current SDP negotiation and moving the SDP [SDP] offer back to what it was in the previous stable state.

RTCPeerConfiguration

Defines a set of parameters to configure how the peer-to-peer communication established via RTCPeerConnection

iceServers of type sequence
array of objects describing servers available to be used by ICE, such as STUN and TURN servers.

iceTransportPolicy of type RTCIceTransportPolicy.

bundle policy affects which media tracks are negotiated if the remote endpoint is not bundle-aware, and what ICE candidates are gathered. If the remote endpoint is bundle-aware, all media tracks and data channels are bundled onto the same transport.

  • relay
    ICE Agent uses only media relay candidates such as candidates passing through a TURN server.
  • all
    The ICE Agent can use any type of candidate when this value is specified.

bundlePolicy of type RTCBundlePolicy.
media-bundling policy to use when gathering ICE candidates.
Types :

  • balanced
    Gather ICE candidates for each media type in use (audio, video, and data). If the remote endpoint is not bundle-aware, negotiate only one audio and video track on separate transports.
  • max-compat
    Gather ICE candidates for each track. If the remote endpoint is not bundle-aware, negotiate all media tracks on separate transports.
  • max-bundle
    Gather ICE candidates for only one track. If the remote endpoint is not bundle-aware, negotiate only one media track.

rtcpMuxPolicy of type RTCRtcpMuxPolicy.
rtcp-mux policy to use when gathering ICE candidates.

certificates of type sequence
A set of certificates that the RTCPeerConnection uses to authenticate.

iceCandidatePoolSize of type octet, defaulting to 0
Size of the prefetched ICE pool as defined in [JSEP]

RTCDataChannel

allows bidirectional communication of arbitrary data between peers. It uses the same API as WebSockets and has very low latency.

getStats

allows the web application to retrieve a set of statistics about WebRTC sessions. These statistics data are being described in a separate W3C document

Peer to Peer DTMF

-tbd

Call Setup betweeb WebRTC Endpoints

updates in W3C 13 Dec , 2019

Over the years since its adoption many of the associated tech were depricated from the Webrtc based platforms and enviornments , some of which are: OAuth as a credential method for ICE servers
Negotiated RTCRtcpMuxPolicy (previously marked at risk)
voiceActivityDetection
RTCCertificate.getSupportedAlgorithms()
RTCRtpEncodingParameters: ptime, maxFrameRate, codecPayloadType, dtx, degradationPreference
RTCRtpDecodingParameters: encodings
RTCDatachannel.priority

Some of the newly added featufres include:

restartIce() method added to RTCPeerConnection
Introduced the concept of “perfect negotiation”, with an example to solve signalling races.
Implicit rollback in setRemoteDescription to solve races.
Implicit offer/answer creation in setLocalDescription to solve races.

References :

WebRTC 1.0: Real-time Communication Between Browsers – W3C Candidate Recommendation 13 December 2019https://www.w3.org/TR/webrtc/

WebRTC Stack Architecture and layers

WebRTC stands for Web Real-Time Communications and introduces a real-time media framework in the browser core alongside associated JavaScript APIs for controlling the media frame and HTML5 tags for displaying.

If you are new to WebRTC , read what is WebRTC ? From a technical point of view, WebRTC will hide all the complexity of real-time media behind a very simple JavaScript API. 

Codec Confusion :

Video Codecs

Currently VP8 is the codec of choice since it is royalty-free. In mobility today, the codec of choice is h264. H264 is not royalty-free. But it is native in most mobile handsets due to its high performance.

Audio Codecs

Opus is a lossy audio compression format developed by the Internet Engineering Task Force (IETF) targeting a broad range of interactive real-time applications over the Internet, from speech to music. As an open format standardized through RFC 6716, a reference implementation is provided under the 3-clause BSD license. All known software patents Which cover Opus are licensed under royalty-free terms.

G.711 is an ITU (International Telecommunications Union) standard for  audio compression. It is primarily used in telephony. The standard was released in 1972. It is the required standard in many voice-based systems  and technologies, for example in H.320 and H.323 specifications.
Speex is a patent-free audio compression format designed for speech and also  a free software speech codec that is used in VoIP applications and podcasts. Some consider Speex obsolete, with Opus as its official successor, but since
significant content is out there using Speex, it will not disappear anytime soon.

G.722 is an ITU standard 7 kHz Wideband audio codec operating at 48, 56 and 64 kbit/s. It was approved by ITU-T in 1988. G722 provides improved speech quality due to a wider speech bandwidth of up to 50-7000 Hz compared to G.711 of 300–3400 Hz.

AMR-WB Adaptive Multi-rate Wideband is a patented wideband speech coding standard that provides improved speech quality due to a wider speech bandwidth of 50–7000 Hz. Its data rate is between 6-12 kbit/s, and the codec is generally available on mobile phones.

Architecture :

WebRTC offers web application developers the ability to write rich, realtime multimedia applications (think video chat) on the web, without requiring plugins, downloads or installs. It’s purpose is to help build a strong RTC platform that works across multiple web browsers, across multiple platforms.

WebRTCpublicdiagramforwebsite

Web API – An API to be used by third-party developers for developing web-based video chat-like applications.

WebRTC Native C++ API – An API layer that enables browser makers to easily implement the Web API proposal

Transport / Session – The session components are built by re-using components from libjingle, without using or requiring the XMPP/jingle protocol.

RTP Stack – A network stack for RTP, the Real-Time Protocol.

STUN/ICE – A component allowing calls to use the STUN and ICE mechanisms to establish connections across various types of networks.

Session Management – An abstracted session layer, allowing for call setup and management layer. This leaves the protocol implementation decision to the application developer.

VoiceEngine – VoiceEngine is a framework for the audio media chain, from sound card to the network.

iSAC / iLBC / Opus

iSAC: A wideband and super wideband audio codec for VoIP and streaming audio. iSAC uses 16 kHz or 32 kHz sampling frequency with an adaptive and variable bit rate of 12 to 52 kbps.

iLBC: A narrowband speech codec for VoIP and streaming audio. Uses 8 kHz sampling frequency with a bitrate of 15.2 kbps for 20ms frames and 13.33 kbps for 30ms frames. Defined by IETF RFCs 3951 and 3952.

Opus: Supports constant and variable bitrate encoding from 6 kbit/s to 510 kbit/s, frame sizes from 2.5 ms to 60 ms, and various sampling rates from 8 kHz (with 4 kHz bandwidth) to 48 kHz (with 20 kHz bandwidth, where the entire hearing range of the human auditory system can be reproduced). Defined by IETF RFC 6176.

NetEQ for Voice– A dynamic jitter buffer and error concealment algorithm used for concealing the negative effects of network jitter and packet loss. Keeps latency as low as possible while maintaining the highest voice quality.

Acoustic Echo Canceler (AEC) – The Acoustic Echo Canceler is a software-based signal processing component that removes, in real-time, the acoustic echo resulting from the voice being played out coming into the active microphone.

Noise Reduction (NR) -The Noise Reduction component is a software-based signal processing component that removes certain types of background noise usually associated with VoIP. (Hiss, fan noise, etc…)

Video Engine – VideoEngine is a framework video media chain for video, from the camera to the network, and from network to the screen.

VP8  – Video codec from the WebM Project. Well suited for RTC as it is designed for low latency.

Video Jitter Buffer – Dynamic Jitter Buffer for video. Helps conceal the effects of jitter and packet loss on overall video quality.
Image enhancements -For example, removes video noise from the image capture by the webcam.

W3C contribution


w3c

  • Media Stream Functions

API for connecting processing functions to media devices and network connections, including media manipulation functions.

  • Audio Stream Functions

An extension of the Media Stream Functions to process audio streams (e.g. automatic gain control, mute functions and echo cancellation).

  • Video Stream Functions

An extension of the Media Stream Functions to process video streams (e.g. bandwidth limiting, image manipulation or “video mute“).

  • Functional Component 

 API to query presence of WebRTC components in an implementation, instantiate them and connect them to media streams.

  • P2P Connection Functions

API functions to support establishing signalling protocol-agnostic peer-to-peer connections between Web browsers

  • API specification Availability

WebRTC 1.0: Real-time Communication Between Browsers –  Draft 3 June 2013 available

  • Implementation Library: WebRTC Native APIs

Media Capture and Streams – Draft 16 May 2013

  • Supported by Chrome , Firefox, Opera in desktop of all OS ( Linux, Windows , Mac )
  • Supported by Chrome , Firefox  in Mobile browsers ( android )

IETF contribution

ietf

Communication model

Security model

Firewall and NAT traversal

Media functions

Functionality such as media codecs, security algorithms, etc.,

Media formats

Transport of non media data between clients

Input to W3C for APIs development

Interworking with legacy VoIP equipment

WG RFC   Date

  • draft-ietf-rtcweb-audio-02      2013-08-02
  • draft-ietf-rtcweb-data-channel-05      2013-07-15
  • draft-ietf-rtcweb-data-protocol-00      2013-07-15
  • draft-ietf-rtcweb-jsep-03      2013-02-27
  • draft-ietf-rtcweb-overview-07      2013-08-14
  • draft-ietf-rtcweb-rtp-usage-07     2013-07-15
  • draft-ietf-rtcweb-security-05      2013-07-15
  • draft-ietf-rtcweb-security-arch-07      2013-07-15
  • draft-ietf-rtcweb-transports-00      2013-08-19
  • draft-ietf-rtcweb-use-cases-and-reqs-11      2013-06-27
  • Plus over 20 discussion RFC drafts

What will be the outcome of WebRTC Adoption?

In simple words, it’s a phenomenal change in decentralizing communication platforms from proprietary vendors who heavily depended on patented and royalty bound technologies and protocols.  It will revolutionize internet telephony.  Also it will emerge to be platform-independent ( ie any browser, any desktop operating system any mobile Operating system ).

WebRTC allows anybody to introduce real-time communication to their web page as simple as introducing a table.

Read More about webRTC business benefits


update 2020 – This article was written very early in 2013 while WebRTC was being standardised and not as widely adopted since the inception of WebRTC began in 2012.

There are many more articles written after that to explain and emphasize the detailing and application of WebRTC. List of these is below :

For SIP IMS and WebRTC

Read about STUN and TURN which form a crtical part of any webrtc based communication system

Security of WebRTC based CaaS and CPaaS

WebRTC APIs