ICE Warm-up and Finding The Right m-line Using WebRTC Transceivers

 Highlights

 Impact on my application

 Standardization status

 Details

 

 Highlights

At the API level is it difficult to figure out which SDP “m=” media line is used for which track. This is mainly of importance in cases where an application is sending multiple streams and m-lines are being added and/or removed during the call.

Example:

A and B have a call connected with voice and video. So far, there are 2 bidirectional m-lines, one for audio and one for video.

B now adds a send-only m-line for sending an additional source of video.

A now has one bidirectional video m-line and one receive only video m-line.

Due to congestion A stops sending video in an attempt to improve audio quality, so he now has 2 receive only video m-lines. Later during the call, as network conditions improve, A wants to resume sending video. To do so the browser will need to modify one of the m-lines. If the application cares which m-line should be modified to be both send and receive, it will need to activate the correct RTCRtpSender object, but that assumes there is one, a bad assumption in this case because the second m-line never had a sender (from A’s perspective).

Things can get even more complex if there are more m-lines for different resolutions.

An RTCRtpTransceiver consists of a pair of RTCRtpSender and RTCRtpReceiver objects and is associated with a specific m-line. By explicitly creating the WebRTC transceiver object, and thus an m-line, it becomes possible to control precisely the sender and/or receiver activity on that m-line.

There are other reasons for adding the new WebRTC transceiver objects as well, one of which is the ability to do ICE warm-up, starting the ICE process earlier in order to reduce the time needed to get media flowing later since the connection will already have been established. This is possible since the creation of a WebRTC Transceiver also creates an m-line which is mandatory for starting ICE.

The fact that replaceTrack() was added a while ago and is already implemented allows application code to replace the empty track with a real one later without renegotiating.

 

 Impact on my application

This is an optional new capability.

 

 Standardization status

This is now in the specification.

 

 Details

For quite a while there was a desire ,in the APIs, to hide the fact that SDP media lines are being used in the implementations.  By default m-lines (media lines) in SDP are bidirectional.  This comes from the original SIP design that was intended to replace the traditional bidirectional audio call.  For WebRTC this has been confusing at best, since the APIs assume that unidirectional tracks are added, removed, or replaced even though implementations are expected to be economical in their use of m-lines to support the tracks that are being sent by combining different-direction tracks of the same type into bidirectional m-lines wherever possible.

Now that we have RTCRtpSenders and RTCRtpReceivers for the tracks sent over a Peer Connection, it is becoming increasingly important to figure out exactly which track’s transmission is being modified.

To assist application developers who want low-level control over their RTP media streams (the tracks sent over the Peer Connection), the specification has now introduced the concept of an RTCRtpTransceiver, an object that holds a paired RTCRtpSender and RTCRtpReceiver, and a createRtpTransceiver() method to create them.  Calling this new method will create a brand new m-line in the SDP, effectively creating a new local track and a new remote track.  It is expected that the appropriate events will be triggered in this case.  It is now conceivable that the definition of addTrack() could be given in terms of a combination of createRtpTransceiver() and replaceTrack(). It is expected that this method will be used something like this:

navigator.mediaDevices.getUserMedia("audio")
   .then(function(myStream) {
     var myTrack = myStream.getTracks[0];
     return createRtpTransceiver(myTrack);
   })
   .then(function(transceiver) {
     // then, just for fun, replace this track with another one.
     transceiver.sender.replaceTrack(myOtherTrack);
   });

There are two other really nice properties here.

First, the createRtpTransceiver() method also takes an init object whose receive attribute will now replace the “offerToReceiveAudio” and “offerToReceiveVideo” properties in the RTCPeerConnection config object.  There is a corresponding send attribute as well.  The default value of ‘true’ for these attributes means that, by default, each browser will offer to send and receive media for each m-line.

Second, the creation of an RTCRtpTransceiver (which creates an m-line) allows for ICE and DTLS processing to start, in advance of actually having media to send.  This provides the long-desired “warm-up” for those time-consuming processes.