WebRTC Video Sharpness vs. Motion Prioritization

 Highlights

 Impact on my application

 Standardization status

 Details

 

 Highlights

When things start to go bad on a video call due to bandwidth limitations and congestion there are various actions that can be taken. The ruthless one can be to turn off video altogether and stick to voice only, but before going that route it is possible to reduce the bandwidth a call consumes. A common way to do this is through changes in resolution or frame rate.

In essence, the higher the frame rate is, the better the user experience is in the event of a lot of motion in the call; hence, reducing frame rate can hurt the user experience in cases where there is motion.

On the other side of the equation we have the resolution. Naturally, the higher it is, the sharper the video is and vice versa.

Many video applications make these decisions automatically, but some allow the user (or the application developer in case of an SDK) to set his preference between prioritizing for frame rate vs. resolution.

WebRTC plays with frame rate and resolution based on network conditions automatically, but developers wanted to have the option to set priorities between these two contradicting options. W3C has just given this option to developers through a new API.

 

 Impact on my application

Application developers may now prioritize between frame rate and resolution according to their application needs.

 

 Standardization status

Was added to the specification.

 

 Details

Given the requirement presented in the Highlights section there is a new RtpEncodingParameter called “degradationPreference” that permits applications to give a preference for maintaining frame rate or resolution as bandwidth degrades.

The possible values for it are:

  • maintain-framerate:  Degrade resolution in order to maintain frame rate
  • maintain-resolution:  Degrade frame rate in order to maintain resolution
  • Balanced:  Degrade both frame rate and resolution in a balanced manner

Since the specific control over both frame rate and resolution may be managed by either the browser or the codec itself, depending upon the codec, this parameter ensures that if there is a choice which to degrade, frame rate or resolution, as bandwidth decreases the right thing will happen.

It is important to note that even if, for example, priority was given to resolution, it doesn’t mean that when bandwidth decreases resolution will remain unchanged while frame rate will be reduced to unacceptable levels. It should be assumed that WebRTC implementations will use automatic algorithms for setting the proper frame rate and resolution given the existing bandwidth, but the algorithm will yield different results based on the priority given through this new API.