On September 9 to 10th there was a W3C interim meeting with 2 major areas on the surgery bed:
- PeerConnection media negotiation
- Local media
In each of these areas there were several topics discussed covering better control over media negotiation, media priority settings, adding media streams to an SDP offer and details of local media capturing for screen sharing.
In the details section below we provide the highlights for each item discussed. In our next updates we will be covering some of these items in greater detail.
There are new functionalities and changes related to media control, negotiation and capture.
Some items are in advanced standardization stage while others are not decided yet. Read the Details section for specific information on each item.
PeerConnection media negotiation items
W3C decided to allow an API mechanism to get the list of negotiated codecs and reorder/subset the list in order to select a different one, both for RTCRtpSenders and RTCRtpReceivers. The details are being worked out.
There was a fairly major decision to create a new addMedia() (name TBD) method on a Peer Connection that explicitly creates new RTCRtpSenders and Receivers, essentially creating new m-lines in the SDP. The arguments on this method would allow for controlling which direction (send, recv, or sendrecv) is active. This approach will also let WebRTC to provide a much-requested capability, that of “warming up” the Peer Connection by getting ICE and DTLS processing done in advance of actually having media to send.
An item closely related to addMedia is “early media”. Today, for media that arrives before it is signalled, if it is inconsistent with the current offer/answer it will be dropped, but if the media is otherwise acceptable and arrives before the answer, we expect this will be addressed by use of the new addMedia() (see above) method to create an RTCRtpReceiver to receive it.
WebRTC will enable the sending of client-to-mixer audio levels in SDP and the sending of CSRCs from mixer to client to identify who is speaking. We will also likely add mixer-to-client audio levels as well.
There was a long conversation about privacy of local and behind-VPN IP addresses. There is much to say here, but the short answer is that:
- The browsers have put some protections in place
- Those protections currently would prevent the local-network P2P use cases that are needed for demos and gaming
- There may be some clever technical work the browsers can do to re-enable many of those use cases.
There will be a new RtpEncodingParameter that permits apps to give a preference for maintaining frame rate or resolution as bandwidth degrades. Think of the ability to prioritize sharpness over motion so for sessions that involve more motion of elements in the video you could request to maintain frame-rate while decreasing resolution while for more static sessions request to maintain resolution while reducing frame-rate.
Speed up connection – There will now be an ability to set a non-zero ICE candidate pool size, which will have the effect of allowing ICE to pre-gather addresses, before the offer.
W3C will be adding a new connectionState attribute on Peer Connections which combines state information for both DTLS and ICE processing into a single, simple “connection state for dummies” variable that applications can use. We may eventually deprecate iceState (after 1.0), encouraging developers to check the detailed transport objects if they need more info.
There was a major discussion about simulcast, but in the end no consensus on how to signal/implement it. The issues are complex but based on the fact that the way the browser should signal simulcast in SDP hasn’t yet been finalized at the IETF.
W3C will consider getting codec capabilities pre-negotiation, with the understanding that it is not until createOffer that resources are reserved (and thus guaranteed to be available). This would allow only showing the “join conference” button if the desired codec were available. Selection of codecs pre-negotiation will also be considered.
WebRTC will provide more readonly info about ICE transports in the relevant API objects. As part of this discussion, W3C generally agreed that largely static information will go into the objects and largely dynamic info will go into the getStats interface.
WebRTC will provide more readonly info about the RTP underlying an RTCRtpSender, such as header extension values and the various ssrc values (core, rtx, fec).
An outstanding question regarding replaceTrack was how it should work when negotiation was needed. The decision was that if a negotiation is needed in order to complete the replaceTrack operation the promise will be rejected with an error (rather than automagically triggering onnegotiationneeded handlers and then resolving when those have completed).
W3C has confirmed a decision to have new pc.current/pendingLocal/RemoteDescription variables to augment (and possibly replace) the current pc.Local/RemoteDescription variables.
Local media related items
Screen sharing (screen capture, actually) work is progressing nicely. Discussions recently have been primarily about how to do consent/permissions and how to deal with shared applications that can switch focus away from the user-shared window, such as Power Point going into presentation mode.
Capturing from an element (such as video) is also making progress, but there are some details around capture frequency, how to deal with paused input media, etc. that are being worked through.
The discussion about whether constraints should be registered in an IANA registry remains contentious and unresolved. The chairs will organize a formal, by-member-organization vote.