Jehan Monnier

Existing audio/video conferencing solutions of type “Multipoint control unit” or “Mediamixing mixer”, have proved to be costly in terms of resources in the case of video processing, due to the algorithm complexity of video codecs. This constraint leaded to the emergence of new solutions transmitting a useful subset of the streams received from all the participants, to each participant. For example: only the video of the actively talking participant is dispatched to each participants, together with some very low resolution video streams of the other participants. This technology is referred to as “Selective Forwarding Unit” (SFU) or “Selective Forwarding Middlebox” (SFM). These solutions open the path to conference sessions managed by a server that does not decode the media content. The main benefit of a Selective Forwarding Unit is that it is capable of receiving multiple media streams and then decide which of these media streams should be sent to which participant. That way it is technically possible to enhance the number of participants to a group call.

Linphone is likely to be the first open source SIP-based conferencing solution powered by a modern SFU server. As the server solution already used for Instant Messaging features was already compatible with open standards for the function of managing group chatrooms and their participants., the idea was to enhance this component with media capabilities. Therefore the Linphone team developed a SFU algorithm on top of mediastreamer2 and ortp (Linphone’s media processing libraries), inside of a new software component called “MS-SFU”.

This talk will focus on the following topics:

• compatibility of the conference establishment mechanism with SIP and the RFC 4579 (SIP Call Control - Conferencing for User Agents)

• stream selection performed by the SFU based on the current audio level of participants

• implemented standards to allow the communication between the Linphone's library and the Flexisip conference server in SFU mode

• how this technology paves the way for a future major innovation: end-to-end encryption for real-time audio and video conferences

Current developments in the field of quantum computer science bring a growing threat against the existing cryptographic algorithms used today, for example in secure Voice over IP and instant messaging applications. Although such a quantum computer has not yet been officially announced, some governments recommends protecting data against this type of attack by 2030. The encrypted data shared today could be stored and decrypted soon thanks to this breakthrough innovation.

The National Institute of Standards and Technology (NIST) launched in 2017 an international competition to standardise "post quantum algorithms". Such algorithms are expected to be resilient to an attack made by a generalist post quantum computer. They are meant to replace in the long term the algorithms that are used today in many secure protocols relying on cryptographic key exchange mechanisms. The Linphone application is most likely the first open source communication software in the world to have implemented the NIST finalist algorithm in the encryption key category, CRYSTALS-Kyber, as of today. One of the key steps: the development of a modified version of the standardized ZRTP encryption protocol.

A few challenges we have taken:

• Reach the same level of effectiveness even if cryptographic keys are much larger

• Remain resilient to classic attacks

• Be interoperable with encryption features offered by previous versions

The different steps that have been carried out:

• Integration of KEM in ZRTP protocol: creation of a modified version of ZRTP that accepts a key exchange algorithm of the type of Key Encapsulation Mechanism

• Hybridation : conception of an encryption engine combining a classic (EC)DH and a post quantum encryption. Modification of the ZRTP protocol so that it can negotiate two different key exchange algorithms at the same time and securely combine results.

• Fragmentation: addition of a mechanism to fragment ZRTP packets

• Integration in Linphone of this new ZRTP library with post quantum capacities and of configuration settings to activate/deactivate the post quantum mode

• Building of performance tests