Lorenzo Miniero

WebRTC is not always all about video, and actually applications like Clubhouse, Twitter Spaces and Reddit Talk have proven you can create engaging scenarios also using audio alone. This presentation will give a brief overview on how you can build a similar kind of application using the Janus WebRTC Server, by focusing on the broadcasting and scalability aspects.

While it may seem weird to think of WebRTC applications without video, that shouldn't be the case. In fact, the success of podcasts or platforms like Audible are proof of the fact there's a lot of people definitely interested in consuming content that's only made of audio. When you think of applications like the aforementioned Clubhouse, Twitter Spaces and Reddit Talk, it's clear that the same can be said when there's potential engagement as well, and where the actual conversation becomes more important than any visual aid that may come with it.

This made me think of how one could build a similar application using open source tools, and the Janus WebRTC Server in particular (which as a matter of fact is apparently part of the Twitter Spaces stack itself). Specifically, in the presentation I'll focus mostly on what building blocks you can start from, and how you can build a larger application on top of them in a scalable way (with or without WebRTC as a distribution channel), taking advantage of functionality Janus ships out of the box (like the RTP forwarders that were presented at FOSDEM a couple of years ago), with a few considerations on how the current audio support in WebRTC can be of help.

Spoiler alert: it's a clickbait title, I'm not really a rockstar! :-)

That said, it's usually not that well known, but Linux has an amazing ecosystem for creating music, whatever genre you like, with a ton of fantastic applications to tackle different stages of the music production. In this presentation, I'll share what my experience has been in this past couple of years, talking about the different applications I've used to compose, orchestrate, process, mix, master and so on, with real or virtual instruments, and how that allowed me to publish a few albums just for the fun of it.

The music production ecosystem on Linux is huge, and yet it's unortunately not that well known, possibly because it's seen more as "just" software for entertainment purposes rather than something much more serious and valuable than that. JACK and, more recently, Pipewire provide an excellent environment for working with low latency music, and there's a ton of applications that can help at different stages of music production. There's software for composing and/or orchestrating your music, using music sheets, piano rolls or patterns; software for processing your real instruments and software for simulating virtual instruments, including complex syntethizers; DAWs to work on music projects in a more organized way; a plethora of plugins that can be used in different applications to customize the way each track needs to be processed, and mix/master your work; and so on and so forth. And while I mostly like working on "rock" and orchestral music (and so the talk will focus a bit more on software that helped me do that), there's so much software out there to help with basically any genre that you may be interested in, from classical music to dub and beyond.

I'll also talk a bit of the community aspects of it (for instance introducing the LinuxMusicians forum or the "Made with Ardour" topic that allowed me to meet so many other people using FOSS to create and share their music), and share a few notes about how you can indeed share your music with the world (or at least how I did it!).

The broadcasting industry has for years been dominated by a specific set of technologies (RTMP, HLS, etc.) that, while effective, suffer from high latencies and so are not always a good option. The IETF has started to look into WebRTC for that, starting from ingestion using the WHIP protocol. This presentation will introduce WHIP, some existing implementations, and how this could be expanded to distribute streams to a wide audience via WebRTC as well.

The broadcasting industry has for years been dominated by a well known and specific set of technologies (RTMP, HLS, etc.). These solutions do work and are widely deployed, taking advantage of CDNs for distributing streams to a potentially very large audience in an effective way. That said, they do suffer from some limitations, the stronger one being a high latency that cannot really be reduced, if not slightly. Considering there are scenarios where a very low latency may be desired, WebRTC has started to gain some traction, due to its real-time nature that, although originally conceived for conversational scenarios, has actually often used for streaming purposes as well.

When it comes to broadcasting, there are two main aspects to take into account: ingestion (providing a stream to broadcast) and distribution (feeding the stream to an audience). The IETF has started addressing ingestion in the WISH Working Group, with a draft describing the WHIP protocol. This presentation will introduce what WHIP is, its requirements, and how it's supposed to work, plus some details on existing implementations. It will then move to some considerations on how distribution could be handled via WebRTC as well, possibly at scale, using the open source Janus WebRTC Server as a reference implementation.

Last year, the world changed, and musicians were among those that were hit the most, and music lovers with them. Can WebRTC help, here? This presentation will try to cover different areas where it could, and in some cases already is.

WebRTC is well known and widely deployed as a communication technology. Most of the times, though, we hear about it in the context of those same boring and traditional use cases that we've all seen thousands of times: conferencing, e-learning, contact centers, webinars, etc. As a very amateur and hobbyist musician, I've always wondered whether WebRTC could do more to support the arts instead: the answer is obviously a strong yes, and I've always been pleasantly surprised when I've seen it used to that effect.

Music is indeed one of the areas where WebRTC could do so much more. Since the pandemic started, musicians all over the world were affected in different ways: no more concerts, or meeting the fans, or jam session, and so on and so forth, basically making it much harder for them to earn a living; the same could be said for those who simply love to listen to music, attend concerts and engage with musicians. What can WebRTC do to help, here, and what are the technical challenges that may need to be overcome?

This presentation will try to cover a few different use cases, ranging from plain streaming/broadcasting (and how WebRTC audio could be improved), tinkering with remote music equipment, playing with friends online and so on. I'll also share some of the experiments I've carried on myself just for fun in this area, the challenges I've identified, and what I'd like to work on next, possibly using the Janus WebRTC Server as a support.

This talk will cover several aspects related to Janus as a WebRTC "enabler" for non-WebRTC applications. In particular, it will focus on the RTP management in Janus, namely how to use it as input/output to interact with external applications for different use cases.

Janus is an open source and general purpose WebRTC server. Its modular nature makes it easy to implement heterogeneous multimedia applications based on WebRTC, whether it's for conferencing, talking to a SIP infrastructure, broadcast a stream or interacting with an IoT device. One of its strongest points is the ability to seamlessly involve plain RTP within the context of a WebRTC communication, whether it's for feeding media to a WebRTC endpoint, or use a WebRTC stream somewhere else: this makes Janus a good WebRTC "enabler" for platforms that may not be aware of, or be compliant with, the WebRTC specification.

This talk will cover the different features Janus provides implementers with, when it comes to RTP. In particular, it will introduce the Streaming plugin (RTP- and RTSP-to-WebRTC broadcaster), the SIP/NoSIP plugins (for legacy VoIP integration) and the so-called RTP forwarders (to relay media coming from WebRTC sources as plain RTP to external endpoints), and explain how these different components can be used together in different scenarios, whether it's just to increase scalability or to implement a complex and rich multimedia application. Besides, it will spend a few words on how simulcast, SRTP and recordings can be part of the picture.

Janus is written in C, and so are its plugins. That said, we recently implemented a Janus Lua plugin, that allows developers to use Lua scripts to drive the media/application logic instead.

Janus is a general purpose and modular WebRTC server. It can be easily extended by writing new plugins, whether for new transports for the Janus API, or for handling the media related to a PeerConnection in a custom way. Out of the box, Janus ships several plugins implementing different features (e.g., an SFU, an audio MCU, a SIP gateway, etc.), but developers interested in adding custom functionality can write their own plugins to handle media their own way. That said, Janus is written in C, which means that plugins are typically expected to be written in C as well. This can be an impediment for those not proficient with C as a programming language. As such, we recently released a new Janus plugin that acts as a bridge between the C code and Lua scripts: developers can write their logic in Lua, while leaving the media manipulation and routing to C helper methods that are exposed as Lua functions scripts can take advantage of. This presentation will introduce this plugin, how it works, and how it can be used to write custom logic with different Lua scripts: a couple of real use cases will be presented as a proof of concept.

This presentation introduces a new modular mechanism for implementing live state and event notifications within the context of Janus WebRTC server instances, for the purpose of allowing administrators and developers to monitor and handle WebRTC sessions more effectively.

Janus is a general purpose open source WebRTC server and gateway. It allows you to implement heterogeneous and complex WebRTC multimedia applications using the different functionality it provides as modules, and can interact with legacy technologies as well in the process (e.g., SIP or RTSP). An introduction to Janus was made ad Fosdem '16. This new presentation is aimed to address a new feature recently added to Janus, and currently being worked on in a separate branch that will soon be merged.

Specifically, a modular mechanism has been designed and implemented to allow modules to receive asynchronous and live events of several different types from Janus and its plugins, in order to then handle them accordingly. Notifications can be about whatever is happening in a Janus instance, e.g., Janus sessions/handles and their states, PeerConnections and their lifecycle (SDP exchanges, ICE and DTLS state changes, etc.), application plugin internals and related events, and so on. These new modules can then decide whether to just forward these events, or a subset of them, to an external backend for further processing, or process them themselves. Use case examples are simple debugging, live monitoring, troubleshooting, storing on DBs or other systems in a structured way, CDRs, etc. Within the context of Janus, this is particularly useful as so far the only API available to administrators has been a query/response API to monitor the WebRTC internals of Janus itself and a few other aspects, which limited the scope of what admins could do to effectively manage a Janus instance.

At the time of writing, a single module implementing this mechanism exists, one we implemented ourselves as a proof of concept and that simply forwards all events as JSON objects to an external web server. In the future more modules will be implemented to cover more practical scenarios, hopefully by third-party developers as well.

Janus is an open source WebRTC Gateway developed by Meetecho conceived to be a general purpose one. As such, it doesn't provide any functionality per se other than implementing the means to set up a WebRTC media communication with a browser, exchanging JSON messages with it, and relaying RTP/RTCP and messages between browsers and the server-side application logic they're attached to.

Any specific feature/application is provided by server side plugins, that browsers can then contact via the gateway to take advantage of the functionality they provide. Example of such plugins can be implementations of applications like echo tests, conference bridges, media recorders, SIP gateways and the like. A few examples describing how these plugins can be used as "bricks" to build a more comprehensive and complex application will be provided. Some references to existing usages of Janus and third-party imlementations will be presented too, as well as some notes on future plans and next steps.