Keeping It Realtime

We believe WebRTC is one of the most important technologies to hit the web in many years. There is a significant opportunity for WebRTC to help deliver on the promise of the open web—but for communication and collaboration.

In several of our colleague Henrik Joreteg's talks the past couple years, he's said, "WebRTC needs more Open Web hackers!" It's very true.

But WebRTC is much more complicated than other browser APIs that web developers deal with.

It seems, from our vantage point, like a large portion of the users of WebRTC have not been Open Web hackers.

Yesterday Fippo talked about applying the principles of the open web to realtime communication with WebRTC. Consider this post the first installment of providing in-depth descriptions of what that means in practice.

Where codecs fit in

A key component of realtime audio and video systems (like Talky) consists of methods for encoding the light waves and sound waves that our eyes and ears can understand into digital information that computers can process. Such a method is called a codec (shorthand for code/decode).

Because there are many different audio codecs and video codecs, if we want interoperability, then it's important for different software implementations to "speak the same language" by having at least one codec in common. In the parlance of industry standards, such a codec is specified as "mandatory to implement" or MTI (often an MTI codec is a lowest common denominator, but software can use better codecs if available - a model that has also worked well for cipher suites in SSL/TLS and other technologies.)

There's a lot of talk about this topic of "the web we want," and a lot of it has focused around WebRTC lately.

I have been working with WebRTC since mid-2012, both in the Chrome and Firefox browsers, as well as the native webrtc.org library. So far I have filed more than sixty issues in the WebRTC project issue tracker and the Chrome bugtracker. I'm especially proud that I've crashed the production version of Chrome eight times.

I am among the top non-Google people to file WebRTC issues. And I managed to get quite a few of them fixed, too. I visited Google's Stockholm office in September and had a conversation with the team there about how I use the issue tracker and how that process works. Full disclosure: I got a t-shirt (even though it turned out to be too large). And I even started reviewing the Chrome WebRTC release notes before they're sent out.

Justin Uberti's description of WebRTC as a "project to bring realtime communication to the open web platform" is still the vision I cling to.

Last week, Google Chrome started to support the next generation video codec VP9 for WebRTC (which is highly experimental in the developer version and you need to enable it, so the ETA for this is probably going to be mid-2015). That is good news because VP9 is going to offer better video quality at the same bandwidth (another way to look at it is that VP9 gives you the same quality at lower bandwidth, although at the expense of computational power, i.e. your mobile device gets warmer).

I immediately wanted to implement and test the feature, if only so I could add Chrome 41 to the long list of versions that I managed to crash. However, this turned out to be harder than it initially seemed.

Let me try to explain the issue with an example of a videochat that has three participants, two of which support VP9: Suppose that those clients use an XMPP MUC for having the conference, supported by an SFU like the Jitsi videobridge (which is what we do in the next version of Talky). The browsers announce their capabilities ("I implement VP8," "I implement VP9," a Firefox browser might even do H.264 and VP8) through Entity Capabilites when joining the MUC.

So the first two browsers know they both implement VP8 and VP9 and use that list of codecs when calling the RtpSenders .send method for the first time. That RtpSender object is another improvement that came from ORTC. If everything goes well, they will enjoy the superior quality of VP9.

Microsoft recently announced they will support Object RTC and now everyone is talking about ORTC and how they will support it.

What is this all about and what is ORTC anyway?

In essence, ORTC is an alternative API for WebRTC. It is object-oriented and protects developers from all that ugly Session Description Protocol (SDP) madness. Some people call it WebRTC 1.1, or maybe WebRTC 2.0.

So... will &yet (and Otalk) (and Talky) support ORTC? Of course!

As you probably know, we run Talky, a free videochat service powered by WebRTC. Since WebRTC is still evolving quickly, we add new features to Talky roughly every two weeks. So far, this has required manual testing in Chrome, Opera, and Firefox each time to verify that the deployed changes are working. Since the goal of any deploy is to avoid breaking the system, each time we make a change we run it through a post-commit set of unit tests, as well as an integration test using a browser test-runner script as outlined in this post.

All that manual testing is pretty old-fashioned, though. Since WebRTC is supposed to be for the web, we decided it was time to apply modern web testing methods to the problem.

The trigger was reading two blog posts published recently by Patrik Höglund of the Google WebRTC team, describing how they do automated interop testing between Chrome and Firefox. This motivated me to spend some time on the post-deploy process of testing we do for Talky. The result is now available on github.

Let's review how Talky works and what we need to test. Basically we need to verify that two browsers can connect to our signaling service and establish a direct connection. The test consists of three simple steps:

More and more application developers have come to rely on platform-as-a-service providers for building and scaling software.

WebRTC's complexity makes it ripe for this kind of approach, so it's no surprise that so many early WebRTC companies have been platform service providers. Unfortunately for customers, the nascent Rent-Your-WebRTC-Solution market has proven pretty unstable.

News came yesterday that yet another provider of WebRTC hosted services—in this case, Requestec—has been acquired. We've seen this movie before with Snapchat's acquisition of AddLive and we'll probably see it again, maybe multiple times.

At &yet, we've been working steadily at creating open source software and approaches to infrastructure to help our clients avoid the volatile WebRTC rental market.

About a year ago, our friends at TokBox published a blog post entitled WebRTC and Signaling: What Two Years Has Taught Us. It's a great overview of their experience with technologies like SIP and XMPP for stitching together WebRTC endpoints. And we couldn't agree more with their observation that trying to settle on one signaling method for WebRTC would have been even more contentious than the endless video codec debates. However, our experience with XMPP differs enough from theirs that we thought it would be good to set our thoughts down in writing.

First, we're always a bit suspicious when someone says they abandoned XMPP because it didn't scale for them. As a general rule, protocols don't scale - implementations do. We know of several XMPP server implementations that can route, distribute, and deliver tens of thousands of messages per second without breaking a sweat. After all, that's what a messaging server is designed to do. Sure, some server implementations don't scale up that high; the answer is to use a better server. We're partial to Prosody, but Tigase and a few others are well worth researching, too.

Second, pure messaging is the easy part. The hard part is defining the payloads for those messages to build out a wide variety of features and functionality. What we like best about the XMPP community is that over the last 15 years they have defined just about everything you need for a modern collaboration system: presence, contact lists, 1:1 messaging, group messaging, pubsub notifications, service discovery, device capabilities, strong authentication and encryption, audio/video session negotiation, file transfer, you name it. Why put a lot of work into recreating those wheels when they're ready-made for you?

An added benefit of having so many building blocks is that it's straightforward to put them all together in productive ways. For example, XMPP includes extensions for both multi-user chat ("MUC") and multimedia session management (Jingle). If we need multiparty signaling for video conferencing, we can easily combine the two to create what we need. Plus we get a number of advanced solutions for free this way, since MUC includes in-room presence along with a helpful authorization model and Jingle supports helpful features like renegotiation and file transfer. Not to mention that the ability to communicate device capabilities in XMPP enables us to avoid monstrosities like SDP bundling.

We are very excited to announce that the iOS app for Talky is now available for download on the App Store.

(Can I get a woohoo?)

Take a quick look

A few months ago, WebRTC agitator and yeti-friend Chris Koehnke wrote an excellent blog post explaining that browser-based videochat won't work 100% of the time without a little help from something called a "TURN Server". As he put it:

If you're going to launch a WebRTC powered service for financial 
gain, then you need to have done everything within your power to
ensure it works reliably across as many cases as possible.

Chris was satisfied when a few simple tests worked and stopped after that. Well, he skipped the next step. But that's reasonable because he was probably bored already (does anyone get excited about TURN servers?) and he doesn't run a WebRTC powered service himself.

The next step is looking for cases where things did not work and figure out what we can do about it. But hey, we run a WebRTC service called Talky and connection failures are frustrating, so we decided to dig a little deeper.