Today I went up to San Francisco to attend Kranky Geek’s WebRTC Show 2015, held at Google’s office. There were handful of speakers and the topics ranged from tips and tricks for a good WebRTC user experience, to more technical details of how to resume sessions and take advantage of the emerging platform.

Here is a recap of several of the sessions I found interesting from a beginner’s perspective.

User Experience with WebRTC

Arin Sime from WebRTC.ventures started the afternoon off by talking about how important the user experience is when using WebRTC.

Customers and users of our apps don’t care about the technology. Developers care about the technology. Users care about the experience in the application.

WebRTC provides interactions like block allow permission out of the box. The red circle in the tab alerts the user that the video/audio is live and reminds user which tab is using the camera. It also alerts them when they return to your application so they aren’t caught off guard having the camera automatically enabled.

Pre-call check: shows video of what you will look like to others. Gives you a chance to check your hair and hide anything you don’t want visible. This screen displays names of the callers and shows the volume indicator to avoid being on mute by accident. Showing your own camera also helps the user know which camera is being used.

Desktop and mobile experiences are different. Desktop allows more real estate for larger videos. Mobile devices stack videos and use overlays to present buttons. Like in the Periscope app, use overlays wisely and temporarily.

Multiparty chat: each party receives equal real estate in the grid. Making the speaker’s video larger helps users distinguish who is currently speaking.

Tell the user how their connection is before the call, and also during the call with a Poor Connection warning. Before the call, show a testing stream capability screen to improve the experience.

During the call, determine what is the most important thing. During screensharing, participant videos are located in bottom corners, the screen being shared takes the most space and offers the best quality.

Because mobile is more constrained, everything matters more with mobile. Features include showing video of face to help framing, name of person to identify who is the active speaker, and conventional parts of calling including the disconnect button. Most important features dominate and don’t distract.

Doctor, patient video conference using Twilio and IBM Bluemix’s Watson Transcription Service

Next up was Jeff Sloyer, a developer advocate with IBM Bluemix, who showed a demo of a video conferencing flow between a patient and a doctor.

The demo used Twilio’s WebRTC service, along with IBM’s Watson to transcribe the conversation between a doctor and a patient.

The audio of the conference is streamed to Watson over a socket and is translated into text. The text is shown on the doctor’s screen to help transcribe what was discussed.

As for the HIPAA compliance, IBM Bluemix is compliant and the data is transmitted over a secure connection.

Streaming a HTML5 canvas to WebRTC

Nils Ohlmeier from Mozilla showed a demo using the HTML5 canvas to draw onto and then stream the canvas to the other participants in a WebRTC session.

It is even possible to do video mixing by combining multiple video streams into a canvas, and then streaming the canvas out to other participants.

There were no performance numbers on the video mixing, and Ohlmeier recommended running your own tests to see how well it performed.Ohlmeier said in other tests, the demo was successful in video mixing up to eight streams using an Amazon EC2 instance.

Customer Experience

Amitha Pulijala from Oracle talked about how customer experience is important and to enable seamless journeys. Customers reloads the browser using the reload and the back buttons when things don’t appear to work correctly. Other things that cause reliability problems include native app crashes, IP network connectivity changes when switching between WiFi and 4G connections, and server-side failures.

Using Session Rehydration, it is possible to keep an existing session alive, via a process called Session Rehydration. Upon reconnection, resurrect the session (voice, video and data channel) using client information stored locally (sessionId is stored in LocalStorage).

The WebSocket connection is kept for a short time and message resynchronization happens when a client is reconnected.

Customer expects the call setup time to be fast. However, WebRTC call setup takes a considerable amount of time. This takes time to gather candidates, prioritize, exchange with remote party, and perform connectivity checks.

Dynamic Media Peering prioritizes the candidates that are most likely to work first.

Another tip she talked about is to identify and solve weak points. Patchy videos and mid call-drops impair customer experience. Use the WebRTC-stats API to measure jitter, packets lost, and bandwidth.

Customers do not like if their video calls quickly drain the battery on their device. Using the WebRTC API, applications can measure the amount of jitter, packets lost, and bandwidth used and dynamically adjust to improve the experience. For example, to reduce packet loss, reduce the resolution of video and audio.

Lastly, when users wander away from the app, they expect to still be engaged, but not have their battery power sacrificed. Hibernating the session and using Push Notifications can help keep the user’s session available for when they return.

The sessions were recorded and have been posted on YouTube.