Recherche avancée

Sur d’autres sites (5789)

• Video Conferencing in HTML5 : WebRTC via Web Sockets

  1er janvier 2014, par silvia

  A bit over a week ago I gave a presentation at Web Directions Code 2012 in Melbourne. Maxine and John asked me to speak about something related to HTML5 video, so I went for the new shiny : WebRTC – real-time communication in the browser.

  Presentation slides

  I only had 20 min, so I had to make it tight. I wanted to show off video conferencing without special plugins in Google Chrome in just a few lines of code, as is the promise of WebRTC. To a large extent, I achieved this. But I made some interesting discoveries along the way. Demos are in the slide deck.

  UPDATE : Opera 12 has been released with WebRTC support.

  Housekeeping : if you want to replicate what I have done, you need to install a Google Chrome Web Browser 19+. Then make sure you go to chrome ://flags and activate the MediaStream and PeerConnection experiment(s). Restart your browser and now you can experiment with this feature. Big warning up-front : it’s not production-ready, since there are still changes happening to the spec and there is no compatible implementation by another browser yet.

  Here is a brief summary of the steps involved to set up video conferencing in your browser :

  1. Set up a video element each for the local and the remote video stream.
  2. Grab the local camera and stream it to the first video element.
  3. (*) Establish a connection to another person running the same Web page.
  4. Send the local camera stream on that peer connection.
  5. Accept the remote camera stream into the second video element.

  Now, the most difficult part of all of this – believe it or not – is the signalling part that is required to build the peer connection (marked with (*)). Initially I wanted to run completely without a server and just enter the remote’s IP address to establish the connection. This is, however, not a functionality that the PeerConnection object provides [might this be something to add to the spec ?].

  So, you need a server known to both parties that can provide for the handshake to set up the connection. All the examples that I have seen, such as https://apprtc.appspot.com/, use a channel management server on Google’s appengine. I wanted it all working with HTML5 technology, so I decided to use a Web Socket server instead.

  I implemented my Web Socket server using node.js (code of websocket server). The video conferencing demo is in the slide deck in an iframe – you can also use the stand-alone html page. Works like a treat.

  While it is still using Google’s STUN server to get through NAT, the messaging for setting up the connection is running completely through the Web Socket server. The messages that get exchanged are plain SDP message packets with a session ID. There are OFFER, ANSWER, and OK packets exchanged for each streaming direction. You can see some of it in the below image :

  

  I’m not running a public WebSocket server, so you won’t be able to see this part of the presentation working. But the local loopback video should work.

  At the conference, it all went without a hitch (while the wireless played along). I believe you have to host the WebSocket server on the same machine as the Web page, otherwise it won’t work for security reasons.

  A whole new world of opportunities lies out there when we get the ability to set up video conferencing on every Web page – scary and exciting at the same time !

• Video Conferencing in HTML5 : WebRTC via Web Sockets

  14 juin 2012, par silvia

  A bit over a week ago I gave a presentation at Web Directions Code 2012 in Melbourne. Maxine and John asked me to speak about something related to HTML5 video, so I went for the new shiny : WebRTC – real-time communication in the browser.

  Presentation slides

  I only had 20 min, so I had to make it tight. I wanted to show off video conferencing without special plugins in Google Chrome in just a few lines of code, as is the promise of WebRTC. To a large extent, I achieved this. But I made some interesting discoveries along the way. Demos are in the slide deck.

  UPDATE : Opera 12 has been released with WebRTC support.

  Housekeeping : if you want to replicate what I have done, you need to install a Google Chrome Web Browser 19+. Then make sure you go to chrome ://flags and activate the MediaStream and PeerConnection experiment(s). Restart your browser and now you can experiment with this feature. Big warning up-front : it’s not production-ready, since there are still changes happening to the spec and there is no compatible implementation by another browser yet.

  Here is a brief summary of the steps involved to set up video conferencing in your browser :

  1. Set up a video element each for the local and the remote video stream.
  2. Grab the local camera and stream it to the first video element.
  3. (*) Establish a connection to another person running the same Web page.
  4. Send the local camera stream on that peer connection.
  5. Accept the remote camera stream into the second video element.

  Now, the most difficult part of all of this – believe it or not – is the signalling part that is required to build the peer connection (marked with (*)). Initially I wanted to run completely without a server and just enter the remote’s IP address to establish the connection. This is, however, not a functionality that the PeerConnection object provides [might this be something to add to the spec ?].

  So, you need a server known to both parties that can provide for the handshake to set up the connection. All the examples that I have seen, such as https://apprtc.appspot.com/, use a channel management server on Google’s appengine. I wanted it all working with HTML5 technology, so I decided to use a Web Socket server instead.

  I implemented my Web Socket server using node.js (code of websocket server). The video conferencing demo is in the slide deck in an iframe – you can also use the stand-alone html page. Works like a treat.

  While it is still using Google’s STUN server to get through NAT, the messaging for setting up the connection is running completely through the Web Socket server. The messages that get exchanged are plain SDP message packets with a session ID. There are OFFER, ANSWER, and OK packets exchanged for each streaming direction. You can see some of it in the below image :

  

  I’m not running a public WebSocket server, so you won’t be able to see this part of the presentation working. But the local loopback video should work.

  At the conference, it all went without a hitch (while the wireless played along). I believe you have to host the WebSocket server on the same machine as the Web page, otherwise it won’t work for security reasons.

  A whole new world of opportunities lies out there when we get the ability to set up video conferencing on every Web page – scary and exciting at the same time !

• Gallery of VP8 Encoding Naivete

  15 octobre 2010, par Multimedia Mike — VP8

  I’ve been toiling away as a multimedia technology generalist for so long that it’s easy for me to forget that not everyone is as versed in the minutiae of the domain as I am. But I recently experienced what it’s like to be such an outsider when I posted about my toy VP8 encoder, expressing that it’s one of the hardest things I have ever tried to do. I heard of from number of people who do have extensive experience in video encoding, particularly with the H.264 and VP8 codecs. Their reactions were predictable : What’s so hard ? Look, you might be a little too immersed in the area to really understand a relative beginner’s perspective.

  And to all the people who suggested that I should get the encoder into FFmpeg ASAP : Are you crazy ?! Did you see what the first pass of the encoder produced ? Do you have lower standards than even I do ?

  
  
  

  Not Giving Up
  I worked a little more on the toy encoder. Remember that the above image is what I’m hoping to encode somewhat faithfully for this experiment. In my first pass, I attempted vertical prediction for all planes. For my next pass, I forced the chroma planes to mid-level (which results in a greyscale image) and played with the 16×16 luma prediction modes. When implementing an extremely naive algorithm to decide which 16×16 prediction mode would be the best for a particular block, this is what the program produced :

  
  
  

  For fun, here is what the image encodes to when forcing various prediction modes :
  

  I think the DC-only prediction mode actually looks a little better than the image that the naive algorithm produced :

  
  
  

  Vertical 16×16 prediction, similar to the image from the last post (just in black and white) :
  

  
  
  

  Horizontal 16×16 prediction :
  

  
  
  

  This is the 16×16 prediction mode unique to VP8, the TrueMotion mode (based on On2/Duck’s very first video codec) :
  

  
  
  

  Wow, these encodings really bring down the cheerful tone of the original image.

  Next Steps
  I have little reason to believe that I am encoding and subsequently reconstructing the image correctly (i.e., error is likely propagating through the entire encoding). If I have time, the next step is to validate my reconstruction against the encoder. Then I need to get the entropy considerations correct so that I actually get some compression out of this format.