Recherche avancée

Sur d’autres sites (4717)

• Crossdevice encoding static file to stream in browser using FFMPEG (segmented h264 ?)

  20 mars 2014, par Vprnl

  I'm building a mediacenter application in NodeJS which is going pretty ok.
  (you can check it out on Github : https://github.com/jansmolders86/mediacenterjs )

  I'm using FFMPEG to transcode local (static) movies to a stream which I then send to the browser.

  At first I used h264 with Flash which worked in browsers, but I really need it to work on Android an iOS (so no Flash) and preferably working on a Raspberry Pi.

  But getting it to play on all devices is driving me absolutely insane !

  I have all these bits of the puzzle I've gathered from countless hours reading articles, tutorials and stack overflow posts, which led me to the conclusion that I need to produce the following :

  • Use video codec H264 to transcode to MP4
  • Move the moovatom '-movflags' to make a MP4 streamable
  • Segment the stream so Apple can play the stream as well.

  But getting nowhere with this. Every time I produce a series of FFMPEG settings that either don't work, or work on some devices rather than all.

  Some of my failed attempt where :

  My flash attempt -> Main problem (not running in IOS) :

      '-y','-ss 0','-b 800k','-vcodec libx264','-acodec mp3'\ 
  
      '-ab 128','-ar 44100','-bufsize 62000', '-maxrate 620k'\
  
      metaDuration,tDuration,'-f flv

  my HLS attempt -> Main problem (not running in browser) :

          '-r 15','-b:v 128k','-c:v libx264','-x264opts level=41'\
  
          '-threads 4','-s 640x480','-map 0:v','-map 0:a:0','-c:a mp3'\
  
          '-b:a 160000','-ac 2','-f hls','-hls_time 10','-hls_list_size 6'\
  
          '-hls_wrap 18','-start_number 1'

  My MP4 attempt -> Main problem (duration is shortened and the later part of the video is speeding by)

         '-y','-ss 0','-b 800k','-vcodec libx264','-acodec mp3'\
  
         '-ab 128','-ar 44100','-bufsize 62000', '-maxrate 620k'\
  
         metaDuration,tDuration,'-f mp4','-movflags','frag_keyframe+empty_moov'

  Second MP4 attempt : -> Main problem (duration is shortened and the later part of the video is speeding by)

      '-y','-vcodec libx264','-pix_fmt yuv420p','-b 1200k','-flags +loop+mv4'\
  
      '-cmp 256','-partitions +parti4x4+parti8x8+partp4x4+partp8x8+partb8x8'\
  
      '-me_method hex','-subq 7','-trellis 1','-refs 5','-bf 3','-coder 1'\
  
      '-me_range 16','-g 150','-keyint_min 25','-sc_threshold 40'\
  
      '-i_qfactor 0.71','-acodec mp3','-qmin 10','-qdiff 4','-qmax 51'\
  
      '-ab 128k','-ar 44100','-threads 2','-f mp4','-movflags','frag_keyframe+empty_moov'])

  Here is an example of the FFMPEG log running with these settings :

          file conversion error ffmpeg version N-52458-gaa96439 Copyright (c) 2000-2013 the FFmpeg developers
  
            built on Apr 24 2013 22:19:32 with gcc 4.8.0 (GCC)
  
            configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --e
  
          nable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable
  
          -libgsm --enable-libilbc --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --ena
  
          ble-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwola
  
          me --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enabl
  
          e-libxvid --enable-zlib
  
            libavutil      52. 27.101 / 52. 27.101
  
            libavcodec     55.  6.100 / 55.  6.100
  
            libavformat    55.  3.100 / 55.  3.100
  
            libavdevice    55.  0.100 / 55.  0.100
  
            libavfilter     3. 60.101 /  3. 60.101
  
            libswscale      2.  2.100 /  2.  2.100
  
            libswresample   0. 17.102 /  0. 17.102
  
            libpostproc    52.  3.100 / 52.  3.100
  
          [avi @ 02427900] non-interleaved AVI
  
          Guessed Channel Layout for  Input Stream #0.1 : mono
  
          Input #0, avi, from 'C:/temp/the avengers.avi':
  
            Duration: 00:00:34.00, start: 0.000000, bitrate: 1433 kb/s
  
              Stream #0:0: Video: cinepak (cvid / 0x64697663), rgb24, 320x240, 15 tbr, 15 tbn, 15 tbc
  
              Stream #0:1: Audio: pcm_u8 ([1][0][0][0] / 0x0001), 22050 Hz, mono, u8, 176 kb/s
  
          Please use -b:a or -b:v, -b is ambiguous
  
          [libx264 @ 02527c60] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
  
          [libx264 @ 02527c60] profile High, level 2.0
  
          [libx264 @ 02527c60] 264 - core 130 r2274 c832fe9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2013 - http://www.videolan.org/x26
  
          4.html - options: cabac=1 ref=5 deblock=1:0:0 analyse=0x3:0x133 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16
  
           chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=2 lookahead_threads=1 sliced_th
  
          reads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 we
  
          ightb=1 open_gop=0 weightp=2 keyint=150 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=120
  
          0 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.41 aq=1:1.00
  
          Output #0, mp4, to 'pipe:1':
  
            Metadata:
  
              encoder         : Lavf55.3.100
  
              Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 320x240, q=10-51, 1200 kb/s, 15360 tbn, 15 tbc
  
              Stream #0:1: Audio: mp3 (i[0][0][0] / 0x0069), 44100 Hz, mono, s16p, 128 kb/s
  
          Stream mapping:
  
            Stream #0:0 -> #0:0 (cinepak -> libx264)
  
            Stream #0:1 -> #0:1 (pcm_u8 -> libmp3lame)
  
          Press [q] to stop, [?] for help
  
          frame=  106 fps=0.0 q=10.0 size=       1kB time=00:00:06.94 bitrate=   1.4kbits/s
  
          frame=  150 fps=149 q=14.0 size=       1kB time=00:00:09.87 bitrate=   1.0kbits/s
  
          frame=  191 fps=126 q=16.0 size=       1kB time=00:00:12.61 bitrate=   0.8kbits/s
  
          frame=  244 fps=121 q=16.0 size=    2262kB time=00:00:16.14 bitrate=1147.6kbits/s
  
          frame=  303 fps=120 q=14.0 size=    2262kB time=00:00:20.08 bitrate= 922.2kbits/s
  
          frame=  354 fps=117 q=15.0 size=    3035kB time=00:00:23.48 bitrate=1058.6kbits/s
  
          frame=  402 fps=113 q=15.0 size=    3035kB time=00:00:26.67 bitrate= 932.1kbits/s
  
          frame=  459 fps=113 q=16.0 size=    4041kB time=00:00:30.43 bitrate=1087.7kbits/s
  
          frame=  510 fps=103 q=2686559.0 Lsize=    5755kB time=00:00:33.93 bitrate=1389.3kbits/s
  
  
  
          video:5211kB audio:531kB subtitle:0 global headers:0kB muxing overhead 0.235111%
  
          [libx264 @ 02527c60] frame I:6     Avg QP:10.55  size: 25921
  
          [libx264 @ 02527c60] frame P:245   Avg QP:12.15  size: 14543
  
          [libx264 @ 02527c60] frame B:259   Avg QP:15.55  size:  6242
  
          [libx264 @ 02527c60] consecutive B-frames:  6.1% 73.7% 14.7%  5.5%
  
          [libx264 @ 02527c60] mb I  I16..4: 19.9%  6.2% 73.9%
  
          [libx264 @ 02527c60] mb P  I16..4:  6.0%  0.2% 12.0%  P16..4: 35.4%  9.6% 16.3%  7.0%  5.6%    skip: 7.8%
  
          [libx264 @ 02527c60] mb B  I16..4:  0.7%  0.0%  4.3%  B16..8: 27.6% 17.2% 17.0%  direct:17.3%  skip:15.9%  L0:39.4% L1:43.2%
  
          BI:17.4%
  
          [libx264 @ 02527c60] final ratefactor: 11.41
  
          [libx264 @ 02527c60] 8x8 transform intra:1.6% inter:4.0%
  
          [libx264 @ 02527c60] coded y,uvDC,uvAC intra: 93.0% 97.0% 94.9% inter: 58.4% 58.7% 50.6%
  
          [libx264 @ 02527c60] i16 v,h,dc,p: 15% 26% 54%  5%
  
          [libx264 @ 02527c60] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 17% 39%  4%  4%  3%  1%  6%  9%
  
          [libx264 @ 02527c60] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 34% 21%  4%  2%  2%  2%  2%  5%
  
          [libx264 @ 02527c60] i8c dc,h,v,p: 51% 24% 19%  6%
  
          [libx264 @ 02527c60] Weighted P-Frames: Y:4.1% UV:1.2%
  
          [libx264 @ 02527c60] ref P L0: 68.2%  9.8% 11.0%  5.6%  4.6%  0.8%  0.0%
  
          [libx264 @ 02527c60] ref B L0: 87.7%  8.0%  3.9%  0.4%
  
          [libx264 @ 02527c60] ref B L1: 97.8%  2.2%
  
          [libx264 @ 02527c60] kb/s:1255.36

  Lastly this is my node code fireing up FFMPEG. (I use the module Fluent-ffmpeg : https://github.com/schaermu/node-fluent-ffmpeg )

      var proc = new ffmpeg({ source: movie, nolog: true, timeout:15000})                         
  
          .addOptions(['-r 15','-b:v 128k','-c:v libx264','-x264opts level=41','-threads 4','-s 640x480','-map 0:v','-map 0:a:0','-c:a mp3','-b:a 160000','-ac 2','-f hls','-hls_time 10','-hls_list_size 6','-hls_wrap 18','-start_number 1 stream.m3u8'])
  
          .writeToStream(res, function(retcode, error){
  
              if (!error){
  
                  console.log('file has been converted succesfully',retcode .green);
  
              }else{
  
                  console.log('file conversion error',error .red);
  
              }
  
          });

  So to conclude this very long and code heavy question :

  I hope this does not come off as a lazy request, but could someone show/explain to me which FFMPEG settings could/should work on all platforms (modern browsers, Android and iOS) producing a stream of a static file which I can send to a HTML5 player.

  [EDIT] what I need if a generic option isn't available

  And if this is not possible as some posts might suggest, I would love to see a set of FFMPEG settings that get's the job done properly as far as mp4 streaming is concerned. (e.g encoding a streamable mp4).

  The streaming mp4 needs the following

  • A shifted moovAtom
  • It needs to be h264

  Thanks very much for your help !

• FFmpeg h264_v4l2m2m to rtmp

  6 juillet 2021, par KnightRex

  On an up to date Raspberry Pi 4 B+ 4GB with the Raspberry OS 32bit I'm attempting to stream to an rtmp service, like Youtube or Twitch with FFmpeg using the hardware encoder h264_v4l2m2m but it fails.

  


  Issue

  


  When streaming the streaming service it ingests the input, bitrate and goes live but no output is shown. When connecting to Twitch the stream inspector correctly detects video resolution but fails to detect the codec or fps.

  


  When I replace the stream output to for example a file, test.flv, it records a viewable video with the correct codec (h264/aac) codecs in vlc and ffplay.

  


  When I replace the h264_v4l2m2m encoder with h264_omx it streams the video correctly.

  


  Steps

  


  


  • Installed a clean version of the latest Raspi OS 32bit

    


  


  • Recompiled FFmpeg to 4.3.2 (to the solve green screen)

    


  


  • Record video, creates viewable video :

    


    ffmpeg \
-f v4l2 -input_format mjpeg -video_size 1280x720 -framerate 30 -i /dev/video0 \
-c:v h264_v4l2m2m -g 60 -pix_fmt yuv420p -b:v 3000k -minrate 3000k -maxrate 3000k -r 30 -an \
-t 15 test.flv


    


  


  • Start stream, unable to view :

    


    ffmpeg \&#xA;-f v4l2 -input_format mjpeg -video_size 1280x720 -framerate 30 -i /dev/video0 \&#xA;-c:v h264_v4l2m2m -g 60 -pix_fmt yuv420p -b:v 3000k -minrate 3000k -maxrate 3000k -r 30 -an \&#xA;-f flv "<destination uri="uri">"&#xA;</destination>

    


  


  • Start stream, unable to view (Extended based on comments below) :

    


    ffmpeg \&#xA; -re -f lavfi -i anullsrc \&#xA; -f v4l2 -input_format mjpeg -video_size 1280x720 -framerate 30 -i /dev/video0 \&#xA; -c:v h264_v4l2m2m -g 60 -pix_fmt yuv420p -b:v 3000k -minrate 3000k -maxrate 3000k -r 30 \&#xA; -c:a aac \&#xA; -f flv "<destination uri="uri">"&#xA;</destination>

    


  


  • Start stream, with omx works :

    


    ffmpeg \&#xA;-f v4l2 -input_format mjpeg -video_size 1280x720 -framerate 30 -i /dev/video0 \&#xA;-c:v h264_omx -preset ultrafast -pix_fmt yuv420p -g 60 -b:v 3000k -minrate 3000k -maxrate 3000k -r 30 -an \&#xA;-f flv "<destination uri="uri">"&#xA;</destination>

    


  


  


  Output

  


  With h264_v4l2m2m (invalid output - loglevel verbose) :

  


  $ ffmpeg -re -f lavfi -i anullsrc  -f v4l2 -input_format mjpeg -video_size 1280x720 -framerate 30 -i /dev/video0  -c:v h264_v4l2m2m -g 60 -pix_fmt yuv420p -b:v 3000k -minrate 3000k -maxrate 3000k -r 30 -c:a aac -f flv <destination url="url">&#xA;ffmpeg version n4.3.2 Copyright (c) 2000-2021 the FFmpeg developers&#xA;  built with gcc 8 (Raspbian 8.3.0-6&#x2B;rpi1)&#xA;  configuration: --prefix=/home/pi/ffmpeg_build --bindir=/home/pi/bin --extra-cflags=-I/usr/local/include --extra-ldflags=-L/usr/local/lib --extra-libs=&#x27;-lpthread -lm -latomic&#x27; --enable-omx --enable-encoder=h264_omx&#xA;  libavutil      56. 51.100 / 56. 51.100&#xA;  libavcodec     58. 91.100 / 58. 91.100&#xA;  libavformat    58. 45.100 / 58. 45.100&#xA;  libavdevice    58. 10.100 / 58. 10.100&#xA;  libavfilter     7. 85.100 /  7. 85.100&#xA;  libswscale      5.  7.100 /  5.  7.100&#xA;  libswresample   3.  7.100 /  3.  7.100&#xA;[Parsed_anullsrc_0 @ 0x7215c0] sample_rate:44100 channel_layout:&#x27;stereo&#x27; nb_samples:1024&#xA;Input #0, lavfi, from &#x27;anullsrc&#x27;:&#xA;  Duration: N/A, start: 0.000000, bitrate: 705 kb/s&#xA;    Stream #0:0: Audio: pcm_u8, 44100 Hz, stereo, u8, 705 kb/s&#xA;[video4linux2,v4l2 @ 0x725350] fd:3 capabilities:84a00001&#xA;[mjpeg @ 0x1e10210] EOI missing, emulating&#xA;Input #1, video4linux2,v4l2, from &#x27;/dev/video0&#x27;:&#xA;  Duration: N/A, start: 42396.344658, bitrate: N/A&#xA;    Stream #1:0: Video: mjpeg (Baseline), yuvj422p(pc, bt470bg/unknown/unknown), 1280x720, 30 fps, 30 tbr, 1000k tbn, 1000k tbc&#xA;[tcp @ 0x72cac0] Starting connection attempt to <destination ip="ip"> port 1935&#xA;[tcp @ 0x72cac0] Successfully connected to <destination ip="ip"> port 1935&#xA;Stream mapping:&#xA;  Stream #1:0 -> #0:0 (mjpeg (native) -> h264 (h264_v4l2m2m))&#xA;  Stream #0:0 -> #0:1 (pcm_u8 (native) -> aac (native))&#xA;Press [q] to stop, [?] for help&#xA;[graph 0 input from stream 1:0 @ 0x73de00] w:1280 h:720 pixfmt:yuvj422p tb:1/1000000 fr:30/1 sar:0/1 sws_param:flags=2&#xA;[auto_scaler_0 @ 0x745400] w:iw h:ih flags:&#x27;bicubic&#x27; interl:0&#xA;[format @ 0x73e750] auto-inserting filter &#x27;auto_scaler_0&#x27; between the filter &#x27;Parsed_null_0&#x27; and the filter &#x27;format&#x27;&#xA;[swscaler @ 0x1e300b0] deprecated pixel format used, make sure you did set range correctly&#xA;[auto_scaler_0 @ 0x745400] w:1280 h:720 fmt:yuvj422p sar:0/1 -> w:1280 h:720 fmt:yuv420p sar:0/1 flags:0x4&#xA;[h264_v4l2m2m @ 0x72a500] driver &#x27;bcm2835-codec&#x27; on card &#x27;bcm2835-codec-isp&#x27;&#xA;[h264_v4l2m2m @ 0x72a500] driver &#x27;uvcvideo&#x27; on card &#x27;USB Video: USB Video&#x27;&#xA;    Last message repeated 1 times&#xA;[h264_v4l2m2m @ 0x72a500] driver &#x27;bcm2835-codec&#x27; on card &#x27;bcm2835-codec-encode&#x27;&#xA;[h264_v4l2m2m @ 0x72a500] Using device /dev/video11&#xA;[h264_v4l2m2m @ 0x72a500] driver &#x27;bcm2835-codec&#x27; on card &#x27;bcm2835-codec-encode&#x27;&#xA;[h264_v4l2m2m @ 0x72a500] Failed to set number of B-frames&#xA;    Last message repeated 1 times&#xA;[h264_v4l2m2m @ 0x72a500] Failed to set gop size&#xA;[h264_v4l2m2m @ 0x72a500] h264 profile not found&#xA;[h264_v4l2m2m @ 0x72a500] Encoder adjusted: qmin (0), qmax (51)&#xA;[h264_v4l2m2m @ 0x72a500] Failed to set minimum video quantizer scale&#xA;[h264_v4l2m2m @ 0x72a500] Failed to set maximum video quantizer scale&#xA;[graph_1_in_0_0 @ 0x786f90] tb:1/44100 samplefmt:u8 samplerate:44100 chlayout:0x3&#xA;[format_out_0_1 @ 0x7871d0] auto-inserting filter &#x27;auto_resampler_0&#x27; between the filter &#x27;Parsed_anull_0&#x27; and the filter &#x27;format_out_0_1&#x27;&#xA;[auto_resampler_0 @ 0x7887f0] ch:2 chl:stereo fmt:u8 r:44100Hz -> ch:2 chl:stereo fmt:fltp r:44100Hz&#xA;Output #0, flv, to &#x27;<destination url="url">&#x27;:&#xA;  Metadata:&#xA;    encoder         : Lavf58.45.100&#xA;    Stream #0:0: Video: h264 (h264_v4l2m2m) ([7][0][0][0] / 0x0007), yuv420p(progressive), 1280x720, q=-1--1, 3000 kb/s, 30 fps, 1k tbn, 30 tbc&#xA;    Metadata:&#xA;      encoder         : Lavc58.91.100 h264_v4l2m2m&#xA;    Stream #0:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, fltp, 128 kb/s&#xA;    Metadata:&#xA;      encoder         : Lavc58.91.100 aac&#xA;[video4linux2,v4l2 @ 0x1e0f410] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)&#xA;frame= 1966 fps= 24 q=-0.0 size=   13016kB time=00:01:23.60 bitrate=1275.4kbits/s speed=   1x&#xA;</destination></destination></destination></destination>

  


  

  • deprecated pixel format yuvy422p : video0 has mjpeg and raw output in pixel format yuvy422p.

  


  


  With h264 omx (works)

  


    libavutil      56. 22.100 / 56. 22.100&#xA;  libavcodec     58. 35.100 / 58. 35.100&#xA;  libavformat    58. 20.100 / 58. 20.100&#xA;  libavdevice    58.  5.100 / 58.  5.100&#xA;  libavfilter     7. 40.101 /  7. 40.101&#xA;  libavresample   4.  0.  0 /  4.  0.  0&#xA;  libswscale      5.  3.100 /  5.  3.100&#xA;  libswresample   3.  3.100 /  3.  3.100&#xA;  libpostproc    55.  3.100 / 55.  3.100&#xA;Input #0, lavfi, from &#x27;anullsrc&#x27;:&#xA;  Duration: N/A, start: 0.000000, bitrate: 705 kb/s&#xA;    Stream #0:0: Audio: pcm_u8, 44100 Hz, stereo, u8, 705 kb/s&#xA;[mjpeg @ 0x7c7320] EOI missing, emulating&#xA;Input #0, video4linux2,v4l2, from &#x27;/dev/video0&#x27;:&#xA;  Duration: N/A, start: 71.378330, bitrate: N/A&#xA;    Stream #0:0: Video: mjpeg, yuvj422p(pc, bt470bg/unknown/unknown), 1280x720, 30 fps, 30 tbr, 1000k tbn, 1000k tbc&#xA;Codec AVOption preset (Configuration preset) specified for output file #0 (<destination url="url">) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.&#xA;Stream mapping:&#xA;  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (h264_omx))&#xA;  Stream #0:0 -> #0:1 (pcm_u8 (native) -> aac (native))&#xA;Press [q] to stop, [?] for help&#xA;[swscaler @ 0x7e5180] deprecated pixel format used, make sure you did set range correctly&#xA;[h264_omx @ 0x7cb800] Using OMX.broadcom.video_encode&#xA;Output #0, flv, to &#x27;<destination url="url">&#x27;:&#xA;  Metadata:&#xA;    encoder         : Lavf58.20.100&#xA;    Stream #0:0: Video: h264 (h264_omx) ([7][0][0][0] / 0x0007), yuv420p(progressive), 1280x720, q=2-31, 3000 kb/s, 30 fps, 1k tbn, 30 tbc&#xA;    Metadata:&#xA;      encoder         : Lavc58.35.100 h264_omx&#xA;    Stream #0:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, fltp, 128 kb/s&#xA;    Metadata:&#xA;      encoder         : Lavc58.35.100 aac&#xA;[flv @ 0x7ca190] Failed to update header with correct duration.rate=1438.3kbits/s speed=   1x&#xA;[flv @ 0x7ca190] Failed to update header with correct filesize.&#xA;frame= 2190 fps= 24 q=-0.0 Lsize=   16424kB time=00:01:33.20 bitrate=1443.6kbits/s speed=   1x&#xA;</destination></destination>

  


• My journey to Coviu

  27 octobre 2015, par silvia

  My new startup just released our MVP – this is the story of what got me here.

  I love creating new applications that let people do their work better or in a manner that wasn’t possible before.

  My first such passion was as a student intern when I built a system for a building and loan association’s monthly customer magazine. The group I worked with was managing their advertiser contacts through a set of paper cards and I wrote a dBase based system (yes, that long ago) that would manage their customer relationships. They loved it – until it got replaced by an SAP system that cost 100 times what I cost them, had really poor UX, and only gave them half the functionality. It was a corporate system with ongoing support, which made all the difference to them.

  The story repeated itself with a CRM for my Uncle’s construction company, and with a resume and quotation management system for Accenture right after Uni, both of which I left behind when I decided to go into research.

  Even as a PhD student, I never lost sight of challenges that people were facing and wanted to develop technology to overcome problems. The aim of my PhD thesis was to prepare for the oncoming onslaught of audio and video on the Internet (yes, this was 1994 !) by developing algorithms to automatically extract and locate information in such files, which would enable users to structure, index and search such content.

  Many of the use cases that we explored are now part of products or continue to be challenges : finding music that matches your preferences, identifying music or video pieces e.g. to count ads on the radio or to mark copyright infringement, or the automated creation of video summaries such as trailers.

  

  This continued when I joined the CSIRO in Australia – I was working on segmenting speech into words or talk spurts since that would simplify captioning & subtitling, and on MPEG-7 which was a (slightly over-engineered) standard to structure metadata about audio and video.

  In 2001 I had the idea of replicating the Web for videos : i.e. creating hyperlinked and searchable video-only experiences. We called it “Annodex” for annotated and indexed video and it needed full-screen hyperlinked video in browsers – man were we ahead of our time ! It was my first step into standards, got several IETF RFCs to my name, and started my involvement with open codecs through Xiph.

  Around the time that YouTube was founded in 2006, I founded Vquence – originally a video search company for the Web, but pivoted to a video metadata mining company. Vquence still exists and continues to sell its data to channel partners, but it lacks the user impact that has always driven my work.

  As the video element started being developed for HTML5, I had to get involved. I contributed many use cases to the W3C, became a co-editor of the HTML5 spec and focused on video captioning with WebVTT while contracting to Mozilla and later to Google. We made huge progress and today the technology exists to publish video on the Web with captions, making the Web more inclusive for everybody. I contributed code to YouTube and Google Chrome, but was keen to make a bigger impact again.

  The opportunity came when a couple of former CSIRO colleagues who now worked for NICTA approached me to get me interested in addressing new use cases for video conferencing in the context of WebRTC. We worked on a kiosk-style solution to service delivery for large service organisations, particularly targeting government. The emerging WebRTC standard posed many technical challenges that we addressed by building rtc.io , by contributing to the standards, and registering bugs on the browsers.

  Fast-forward through the development of a few further custom solutions for customers in health and education and we are starting to see patterns of need emerge. The core learning that we’ve come away with is that to get things done, you have to go beyond “talking heads” in a video call. It’s not just about seeing the other person, but much more about having a shared view of the things that need to be worked on and a shared way of interacting with them. Also, we learnt that the things that are being worked on are quite varied and may include multiple input cameras, digital documents, Web pages, applications, device data, controls, forms.

  So we set out to build a solution that would enable productive remote collaboration to take place. It would need to provide an excellent user experience, it would need to be simple to work with, provide for the standard use cases out of the box, yet be architected to be extensible for specialised data sharing needs that we knew some of our customers had. It would need to be usable directly on Coviu.com, but also able to integrate with specialised applications that some of our customers were already using, such as the applications that they spend most of their time in (CRMs, practice management systems, learning management systems, team chat systems). It would need to require our customers to sign up, yet their clients to join a call without sign-up.

  Collaboration is a big problem. People are continuing to get more comfortable with technology and are less and less inclined to travel distances just to get a service done. In a country as large as Australia, where 12% of the population lives in rural and remote areas, people may not even be able to travel distances, particularly to receive or provide recurring or specialised services, or to achieve work/life balance. To make the world a global village, we need to be able to work together better remotely.

  The need for collaboration is being recognised by specialised Web applications already, such as the LiveShare feature of Invision for Designers, Codassium for pair programming, or the recently announced Dropbox Paper. Few go all the way to video – WebRTC is still regarded as a complicated feature to support.

  

  With Coviu, we’d like to offer a collaboration feature to every Web app. We now have a Web app that provides a modern and beautifully designed collaboration interface. To enable other Web apps to integrate it, we are now developing an API. Integration may entail customisation of the data sharing part of Coviu – something Coviu has been designed for. How to replicate the data and keep it consistent when people collaborate remotely – that is where Coviu makes a difference.

  We have started our journey and have just launched free signup to the Coviu base product, which allows individuals to own their own “room” (i.e. a fixed URL) in which to collaborate with others. A huge shout out goes to everyone in the Coviu team – a pretty amazing group of people – who have turned the app from an idea to reality. You are all awesome !

  With Coviu you can share and annotate :

  • images (show your mum photos of your last holidays, or get feedback on an architecture diagram from a customer),
  • pdf files (give a presentation remotely, or walk a customer through a contract),
  • whiteboards (brainstorm with a colleague), and
  • share an application window (watch a YouTube video together, or work through your task list with your colleagues).

  All of these are regarded as “shared documents” in Coviu and thus have zooming and annotations features and are listed in a document tray for ease of navigation.

  This is just the beginning of how we want to make working together online more productive. Give it a go and let us know what you think.

  http://coviu.com/