Recherche avancée

Sur d’autres sites (4770)

• Build latest version of FFMPEG with libx264 Android

  12 novembre 2016, par Biraj Zalavadia

  I was able to Build FFMPEG with libx264 Android successfully and was working well before 2 months.

  I uses halfninja for building.

  Today I take latest source of ffmpeg and libX264 from here.

  git clone git://git.videolan.org/x264.git
  
  
  
  git clone git://git.videolan.org/ffmpeg.git

  And try to rebuild whole project. But now it throws following errors.

   ffmpeg.c:2406: error: undefined reference to 'hwaccels'
  
   ffmpeg.c:3390: error: undefined reference to 'qp_hist'
  
   ffmpeg.c:3390: error: undefined reference to 'do_hex_dump'
  
   ffmpeg.c:3390: error: undefined reference to 'do_pkt_dump'
  
   ffmpeg.c:576: error: undefined reference to 'do_benchmark_all'
  
   ffmpeg.c:657: error: undefined reference to 'audio_sync_method'
  
   ffmpeg.c:1222: error: undefined reference to 'vstats_filename'
  
   ffmpeg.c:972: error: undefined reference to 'frame_drop_threshold'
  
   ffmpeg.c:1181: error: undefined reference to 'debug_ts'
  
   ffmpeg.c:2742: error: undefined reference to 'copy_tb'
  
   ffmpeg.c:2846: error: undefined reference to 'audio_volume'
  
   ffmpeg.c:2862: error: undefined reference to 'init_simple_filtergraph'
  
   ffmpeg.c:2863: error: undefined reference to 'configure_filtergraph'
  
   ffmpeg.c:2923: error: undefined reference to 'video_sync_method'
  
   ffmpeg.c:2538: error: undefined reference to 'frame_bits_per_raw_sample'
  
   ffmpeg.c:3258: error: undefined reference to 'sdp_filename'
  
   ffmpeg.c:2087: error: undefined reference to 'ist_in_filtergraph'
  
   ffmpeg.c:4069: error: undefined reference to 'ffmpeg_parse_options'
  
   ffmpeg.c:4074: error: undefined reference to 'show_usage'
  
   ffmpeg.c:3303: error: undefined reference to 'options'
  
   ffmpeg.c:3303: error: undefined reference to 'stdin_interaction'
  
   ffmpeg.c:2553: error: undefined reference to 'exit_on_error'
  
   ffmpeg.c:2553: error: undefined reference to 'do_pkt_dump'
  
   ffmpeg.c:2553: error: undefined reference to 'do_hex_dump'
  
   ffmpeg.c:3727: error: undefined reference to 'copy_ts'
  
   ffmpeg.c:3727: error: undefined reference to 'dts_delta_threshold'
  
   ffmpeg.c:268: error: undefined reference to 'dts_error_threshold'
  
   ffmpeg.c:3976: error: undefined reference to 'print_stats'
  
   ffmpeg.c:1694: error: undefined reference to 'do_benchmark'
  
   ffmpeg.c:1694: error: undefined reference to 'max_error_rate'

  Please help me out to resolve this.

  Please look at my script.

  build_android.sh

  #!/bin/bash
  
  #
  
  # build_android.sh
  
  # Copyright (c) 2012 Jacek Marchwicki
  
  #
  
  # Licensed under the Apache License, Version 2.0 (the "License");
  
  # you may not use this file except in compliance with the License.
  
  # You may obtain a copy of the License at
  
  #
  
  #     http://www.apache.org/licenses/LICENSE-2.0
  
  #
  
  # Unless required by applicable law or agreed to in writing, software
  
  # distributed under the License is distributed on an "AS IS" BASIS,
  
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  
  # See the License for the specific language governing permissions and
  
  # limitations under the License.
  
  
  
  export NDK=/home/tasol/DEVELOPMENT/IJOOMER_DEVELOPMENT/NDK/android-ndk-r10d
  
  
  
  if [ "$NDK" = "" ]; then
  
      echo NDK variable not set, exiting
  
      echo "Use: export NDK=/your/path/to/android-ndk"
  
      exit 1
  
  fi
  
  
  
  OS=`uname -s | tr '[A-Z]' '[a-z]'`
  
  function build_x264
  
  {
  
      PLATFORM=$NDK/platforms/$PLATFORM_VERSION/arch-$ARCH/
  
      export PATH=${PATH}:$PREBUILT/bin/
  
      CROSS_COMPILE=$PREBUILT/bin/$EABIARCH-
  
      CFLAGS=$OPTIMIZE_CFLAGS
  
  #CFLAGS=" -I$ARM_INC -fpic -DANDROID -fpic -mthumb-interwork -ffunction-sections -funwind-tables -fstack-protector -fno-short-enums -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__  -Wno-psabi -march=armv5te -mtune=xscale -msoft-float -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -DANDROID  -Wa,--noexecstack -MMD -MP "
  
      export CPPFLAGS="$CFLAGS"
  
      export CFLAGS="$CFLAGS"
  
      export CXXFLAGS="$CFLAGS"
  
      export CXX="${CROSS_COMPILE}g++ --sysroot=$PLATFORM"
  
      export AS="${CROSS_COMPILE}gcc --sysroot=$PLATFORM"
  
      export CC="${CROSS_COMPILE}gcc --sysroot=$PLATFORM"
  
      export NM="${CROSS_COMPILE}nm"
  
      export STRIP="${CROSS_COMPILE}strip"
  
      export RANLIB="${CROSS_COMPILE}ranlib"
  
      export AR="${CROSS_COMPILE}ar"
  
      #export LDFLAGS="-Wl,-rpath-link=$PLATFORM/usr/lib -L$PLATFORM/usr/lib -nostdlib -lc -lm -ldl -llog"
  
  
  
      cd x264
  
      ./configure --disable-asm --prefix=$(pwd)/$PREFIX --host=$ARCH-linux --enable-static $ADDITIONAL_CONFIGURE_FLAG || exit 1
  
  
  
      make clean || exit 1
  
      make -j4 install || exit 1
  
      cd ..
  
  }
  
  
  
  function build_ffmpeg
  
  {
  
      PLATFORM=$NDK/platforms/$PLATFORM_VERSION/arch-$ARCH/
  
      CC=$PREBUILT/bin/$EABIARCH-gcc
  
      CROSS_PREFIX=$PREBUILT/bin/$EABIARCH-
  
      PKG_CONFIG=${CROSS_PREFIX}pkg-config
  
      if [ ! -f $PKG_CONFIG ];
  
      then
  
          cat > $PKG_CONFIG << EOF
  
  #!/bin/bash
  
  pkg-config \$*
  
  EOF
  
          chmod u+x $PKG_CONFIG
  
      fi
  
      NM=$PREBUILT/bin/$EABIARCH-nm
  
      cd ffmpeg
  
      export PKG_CONFIG_LIBDIR=$(pwd)/$PREFIX/lib/pkgconfig/
  
      export PKG_CONFIG_PATH=$(pwd)/$PREFIX/lib/pkgconfig/
  
      ./configure --target-os=linux \
  
          --prefix=$PREFIX \
  
          --enable-cross-compile \
  
          --extra-libs="-lgcc" \
  
          --arch=$ARCH \
  
          --cc=$CC \
  
          --cross-prefix=$CROSS_PREFIX \
  
          --nm=$NM \
  
          --sysroot=$PLATFORM \
  
          --extra-cflags=" -O3 -fpic -DANDROID -DHAVE_SYS_UIO_H=1 -Dipv6mr_interface=ipv6mr_ifindex -fasm -Wno-psabi -fno-short-enums  -fno-strict-aliasing -finline-limit=300 $OPTIMIZE_CFLAGS " \
  
          --disable-shared \
  
          --disable-stripping \
  
          --enable-static \
  
          --enable-runtime-cpudetect \
  
          --extra-ldflags="-Wl,-rpath-link=$PLATFORM/usr/lib -L$PLATFORM/usr/lib  -nostdlib -lc -lm -ldl -llog -L$PREFIX/lib" \
  
          --extra-cflags="-I$PREFIX/include" \
  
              --enable-gpl \
  
          --enable-encoder=libx264 \
  
          --enable-libx264 \
  
          --enable-parser=mjpeg \
  
          --enable-demuxer=image2 \
  
          --enable-hwaccel=h264_vaapi \
  
          --enable-hwaccel=h264_vaapi \
  
          --enable-hwaccel=h264_dxva2 \
  
          --enable-hwaccel=mpeg4_vaapi \
  
          --enable-demuxer=mov \
  
          --enable-demuxer=h264 \
  
          --enable-demuxer=mpegvideo \
  
          --enable-demuxer=h263 \
  
          --enable-demuxer=mpegps \
  
          --enable-demuxer=mjpeg \
  
          --enable-demuxer=rtsp \
  
          --enable-demuxer=rtp \
  
          --enable-demuxer=hls \
  
          --enable-demuxer=matroska \
  
          --enable-muxer=rtsp \
  
          --enable-muxer=mp4 \
  
          --enable-muxer=mov \
  
          --enable-muxer=mjpeg \
  
          --enable-muxer=matroska \
  
          --enable-protocol=crypto \
  
          --enable-protocol=jni \
  
          --enable-protocol=file \
  
          --disable-network \
  
              --enable-filter=buffer \
  
          --enable-filter=buffersink \
  
          --enable-filter=scale \
  
          --enable-protocol=rtp \
  
          --enable-protocol=tcp \
  
          --enable-protocol=udp \
  
          --enable-protocol=applehttp \
  
          --enable-protocol=hls \
  
          --enable-protocol=http \
  
          --enable-decoder=xsub \
  
          --enable-decoder=jacosub \
  
          --enable-decoder=dvdsub \
  
          --enable-decoder=dvbsub \
  
          --enable-decoder=subviewer \
  
          --enable-decoder=rawvideo \
  
          --enable-encoder=rawvideo \
  
          --enable-decoder=mjpeg \
  
          --enable-encoder=mjpeg \
  
          --enable-decoder=h263 \
  
          --enable-decoder=mpeg4 \
  
          --enable-encoder=mpeg4 \
  
          --enable-decoder=h264 \
  
          --enable-encoder=h264 \
  
          --enable-decoder=aac \
  
          --enable-encoder=aac \
  
          --enable-parser=h264 \
  
          --enable-encoder=mp2 \
  
          --enable-decoder=mp2 \
  
          --enable-encoder=libvo_amrwbenc \
  
          --enable-decoder=amrwb \
  
          --enable-muxer=mp2 \
  
          --enable-bsfs \
  
          --enable-decoders \
  
          --enable-encoders \
  
          --enable-parsers \
  
          --enable-hwaccels \
  
          --enable-muxers \
  
          --enable-avformat \
  
          --enable-avcodec \
  
          --disable-demuxer=v4l \
  
          --disable-demuxer=v4l2 \
  
          --disable-indev=v4l \
  
          --disable-indev=v4l2 \
  
          --disable-ffplay \
  
          --disable-ffmpeg \
  
          --disable-ffplay \
  
          --disable-ffprobe \
  
          --disable-ffserver \
  
          --enable-nonfree \
  
          --enable-version3 \
  
          --enable-memalign-hack \
  
          --disable-asm \
  
          $ADDITIONAL_CONFIGURE_FLAG \
  
          || exit 1
  
      make clean || exit 1
  
      make -j4 install || exit 1
  
  
  
      cd ..
  
  }
  
  
  
  function build_one {
  
      cd ffmpeg
  
      cd ..
  
  }
  
  
  
  #arm v5
  
  EABIARCH=arm-linux-androideabi
  
  ARCH=arm
  
  CPU=armv5
  
  OPTIMIZE_CFLAGS="-marm -march=$CPU"
  
  PREFIX=../ffmpeg-build/armeabi
  
  OUT_LIBRARY=$PREFIX/libffmpeg.so
  
  ADDITIONAL_CONFIGURE_FLAG=
  
  SONAME=libffmpeg.so
  
  PREBUILT=$NDK/toolchains/arm-linux-androideabi-4.8/prebuilt/$OS-x86_64
  
  #PREBUILT=$NDK/toolchains/arm-linux-androideabi-4.6/prebuilt/$OS-x86
  
  PLATFORM_VERSION=android-5
  
  build_x264
  
  build_ffmpeg
  
  build_one
  
  
  
  
  
  #arm v7vfpv3
  
  #EABIARCH=arm-linux-androideabi
  
  #ARCH=arm
  
  #CPU=armv7-a
  
  #OPTIMIZE_CFLAGS="-mfloat-abi=softfp -mfpu=vfpv3-d16 -marm -march=$CPU "
  
  #PREFIX=../ffmpeg-build/armeabi-v7a
  
  #OUT_LIBRARY=$PREFIX/libffmpeg.so
  
  #ADDITIONAL_CONFIGURE_FLAG=
  
  #SONAME=libffmpeg.so
  
  #PREBUILT=$NDK/toolchains/arm-linux-androideabi-4.8/prebuilt/$OS-x86_64
  
  #PREBUILT=$NDK/toolchains/arm-linux-androideabi-4.6/prebuilt/$OS-x86
  
  #PLATFORM_VERSION=android-5
  
  #build_x264
  
  #build_ffmpeg
  
  #build_one
  
  
  
  #arm v7 + neon (neon also include vfpv3-32)
  
  EABIARCH=arm-linux-androideabi
  
  ARCH=arm
  
  CPU=armv7-a
  
  OPTIMIZE_CFLAGS="-mfloat-abi=softfp -mfpu=neon -marm -march=$CPU -mtune=cortex-a8 -mthumb -D__thumb__ "
  
  PREFIX=../ffmpeg-build/armeabi-v7a
  
  #PREFIX=../ffmpeg-build/armeabi-v7a-neon
  
  OUT_LIBRARY=../ffmpeg-build/armeabi-v7a/libffmpeg-neon.so
  
  ADDITIONAL_CONFIGURE_FLAG=--enable-neon
  
  SONAME=libffmpeg-neon.so
  
  PREBUILT=$NDK/toolchains/arm-linux-androideabi-4.8/prebuilt/$OS-x86_64
  
  #PREBUILT=$NDK/toolchains/arm-linux-androideabi-4.6/prebuilt/$OS-x86
  
  PLATFORM_VERSION=android-9
  
  build_x264
  
  build_ffmpeg
  
  build_one
  
  
  
  #x86
  
  EABIARCH=i686-linux-android
  
  ARCH=x86
  
  OPTIMIZE_CFLAGS="-m32"
  
  PREFIX=../ffmpeg-build/x86
  
  OUT_LIBRARY=$PREFIX/libffmpeg.so
  
  ADDITIONAL_CONFIGURE_FLAG=--disable-asm
  
  SONAME=libffmpeg.so
  
  PREBUILT=$NDK/toolchains/x86-4.8/prebuilt/$OS-x86_64
  
  #PREBUILT=$NDK/toolchains/x86-4.6/prebuilt/$OS-x86
  
  PLATFORM_VERSION=android-9
  
  build_x264
  
  build_ffmpeg
  
  build_one
  
  
  
  #mips
  
  EABIARCH=mipsel-linux-android
  
  ARCH=mips
  
  OPTIMIZE_CFLAGS="-EL -march=mips32 -mips32 -mhard-float"
  
  PREFIX=../ffmpeg-build/mips
  
  OUT_LIBRARY=$PREFIX/libffmpeg.so
  
  ADDITIONAL_CONFIGURE_FLAG=""
  
  SONAME=libffmpeg.so
  
  PREBUILT=$NDK/toolchains/mipsel-linux-android-4.8/prebuilt/$OS-x86_64
  
  #PREBUILT=$NDK/toolchains/mipsel-linux-android-4.6/prebuilt/$OS-x86
  
  PLATFORM_VERSION=android-9
  
  build_x264
  
  build_ffmpeg
  
  build_one

  build_all.sh

  #!/bin/bash
  
  #
  
  # build_all.sh
  
  
  
  export NDK=/home/tasol/DEVELOPMENT/IJOOMER_DEVELOPMENT/NDK/android-ndk-r10d
  
  sudo ./build_android.sh
  
  $NDK/ndk-build
  
  
  
  echo "Done Dona Done!!!"

  Android.mk

  LOCAL_PATH := $(call my-dir)
  
  
  
  include $(CLEAR_VARS)
  
  LOCAL_MODULE  := videokit
  
  # These need to be in the right order
  
  FFMPEG_LIBS := $(addprefix ffmpeg-build/$(TARGET_ARCH_ABI)/lib/, \
  
   libavdevice.a \
  
   libavformat.a \
  
   libavfilter.a \
  
   libavcodec.a \
  
   libswscale.a \
  
   libavutil.a \
  
   libswresample.a \
  
   libpostproc.a )
  
  # ffmpeg uses its own deprecated functions liberally, so turn off that annoying noise
  
  LOCAL_CFLAGS += -g -Iffmpeg -Ivideokit -Wno-deprecated-declarations 
  
  LOCAL_LDLIBS += -llog -lz $(FFMPEG_LIBS) ffmpeg-build/$(TARGET_ARCH_ABI)/lib/libx264.a
  
  LOCAL_SRC_FILES := videokit/uk_co_halfninja_videokit_Videokit.c videokit/ffmpeg.c videokit/cmdutils.c
  
  include $(BUILD_SHARED_LIBRARY)
  
  
  
  
  
  include $(CLEAR_VARS)
  
  LOCAL_MODULE  := ffmpeg
  
  FFMPEG_LIBS := $(addprefix ffmpeg-build/$(TARGET_ARCH_ABI)/lib/, \
  
   libavdevice.a \
  
   libavformat.a \
  
   libavfilter.a \
  
   libavcodec.a \
  
   libswscale.a \
  
   libavutil.a \
  
   libswresample.a \
  
   libpostproc.a )
  
  LOCAL_CFLAGS += -g -Iffmpeg -Ivideokit -Wno-deprecated-declarations 
  
  LOCAL_LDLIBS += -llog -lz $(FFMPEG_LIBS) ffmpeg-build/$(TARGET_ARCH_ABI)/lib/libx264.a
  
  LOCAL_SRC_FILES := ffmpeg/ffmpeg.c ffmpeg/cmdutils.c
  
  include $(BUILD_EXECUTABLE)

  Application.mk

  APP_ABI := armeabi,armeabi-v7a,x86,mips
  
  APP_PLATFORM := android-10
  
  APP_STL := gnustl_static

  UPDATE

  Finally Got success

• RTP packets detected as UDP

  8 juillet 2024, par fritz

  Here is what I am trying to do :

  



  WebRTC endpoint > RTP Endpoint > ffmpeg > RTMP server.


  



  This is what my SDP file looks like.

  



  var cm_offer = "v=0\n" +
              "o=- 3641290734 3641290734 IN IP4 127.0.0.1\n" +
              "s=nginx\n" +
              "c=IN IP4 127.0.0.1\n" +
              "t=0 0\n" +
              "m=audio 60820 RTP/AVP 0\n" +
              "a=rtpmap:0 PCMU/8000\n" +
              "a=recvonly\n" +
              "m=video 59618 RTP/AVP 101\n" +
              "a=rtpmap:101 H264/90000\n" +
              "a=recvonly\n";


  



  What's happening is that wireshark can detect the incoming packets at port 59618, but not as RTP packets but UDP packets. I am trying to capture the packets using ffmpeg with the following command :

  



  ubuntu@ip-132-31-40-100:~$ ffmpeg -i udp://127.0.0.1:59618 -vcodec copy stream.mp4
ffmpeg version git-2017-01-22-f1214ad Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libfreetype --enable-gnutls --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvidstab --enable-libwavpack --enable-nvenc
  libavutil      55. 44.100 / 55. 44.100
  libavcodec     57. 75.100 / 57. 75.100
  libavformat    57. 63.100 / 57. 63.100
  libavdevice    57.  2.100 / 57.  2.100
  libavfilter     6. 69.100 /  6. 69.100
  libavresample   3.  2.  0 /  3.  2.  0
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100 


  



  All I get is a blinking cursor and The stream.mp4 file is not written to disk after I exit (ctrl+c).

  



  So can you help me figure out :

  



  

  1. why wireshark cannot detect the packets as RTP (I suspect it has something to do with SDP)

  


  2. How to handle SDP answer when the RTP endpoint is pushing to ffmpeg which doesn't send an answer back.

  


  



  Here is the entire code (hello world tutorial modified)

  



  /*
     * (C) Copyright 2014-2015 Kurento (http://kurento.org/)
     *
     * Licensed under the Apache License, Version 2.0 (the "License");
     * you may not use this file except in compliance with the License.
     * You may obtain a copy of the License at
     *
     *   http://www.apache.org/licenses/LICENSE-2.0
     *
     * Unless required by applicable law or agreed to in writing, software
     * distributed under the License is distributed on an "AS IS" BASIS,
     * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     * See the License for the specific language governing permissions and
     * limitations under the License.
     */

    function getopts(args, opts)
    {
      var result = opts.default || {};
      args.replace(
          new RegExp("([^?=&]+)(=([^&]*))?", "g"),
          function($0, $1, $2, $3) { result[$1] = decodeURI($3); });

      return result;
    };

    var args = getopts(location.search,
    {
      default:
      {
        ws_uri: 'wss://' + location.hostname + ':8433/kurento',
        ice_servers: undefined
      }
    });

    function setIceCandidateCallbacks(webRtcPeer, webRtcEp, onerror)
    {
      webRtcPeer.on('icecandidate', function(candidate) {
        console.log("Local candidate:",candidate);

        candidate = kurentoClient.getComplexType('IceCandidate')(candidate);

        webRtcEp.addIceCandidate(candidate, onerror)
      });

      webRtcEp.on('OnIceCandidate', function(event) {
        var candidate = event.candidate;

        console.log("Remote candidate:",candidate);

        webRtcPeer.addIceCandidate(candidate, onerror);
      });
    }


    function setIceCandidateCallbacks2(webRtcPeer, rtpEp, onerror)
    {
      webRtcPeer.on('icecandidate', function(candidate) {
        console.log("Localr candidate:",candidate);

        candidate = kurentoClient.getComplexType('IceCandidate')(candidate);

        rtpEp.addIceCandidate(candidate, onerror)
      });
    }


    window.addEventListener('load', function()
    {
      console = new Console();

      var webRtcPeer;
      var pipeline;
      var webRtcEpt;

      var videoInput = document.getElementById('videoInput');
      var videoOutput = document.getElementById('videoOutput');

      var startButton = document.getElementById("start");
      var stopButton = document.getElementById("stop");

      startButton.addEventListener("click", function()
      {
        showSpinner(videoInput, videoOutput);

        var options = {
          localVideo: videoInput,
          remoteVideo: videoOutput
        };


        if (args.ice_servers) {
         console.log("Use ICE servers: " + args.ice_servers);
         options.configuration = {
           iceServers : JSON.parse(args.ice_servers)
         };
        } else {
         console.log("Use freeice")
        }

        webRtcPeer = kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options, function(error)
        {
          if(error) return onError(error)

          this.generateOffer(onOffer)
        });

        function onOffer(error, sdpOffer)
        {
          if(error) return onError(error)

          kurentoClient(args.ws_uri, function(error, client)
          {
            if(error) return onError(error);

            client.create("MediaPipeline", function(error, _pipeline)
            {
              if(error) return onError(error);

              pipeline = _pipeline;

              pipeline.create("WebRtcEndpoint", function(error, webRtc){
                if(error) return onError(error);

                webRtcEpt = webRtc;

                setIceCandidateCallbacks(webRtcPeer, webRtc, onError)

                webRtc.processOffer(sdpOffer, function(error, sdpAnswer){
                  if(error) return onError(error);

                  webRtcPeer.processAnswer(sdpAnswer, onError);
                });
                webRtc.gatherCandidates(onError);

                webRtc.connect(webRtc, function(error){
                  if(error) return onError(error);

                  console.log("Loopback established");
                });
              });



            pipeline.create("RtpEndpoint", function(error, rtp){
                if(error) return onError(error);

                //setIceCandidateCallbacks2(webRtcPeer, rtp, onError)


                var cm_offer = "v=0\n" +
                      "o=- 3641290734 3641290734 IN IP4 127.0.0.1\n" +
                      "s=nginx\n" +
                      "c=IN IP4 127.0.0.1\n" +
                      "t=0 0\n" +
                      "m=audio 60820 RTP/AVP 0\n" +
                      "a=rtpmap:0 PCMU/8000\n" +
                      "a=recvonly\n" +
                      "m=video 59618 RTP/AVP 101\n" +
                      "a=rtpmap:101 H264/90000\n" +
                      "a=recvonly\n";



                rtp.processOffer(cm_offer, function(error, cm_sdpAnswer){
                  if(error) return onError(error);

                  //webRtcPeer.processAnswer(cm_sdpAnswer, onError);
                });
                //rtp.gatherCandidates(onError);

                webRtcEpt.connect(rtp, function(error){
                  if(error) return onError(error);

                  console.log("RTP endpoint connected to webRTC");
                });
              });









            });
          });
        }
      });
      stopButton.addEventListener("click", stop);


      function stop() {
        if (webRtcPeer) {
          webRtcPeer.dispose();
          webRtcPeer = null;
        }

        if(pipeline){
          pipeline.release();
          pipeline = null;
        }

        hideSpinner(videoInput, videoOutput);
      }

      function onError(error) {
        if(error)
        {
          console.error(error);
          stop();
        }
      }
    })


    function showSpinner() {
      for (var i = 0; i < arguments.length; i++) {
        arguments[i].poster = 'img/transparent-1px.png';
        arguments[i].style.background = "center transparent url('img/spinner.gif') no-repeat";
      }
    }

    function hideSpinner() {
      for (var i = 0; i < arguments.length; i++) {
        arguments[i].src = '';
        arguments[i].poster = 'img/webrtc.png';
        arguments[i].style.background = '';
      }
    }

    /**
     * Lightbox utility (to display media pipeline image in a modal dialog)
     */
    $(document).delegate('*[data-toggle="lightbox"]', 'click', function(event) {
      event.preventDefault();
      $(this).ekkoLightbox();
    });


  


• FFmpeg + iOS + lossy cellular connections

  9 novembre 2014, par Moss

  I am able to play an RTMP audio + video real-time stream on iOS with FFmpeg. Works fantastic when everything is on a solid WiFi connection.

  When I switch to a cellular connection (great signal strength and LTE/4G), av_read_frame() will intermittently block for an unacceptable amount of time. From what I can tell, it’s not that the cellular data connection just dropped, because I can reconnect immediately and start downloading more packets. In some cases, I’ve clocked 30+ seconds of hang time before it returns the next frame. When the next frame finally comes in, my real-time video stream is permanently delayed by the amount of time that av_read_frame() blocked.

  I attempted a work-around by using the AVIOInterruptCB interrupt callback to abort av_read_frame() if the function takes longer than 1 second to return. Here’s what that code looks like :

  - (void)readPackets {
  
      // Make sure FFmpeg calls our interrupt periodically
  
      _context->interrupt_callback.callback = interrupt_cb;
  
      _context->interrupt_callback.opaque = self;
  
  
  
      dispatch_async(_readPacketQueue, ^(void) {
  
          int err;
  
  
  
          while(true) {
  
              _readFrameTimeStamp = [[NSDate date] timeIntervalSince1970];
  
              err = av_read_frame(_context, &packet);
  
              _readFrameTimeStamp = 0;
  
  
  
              if(err) {
  
                  // Error - Reconnect the entire stream from scratch, taking 5-10 seconds
  
                  // And we know when av_read_frame() was aborted
  
                  // because its error code is -1414092869 ("EXIT")
  
              }
  
              else {
  
                  // Play this audio or video packet
  
              }
  
          }
  
     });
  
  }
  
  
  
  /**
  
   * Interrupt
  
   * @return 1 to abort the current operation
  
   */
  
  static int interrupt_cb(void *decoder) {
  
      if(decoder) {
  
          if(_readFrameTimeStamp != 0) {
  
              if([[NSDate date] timeIntervalSince1970] - _readFrameTimeStamp > 1) {
  
                  // Abort av_read_frame(), it's taking longer than 1 second
  
                  return 1;
  
              }
  
          }
  
      }
  
  }

  This definitely aborts av_read_frame() after 1 second, but unfortunately after I do this, future attempts to call av_read_frame() result in EIO errors (-5), which indicates that the connection has been severed.

  As a result, I am forced to fully reconnect the viewer, which takes 5-10 seconds. (avformat_open_input() takes 3-4 seconds, and then find the stream info again takes 2-3 seconds, and then start reading frames).

  The 5-10 second delay to fully reconnect is much better than waiting more than 10 seconds for av_read_frame() to unblock, and it’s much better than the real-time stream being delayed by a significant amount. But it’s much worse than being able to retry av_read_frame() immediately.

  From a cellular user’s perspective, their video locks up intermittently for 5-10 seconds while we reconnect the stream in the background from scratch, which isn’t a good user experience.

  What strategies are there to better way to manage av_read_frame() on a lossy cellular connection ?
  (Or strategies to improve the reconnect time ?)