Recherche avancée

Sur d’autres sites (4094)

• texture rendering issue on iOS using OpenGL ES in Unity project

  28 mars 2016, par Time1ess

  I’m working on a project, part of it is to streaming video to my iPhone, currently I use my laptop to create the video stream to my iPhone with ffmpeg.The stream code in shell is below :

  ffmpeg \
  
      -f avfoundation -i "1" -s 1280*720 -r 29.97 \
  
      -c:v mpeg2video -q:v 20 -pix_fmt yuv420p -g 1 -threads 4\
  
      -f mpegts udp://192.168.1.102:6666

  with this, I successfully create my video stream.

  In Unity, I want to decode the video stream to create a texture. After I have gone through with some ffmpeg tutorial and Unity tutorial(Since I’m new to both of them), I followed tutorials and created my link library. Some of these codes are below(ask me if more is needed) :

  In my library :

  buffer alloc :

  uint8_t *buffer;
  
  int buffer_size;
  
  buffer_size = avpicture_get_size(AV_PIX_FMT_RGBA, VIEW_WIDTH, VIEW_HEIGHT);
  
  
  
  buffer = (uint8_t *) av_malloc(buffer_size*sizeof(uint8_t));
  
  
  
  avpicture_fill((AVPicture *) pFrameRGB, buffer, AV_PIX_FMT_RGBA,
  
                 VIEW_WIDTH, VIEW_HEIGHT);

  getContext :

      is->sws_ctx = sws_getContext
  
      (
  
       is->video_st->codec->width,
  
       is->video_st->codec->height,
  
       is->video_st->codec->pix_fmt,
  
       VIEW_WIDTH,
  
       VIEW_HEIGHT,
  
       AV_PIX_FMT_RGBA,
  
       SWS_BILINEAR,
  
       NULL,
  
       NULL,
  
       NULL
  
       );

  sws_scale :

  sws_scale(
  
            is->sws_ctx,
  
            (uint8_t const * const *)pFrame->data,
  
            pFrame->linesize,
  
            0,
  
            is->video_st->codec->height,
  
            pFrameRGB->data,
  
            pFrameRGB->linesize
  
            );

  texture render :

  static void UNITY_INTERFACE_API OnRenderEvent(int texID)
  
  {
  
      GLuint gltex = (GLuint)(size_t)(texID);
  
  
  
      glBindTexture(GL_TEXTURE_2D, gltex);
  
  
  
      glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, VIEW_WIDTH, VIEW_HEIGHT,
  
                      GL_RGBA, GL_UNSIGNED_BYTE, pFrameRGB->data[0]);
  
  
  
      glGetError();
  
      return;
  
  }
  
  
  
  extern "C" UnityRenderingEvent UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API GetRenderEventFunc()
  
  {
  
      return OnRenderEvent;
  
  }

  In Unity :

  texture created :

      private Texture2D texture;
  
      private int texID;
  
      texture = new Texture2D (width, height, TextureFormat.RGBA32, false);
  
      texture.filterMode = FilterMode.Point;
  
      texture.Apply ();
  
      GetComponent<renderer> ().material.mainTexture = texture;
  
      texID = texture.GetNativeTexturePtr ().ToInt32();
  
  </renderer>

  update func :

      void Update ()
  
      {
  
          GL.IssuePluginEvent(GetRenderEventFunc(), texID);
  
      }

  Video stream info :

  Input #0, mpegts, from 'udp://0.0.0.0:6666':
  
    Duration: N/A, start: 2.534467, bitrate: N/A
  
    Program 1 
  
      Metadata:
  
        service_name    : Service01
  
        service_provider: FFmpeg
  
      Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], max. 104857 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

  Leave other details, my library works fine on the Unity simulator, but when I compiled all my libraries for arm64 and used the xcode project that Unity created to build my app and ran it, I couldn’t get any texture rendered in my iPhone, I checkd my network and I’m sure that data had been sent to my iPhone and the Debug log showed me that frame has been successfully decoded also the OnRenderEvent func had been called.

  I’m confused and try to find answer on stackoverflow, maybe I’m a beginer cause I can’t find answers, so I ask you guys to help me plz.

  FYI :

  Unity 5.3.2f1 Personal

  Xcode 7.2.1

  iOS 9.2.1

  ffmpeg 3.0

• swscale/arm : re-enable neon rgbx to nv12 routines

  22 février 2016, par Xiaolei Yu

  swscale/arm : re-enable neon rgbx to nv12 routines
  Commit ’842b8f4ba2e79b9c004a67f6fdb3d5c5d05805d3’ fixed clang/iphone
  
  build but failed on some versions of cygwin. It has now been verified
  
  to work on both platforms.
  Signed-off-by : Michael Niedermayer <michael@niedermayer.cc>
  

  • [DH] libswscale/arm/Makefile
  • [DH] libswscale/arm/swscale_unscaled.c

• Detect rotation angle and rotate final video using ffmpeg version 2.8

  3 février 2016, par Hemant Kumar

  I used to work on ffmpeg 2.2 until now and was detecting rotation angle of video uploaded from android / iPhone mobiles and rotate the resulting video so that it run perfectly on a correct angle.

  But since I have updated the ffmpeg to version 2.8 I am getting this rotation problem. My queries are not rotating the videos as they were earlier.

  Here’r the commands I was using :

  To check rotation angle :

  ffprobe -of json -show_streams {$input} | grep rotate

  below is my final command to convert a video to mp4

  "ffmpeg -i {$input} -strict -2 -vcodec libx264 -preset slow -vb 500k -maxrate 500k -bufsize 1000k -vf 'scale=-1:480 ".fix_video_orientation($input)."' -threads 0 -ab 64k -s {$resolution}  -movflags faststart -metadata:s:v:0 rotate=0 {$output}";

  "fix_video_orientation" function is given below. It detect the angle of rotation of the initial video and output optimal option for rotating the final video.

  function fix_video_orientation($input){
  
  
  
  $return= ", transpose=1 ";
  
  
  
  $dd= exec("ffprobe -of json -show_streams  {$input}   | grep rotate");
  
  
  
  if(!empty($dd)){
  
  
  
  $dd=explode(":",$dd);
  
  $rotate=str_replace(",","",str_replace('"',"",$dd[1]));
  
  
  
  if($rotate=="90")return $return;
  
  
  
  else if ($rotate=="180") return ", transpose=2,transpose=2 ";
  
  
  
  else if($rotate == "270") return ", transpose=2 ";
  
  }

  Currently above script is supporting "flv","avi","mp4","mkv","mpg","wmv","asf","webm","mov","3gp","3gpp" extensions, also the script is supporting the resulting .mp4 file to play on all browsers and devices.

  Now the ffprobe command is not returning rotation angle and so a portrait video if uploaded from iphone is showing as landscape on the website.

  Output console of ffprobe command :

  ffprobe version N-77455-g4707497 Copyright (c) 2007-2015 the FFmpeg developers
  
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04)
  
  configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libdcadec --enable-libfreetype --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvo-aacenc --enable-libvidstab
  
  libavutil 55. 11.100 / 55. 11.100
  
  libavcodec 57. 20.100 / 57. 20.100
  
  libavformat 57. 20.100 / 57. 20.100
  
  libavdevice 57. 0.100 / 57. 0.100
  
  libavfilter 6. 21.101 / 6. 21.101
  
  libavresample 3. 0. 0 / 3. 0. 0
  
  libswscale 4. 0.100 / 4. 0.100
  
  libswresample 2. 0.101 / 2. 0.101
  
  libpostproc 54. 0.100 / 54. 0.100
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/standard/PORTRAIT.m4v':
  
  Metadata:
  
  major_brand : qt
  
  
  
  minor_version : 0
  
  compatible_brands: qt
  
  
  
  creation_time : 2016-02-03 05:25:18
  
  com.apple.quicktime.make: Apple
  
  com.apple.quicktime.model: iPhone 4S
  
  com.apple.quicktime.software: 9.2.1
  
  com.apple.quicktime.creationdate: 2016-02-03T10:52:11+0530
  
  Duration: 00:00:03.34, start: 0.000000, bitrate: 7910 kb/s
  
  Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 63 kb/s (default)
  
  Metadata:
  
  creation_time : 2016-02-03 05:25:18
  
  handler_name : Core Media Data Handler
  
  Stream #0:1(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 720x1280 [SAR 1:1 DAR 9:16], 7832 kb/s, 29.97 fps, 29.97 tbr, 600 tbn, 50 tbc (default)
  
  Metadata:
  
  creation_time : 2016-02-03 05:25:18
  
  handler_name : Core Media Data Handler
  
  encoder : H.264
  
  Stream #0:2(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
  
  Metadata:
  
  creation_time : 2016-02-03 05:25:18
  
  handler_name : Core Media Data Handler
  
  Stream #0:3(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
  
  Metadata:
  
  creation_time : 2016-02-03 05:25:18
  
  handler_name : Core Media Data Handler
  
  Unsupported codec with id 0 for input stream 2
  
  Unsupported codec with id 0 for input stream 3

  If latest version of ffmpeg (2.8) is used to auto rotate the video, can you please suggest me what option I need to add or remove from my final command.