Recherche avancée

Sur d’autres sites (6213)

• Android sws_scale RGB0 Frame taking long time and cause video latency [duplicate]

  17 avril 2018, par AJit

  This question already has an answer here :

  • Hardware scaling the OpenGL surface with NativeActivity
    
    1 answer
    

  Reading frames from FFMPEG and try to directly draw on a surface window in Native android, If scale the image to exact size what we are getting from the camera and apply YUV420P to RGBA it take 1ms to scale through av_image_fill_arrays but if scale image surface size, then It took 25 to 30ms to scale a same frame.

  Low latency :[ 1ms by sws_scale]

  swsContext = sws_getContext(videoCodecContext->width,
  
                                      videoCodecContext->height,
  
                                      videoCodecContext->pix_fmt,
  
                                      videoCodecContext->width,
  
                                      videoCodecContext->height,
  
                                      AV_PIX_FMT_RGB0,
  
                                      SWS_FAST_BILINEAR, NULL, NULL, NULL);
  
  av_image_fill_arrays()
  
  av_read_frame()
  
  avcodec_decode_video2(
  
  sws_scale(swsContext,
  
            (const uint8_t *const *) videoFrame->data,
  
            videoFrame->linesize,
  
            0,
  
            videoCodecContext->height,
  
            pictureFrame->data,
  
            pictureFrame->linesize);
  
  ANativeWindow_lock()
  
  Write all buffer bytes to window.
  
  ANativeWindow_unlockAndPost()

  Low latency :[ 30ms by sws_scale]

  [videoContext Width: 848 Height: 608]
  
  swsContext = sws_getContext(videoCodecContext->width,
  
                                      videoCodecContext->height,
  
                                      videoCodecContext->pix_fmt,
  
                                      1080,
  
                                      608,
  
                                      AV_PIX_FMT_RGB0,
  
                                      SWS_FAST_BILINEAR, NULL, NULL, NULL);

  NOTE : we are getting YUV420P pixel format.

  Whenever we change context width and height other than videoContext it will take more than 30ms so result is video delaying.

  TRY 1 : Pass the buffer from JNI to Java and create bitmap there to scale later on but createBitmap itself take ~500ms which is not good enough.

  TRY 2 : Direct YUV420P to RGB conversion, but still sws_scale is greater.

  TRY 3 : Direct writes YUV to window bites, but it shows without color (If anyone has a solution here will might helpful).

  TRY 4 : yuvlib, still not worth it.

  TRY 5 : Different pixel formats and flags in swsContext.

  

  Any help would appreciated.

• C pointer casting to and from Golang

  26 avril 2018, par nevernew

  I’m writing an app for the windows platform using FFmpeg and it’s golang wrapper goav, but I’m having trouble understanding how to pass the C pointers between C and Golang.

  I’ve stripped out all the relevant parts of the C code, the wrapper and my code, shown below :

  C code - libavutil/frame.h

  #include 
  
  
  
  typedef struct AVFrame {
  
  #define AV_NUM_DATA_POINTERS 8
  
      uint8_t *data[AV_NUM_DATA_POINTERS];
  
  }

  Golang goav wrapper

  package avutil
  
  
  
  /*
  
      #cgo pkg-config: libavutil
  
      #include <libavutil></libavutil>frame.h>
  
      #include 
  
  
  
      // C code I added:
  
      #include 
  
  
  
      void SaveFrame(const char* location, uint8_t *data, int width, int height) {
  
          FILE *pFile;
  
          int  y;
  
  
  
          // Open file
  
          pFile=fopen(location, "wb");
  
          if(pFile==NULL)
  
              return;
  
  
  
          // Write header
  
          fprintf(pFile, "P6\n%d %d\n255\n", width, height);
  
  
  
          // Write pixel data
  
          for(y=0; y/ Close file
  
          fclose(pFile);
  
      }
  
  */
  
  import "C"
  
  import (
  
      "unsafe"
  
  )
  
  
  
  type Frame C.struct_AVFrame
  
  
  
  func Data(f *Frame) *uint8 {
  
      // i think this is the function thats not working?
  
      return (*uint8)(unsafe.Pointer((*C.uint8_t)(unsafe.Pointer(&amp;f.data))))
  
  }
  
  func SaveFrame(location string, data *uint8, width int, height int) {
  
      C.SaveFrame(C.CString(location), unsafe.Pointer(data), C.int(width), C.int(height))
  
  }

  My Golang code

  package main
  
  
  
  import "github.com/giorgisio/goav/avutil"
  
  
  
  func main() {
  
      var frame *avutil.Frame
  
      var data *uint8
  
  
  
      //... initialize frame
  
  
  
      data = avutil.Data(frame)
  
      avutil.SaveFrame("frame0.ppm", data, 1920, 1080)
  
  }

  When I try to save the frame, the resulting image is garbled because the pointer is wrong, how do i fix this ?

• Android sws_scale RGB Frame taking long time and cause of latency in the video

  17 avril 2018, par AJit

  I am reading frames from FFMPEG and try to directly draw to surface window in Native android, If i scale image to exact size what we are getting from camera and apply YUV420P to RGBA it take 1ms to scale through av_image_fill_arrays but if i try to scale image what i need for surface, then i take 25 to 30ms to scale same frame. so latency is the problem.

  In below example getting YUV420P pixel format.

  Low latency :[ 1ms by sws_scale]

  swsContext = sws_getContext(videoCodecContext->width,
  
                                      videoCodecContext->height,
  
                                      videoCodecContext->pix_fmt,
  
                                      videoCodecContext->width,
  
                                      videoCodecContext->height,
  
                                      AV_PIX_FMT_RGB0,
  
                                      SWS_FAST_BILINEAR, NULL, NULL, NULL);
  
  av_image_fill_arrays()
  
  av_read_frame()
  
  avcodec_decode_video2(
  
  sws_scale(swsContext,
  
            (const uint8_t *const *) videoFrame->data,
  
            videoFrame->linesize,
  
            0,
  
            videoCodecContext->height,
  
            pictureFrame->data,
  
            pictureFrame->linesize);
  
  ANativeWindow_lock()
  
  Write all buffer bytes to window.
  
  ANativeWindow_unlockAndPost()

  Low latency :[ 30ms by sws_scale]

  [videoContext Width: 848 Height: 608]
  
  swsContext = sws_getContext(videoCodecContext->width,
  
                                      videoCodecContext->height,
  
                                      videoCodecContext->pix_fmt,
  
                                      1080,
  
                                      608,
  
                                      AV_PIX_FMT_RGB0,
  
                                      SWS_FAST_BILINEAR, NULL, NULL, NULL);

  Whenever we change context width and height other then videoContext it will take more then 30ms and we are delaying the video.

  TRY 1 : Pass the buffer from JNI to Java and create bitmap there to scale later on but createBitmap itself take 500ms so no useful.

  TRY 2 : Direct YUV420P to RGB conversion. still long time then sws_scale.

  TRY 3 : Direct write YUV to window bites but it show without color(If anyone have solution here will might helpful).

  TRY 4 : Use yuvlib, no luck.

  TRY 5 : Different color combinations and flags in swsContext.

  Any help would appreciated.