Recherche avancée

Sur d’autres sites (4725)

• Android sws_scale RGB0 Frame taking long time and cause video latency [duplicate]

  17 avril 2018, par AJit

  This question already has an answer here :

  • Hardware scaling the OpenGL surface with NativeActivity
    
    1 answer
    

  Reading frames from FFMPEG and try to directly draw on a surface window in Native android, If scale the image to exact size what we are getting from the camera and apply YUV420P to RGBA it take 1ms to scale through av_image_fill_arrays but if scale image surface size, then It took 25 to 30ms to scale a same frame.

  Low latency :[ 1ms by sws_scale]

  swsContext = sws_getContext(videoCodecContext->width,
  
                                      videoCodecContext->height,
  
                                      videoCodecContext->pix_fmt,
  
                                      videoCodecContext->width,
  
                                      videoCodecContext->height,
  
                                      AV_PIX_FMT_RGB0,
  
                                      SWS_FAST_BILINEAR, NULL, NULL, NULL);
  
  av_image_fill_arrays()
  
  av_read_frame()
  
  avcodec_decode_video2(
  
  sws_scale(swsContext,
  
            (const uint8_t *const *) videoFrame->data,
  
            videoFrame->linesize,
  
            0,
  
            videoCodecContext->height,
  
            pictureFrame->data,
  
            pictureFrame->linesize);
  
  ANativeWindow_lock()
  
  Write all buffer bytes to window.
  
  ANativeWindow_unlockAndPost()

  Low latency :[ 30ms by sws_scale]

  [videoContext Width: 848 Height: 608]
  
  swsContext = sws_getContext(videoCodecContext->width,
  
                                      videoCodecContext->height,
  
                                      videoCodecContext->pix_fmt,
  
                                      1080,
  
                                      608,
  
                                      AV_PIX_FMT_RGB0,
  
                                      SWS_FAST_BILINEAR, NULL, NULL, NULL);

  NOTE : we are getting YUV420P pixel format.

  Whenever we change context width and height other than videoContext it will take more than 30ms so result is video delaying.

  TRY 1 : Pass the buffer from JNI to Java and create bitmap there to scale later on but createBitmap itself take ~500ms which is not good enough.

  TRY 2 : Direct YUV420P to RGB conversion, but still sws_scale is greater.

  TRY 3 : Direct writes YUV to window bites, but it shows without color (If anyone has a solution here will might helpful).

  TRY 4 : yuvlib, still not worth it.

  TRY 5 : Different pixel formats and flags in swsContext.

  

  Any help would appreciated.

• DirectX D3D11 Hardware Scaling Filter [closed]

  30 décembre 2024, par Dash Santosh

  I'm trying to implement a Hardware scaling filter (vf_scale_d3d11) using the VideoProcessBlt D3D11 DDI, and also add PIX_FMT_D3D11 support for the MediaFoundation Encoder (mfenc).

  


  Initially, there were a lot of "Static surface pool size exceeded".

  


  decode_receive_frame_internal failed = Invalid data found when processing input
Static surface pool size exceeded
get_buffer() failed
thread_get_buffer() failed
decode_slice_headec error
no frame!


  


  This got fixed by incrementing initial_pool_size by 3. Now the following two flows works as expected :

  


  

  1. ./ffmpeg.exe -hwaccel d3d11va -hwaccel_output_format d3d11 -hide_banner -i bbb_sunflower_2160p_30fps_normal.mp4 -hw_encoding 1 -c:v h264_mf -vframes 100 -loglevel verbose sample.mp4 -y

  


  2. ./ffmpeg.exe -hwaccel d3d11va -hwaccel_output_format d3d11 -hide_banner -i bbb_sunflower_2160p_30fps_normal.mp4 -hw_encoding 1 -c:v h264_mf -vf "scale_d3d11=width=1920:height=1080" -vframes 100 -loglevel verbose sample.mp4 -y

  


  


  When I try to call av_hwframe_transfer_data() within the if(encoder_requires_software_frame) block for the following flow, "Static surface pool size exceeded" occurs once again.

  


  

  3. ./ffmpeg.exe -hwaccel d3d11va -hwaccel_output_format d3d11 -hide_banner -i bbb_sunflower_2160p_30fps_normal.mp4 -c:v rawvideo -vframes 10 -loglevel verbose sample.yuv -y

  


  


  Increasing the initial_pool_size to any higher value doesn't seem to resolve this issue. The same error can be observed when the changes are integrated in HandBrake application, even for use cases (1) and (2). Any insights on why this would occur and how to fix this would be appreciated. Link to patch : https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20241230/ce28c8db/attachment.obj

  


• Decoded H.264 gives different frame and context size

  28 août 2015, par Deanna

  We’re using avcodec to decode H.264, and in some circumstances, after changing the resolution, avcodec gets confused, and gives two different sizes for the decoded frame :

  if (av_init_packet_dll)
  
      av_init_packet_dll(&avpkt);
  
  
  
  avpkt.data  = pBuffer;
  
  avpkt.size  = lBuffer;
  
  
  
  //  Make sure the output frame has NULLs for the data lines
  
  pAVFrame->data[0]   = NULL;
  
  pAVFrame->data[1]   = NULL;
  
  pAVFrame->data[2]   = NULL;
  
  pAVFrame->data[3]   = NULL;
  
  
  
  res = avcodec_decode_video2_dll(pCodecCtx, pAVFrame, &FrameFinished, &avpkt);
  
  
  
  DEBUG_LOG("Decoded frame: %d, %d, resulting dimensions: context: %dx%d, frame: %dx%d\n", res, FrameFinished, pCodecCtx->width, pCodecCtx->height, pAVFrame->width, pAVFrame->height);
  
  
  
  if (pCodecCtx->width != pAVFrame->width || pCodecCtx->height != pAVFrame->height) {
  
      OutputDebugStringA("Size mismatch, ignoring frame!\n");
  
      FrameFinished = 0;
  
  }
  
  
  
  if (FrameFinished == 0)
  
      OutputDebugStringA("Unfinished frame\n");

  This results in this log (with some surrounding lines) :

  [5392] Decoded frame: 18690, 1, resulting dimensions: context: 640x480, frame: 640x480
  
  [5392] Set dimensions to 640x480 in DecodeFromMap
  
  [5392] checking size 640x480 against 640x480
  
  [5392] Drawing 640x480, 640x480, 640x480, 0x05DB0060, 0x05DFB5C0, 0x05E0E360, 0x280, to surface 0x03198100, 1280x800
  
  [5392] Drawing 640x480, 640x480, 640x480, 0x05DB0060, 0x05DFB5C0, 0x05E0E360, 0x280, to surface 0x03198100, 1280x800
  
  [5392] Delayed frames seen. Reenabling low delay requires a codec flush.
  
  [5392] Reinit context to 1280x800, pix_fmt: yuvj420p
  
  *[5392] Decoded frame: 54363, 1, resulting dimensions: context: 1280x800, frame: 640x480
  
  [5392] Set dimensions to 1280x800 in DecodeFromMap
  
  [5392] checking size 1280x800 against 640x480
  
  [5392] Found adapter NVIDIA GeForce GTX 650 ({D7B71E3E-4C86-11CF-4E68-7E291CC2C435}) on monitor 00020003
  
  [5392] Found adapter NVIDIA GeForce GTX 650 ({D7B71E3E-4C86-11CF-4E68-7E291CC2C435}) on monitor FA650589
  
  [5392] Creating Direct3D interface on adapter 1 at 1280x800 window 0015050C
  
  [5392] Direct3D created using hardware vertex processing on HAL.
  
  [5392] Creating D3D surface of 1280x800
  
  [5392] Result 0x00000000, got surface 0x03210C40
  
  [5392] Drawing 1280x800, 1280x800, 640x480, 0x02E3B0A0, 0x02E86600, 0x02E993A0, 0x280, to surface 0x03210C40, 1280x800

  The line where this breaks is marked with a *. pAVFrame contains the old frame dimensions, while pCodecCtx contains the new dimensions. When the drawing code than tries to access the data as a 1280x800 image, it hits an access violation.

  When going down a size, avcodec transitions correctly, and sets FrameFinished to 0 and leaves pAVFrame resolution at 0x0.

  Can anyone think what is causing this, why avcodec is reporting success, yet not doing anything, and what I can do to correctly resolve this ?

  For now, the mismatch check is protecting against this.

  The avcodec in use is built from git-5cba529 by Zeranoe.

  FFmpeg version: 2015-03-31 git-5cba529
  
    libavutil      54. 21.100 / 54. 21.100
  
    libavcodec     56. 32.100 / 56. 32.100