Recherche avancée

Sur d’autres sites (8258)

• Need help using libavfilter for adding overlay to frames [closed]

  30 juillet 2024, par Michael Werner

  Hello gentlemen and ladies,

  


  I am working with libavfilter and I am getting crazy.

  


  On Windows 11 OS with latest libav (full build) a C/C++ app reads YUV420P frames from a frame grabber card.

  


  I want to draw a bitmap (BGR24) overlay image from file on every frame via libavfilter. First I convert the BGR24 overlay image via format filter to YUV420P. Then I feed the YUV420P frame from frame grabber and the YUV420P overlay into the overlay filter.

  


  Everything seems to be fine but when I try to get the frame out of the filter graph I always get an "Resource is temporary not available" (EAGAIN) return code, independent on how many frames I put into the graph.

  


  The frames from the frame grabber card are fine, I could encode them or write them to a .yuv file. The overlay frame looks fine too.

  


  My current initialization code looks like below. It does not report any errors or warnings but when I try to get the filtered frame out of the graph via av_buffersink_get_frame I always get an EAGAIN return code.

  


  Here is my current initialization code :

  


  int init_overlay_filter(AVFilterGraph** graph, AVFilterContext** src_ctx, AVFilterContext** overlay_src_ctx,
                        AVFilterContext** sink_ctx)
{
    AVFilterGraph* filter_graph;
    AVFilterContext* buffersrc_ctx;
    AVFilterContext* overlay_buffersrc_ctx;
    AVFilterContext* buffersink_ctx;
    AVFilterContext* overlay_ctx;
    AVFilterContext* format_ctx;
    const AVFilter *buffersrc, *buffersink, *overlay_buffersrc, *overlay_filter, *format_filter;
    int ret;

    // Create the filter graph
    filter_graph = avfilter_graph_alloc();
    if (!filter_graph)
    {
        fprintf(stderr, "Unable to create filter graph.\n");
        return AVERROR(ENOMEM);
    }

    // Create buffer source filter for main video
    buffersrc = avfilter_get_by_name("buffer");
    if (!buffersrc)
    {
        fprintf(stderr, "Unable to find buffer filter.\n");
        return AVERROR_FILTER_NOT_FOUND;
    }

    // Create buffer source filter for overlay image
    overlay_buffersrc = avfilter_get_by_name("buffer");
    if (!overlay_buffersrc)
    {
        fprintf(stderr, "Unable to find buffer filter.\n");
        return AVERROR_FILTER_NOT_FOUND;
    }

    // Create buffer sink filter
    buffersink = avfilter_get_by_name("buffersink");
    if (!buffersink)
    {
        fprintf(stderr, "Unable to find buffersink filter.\n");
        return AVERROR_FILTER_NOT_FOUND;
    }

    // Create overlay filter
    overlay_filter = avfilter_get_by_name("overlay");
    if (!overlay_filter)
    {
        fprintf(stderr, "Unable to find overlay filter.\n");
        return AVERROR_FILTER_NOT_FOUND;
    }

    // Create format filter
    format_filter = avfilter_get_by_name("format");
    if (!format_filter) 
    {
        fprintf(stderr, "Unable to find format filter.\n");
        return AVERROR_FILTER_NOT_FOUND;
    }

    // Initialize the main video buffer source
    char args[512];
    snprintf(args, sizeof(args),
             "video_size=1920x1080:pix_fmt=yuv420p:time_base=1/25:pixel_aspect=1/1");
    ret = avfilter_graph_create_filter(&buffersrc_ctx, buffersrc, "in", args, NULL, filter_graph);
    if (ret < 0)
    {
        fprintf(stderr, "Unable to create buffer source filter for main video.\n");
        return ret;
    }

    // Initialize the overlay buffer source
    snprintf(args, sizeof(args),
             "video_size=165x165:pix_fmt=bgr24:time_base=1/25:pixel_aspect=1/1");
    ret = avfilter_graph_create_filter(&overlay_buffersrc_ctx, overlay_buffersrc, "overlay_in", args, NULL,
                                       filter_graph);
    if (ret < 0)
    {
        fprintf(stderr, "Unable to create buffer source filter for overlay.\n");
        return ret;
    }

    // Initialize the format filter to convert overlay image to yuv420p
    snprintf(args, sizeof(args), "pix_fmts=yuv420p");
    ret = avfilter_graph_create_filter(&format_ctx, format_filter, "format", args, NULL, filter_graph);

    if (ret < 0) 
    {
        fprintf(stderr, "Unable to create format filter.\n");
        return ret;
    }

    // Initialize the buffer sink
    ret = avfilter_graph_create_filter(&buffersink_ctx, buffersink, "out", NULL, NULL, filter_graph);
    if (ret < 0)
    {
        fprintf(stderr, "Unable to create buffer sink filter.\n");
        return ret;
    }

    // Initialize the overlay filter
    ret = avfilter_graph_create_filter(&overlay_ctx, overlay_filter, "overlay", "W-w:H-h:enable='between(t,0,20)':format=yuv420", NULL, filter_graph);
    if (ret < 0)
    {
        fprintf(stderr, "Unable to create overlay filter.\n");
        return ret;
    }

    // Connect the filters
    ret = avfilter_link(overlay_buffersrc_ctx, 0, format_ctx, 0);

    if (ret >= 0)
    {
        ret = avfilter_link(buffersrc_ctx, 0, overlay_ctx, 0);
    }
    else
    {
        fprintf(stderr, "Unable to configure filter graph.\n");
        return ret;
    }


    if (ret >= 0) 
    {
        ret = avfilter_link(format_ctx, 0, overlay_ctx, 1);
    }
    else
    {
        fprintf(stderr, "Unable to configure filter graph.\n");
        return ret;
    }

    if (ret >= 0) 
    {
        if ((ret = avfilter_link(overlay_ctx, 0, buffersink_ctx, 0)) < 0)
        {
            fprintf(stderr, "Unable to link filter graph.\n");
            return ret;
        }
    }
    else
    {
        fprintf(stderr, "Unable to configure filter graph.\n");
        return ret;
    }

    // Configure the filter graph
    if ((ret = avfilter_graph_config(filter_graph, NULL)) < 0)
    {
        fprintf(stderr, "Unable to configure filter graph.\n");
        return ret;
    }

    *graph = filter_graph;
    *src_ctx = buffersrc_ctx;
    *overlay_src_ctx = overlay_buffersrc_ctx;
    *sink_ctx = buffersink_ctx;

    return 0;
}


  


  Feeding the filter graph is done this way :

  


  av_buffersrc_add_frame_flags(buffersrc_ctx, pFrameGrabberFrame, AV_BUFFERSRC_FLAG_KEEP_REF)
av_buffersink_get_frame(buffersink_ctx, filtered_frame)


  


  av_buffersink_get_frame returns always EAGAIN, no matter how many frames I feed into the graph. The frames (from framegrabber and the overlay frame) itself are looking fine.

  


  I did set libav logging level to maximum but I do not see any warnings or errors or helpful, related information in the log.

  


  Here the log output related to the filter configuration :

  


  [in @ 00000288ee494f40] Setting 'video_size' to value '1920x1080'
[in @ 00000288ee494f40] Setting 'pix_fmt' to value 'yuv420p'
[in @ 00000288ee494f40] Setting 'time_base' to value '1/25'
[in @ 00000288ee494f40] Setting 'pixel_aspect' to value '1/1'
[in @ 00000288ee494f40] w:1920 h:1080 pixfmt:yuv420p tb:1/25 fr:0/1 sar:1/1 csp:unknown range:unknown
[overlay_in @ 00000288ff1013c0] Setting 'video_size' to value '165x165'
[overlay_in @ 00000288ff1013c0] Setting 'pix_fmt' to value 'bgr24'
[overlay_in @ 00000288ff1013c0] Setting 'time_base' to value '1/25'
[overlay_in @ 00000288ff1013c0] Setting 'pixel_aspect' to value '1/1'
[overlay_in @ 00000288ff1013c0] w:165 h:165 pixfmt:bgr24 tb:1/25 fr:0/1 sar:1/1 csp:unknown range:unknown
[format @ 00000288ff1015c0] Setting 'pix_fmts' to value 'yuv420p'
[overlay @ 00000288ff101880] Setting 'x' to value 'W-w'
[overlay @ 00000288ff101880] Setting 'y' to value 'H-h'
[overlay @ 00000288ff101880] Setting 'enable' to value 'between(t,0,20)'
[overlay @ 00000288ff101880] Setting 'format' to value 'yuv420'
[auto_scale_0 @ 00000288ff101ec0] w:iw h:ih flags:'' interl:0
[format @ 00000288ff1015c0] auto-inserting filter 'auto_scale_0' between the filter 'overlay_in' and the filter 'format'
[auto_scale_1 @ 00000288ee4a4cc0] w:iw h:ih flags:'' interl:0
[overlay @ 00000288ff101880] auto-inserting filter 'auto_scale_1' between the filter 'format' and the filter 'overlay'
[AVFilterGraph @ 00000288ee495c80] query_formats: 5 queried, 6 merged, 6 already done, 0 delayed
[auto_scale_0 @ 00000288ff101ec0] w:165 h:165 fmt:bgr24 csp:gbr range:pc sar:1/1 -> w:165 h:165 fmt:yuv420p csp:unknown range:unknown sar:1/1 flags:0x00000004
[auto_scale_1 @ 00000288ee4a4cc0] w:165 h:165 fmt:yuv420p csp:unknown range:unknown sar:1/1 -> w:165 h:165 fmt:yuva420p csp:unknown range:unknown sar:1/1 flags:0x00000004
[overlay @ 00000288ff101880] main w:1920 h:1080 fmt:yuv420p overlay w:165 h:165 fmt:yuva420p
[overlay @ 00000288ff101880] [framesync @ 00000288ff1019a8] Selected 1/25 time base
[overlay @ 00000288ff101880] [framesync @ 00000288ff1019a8] Sync level 2


  


• JavaCV on Android : UnsatisfiedLinkError from libjniavdevice.so / dlopen failed cannot locate symbol "av_input_audio_device_next"

  8 septembre 2014, par liorg

  I am trying to work with JavaCV’s FFmpegFrameGrabber on Android 4.4.2 / 4.3.1 .

  Inside the onCreate(...) method I am calling :

  FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(path)

  This causes the following runtime error :

  dlopen("/data/app-lib/com.example.testjavacv-1/libjniavdevice.so")
  
  failed: dlopen failed: cannot locate symbol "av_input_audio_device_next"
  
  referenced by "libjniavdevice.so"

  Other (possible) relevant information :

  1) I have included all the JavaCV .so libraries inside the libs/armeabi and libs/x86 folders.

  2) I have given the application permissions to read/write to external storage and access the camera and microphone.

  3) I am NOT statically loading any of the JavaCV native libraries before instantiating FFmpegFrameGrabber. This is because,

  • Some of the libraries cause runtime exceptions when I attempt to load them statically by myself.
  • I get the impression that JavaCV handles it’s own native library loading - for example the error I get ocurrs even though I have not requested to load the jniavdevice.so library and happens only if the ... new FFmpegFrameGrabber(...) ... code is present.

  4) I have taken a look at : https://groups.google.com/forum/# !topic/javacv/bgB5rCv_fVM which seems to address a similar issue, but the suggested solutions don’t seem to work.

  Any help on this issue would be much appreciated,
  Cheers !

• avcodec/adpcm : squelch ’mismatch in coded sample count’ warning for AV_CODEC_ID_ADPCM...

  14 mars 2014, par Peter Ross

  avcodec/adpcm : squelch ’mismatch in coded sample count’ warning for AV_CODEC_ID_ADPCM_EA_R2/3
  These ADPCM codecs include a per-frame flag that enables a raw 16-bit mode. Therefore
  
  the the number of samples returned by get_nb_samples() is only ever approximate.
  Fixes ticket #3460.
  Signed-off-by : Peter Ross <pross@xvid.org>
  
  Reviewed-by : Paul B Mahol <onemda@gmail.com>
  
  Signed-off-by : Michael Niedermayer <michaelni@gmx.at>
  

  • [DH] libavcodec/adpcm.c