Recherche avancée

Sur d’autres sites (5363)

• Troubleshooting ffmpeg/ffplay client RTSP RTP UDP * multicast * issue

  6 novembre 2020, par MAXdB

  I'm having problem with using udp_multicast transport method using ffmpeg or ffplay as a client to a webcam.

  


  TCP transport works :

  


  ffplay -rtsp_transport tcp rtsp://192.168.1.100/videoinput_1/mjpeg_3/media.stm


  


  UDP transport works :

  


  ffplay -rtsp_transport udp rtsp://192.168.1.100/videoinput_1/mjpeg_3/media.stm


  


  Multicast transport does not work :

  


  ffplay -rtsp_transport udp_multicast rtsp://192.168.1.100/videoinput_1/mjpeg_3/media.stm


  


  The error message when udp_multicast is chosen reads :

  


  [rtsp @ 0x7fd6a8000b80] Could not find codec parameters for stream 0 (Video: mjpeg, none(bt470bg/unknown/unknown)): unspecified size


  


  Run with -v debug : Observe that the UDP multicast information appears in the SDP even though the chosen transport is unicast for this run. The SDP content is unchanged for unicast or multicast.

  


  [tcp @ 0x7f648c002f40] Starting connection attempt to 192.168.1.100 port 554
[tcp @ 0x7f648c002f40] Successfully connected to 192.168.1.100 port 554
[rtsp @ 0x7f648c000b80] SDP:
v=0
o=- 621355968671884050 621355968671884050 IN IP4 192.168.1.100
s=/videoinput_1:0/mjpeg_3/media.stm
c=IN IP4 0.0.0.0
m=video 40004 RTP/AVP 26
c=IN IP4 237.0.0.3/1
a=control:trackID=1
a=range:npt=0-
a=framerate:25.0

Failed to parse interval end specification ''
[rtp @ 0x7f648c008e00] No default whitelist set
[udp @ 0x7f648c009900] No default whitelist set
[udp @ 0x7f648c009900] end receive buffer size reported is 425984
[udp @ 0x7f648c019c80] No default whitelist set
[udp @ 0x7f648c019c80] end receive buffer size reported is 425984
[rtsp @ 0x7f648c000b80] setting jitter buffer size to 500
[rtsp @ 0x7f648c000b80] hello state=0
Failed to parse interval end specification ''
[mjpeg @ 0x7f648c0046c0] marker=d8 avail_size_in_buf=145103 
[mjpeg @ 0x7f648c0046c0] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x7f648c0046c0] marker=e0 avail_size_in_buf=145101
[mjpeg @ 0x7f648c0046c0] marker parser used 16 bytes (128 bits)
[mjpeg @ 0x7f648c0046c0] marker=db avail_size_in_buf=145083
[mjpeg @ 0x7f648c0046c0] index=0
[mjpeg @ 0x7f648c0046c0] qscale[0]: 5
[mjpeg @ 0x7f648c0046c0] index=1
[mjpeg @ 0x7f648c0046c0] qscale[1]: 10
[mjpeg @ 0x7f648c0046c0] marker parser used 132 bytes (1056 bits)
[mjpeg @ 0x7f648c0046c0] marker=c4 avail_size_in_buf=144949
[mjpeg @ 0x7f648c0046c0] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x7f648c0046c0] marker=c0 avail_size_in_buf=144529
[mjpeg @ 0x7f648c0046c0] Changing bps from 0 to 8
[mjpeg @ 0x7f648c0046c0] sof0: picture: 1920x1080
[mjpeg @ 0x7f648c0046c0] component 0 2:2 id: 0 quant:0
[mjpeg @ 0x7f648c0046c0] component 1 1:1 id: 1 quant:1
[mjpeg @ 0x7f648c0046c0] component 2 1:1 id: 2 quant:1
[mjpeg @ 0x7f648c0046c0] pix fmt id 22111100
[mjpeg @ 0x7f648c0046c0] Format yuvj420p chosen by get_format().
[mjpeg @ 0x7f648c0046c0] marker parser used 17 bytes (136 bits)
[mjpeg @ 0x7f648c0046c0] escaping removed 676 bytes
[mjpeg @ 0x7f648c0046c0] marker=da avail_size_in_buf=144510
[mjpeg @ 0x7f648c0046c0] marker parser used 143834 bytes (1150672 bits)
[mjpeg @ 0x7f648c0046c0] marker=d9 avail_size_in_buf=2
[mjpeg @ 0x7f648c0046c0] decode frame unused 2 bytes
[rtsp @ 0x7f648c000b80] All info found vq=    0KB sq=    0B f=0/0
[rtsp @ 0x7f648c000b80] rfps: 24.416667 0.018101
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 24.500000 0.013298
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 24.583333 0.009235
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 24.666667 0.005910
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 24.750000 0.003324
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 24.833333 0.001477
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 24.916667 0.000369
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 25.000000 0.000000
[rtsp @ 0x7f648c000b80] rfps: 25.083333 0.000370
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 25.166667 0.001478
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 25.250000 0.003326
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 25.333333 0.005912
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 25.416667 0.009238
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 25.500000 0.013302
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 25.583333 0.018105
    Last message repeated 1 times
[rtsp @ 0x7f648c000b80] rfps: 50.000000 0.000000
[rtsp @ 0x7f648c000b80] Setting avg frame rate based on r frame rate
Input #0, rtsp, from 'rtsp://192.168.1.100/videoinput_1/mjpeg_3/media.stm':
  Metadata:
    title           : /videoinput_1:0/mjpeg_3/media.stm
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0, 21, 1/90000: Video: mjpeg (Baseline), 1 reference frame, yuvj420p(pc, bt470bg/unknown/unknown, center), 1920x1080 [SAR 1:1 DAR 16:9], 0/1, 25 fps, 25 tbr, 90k tbn, 90k tbc
[mjpeg @ 0x7f648c02ad80] marker=d8 avail_size_in_buf=145103


  


  Here is the same debug section when using udp_multicast. The SDP is identical as mentioned, and the block after the SDP containing [mjpeg] codec info is entirely missing (beginning with marker=d8)—the stream is never identified. This happens (to the eye) instantaneously, there's no indication of a timeout waiting unsuccessfully for an RTP packet, though this, too, could just be insufficient debug info in the driver. Also note that ffmpeg knows that the frames are MJPEG frames and the color primaries are PAL, it just doesn't know the size. Also curious, but not relevant to the problem, the unicast UDP transport destination port utilized for the stream does not appear in the ffmpeg debug dump shown above, meaning part of the RTSP/RTP driver is hiding important information under the kimono, that port number and how it knows that the frames will be MJPEG.

  


  [tcp @ 0x7effe0002f40] Starting connection attempt to 192.168.1.100 port 554
[tcp @ 0x7effe0002f40] Successfully connected to 192.168.1.100 port 554
[rtsp @ 0x7effe0000b80] SDP:aq=    0KB vq=    0KB sq=    0B f=0/0
v=0
o=- 621355968671884050 621355968671884050 IN IP4 192.168.1.100
s=/videoinput_1:0/mjpeg_3/media.stm
c=IN IP4 0.0.0.0
m=video 40004 RTP/AVP 26
c=IN IP4 237.0.0.3/1
a=control:trackID=1
a=range:npt=0-
a=framerate:25.0

Failed to parse interval end specification ''
[rtp @ 0x7effe0008e00] No default whitelist set
[udp @ 0x7effe0009900] No default whitelist set
[udp @ 0x7effe0009900] end receive buffer size reported is 425984
[udp @ 0x7effe0019c40] No default whitelist set
[udp @ 0x7effe0019c40] end receive buffer size reported is 425984
[rtsp @ 0x7effe0000b80] setting jitter buffer size to 500
[rtsp @ 0x7effe0000b80] hello state=0
Failed to parse interval end specification '' 
[rtsp @ 0x7effe0000b80] Could not find codec parameters for stream 0 (Video: mjpeg, 1 reference frame, none(bt470bg/unknown/unknown, center)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, rtsp, from 'rtsp://192.168.1.100/videoinput_1/mjpeg_3/media.stm':
  Metadata:
    title           : /videoinput_1:0/mjpeg_3/media.stm
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0, 0, 1/90000: Video: mjpeg, 1 reference frame, none(bt470bg/unknown/unknown, center), 90k tbr, 90k tbn, 90k tbc
    nan M-V:    nan fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0


  


  This is the TCPDUMP of the traffic. The information in both streams appears identical.

  


  19:21:30.703599 IP 192.168.1.100.64271 > 192.168.1.98.5239: UDP, length 60
19:21:30.703734 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 1400
19:21:30.703852 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 1400
19:21:30.704326 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 1400
19:21:30.704326 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 1400
19:21:30.704327 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 1400
19:21:30.704327 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 1400
19:21:30.704504 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 1400
19:21:30.704813 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 1400
19:21:30.704814 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 1400
19:21:30.704872 IP 192.168.1.100.64270 > 192.168.1.98.5238: UDP, length 732
19:21:30.704873 IP 192.168.1.100.59869 > 237.0.0.3.40005: UDP, length 60
19:21:30.705513 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 1400
19:21:30.705513 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 1400
19:21:30.705513 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 1400
19:21:30.705513 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 1400
19:21:30.705594 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 1400
19:21:30.705774 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 1400
19:21:30.706236 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 1400
19:21:30.706236 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 1400
19:21:30.706236 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 1400
19:21:30.706236 IP 192.168.1.100.59868 > 237.0.0.3.40004: UDP, length 732


  


  I hope this is a configuration problem, that I can fix this in my ffplay/ffmpeg line, and it's not a bug in ffmpeg. Thanks for any tips.

  


• libswresample : Why does swr_init() change |in_ch_layout| order so it no longer matches my decoded AVFrames, causing resampling to fail ?

  20 novembre 2023, par CheekyChips

  I am trying to write some code that resamples an audio file to 16kHz and 1 channel and then encodes it to PCM, but I am having an issue with channel layouts.

  


  In a nutshell :

  


  My AVCodecContext and the frames I get from the stream via avcodec_receive_frame() have a channel layout order of AV_CHANNEL_ORDER_UNSPEC. But when I call swr_init() it changes the in_ch_layout order to AV_CHANNEL_ORDER_NATIVE. Then when I call swr_convert_frame() with my AVFrames, because the channel layout orders don't match, the resampling fails because it thinks the input changed.

  


  More details :

  


  I create an AVCodecContext from my audio stream's codec, and it has a channel layout of AV_CHANNEL_ORDER_UNSPEC with 2 channels, and any frames I decode from the stream via avcodec_receive_frame() also have a channel layout order of AV_CHANNEL_ORDER_UNSPEC.

  


  I set SwrContext's |in_ch_layout| to the sample channel layout from the codec context :

  


      AVChannelLayout in_ch_layout = in_codec_context->ch_layout,
    ...
    int ret = swr_alloc_set_opts2(&swr_ctx, ...
                      &in_ch_layout,
                      ...);


  


  But SwrContext->init() changes its internal in_ch_layout from AV_CHANNEL_ORDER_UNSPEC to AV_CHANNEL_ORDER_NATIVE meaning it fails the next time I call swr_convert_frame() because the input frame has a different channel layout to the SwrContext. When swr_init() is called (in my case indirectly by swr_convert_frame(), but also if I alternatively call it directly) the SwrContext->used_ch_layout and SwrContext->in_ch_layout are updated to have channel layout order of AV_CHANNEL_ORDER_NATIVE :

  


      // swresample.c
    av_cold int swr_init(struct SwrContext *s){
        ...
        if (!av_channel_layout_check(&s->used_ch_layout))       <-- This hits if I don't set anything for used_ch_layout
            av_channel_layout_default(&s->used_ch_layout, s->in.ch_count);      <-- default is AV_CHANNEL_ORDER_NATIVE
        ...
        if (s->used_ch_layout.order == AV_CHANNEL_ORDER_UNSPEC) <-- This hits if I do set used_ch_layout
            av_channel_layout_default(&s->used_ch_layout, s->used_ch_layout.nb_channels);   <-- default is AV_CHANNEL_ORDER_NATIVE


  


  Then when I next call swr_convert_frame(), because the frame has the same layout as the audio stream's codec (AV_CHANNEL_ORDER_UNSPEC), and this is different to SwrContext->in_ch_layout (AV_CHANNEL_ORDER_NATIVE), it early exits with ret |= AVERROR_INPUT_CHANGED.

  


  // swresample_frame.c
    int swr_convert_frame(SwrContext *s,
                      AVFrame *out, const AVFrame *in)
    {
        ...
        if ((ret = config_changed(s, out, in)))
            return ret;
        ...


  


      static int config_changed(SwrContext *s,
                            const AVFrame *out, const AVFrame *in)
    {
        ...
        if ((err = av_channel_layout_copy(&ch_layout, &in->ch_layout)) < 0)
            ...
        if (av_channel_layout_compare(&s->in_ch_layout, &ch_layout) || ...) {   <-- This hits the next time I call swr_convert_frame()
            ret |= AVERROR_INPUT_CHANGED;
        }


  


      // channel_layout.c
    int av_channel_layout_compare(const AVChannelLayout *chl, const AVChannelLayout *chl1)
    {
        ...
        // if only one is unspecified -> not equal
        if ((chl->order  == AV_CHANNEL_ORDER_UNSPEC) !=
            (chl1->order == AV_CHANNEL_ORDER_UNSPEC))
            return 1;


  


  If I hardcode the channel layout order of each input AVFrame to AV_CHANNEL_ORDER_NATIVE before resampling, then the resampling and subsequent encoding works, but this feels like a really bad idea and of course wouldn't work as soon as I resample an audio file with a different channel layout.

  


      avcodec_receive_frame(in_codec_context, input_frame);

    AVChannelLayout input_frame_ch_layout;
    av_channel_layout_default(&input_frame_ch_layout, 2 /* = nb_channels*/);
    input_frame->ch_layout = input_frame_ch_layout;
    // Bad idea - but "fixes" my issue!


  


  My questions

  


  What do I need to do to the resampler OR/AND the decoded audio frame to make sure they have the same channel layout order and the resampling works ?

  


  How can I make the channel order of the AVFrames that I get from avcodec_receive_frame() match the input channel order of SwrContext so the resampling works ? My understanding is that the decoded frames should be 'correct' already and I shouldn't need to change any of their values, only values of the output (resampled) frames that I create.

  


  Is there something I need to set on the AVFrame before I resample it ?

  


  Why does the SwrContext choose to change the channel order to AV_CHANNEL_ORDER_NATIVE ?

  


  Note :
A workaround could be to use swr_convert() with the raw data buffer instead of swr_convert_frame(), since it looks like it bypasses this check (since there are no frames involved). I haven't tried this but this shouldn't be necessary and I would like to use swr_convert_frame() as I am working with input and output frames.

  


  Unfortunately I can't find example code using swr_convert_frame() (not even the ffmpeg code seems to ever call it).

  


  My full c++ source code
(error handling omitted for readability) :

  


  std::string fileToUse = "/home/projects/audioFileProject/Audio files/14 Black Cadillacs.wma";
const std::string outputFilename = "out.wav";
const std::string PCMS16BE_encoder_name = "pcm_f32le";

int main()
{
    // Open audio file
    AVFormatContext* in_format_context = avformat_alloc_context();
    avformat_open_input(&in_format_context, fileToUse.c_str(), NULL, NULL);
    avformat_find_stream_info(in_format_context, NULL);
    
    // Get audio stream from file and corresponding decoder
    AVStream* in_stream = in_format_context->streams[0];
    AVCodecParameters* codec_params = in_stream->codecpar;
    const AVCodec* in_codec = avcodec_find_decoder(codec_params->codec_id);
    AVCodecContext *in_codec_context = avcodec_alloc_context3(in_codec);
    avcodec_parameters_to_context(in_codec_context, codec_params);
    avcodec_open2(in_codec_context, in_codec, NULL);

    // Prepare output stream and output encoder (PCM)
    AVFormatContext* out_format_context = nullptr;
    avformat_alloc_output_context2(&out_format_context, NULL, NULL, outputFilename.c_str());
    AVStream* out_stream = avformat_new_stream(out_format_context, NULL);
    const AVCodec* output_codec = avcodec_find_encoder_by_name(PCMS16BE_encoder_name.c_str());
    AVCodecContext* output_codec_context = avcodec_alloc_context3(output_codec);

    // -------------------------------
    
    AVChannelLayout output_ch_layout;
    av_channel_layout_default(&output_ch_layout, 1);    // AV_CHANNEL_LAYOUT_MONO
    output_codec_context->ch_layout = output_ch_layout;
    
    auto out_sample_rate = 16000;
    output_codec_context->sample_rate = out_sample_rate;
    output_codec_context->sample_fmt = output_codec->sample_fmts[0];
    //output_codec_context->bit_rate = output_codec_context->bit_rate;  // TODO Do we need to set the bit rate?
    output_codec_context->time_base = (AVRational){1, out_sample_rate};
    out_stream->time_base = output_codec_context->time_base;

    auto in_sample_rate = in_codec_context->sample_rate;
    AVChannelLayout in_ch_layout = in_codec_context->ch_layout,
                    out_ch_layout = output_ch_layout;   // AV_CHANNEL_LAYOUT_MONO;
    enum AVSampleFormat in_sample_fmt = in_codec_context->sample_fmt,
                        out_sample_fmt = in_codec_context->sample_fmt;

    SwrContext *swr_ctx = nullptr;
    int ret = swr_alloc_set_opts2(&swr_ctx,
                      &out_ch_layout,
                      out_sample_fmt,
                      out_sample_rate,
                      &in_ch_layout,
                      in_sample_fmt,
                      in_sample_rate,
                      0,                    // log_offset
                      NULL);                // log_ctx

    // Probably not necessary - documentation says "This option is
only used for special remapping."
    av_opt_set_chlayout(swr_ctx,    "used_chlayout",     &in_ch_layout, 0);

    // Open output file for writing
    avcodec_open2(output_codec_context, output_codec, NULL);
    avcodec_parameters_from_context(out_stream->codecpar, output_codec_context);
    
    if (out_format_context->oformat->flags & AVFMT_GLOBALHEADER)
        out_format_context->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

    avio_open(&out_format_context->pb, outputFilename.c_str(), AVIO_FLAG_WRITE);
    AVDictionary* muxer_opts = nullptr;
    avformat_write_header(out_format_context, &muxer_opts);

    AVFrame* input_frame = av_frame_alloc();
    AVPacket* in_packet = av_packet_alloc();

    // Loop through decoded input frames. Resample and get resulting samples in a new output frame.
    // I think PCM supports variable number of samples in frames so probably can immediately write out
    while (av_read_frame(in_format_context, in_packet) >= 0) {
        avcodec_send_packet(in_codec_context, in_packet);
        avcodec_receive_frame(in_codec_context, input_frame);

        // I don't want to do this, but it 'fixes' the error where channel layout of input frames
        // doesn't match what the resampler expects - hardcoded the number 2 to fit my sample audio file.
        AVChannelLayout input_frame_ch_layout;
        av_channel_layout_default(&input_frame_ch_layout, 2 /* = nb_channels*/);
        input_frame->ch_layout = input_frame_ch_layout;

        AVFrame* output_frame = av_frame_alloc();
        output_frame->sample_rate = out_sample_rate;
        output_frame->format = out_sample_fmt;
        output_frame->ch_layout = out_ch_layout;
        output_frame->nb_samples = output_codec_context->frame_size;
        
        // TODO Probably need to do maths to calculate new pts properly
        output_frame->pts = input_frame->pts;

        if (swr_convert_frame(swr_ctx, output_frame, input_frame))
            {   logging("Swr Convert failed");  return -1;   }          
            /// ^ Fails here, the second time (since the first time init() is called internally)

        AVPacket *output_packet = av_packet_alloc();
        int response = avcodec_send_frame(output_codec_context, output_frame);

        while (response >= 0) {
            response = avcodec_receive_packet(output_codec_context, output_packet);

            if (response == AVERROR(EAGAIN) || response == AVERROR_EOF) {
                break;
            }

            output_packet->stream_index = 0;
            av_packet_rescale_ts(output_packet, in_stream->time_base, out_stream->time_base);
            av_interleaved_write_frame(out_format_context, output_packet);
        }
        av_packet_unref(output_packet);
        av_packet_free(&output_packet);
        av_frame_unref(input_frame);    // Free references held by the frame before reading new data into it.
        av_frame_unref(output_frame);
    }
    // TODO write last output packet flushing the buffer

    avformat_close_input(&in_format_context);
    return 0;
}


  


• av_read_frame of ffmpeg in windows always returning packet.stream_index as 0

  17 février 2015, par yanqing lu

  I’m using ffmpeg to demuxer, for some video like the windows’s sample Wildlife.wmv, the av_read_frame is always returning packet.stream_index=0. This leads the video queue very big and memory overflow. I get 15 frames of video, but only 1 frame of audio. Is there some options to control the av_read_frame, so I can read frames like : video-audio-video-audio, in this way it’s easy to sync AV and keep queue balance.

  Thanks in advance.