Recherche avancée

Sur d’autres sites (3690)

• ffmpeg aresample has no effect while it works in ffplay

  11 novembre 2020, par Lemon Sky

  I have a .ts file which is missing frames 20 seconds into the file. I'd like to fill the missing parts with silence. This works using the following command with ffplay :

  


  ffplay -i http://tyberis.com/KFYI-AM_2020-10-30_09-05-26.ts -af aresample=async=1

  


  The same should work with ffmpeg but no silence is injected. Instead, the audio just continues playing. Example :

  


  ffmpeg -i http://tyberis.com/KFYI-AM_2020-10-30_09-05-26.ts -af aresample=async=1 -f wav -acodec pcm_s16le -ac 2 - | ffplay -i -

  


  What am I doing wrong ?

  


  Log :

  


  >ffmpeg -y -nostats -loglevel trace -i http://tyberis.com/KFYI-AM_2020-10-30_09-05-26.ts -af aresample=async=1 -f wav -acodec pcm_s16le -ac 2 out.wav
ffmpeg version 4.3.1-2020-11-08-full_build-www.gyan.dev Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.0 (Rev3, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-nostats' ... matched as option 'stats' (print progress report during encoding) with argument 0.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'trace'.
Reading option '-i' ... matched as input url with argument 'http://tyberis.com/KFYI-AM_2020-10-30_09-05-26.ts'.
Reading option '-af' ... matched as option 'af' (set audio filters) with argument 'aresample=async=1'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'wav'.
Reading option '-acodec' ... matched as option 'acodec' (force audio codec ('copy' to copy stream)) with argument 'pcm_s16le'.
Reading option '-ac' ... matched as option 'ac' (set number of audio channels) with argument '2'.
Reading option 'out.wav' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option nostats (print progress report during encoding) with argument 0.
Applying option loglevel (set logging level) with argument trace.
Successfully parsed a group of options.
Parsing a group of options: input url http://tyberis.com/KFYI-AM_2020-10-30_09-05-26.ts.
Successfully parsed a group of options.
Opening an input file: http://tyberis.com/KFYI-AM_2020-10-30_09-05-26.ts.
[NULL @ 000001d0e7bbd200] Opening 'http://tyberis.com/KFYI-AM_2020-10-30_09-05-26.ts' for reading
[http @ 000001d0e7bbe2c0] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy,data'
[tcp @ 000001d0e7bc0d00] Original list of addresses:
[tcp @ 000001d0e7bc0d00] Address 144.2.127.86 port 80
[tcp @ 000001d0e7bc0d00] Interleaved list of addresses:
[tcp @ 000001d0e7bc0d00] Address 144.2.127.86 port 80
[tcp @ 000001d0e7bc0d00] Starting connection attempt to 144.2.127.86 port 80
[tcp @ 000001d0e7bc0d00] Successfully connected to 144.2.127.86 port 80
[http @ 000001d0e7bbe2c0] request: GET /KFYI-AM_2020-10-30_09-05-26.ts HTTP/1.1
User-Agent: Lavf/58.45.100
Accept: */*
Range: bytes=0-
Connection: close
Host: tyberis.com
Icy-MetaData: 1


[http @ 000001d0e7bbe2c0] header='HTTP/1.1 206 Partial Content'
[http @ 000001d0e7bbe2c0] http_code=206
[http @ 000001d0e7bbe2c0] header='Content-Type: video/vnd.dlna.mpeg-tts'
[http @ 000001d0e7bbe2c0] header='Last-Modified: Tue, 10 Nov 2020 19:12:18 GMT'
[http @ 000001d0e7bbe2c0] header='Accept-Ranges: bytes'
[http @ 000001d0e7bbe2c0] header='ETag: "7c6a9e6895b7d61:0"'
[http @ 000001d0e7bbe2c0] header='Server: Microsoft-IIS/10.0'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Origin: *'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization, Range'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Methods: GET,HEAD,OPTIONS,POST,PUT'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Credentials: true'
[http @ 000001d0e7bbe2c0] header='Date: Wed, 11 Nov 2020 15:52:32 GMT'
[http @ 000001d0e7bbe2c0] header='Connection: close'
[http @ 000001d0e7bbe2c0] header='Content-Length: 6531496'
[http @ 000001d0e7bbe2c0] header='Content-Range: bytes 0-6531495/6531496'
[http @ 000001d0e7bbe2c0] header=''
Probing mpegts score:50 size:2048
[mpegts @ 000001d0e7bbd200] Format mpegts probed with size=2048 and score=50
[mpegts @ 000001d0e7bbd200] Probe: 7648, score: 41, dvhs_score: -2, fec_score: -2
[mpegts @ 000001d0e7bbd200] Filter: pid=0x11 type=1
[mpegts @ 000001d0e7bbd200] Filter: pid=0x0 type=1
[mpegts @ 000001d0e7bbd200] Filter: pid=0x12 type=1
[mpegts @ 000001d0e7bbd200] SDT:
[mpegts @ 000001d0e7bbd200] tag: 0x48 len=18
[mpegts @ 000001d0e7bbd200] new_program: id=0x0001
[mpegts @ 000001d0e7bbd200] PAT:
[mpegts @ 000001d0e7bbd200] sid=0x1 pid=0x1000
[mpegts @ 000001d0e7bbd200] new_program: id=0x0001
[mpegts @ 000001d0e7bbd200] Filter: pid=0x1000 type=1
[mpegts @ 000001d0e7bbd200] PMT: len 21
[mpegts @ 000001d0e7bbd200] sid=0x1 sec_num=0/0 version=0 tid=2
[mpegts @ 000001d0e7bbd200] pcr_pid=0x100
[mpegts @ 000001d0e7bbd200] Filter: pid=0x100 type=0
[mpegts @ 000001d0e7bbd200] stream=0 stream_type=f pid=100 prog_reg_desc=
[mpegts @ 000001d0e7bbd200] tuning done
[mpegts @ 000001d0e7bbd200] Before avformat_find_stream_info() pos: 0 bytes read:7648 seeks:0 nb_streams:1
[mpegts @ 000001d0e7bbd200] Skipping after seek
[mpegts @ 000001d0e7bbd200] SDT:
[mpegts @ 000001d0e7bbd200] tag: 0x48 len=18
[mpegts @ 000001d0e7bbd200] new_program: id=0x0001
[mpegts @ 000001d0e7bbd200] PAT:
[mpegts @ 000001d0e7bbd200] sid=0x1 pid=0x1000
[mpegts @ 000001d0e7bbd200] new_program: id=0x0001
[mpegts @ 000001d0e7bbd200] PMT: len 21
[mpegts @ 000001d0e7bbd200] sid=0x1 sec_num=0/0 version=0 tid=2
[mpegts @ 000001d0e7bbd200] pcr_pid=0x100
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[mpegts @ 000001d0e7bbd200] probing stream 0 pp:2500
Probing aac score:51 size:2640
Probing flac score:13 size:2640
[mpegts @ 000001d0e7bbd200] Probe with size=2640, packets=1 detected aac with score=51
[mpegts @ 000001d0e7bbd200] probed stream 0
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
    Last message repeated 11 times
[mpegts @ 000001d0e7bbd200] max_analyze_duration 5000000 reached at 5014800 microseconds st:0
[tcp @ 000001d0e7c31480] Original list of addresses:
[tcp @ 000001d0e7c31480] Address 144.2.127.86 port 80
[tcp @ 000001d0e7c31480] Interleaved list of addresses:
[tcp @ 000001d0e7c31480] Address 144.2.127.86 port 80
[tcp @ 000001d0e7c31480] Starting connection attempt to 144.2.127.86 port 80
[tcp @ 000001d0e7c31480] Successfully connected to 144.2.127.86 port 80
[http @ 000001d0e7bbe2c0] request: GET /KFYI-AM_2020-10-30_09-05-26.ts HTTP/1.1
User-Agent: Lavf/58.45.100
Accept: */*
Range: bytes=6281496-
Connection: close
Host: tyberis.com
Icy-MetaData: 1


[http @ 000001d0e7bbe2c0] header='HTTP/1.1 206 Partial Content'
[http @ 000001d0e7bbe2c0] http_code=206
[http @ 000001d0e7bbe2c0] header='Content-Type: video/vnd.dlna.mpeg-tts'
[http @ 000001d0e7bbe2c0] header='Last-Modified: Tue, 10 Nov 2020 19:12:18 GMT'
[http @ 000001d0e7bbe2c0] header='Accept-Ranges: bytes'
[http @ 000001d0e7bbe2c0] header='ETag: "7c6a9e6895b7d61:0"'
[http @ 000001d0e7bbe2c0] header='Server: Microsoft-IIS/10.0'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Origin: *'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization, Range'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Methods: GET,HEAD,OPTIONS,POST,PUT'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Credentials: true'
[http @ 000001d0e7bbe2c0] header='Date: Wed, 11 Nov 2020 15:52:32 GMT'
[http @ 000001d0e7bbe2c0] header='Connection: close'
[http @ 000001d0e7bbe2c0] header='Content-Length: 250000'
[http @ 000001d0e7bbe2c0] header='Content-Range: bytes 6281496-6531495/6531496'
[http @ 000001d0e7bbe2c0] header=''
[mpegts @ 000001d0e7bbd200] Skipping after seek
[mpegts @ 000001d0e7bbd200] Probe: 763, score: 5, dvhs_score: 1, fec_score: 1
[mpegts @ 000001d0e7bbd200] Probe: 8192, score: 44, dvhs_score: -3, fec_score: -3
[tcp @ 000001d0e7c31580] Original list of addresses:
[tcp @ 000001d0e7c31580] Address 144.2.127.86 port 80
[tcp @ 000001d0e7c31580] Interleaved list of addresses:
[tcp @ 000001d0e7c31580] Address 144.2.127.86 port 80
[tcp @ 000001d0e7c31580] Starting connection attempt to 144.2.127.86 port 80
[tcp @ 000001d0e7c31580] Successfully connected to 144.2.127.86 port 80
[http @ 000001d0e7bbe2c0] request: GET /KFYI-AM_2020-10-30_09-05-26.ts HTTP/1.1
User-Agent: Lavf/58.45.100
Accept: */*
Range: bytes=6266023-
Connection: close
Host: tyberis.com
Icy-MetaData: 1


[http @ 000001d0e7bbe2c0] header='HTTP/1.1 206 Partial Content'
[http @ 000001d0e7bbe2c0] http_code=206
[http @ 000001d0e7bbe2c0] header='Content-Type: video/vnd.dlna.mpeg-tts'
[http @ 000001d0e7bbe2c0] header='Last-Modified: Tue, 10 Nov 2020 19:12:18 GMT'
[http @ 000001d0e7bbe2c0] header='Accept-Ranges: bytes'
[http @ 000001d0e7bbe2c0] header='ETag: "7c6a9e6895b7d61:0"'
[http @ 000001d0e7bbe2c0] header='Server: Microsoft-IIS/10.0'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Origin: *'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization, Range'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Methods: GET,HEAD,OPTIONS,POST,PUT'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Credentials: true'
[http @ 000001d0e7bbe2c0] header='Date: Wed, 11 Nov 2020 15:52:32 GMT'
[http @ 000001d0e7bbe2c0] header='Connection: close'
[http @ 000001d0e7bbe2c0] header='Content-Length: 265473'
[http @ 000001d0e7bbe2c0] header='Content-Range: bytes 6266023-6531495/6531496'
[http @ 000001d0e7bbe2c0] header=''
[tcp @ 000001d0e7c31480] Original list of addresses:
[tcp @ 000001d0e7c31480] Address 144.2.127.86 port 80
[tcp @ 000001d0e7c31480] Interleaved list of addresses:
[tcp @ 000001d0e7c31480] Address 144.2.127.86 port 80
[tcp @ 000001d0e7c31480] Starting connection attempt to 144.2.127.86 port 80
[tcp @ 000001d0e7c31480] Successfully connected to 144.2.127.86 port 80
[http @ 000001d0e7bbe2c0] request: GET /KFYI-AM_2020-10-30_09-05-26.ts HTTP/1.1
User-Agent: Lavf/58.45.100
Accept: */*
Range: bytes=6281644-
Connection: close
Host: tyberis.com
Icy-MetaData: 1


[http @ 000001d0e7bbe2c0] header='HTTP/1.1 206 Partial Content'
[http @ 000001d0e7bbe2c0] http_code=206
[http @ 000001d0e7bbe2c0] header='Content-Type: video/vnd.dlna.mpeg-tts'
[http @ 000001d0e7bbe2c0] header='Last-Modified: Tue, 10 Nov 2020 19:12:18 GMT'
[http @ 000001d0e7bbe2c0] header='Accept-Ranges: bytes'
[http @ 000001d0e7bbe2c0] header='ETag: "7c6a9e6895b7d61:0"'
[http @ 000001d0e7bbe2c0] header='Server: Microsoft-IIS/10.0'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Origin: *'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization, Range'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Methods: GET,HEAD,OPTIONS,POST,PUT'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Credentials: true'
[http @ 000001d0e7bbe2c0] header='Date: Wed, 11 Nov 2020 15:52:33 GMT'
[http @ 000001d0e7bbe2c0] header='Connection: close'
[http @ 000001d0e7bbe2c0] header='Content-Length: 249852'
[http @ 000001d0e7bbe2c0] header='Content-Range: bytes 6281644-6531495/6531496'
[http @ 000001d0e7bbe2c0] header=''
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
    Last message repeated 88 times
[tcp @ 000001d0e7c31580] Original list of addresses:
[tcp @ 000001d0e7c31580] Address 144.2.127.86 port 80
[tcp @ 000001d0e7c31580] Interleaved list of addresses:
[tcp @ 000001d0e7c31580] Address 144.2.127.86 port 80
[tcp @ 000001d0e7c31580] Starting connection attempt to 144.2.127.86 port 80
[tcp @ 000001d0e7c31580] Successfully connected to 144.2.127.86 port 80
[http @ 000001d0e7bbe2c0] request: GET /KFYI-AM_2020-10-30_09-05-26.ts HTTP/1.1
User-Agent: Lavf/58.45.100
Accept: */*
Range: bytes=0-
Connection: close
Host: tyberis.com
Icy-MetaData: 1


[http @ 000001d0e7bbe2c0] header='HTTP/1.1 206 Partial Content'
[http @ 000001d0e7bbe2c0] http_code=206
[http @ 000001d0e7bbe2c0] header='Content-Type: video/vnd.dlna.mpeg-tts'
[http @ 000001d0e7bbe2c0] header='Last-Modified: Tue, 10 Nov 2020 19:12:18 GMT'
[http @ 000001d0e7bbe2c0] header='Accept-Ranges: bytes'
[http @ 000001d0e7bbe2c0] header='ETag: "7c6a9e6895b7d61:0"'
[http @ 000001d0e7bbe2c0] header='Server: Microsoft-IIS/10.0'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Origin: *'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization, Range'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Methods: GET,HEAD,OPTIONS,POST,PUT'
[http @ 000001d0e7bbe2c0] header='Access-Control-Allow-Credentials: true'
[http @ 000001d0e7bbe2c0] header='Date: Wed, 11 Nov 2020 15:52:33 GMT'
[http @ 000001d0e7bbe2c0] header='Connection: close'
[http @ 000001d0e7bbe2c0] header='Content-Length: 6531496'
[http @ 000001d0e7bbe2c0] header='Content-Range: bytes 0-6531495/6531496'
[http @ 000001d0e7bbe2c0] header=''
[mpegts @ 000001d0e7bbd200] stream 0: start_time: 1.4 duration: 1097.98
[mpegts @ 000001d0e7bbd200] format: start_time: 1.4 duration: 1097.98 (estimate from pts) bitrate=47 kb/s
[mpegts @ 000001d0e7bbd200] After avformat_find_stream_info() pos: 0 bytes read:329238 seeks:3 frames:117
Input #0, mpegts, from 'http://tyberis.com/KFYI-AM_2020-10-30_09-05-26.ts':
  Duration: 00:18:17.98, start: 1.400000, bitrate: 47 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100], 117, 1/90000: Audio: aac (HE-AACv2) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 45 kb/s
Successfully opened the file.
Parsing a group of options: output url out.wav.
Applying option af (set audio filters) with argument aresample=async=1.
Applying option f (force format) with argument wav.
Applying option acodec (force audio codec ('copy' to copy stream)) with argument pcm_s16le.
Applying option ac (set number of audio channels) with argument 2.
Successfully parsed a group of options.
Opening an output file: out.wav.
[file @ 000001d0e7c06200] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[mpegts @ 000001d0e7bbd200] Skipping after seek
[mpegts @ 000001d0e7bbd200] SDT:
[mpegts @ 000001d0e7bbd200] tag: 0x48 len=18
[mpegts @ 000001d0e7bbd200] new_program: id=0x0001
[mpegts @ 000001d0e7bbd200] PAT:
[mpegts @ 000001d0e7bbd200] sid=0x1 pid=0x1000
[mpegts @ 000001d0e7bbd200] new_program: id=0x0001
[mpegts @ 000001d0e7bbd200] PMT: len 21
[mpegts @ 000001d0e7bbd200] sid=0x1 sec_num=0/0 version=0 tid=2
[mpegts @ 000001d0e7bbd200] pcr_pid=0x100
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
detected 4 logical cores
[Parsed_aresample_0 @ 000001d0e7be5e80] Setting 'async' to value '1'
[graph_0_in_0_0 @ 000001d0e7beeac0] Setting 'time_base' to value '1/44100'
[graph_0_in_0_0 @ 000001d0e7beeac0] Setting 'sample_rate' to value '44100'
[graph_0_in_0_0 @ 000001d0e7beeac0] Setting 'sample_fmt' to value 'fltp'
[graph_0_in_0_0 @ 000001d0e7beeac0] Setting 'channel_layout' to value '0x3'
[graph_0_in_0_0 @ 000001d0e7beeac0] tb:1/44100 samplefmt:fltp samplerate:44100 chlayout:0x3
[format_out_0_0 @ 000001d0e7bf0080] Setting 'sample_fmts' to value 's16'
[format_out_0_0 @ 000001d0e7bf0080] Setting 'channel_layouts' to value '0x3'
[AVFilterGraph @ 000001d0e7be6540] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed
[Parsed_aresample_0 @ 000001d0e7be5e80] [SWR @ 000001d0e81bb8c0] Using fltp internally between filters
[Parsed_aresample_0 @ 000001d0e7be5e80] ch:2 chl:stereo fmt:fltp r:44100Hz -> ch:2 chl:stereo fmt:s16 r:44100Hz
Output #0, wav, to 'out.wav':
  Metadata:
    ISFT            : Lavf58.45.100
    Stream #0:0, 0, 1/44100: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s
    Metadata:
      encoder         : Lavc58.91.100 pcm_s16le
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 7 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
(...)
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 4 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
timestamp discontinuity for stream #0:0 (id=256, type=audio): 69892061, new offset= -71292061
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
[aac @ 000001d0e7c0ed80] illegal icc
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 7 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
[mpegts @ 000001d0e7bbd200] pid=100 pes_code=0x1c0
[aac @ 000001d0e7c0ed80] ChannelElement 1.0 missing
    Last message repeated 9 times
(...)
size=    5000kB time=00:00:29.02 bitrate=1411.2kbits/s speed=6.11x
video:0kB audio:5000kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.001523%
Input file #0 (http://tyberis.com/KFYI-AM_2020-10-30_09-05-26.ts):
  Input stream #0:0 (audio): 625 packets read (174267 bytes); 625 frames decoded (1280000 samples);
  Total: 625 packets (174267 bytes) demuxed
Output file #0 (out.wav):
  Output stream #0:0 (audio): 625 frames encoded (1280000 samples); 625 packets muxed (5120000 bytes);
  Total: 625 packets (5120000 bytes) muxed
625 frames successfully decoded, 0 decoding errors
[AVIOContext @ 000001d0e7c2e280] Statistics: 4 seeks, 22 writeouts
[AVIOContext @ 000001d0e7bc9780] Statistics: 533494 bytes read, 3 seeks```


  


• Libavformat/FFMPEG : Muxing into mp4 with AVFormatContext drops the final frame, depending on the number of frames

  27 octobre 2020, par Galen Lynch

  I am trying to use libavformat to create a .mp4 video
with a single h.264 video stream, but the final frame in the resulting file
often has a duration of zero and is effectively dropped from the video.
Strangely enough, whether the final frame is dropped or not depends on how many
frames I try to add to the file. Some simple testing that I outline below makes
me think that I am somehow misconfiguring either the AVFormatContext or the
h.264 encoder, resulting in two edit lists that sometimes chop off the final
frame. I will also post a simplified version of the code I am using, in case I'm
making some obvious mistake. Any help would be greatly appreciated : I've been
struggling with this issue for the past few days and have made little progress.

  


  I can recover the dropped frame by creating a new mp4 container using ffmpeg
binary with the copy codec if I use the -ignore_editlist option. Inspecting
the file with a missing frame using ffprobe, mp4trackdump, or mp4file --dump, shows that the final frame is dropped if its sample time is exactly the
same the end of the edit list. When I make a file that has no dropped frames, it
still has two edit lists : the only difference is that the end time of the edit
list is beyond all samples in files that do not have dropped frames. Though this
is hardly a fair comparison, if I make a .png for each frame and then generate
a .mp4 with ffmpeg using the image2 codec and similar h.264 settings, I
produce a movie with all frames present, only one edit list, and similar PTS
times as my mangled movies with two edit lists. In this case, the edit list
always ends after the last frame/sample time.

  


  I am using this command to determine the number of frames in the resulting stream,
though I also get the same number with other utilities :

  


  ffprobe -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 video_file_name.mp4


  


  Simple inspection of the file with ffprobe shows no obviously alarming signs to
me, besides the framerate being affected by the missing frame (the target was
24) :

  


  $ ffprobe -hide_banner testing.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'testing.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
  Duration: 00:00:04.13, start: 0.041016, bitrate: 724 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 100x100, 722 kb/s, 24.24 fps, 24 tbr, 12288 tbn, 48 tbc (default)
    Metadata:
      handler_name    : VideoHandler


  


  The files that I generate programatically always have two edit lists, one of
which is very short. In files both with and without a missing frame, the
duration one of the frames is 0, while all the others have the same duration
(512). You can see this in the ffmpeg output for this file that I tried to put
100 frames into, though only 99 are visible despite the file containing all 100
samples.

  


  $ ffmpeg -hide_banner -y -v 9 -loglevel 99 -i testing.mp4  &#xA;...&#xA;<edited to="to" remove="remove" the="the" class="class" printing="printing">&#xA;type:&#x27;edts&#x27; parent:&#x27;trak&#x27; sz: 48 100 948&#xA;type:&#x27;elst&#x27; parent:&#x27;edts&#x27; sz: 40 8 40&#xA;track[0].edit_count = 2&#xA;duration=41 time=-1 rate=1.000000&#xA;duration=4125 time=0 rate=1.000000&#xA;type:&#x27;mdia&#x27; parent:&#x27;trak&#x27; sz: 808 148 948&#xA;type:&#x27;mdhd&#x27; parent:&#x27;mdia&#x27; sz: 32 8 800&#xA;type:&#x27;hdlr&#x27; parent:&#x27;mdia&#x27; sz: 45 40 800&#xA;ctype=[0][0][0][0]&#xA;stype=vide&#xA;type:&#x27;minf&#x27; parent:&#x27;mdia&#x27; sz: 723 85 800&#xA;type:&#x27;vmhd&#x27; parent:&#x27;minf&#x27; sz: 20 8 715&#xA;type:&#x27;dinf&#x27; parent:&#x27;minf&#x27; sz: 36 28 715&#xA;type:&#x27;dref&#x27; parent:&#x27;dinf&#x27; sz: 28 8 28&#xA;Unknown dref type 0x206c7275 size 12&#xA;type:&#x27;stbl&#x27; parent:&#x27;minf&#x27; sz: 659 64 715&#xA;type:&#x27;stsd&#x27; parent:&#x27;stbl&#x27; sz: 151 8 651&#xA;size=135 4CC=avc1 codec_type=0&#xA;type:&#x27;avcC&#x27; parent:&#x27;stsd&#x27; sz: 49 8 49&#xA;type:&#x27;stts&#x27; parent:&#x27;stbl&#x27; sz: 32 159 651&#xA;track[0].stts.entries = 2&#xA;sample_count=99, sample_duration=512&#xA;sample_count=1, sample_duration=0&#xA;...&#xA;AVIndex stream 0, sample 99, offset 5a0ed, dts 50688, size 3707, distance 0, keyframe 1&#xA;Processing st: 0, edit list 0 - media time: -1, duration: 504&#xA;Processing st: 0, edit list 1 - media time: 0, duration: 50688&#xA;type:&#x27;udta&#x27; parent:&#x27;moov&#x27; sz: 98 1072 1162&#xA;...&#xA;</edited>

  


  The last frame has zero duration :

  


  $ mp4trackdump -v testing.mp4
...
mp4file testing.mp4, track 1, samples 100, timescale 12288
sampleId      1, size  6943 duration      512 time        0 00:00:00.000 S
sampleId      2, size  3671 duration      512 time      512 00:00:00.041 S
...
sampleId     99, size  3687 duration      512 time    50176 00:00:04.083 S
sampleId    100, size  3707 duration        0 time    50688 00:00:04.125 S


  


  Non-mangled videos that I generate have similar structure, as you can see in
this video that had 99 input frames, all of which are visible in the output.
Even though the sample_duration is set to zero for one of the samples in the
stss box, it is not dropped from the frame count or when reading the frames back
in with ffmpeg.

  


  $ ffmpeg -hide_banner -y -v 9 -loglevel 99 -i testing_99.mp4  
...
type:'elst' parent:'edts' sz: 40 8 40
track[0].edit_count = 2
duration=41 time=-1 rate=1.000000
duration=4084 time=0 rate=1.000000
...
track[0].stts.entries = 2
sample_count=98, sample_duration=512
sample_count=1, sample_duration=0
...
AVIndex stream 0, sample 98, offset 5d599, dts 50176, size 3833, distance 0, keyframe 1
Processing st: 0, edit list 0 - media time: -1, duration: 504
Processing st: 0, edit list 1 - media time: 0, duration: 50184
...


  


  $ mp4trackdump -v testing_99.mp4
...
sampleId     98, size  3814 duration      512 time    49664 00:00:04.041 S
sampleId     99, size  3833 duration        0 time    50176 00:00:04.083 S


  


  One difference that jumps out to me is that the mangled file's second edit list
ends at time 50688, which coincides with the last sample, while the non-mangled
file's edit list ends at 50184, which is after the time of the last sample
at 50176. As I mentioned before, whether the last frame is clipped depends on
the number of frames I encode and mux into the container : 100 input frames
results in 1 dropped frame, 99 results in 0, 98 in 0, 97 in 1, etc...

  


  Here is the code that I used to generate these files, which is a MWE script
version of library functions that I am modifying. It is written in Julia,
which I do not think is important here, and calls the FFMPEG library version
4.3.1. It's more or less a direct translation from of the FFMPEG muxing
demo, although the codec
context here is created before the format context. I am presenting the code that
interacts with ffmpeg first, although it relies on some helper code that I will
put below.

  


  The helper code just makes it easier to work with nested C structs in Julia, and
allows . syntax in Julia to be used in place of C's arrow (->) operator for
field access of struct pointers. Libav structs such as AVFrame appear as a
thin wrapper type AVFramePtr, and similarly AVStream appears as
AVStreamPtr etc... These act like single or double pointers for the purposes
of function calls, depending on the function's type signature. Hopefully it will
be clear enough to understand if you are familiar with working with libav in C,
and I don't think looking at the helper code should be necessary if you don't
want to run the code.

  


  # Function to transfer array to AVPicture/AVFrame
function transfer_img_buf_to_frame!(frame, img)
    img_pointer = pointer(img)
    data_pointer = frame.data[1] # Base-1 indexing, get pointer to first data buffer in frame
    for h = 1:frame.height
        data_line_pointer = data_pointer + (h-1) * frame.linesize[1] # base-1 indexing
        img_line_pointer = img_pointer + (h-1) * frame.width
        unsafe_copyto!(data_line_pointer, img_line_pointer, frame.width) # base-1 indexing
    end
end

# Function to transfer AVFrame to AVCodecContext, and AVPacket to AVFormatContext
function encode_mux!(packet, format_context, frame, codec_context; flush = false)
    if flush
        fret = avcodec_send_frame(codec_context, C_NULL)
    else
        fret = avcodec_send_frame(codec_context, frame)
    end
    if fret < 0 && !in(fret, [-Libc.EAGAIN, VIO_AVERROR_EOF])
        error("Error $fret sending a frame for encoding")
    end

    pret = Cint(0)
    while pret >= 0
        pret = avcodec_receive_packet(codec_context, packet)
        if pret == -Libc.EAGAIN || pret == VIO_AVERROR_EOF
             break
        elseif pret < 0
            error("Error $pret during encoding")
        end
        stream = format_context.streams[1] # Base-1 indexing
        av_packet_rescale_ts(packet, codec_context.time_base, stream.time_base)
        packet.stream_index = 0
        ret = av_interleaved_write_frame(format_context, packet)
        ret < 0 && error("Error muxing packet: $ret")
    end
    if !flush && fret == -Libc.EAGAIN && pret != VIO_AVERROR_EOF
        fret = avcodec_send_frame(codec_context, frame)
        if fret < 0 && fret != VIO_AVERROR_EOF
            error("Error $fret sending a frame for encoding")
        end
    end
    return pret
end

# Set parameters of test movie
nframe = 100
width, height = 100, 100
framerate = 24
gop = 0
codec_name = "libx264"
filename = "testing.mp4"

((width % 2 !=0) || (height % 2 !=0)) && error("Encoding error: Image dims must be a multiple of two")

# Make test images
imgstack = map(x->rand(UInt8,width,height),1:nframe);

pix_fmt = AV_PIX_FMT_GRAY8
framerate_rat = Rational(framerate)

codec = avcodec_find_encoder_by_name(codec_name)
codec == C_NULL && error("Codec '$codec_name' not found")

# Allocate AVCodecContext
codec_context_p = avcodec_alloc_context3(codec) # raw pointer
codec_context_p == C_NULL && error("Could not allocate AVCodecContext")
# Easier to work with pointer that acts like a c struct pointer, type defined below
codec_context = AVCodecContextPtr(codec_context_p)

codec_context.width = width
codec_context.height = height
codec_context.time_base = AVRational(1/framerate_rat)
codec_context.framerate = AVRational(framerate_rat)
codec_context.pix_fmt = pix_fmt
codec_context.gop_size = gop

ret = avcodec_open2(codec_context, codec, C_NULL)
ret < 0 && error("Could not open codec: Return code $(ret)")

# Allocate AVFrame and wrap it in a Julia convenience type
frame_p = av_frame_alloc()
frame_p == C_NULL && error("Could not allocate AVFrame")
frame = AVFramePtr(frame_p)

frame.format = pix_fmt
frame.width = width
frame.height = height

# Allocate picture buffers for frame
ret = av_frame_get_buffer(frame, 0)
ret < 0 && error("Could not allocate the video frame data")

# Allocate AVPacket and wrap it in a Julia convenience type
packet_p = av_packet_alloc()
packet_p == C_NULL && error("Could not allocate AVPacket")
packet = AVPacketPtr(packet_p)

# Allocate AVFormatContext and wrap it in a Julia convenience type
format_context_dp = Ref(Ptr{AVFormatContext}()) # double pointer
ret = avformat_alloc_output_context2(format_context_dp, C_NULL, C_NULL, filename)
if ret != 0 || format_context_dp[] == C_NULL
    error("Could not allocate AVFormatContext")
end
format_context = AVFormatContextPtr(format_context_dp)

# Add video stream to AVFormatContext and configure it to use the encoder made above
stream_p = avformat_new_stream(format_context, C_NULL)
stream_p == C_NULL && error("Could not allocate output stream")
stream = AVStreamPtr(stream_p) # Wrap this pointer in a convenience type

stream.time_base = codec_context.time_base
stream.avg_frame_rate = 1 / convert(Rational, stream.time_base)
ret = avcodec_parameters_from_context(stream.codecpar, codec_context)
ret < 0 && error("Could not set parameters of stream")

# Open the AVIOContext
pb_ptr = field_ptr(format_context, :pb)
# This following is just a call to avio_open, with a bit of extra protection
# so the Julia garbage collector does not destroy format_context during the call
ret = GC.@preserve format_context avio_open(pb_ptr, filename, AVIO_FLAG_WRITE)
ret < 0 && error("Could not open file $filename for writing")

# Write the header
ret = avformat_write_header(format_context, C_NULL)
ret < 0 && error("Could not write header")

# Encode and mux each frame
for i in 1:nframe # iterate from 1 to nframe
    img = imgstack[i] # base-1 indexing
    ret = av_frame_make_writable(frame)
    ret < 0 && error("Could not make frame writable")
    transfer_img_buf_to_frame!(frame, img)
    frame.pts = i
    encode_mux!(packet, format_context, frame, codec_context)
end

# Flush the encoder
encode_mux!(packet, format_context, frame, codec_context; flush = true)

# Write the trailer
av_write_trailer(format_context)

# Close the AVIOContext
pb_ptr = field_ptr(format_context, :pb) # get pointer to format_context.pb
ret = GC.@preserve format_context avio_closep(pb_ptr) # simply a call to avio_closep
ret < 0 && error("Could not free AVIOContext")

# Deallocation
avcodec_free_context(codec_context)
av_frame_free(frame)
av_packet_free(packet)
avformat_free_context(format_context)


  


  Below is the helper code that makes accessing pointers to nested c structs not a
total pain in Julia. If you try to run the code yourself, please enter this in
before the logic of the code shown above. It requires
VideoIO.jl, a Julia wrapper to libav.

  


  # Convenience type and methods to make the above code look more like C
using Base: RefValue, fieldindex

import Base: unsafe_convert, getproperty, setproperty!, getindex, setindex!,
    unsafe_wrap, propertynames

# VideoIO is a Julia wrapper to libav
#
# Bring bindings to libav library functions into namespace
using VideoIO: AVCodecContext, AVFrame, AVPacket, AVFormatContext, AVRational,
    AVStream, AV_PIX_FMT_GRAY8, AVIO_FLAG_WRITE, AVFMT_NOFILE,
    avformat_alloc_output_context2, avformat_free_context, avformat_new_stream,
    av_dump_format, avio_open, avformat_write_header,
    avcodec_parameters_from_context, av_frame_make_writable, avcodec_send_frame,
    avcodec_receive_packet, av_packet_rescale_ts, av_interleaved_write_frame,
    avformat_query_codec, avcodec_find_encoder_by_name, avcodec_alloc_context3,
    avcodec_open2, av_frame_alloc, av_frame_get_buffer, av_packet_alloc,
    avio_closep, av_write_trailer, avcodec_free_context, av_frame_free,
    av_packet_free

# Submodule of VideoIO
using VideoIO: AVCodecs

# Need to import this function from Julia's Base to add more methods
import Base: convert

const VIO_AVERROR_EOF = -541478725 # AVERROR_EOF

# Methods to convert between AVRational and Julia's Rational type, because it's
# hard to access the AV rational macros with Julia's C interface
convert(::Type{Rational{T}}, r::AVRational) where T = Rational{T}(r.num, r.den)
convert(::Type{Rational}, r::AVRational) = Rational(r.num, r.den)
convert(::Type{AVRational}, r::Rational) = AVRational(numerator(r), denominator(r))

"""
    mutable struct NestedCStruct{T}

Wraps a pointer to a C struct, and acts like a double pointer to that memory.
The methods below will automatically convert it to a single pointer if needed
for a function call, and make interacting with it in Julia look (more) similar
to interacting with it in C, except '->' in C is replaced by '.' in Julia.
"""
mutable struct NestedCStruct{T}
    data::RefValue{Ptr{T}}
end
NestedCStruct{T}(a::Ptr) where T = NestedCStruct{T}(Ref(a))
NestedCStruct(a::Ptr{T}) where T = NestedCStruct{T}(a)

const AVCodecContextPtr = NestedCStruct{AVCodecContext}
const AVFramePtr = NestedCStruct{AVFrame}
const AVPacketPtr = NestedCStruct{AVPacket}
const AVFormatContextPtr = NestedCStruct{AVFormatContext}
const AVStreamPtr = NestedCStruct{AVStream}

function field_ptr(::Type{S}, struct_pointer::Ptr{T}, field::Symbol,
                           index::Integer = 1) where {S,T}
    fieldpos = fieldindex(T, field)
    field_pointer = convert(Ptr{S}, struct_pointer) +
        fieldoffset(T, fieldpos) + (index - 1) * sizeof(S)
    return field_pointer
end

field_ptr(a::Ptr{T}, field::Symbol, args...) where T =
    field_ptr(fieldtype(T, field), a, field, args...)

function check_ptr_valid(p::Ptr, err::Bool = true)
    valid = p != C_NULL
    err && !valid && error("Invalid pointer")
    valid
end

unsafe_convert(::Type{Ptr{T}}, ap::NestedCStruct{T}) where T =
    getfield(ap, :data)[]
unsafe_convert(::Type{Ptr{Ptr{T}}}, ap::NestedCStruct{T}) where T =
    unsafe_convert(Ptr{Ptr{T}}, getfield(ap, :data))

function check_ptr_valid(a::NestedCStruct{T}, args...) where T
    p = unsafe_convert(Ptr{T}, a)
    GC.@preserve a check_ptr_valid(p, args...)
end

nested_wrap(x::Ptr{T}) where T = NestedCStruct(x)
nested_wrap(x) = x

function getproperty(ap::NestedCStruct{T}, s::Symbol) where T
    check_ptr_valid(ap)
    p = unsafe_convert(Ptr{T}, ap)
    res = GC.@preserve ap unsafe_load(field_ptr(p, s))
    nested_wrap(res)
end

function setproperty!(ap::NestedCStruct{T}, s::Symbol, x) where T
    check_ptr_valid(ap)
    p = unsafe_convert(Ptr{T}, ap)
    fp = field_ptr(p, s)
    GC.@preserve ap unsafe_store!(fp, x)
end

function getindex(ap::NestedCStruct{T}, i::Integer) where T
    check_ptr_valid(ap)
    p = unsafe_convert(Ptr{T}, ap)
    res = GC.@preserve ap unsafe_load(p, i)
    nested_wrap(res)
end

function setindex!(ap::NestedCStruct{T}, i::Integer, x) where T
    check_ptr_valid(ap)
    p = unsafe_convert(Ptr{T}, ap)
    GC.@preserve ap unsafe_store!(p, x, i)
end

function unsafe_wrap(::Type{T}, ap::NestedCStruct{S}, i) where {S, T}
    check_ptr_valid(ap)
    p = unsafe_convert(Ptr{S}, ap)
    GC.@preserve ap unsafe_wrap(T, p, i)
end

function field_ptr(::Type{S}, a::NestedCStruct{T}, field::Symbol,
                           args...) where {S, T}
    check_ptr_valid(a)
    p = unsafe_convert(Ptr{T}, a)
    GC.@preserve a field_ptr(S, p, field, args...)
end

field_ptr(a::NestedCStruct{T}, field::Symbol, args...) where T =
    field_ptr(fieldtype(T, field), a, field, args...)

propertynames(ap::T) where {S, T<:NestedCStruct{S}} = (fieldnames(S)...,
                                                       fieldnames(T)...)


  


  ---

  


  Edit : Some things that I have already tried

  


  

  • Explicitly setting the stream duration to be the same number as the number of frames that I add, or a few more beyond that

  


  • Explicitly setting the stream start time to zero, while the first frame has a PTS of 1

  


  • Playing around with encoder parameters, as well as gop_size, using B frames, etc.

  


  • Setting the private data for the mov/mp4 muxer to set the movflag negative_cts_offsets

  


  • Changing the framerate

  


  • Tried different pixel formats, such as AV_PIX_FMT_YUV420P

  


  


  Also to be clear while I can just transfer the file into another while ignoring the edit lists to work around this problem, I am hoping to not make damaged mp4 files in the first place.

  


• How to play and seek fragmented MP4 audio using MSE SourceBuffer ?

  29 juin 2024, par Stefan Falk

  Note :

  


  


  If you end up here, you might want to take a look at shaka-player and the accompanying shaka-streamer. Use it. Don't implement this yourself unless you really have to.

  


  


  I am trying for quite some time now to be able to play an audio track on Chrome, Firefox, Safari, etc. but I keep hitting brick walls. My problem is currently that I am just not able to seek within a fragmented MP4 (or MP3).

  


  At the moment I am converting audio files such as MP3 to fragmented MP4 (fMP4) and send them chunk-wise to the client. What I do is defining a CHUNK_DURACTION_SEC (chunk duration in seconds) and compute a chunk size like this :

  


  chunksTotal = Math.ceil(this.track.duration / CHUNK_DURATION_SEC);
chunkSize = Math.ceil(this.track.fileSize / this.chunksTotal);


  


  With this I partition the audio file and can fetch it entirely jumping chunkSize-many bytes for each chunk :

  


  -----------------------------------------
| chunk 1 | chunk 2 |   ...   | chunk n |
-----------------------------------------


  


  How audio files are converted to fMP4

  


  ffmpeg -i input.mp3 -acodec aac -b:a 256k -f mp4 \
       -movflags faststart+frag_every_frame+empty_moov+default_base_moof \
        output.mp4


  


  This seems to work with Chrome and Firefox (so far).

  


  How chunks are appended

  


  After following this example, and realizing that it's simply not working as it is explained here, I threw it away and started over from scratch. Unfortunately without success. It's still not working.

  


  The following code is supposed to play a track from the very beginning to the very end. However, I also need to be able to seek. So far, this is simply not working. Seeking will just stop the audio after the seeking event got triggered.

  


  The code

  


  /* Desired chunk duration in seconds. */&#xA;const CHUNK_DURATION_SEC = 20;&#xA;&#xA;const AUDIO_EVENTS = [&#xA;  &#x27;ended&#x27;,&#xA;  &#x27;error&#x27;,&#xA;  &#x27;play&#x27;,&#xA;  &#x27;playing&#x27;,&#xA;  &#x27;seeking&#x27;,&#xA;  &#x27;seeked&#x27;,&#xA;  &#x27;pause&#x27;,&#xA;  &#x27;timeupdate&#x27;,&#xA;  &#x27;canplay&#x27;,&#xA;  &#x27;loadedmetadata&#x27;,&#xA;  &#x27;loadstart&#x27;,&#xA;  &#x27;updateend&#x27;,&#xA;];&#xA;&#xA;&#xA;class ChunksLoader {&#xA;&#xA;  /** The total number of chunks for the track. */&#xA;  public readonly chunksTotal: number;&#xA;&#xA;  /** The length of one chunk in bytes */&#xA;  public readonly chunkSize: number;&#xA;&#xA;  /** Keeps track of requested chunks. */&#xA;  private readonly requested: boolean[];&#xA;&#xA;  /** URL of endpoint for fetching audio chunks. */&#xA;  private readonly url: string;&#xA;&#xA;  constructor(&#xA;    private track: Track,&#xA;    private sourceBuffer: SourceBuffer,&#xA;    private logger: NGXLogger,&#xA;  ) {&#xA;&#xA;    this.chunksTotal = Math.ceil(this.track.duration / CHUNK_DURATION_SEC);&#xA;    this.chunkSize = Math.ceil(this.track.fileSize / this.chunksTotal);&#xA;&#xA;    this.requested = [];&#xA;    for (let i = 0; i &lt; this.chunksTotal; i&#x2B;&#x2B;) {&#xA;      this.requested[i] = false;&#xA;    }&#xA;&#xA;    this.url = `${environment.apiBaseUrl}/api/tracks/${this.track.id}/play`;&#xA;  }&#xA;&#xA;  /**&#xA;   * Fetch the first chunk.&#xA;   */&#xA;  public begin() {&#xA;    this.maybeFetchChunk(0);&#xA;  }&#xA;&#xA;  /**&#xA;   * Handler for the "timeupdate" event. Checks if the next chunk should be fetched.&#xA;   *&#xA;   * @param currentTime&#xA;   *  The current time of the track which is currently played.&#xA;   */&#xA;  public handleOnTimeUpdate(currentTime: number) {&#xA;&#xA;    const nextChunkIndex = Math.floor(currentTime / CHUNK_DURATION_SEC) &#x2B; 1;&#xA;    const hasAllChunks = this.requested.every(val => !!val);&#xA;&#xA;    if (nextChunkIndex === (this.chunksTotal - 1) &amp;&amp; hasAllChunks) {&#xA;      this.logger.debug(&#x27;Last chunk. Calling mediaSource.endOfStream();&#x27;);&#xA;      return;&#xA;    }&#xA;&#xA;    if (this.requested[nextChunkIndex] === true) {&#xA;      return;&#xA;    }&#xA;&#xA;    if (currentTime &lt; CHUNK_DURATION_SEC * (nextChunkIndex - 1 &#x2B; 0.25)) {&#xA;      return;&#xA;    }&#xA;&#xA;    this.maybeFetchChunk(nextChunkIndex);&#xA;  }&#xA;&#xA;  /**&#xA;   * Fetches the chunk if it hasn&#x27;t been requested yet. After the request finished, the returned&#xA;   * chunk gets appended to the SourceBuffer-instance.&#xA;   *&#xA;   * @param chunkIndex&#xA;   *  The chunk to fetch.&#xA;   */&#xA;  private maybeFetchChunk(chunkIndex: number) {&#xA;&#xA;    const start = chunkIndex * this.chunkSize;&#xA;    const end = start &#x2B; this.chunkSize - 1;&#xA;&#xA;    if (this.requested[chunkIndex] == true) {&#xA;      return;&#xA;    }&#xA;&#xA;    this.requested[chunkIndex] = true;&#xA;&#xA;    if ((end - start) == 0) {&#xA;      this.logger.warn(&#x27;Nothing to fetch.&#x27;);&#xA;      return;&#xA;    }&#xA;&#xA;    const totalKb = ((end - start) / 1000).toFixed(2);&#xA;    this.logger.debug(`Starting to fetch bytes ${start} to ${end} (total ${totalKb} kB). Chunk ${chunkIndex &#x2B; 1} of ${this.chunksTotal}`);&#xA;&#xA;    const xhr = new XMLHttpRequest();&#xA;    xhr.open(&#x27;get&#x27;, this.url);&#xA;    xhr.setRequestHeader(&#x27;Authorization&#x27;, `Bearer ${AuthenticationService.getJwtToken()}`);&#xA;    xhr.setRequestHeader(&#x27;Range&#x27;, &#x27;bytes=&#x27; &#x2B; start &#x2B; &#x27;-&#x27; &#x2B; end);&#xA;    xhr.responseType = &#x27;arraybuffer&#x27;;&#xA;    xhr.onload = () => {&#xA;      this.logger.debug(`Range ${start} to ${end} fetched`);&#xA;      this.logger.debug(`Requested size:        ${end - start &#x2B; 1}`);&#xA;      this.logger.debug(`Fetched size:          ${xhr.response.byteLength}`);&#xA;      this.logger.debug(&#x27;Appending chunk to SourceBuffer.&#x27;);&#xA;      this.sourceBuffer.appendBuffer(xhr.response);&#xA;    };&#xA;    xhr.send();&#xA;  };&#xA;&#xA;}&#xA;&#xA;export enum StreamStatus {&#xA;  NOT_INITIALIZED,&#xA;  INITIALIZING,&#xA;  PLAYING,&#xA;  SEEKING,&#xA;  PAUSED,&#xA;  STOPPED,&#xA;  ERROR&#xA;}&#xA;&#xA;export class PlayerState {&#xA;  status: StreamStatus = StreamStatus.NOT_INITIALIZED;&#xA;}&#xA;&#xA;&#xA;/**&#xA; *&#xA; */&#xA;@Injectable({&#xA;  providedIn: &#x27;root&#x27;&#xA;})&#xA;export class MediaSourcePlayerService {&#xA;&#xA;  public track: Track;&#xA;&#xA;  private mediaSource: MediaSource;&#xA;&#xA;  private sourceBuffer: SourceBuffer;&#xA;&#xA;  private audioObj: HTMLAudioElement;&#xA;&#xA;  private chunksLoader: ChunksLoader;&#xA;&#xA;  private state: PlayerState = new PlayerState();&#xA;&#xA;  private state$ = new BehaviorSubject<playerstate>(this.state);&#xA;&#xA;  public stateChange = this.state$.asObservable();&#xA;&#xA;  private currentTime$ = new BehaviorSubject<number>(null);&#xA;&#xA;  public currentTimeChange = this.currentTime$.asObservable();&#xA;&#xA;  constructor(&#xA;    private httpClient: HttpClient,&#xA;    private logger: NGXLogger&#xA;  ) {&#xA;  }&#xA;&#xA;  get canPlay() {&#xA;    const state = this.state$.getValue();&#xA;    const status = state.status;&#xA;    return status == StreamStatus.PAUSED;&#xA;  }&#xA;&#xA;  get canPause() {&#xA;    const state = this.state$.getValue();&#xA;    const status = state.status;&#xA;    return status == StreamStatus.PLAYING || status == StreamStatus.SEEKING;&#xA;  }&#xA;&#xA;  public playTrack(track: Track) {&#xA;    this.logger.debug(&#x27;playTrack&#x27;);&#xA;    this.track = track;&#xA;    this.startPlayingFrom(0);&#xA;  }&#xA;&#xA;  public play() {&#xA;    this.logger.debug(&#x27;play()&#x27;);&#xA;    this.audioObj.play().then();&#xA;  }&#xA;&#xA;  public pause() {&#xA;    this.logger.debug(&#x27;pause()&#x27;);&#xA;    this.audioObj.pause();&#xA;  }&#xA;&#xA;  public stop() {&#xA;    this.logger.debug(&#x27;stop()&#x27;);&#xA;    this.audioObj.pause();&#xA;  }&#xA;&#xA;  public seek(seconds: number) {&#xA;    this.logger.debug(&#x27;seek()&#x27;);&#xA;    this.audioObj.currentTime = seconds;&#xA;  }&#xA;&#xA;  private startPlayingFrom(seconds: number) {&#xA;    this.logger.info(`Start playing from ${seconds.toFixed(2)} seconds`);&#xA;    this.mediaSource = new MediaSource();&#xA;    this.mediaSource.addEventListener(&#x27;sourceopen&#x27;, this.onSourceOpen);&#xA;&#xA;    this.audioObj = document.createElement(&#x27;audio&#x27;);&#xA;    this.addEvents(this.audioObj, AUDIO_EVENTS, this.handleEvent);&#xA;    this.audioObj.src = URL.createObjectURL(this.mediaSource);&#xA;&#xA;    this.audioObj.play().then();&#xA;  }&#xA;&#xA;  private onSourceOpen = () => {&#xA;&#xA;    this.logger.debug(&#x27;onSourceOpen&#x27;);&#xA;&#xA;    this.mediaSource.removeEventListener(&#x27;sourceopen&#x27;, this.onSourceOpen);&#xA;    this.mediaSource.duration = this.track.duration;&#xA;&#xA;    this.sourceBuffer = this.mediaSource.addSourceBuffer(&#x27;audio/mp4; codecs="mp4a.40.2"&#x27;);&#xA;    // this.sourceBuffer = this.mediaSource.addSourceBuffer(&#x27;audio/mpeg&#x27;);&#xA;&#xA;    this.chunksLoader = new ChunksLoader(&#xA;      this.track,&#xA;      this.sourceBuffer,&#xA;      this.logger&#xA;    );&#xA;&#xA;    this.chunksLoader.begin();&#xA;  };&#xA;&#xA;  private handleEvent = (e) => {&#xA;&#xA;    const currentTime = this.audioObj.currentTime.toFixed(2);&#xA;    const totalDuration = this.track.duration.toFixed(2);&#xA;    this.logger.warn(`MediaSource event: ${e.type} (${currentTime} of ${totalDuration} sec)`);&#xA;&#xA;    this.currentTime$.next(this.audioObj.currentTime);&#xA;&#xA;    const currentStatus = this.state$.getValue();&#xA;&#xA;    switch (e.type) {&#xA;      case &#x27;playing&#x27;:&#xA;        currentStatus.status = StreamStatus.PLAYING;&#xA;        this.state$.next(currentStatus);&#xA;        break;&#xA;      case &#x27;pause&#x27;:&#xA;        currentStatus.status = StreamStatus.PAUSED;&#xA;        this.state$.next(currentStatus);&#xA;        break;&#xA;      case &#x27;timeupdate&#x27;:&#xA;        this.chunksLoader.handleOnTimeUpdate(this.audioObj.currentTime);&#xA;        break;&#xA;      case &#x27;seeking&#x27;:&#xA;        currentStatus.status = StreamStatus.SEEKING;&#xA;        this.state$.next(currentStatus);&#xA;        if (this.mediaSource.readyState == &#x27;open&#x27;) {&#xA;          this.sourceBuffer.abort();&#xA;        }&#xA;        this.chunksLoader.handleOnTimeUpdate(this.audioObj.currentTime);&#xA;        break;&#xA;    }&#xA;  };&#xA;&#xA;  private addEvents(obj, events, handler) {&#xA;    events.forEach(event => obj.addEventListener(event, handler));&#xA;  }&#xA;&#xA;}&#xA;</number></playerstate>

  


  Running it will give me the following output :

  


  

  


  


  Apologies for the screenshot but it's not possible to just copy the output without all the stack traces in Chrome.

  


  


  What I also tried was following this example and call sourceBuffer.abort() but that didn't work. It looks more like a hack that used to work years ago but it's still referenced in the docs (see "Example" -> "You can see something similar in action in Nick Desaulnier's bufferWhenNeeded demo ..").

  


  case 'seeking':
  currentStatus.status = StreamStatus.SEEKING;
  this.state$.next(currentStatus);        
  if (this.mediaSource.readyState === 'open') {
    this.sourceBuffer.abort();
  } 
  break;


  


  Trying with MP3

  


  I have tested the above code under Chrome by converting tracks to MP3 :

  


  ffmpeg -i input.mp3 -acodec aac -b:a 256k -f mp3 output.mp3


  


  and creating a SourceBuffer using audio/mpeg as type :

  


  this.mediaSource.addSourceBuffer('audio/mpeg')


  


  I have the same problem when seeking.

  


  The issue wihout seeking

  


  The above code has another issue :

  


  After two minutes of playing, the audio playback starts to stutter and comes to a halt prematurely. So, the audio plays up to a point and then it stops without any obvious reason.

  


  For whatever reason there is another canplay and playing event. A few seconds after, the audio simply stops..