Recherche avancée

Sur d’autres sites (8474)

• libavformat : add RCWT closed caption muxex

  14 janvier 2024, par Marth64

  libavformat : add RCWT closed caption muxex
  Signed-off-by : Marth64 <marth64@proxyid.net>
  Raw Captions With Time (RCWT) is a format native to ccextractor, a commonly
  
  used open source tool for processing 608/708 closed caption (CC) sources.
  
  It can be used to archive the original, raw CC bitstream and to produce
  
  a source file file for later CC processing or conversion. As a result,
  
  it also allows for interopability with ccextractor for processing CC data
  
  extracted via ffmpeg. The format is simple to parse and can be used
  
  to retain all lines and variants of CC.
  A free specification of RCWT can be found here :
  
  https://github.com/CCExtractor/ccextractor/blob/master/docs/BINARY_FILE_FORMAT.TXT
  
  This muxer implements the specification as of 01/05/2024, which has
  
  been stable and unchanged for 10 years as of this writing.
  This muxer will have some nuances from the way that ccextractor muxes RCWT.
  
  No compatibility issues when processing the output with ccextractor
  
  have been observed as a result of this so far, but mileage may vary
  
  and outputs will not be a bit-exact match.
  Specifically, the differences are :
  
  (1)  This muxer will identify as "FF" as the writing program identifier, so
  
  as to be honest about the output's origin.
  (2)  ffmpeg's MPEG-1/2, H264, HEVC, etc. decoders extract closed captioning
  
  data differently than ccextractor from embedded SEI/user data.
  
  For example, DVD captioning bytes will be translated to ATSC A53 format.
  
  This allows ffmpeg to handle 608/708 in a consistant way downstream.
  
  This is a lossless conversion and the meaningful data is retained.
  (3)  This muxer will not alter the extracted data except to remove invalid
  
  packets in between valid CC blocks. On the other hand, ccextractor
  
  will by default remove mid-stream padding, and add padding at the end
  
  of the stream (in order to convey the end time of the source video).
  

  • [DH] Changelog
  • [DH] doc/muxers.texi
  • [DH] libavformat/Makefile
  • [DH] libavformat/allformats.c
  • [DH] libavformat/rcwtenc.c
  • [DH] tests/fate/subtitles.mak

• ffmpeg produces duplicate pts with "wallclock_as_timestamps 1" option on MKV

  15 avril 2024, par Jax2171

  I need to get real time reference of every keyframe captured by an IP camera. The -wallclock_as_timestamps 1 option seems to do the trick for us, however we are forced to replace the TS output container with MKV to get a correct PTS epoch value 1712996356.833000.

  


  Here is the ffmpeg command used :

  


  ffmpeg -report -use_wallclock_as_timestamps 1 -rtsp_transport tcp -i rtsp://user:password1@192.168.5.21/cam/realmonitor?channel=1channel1[1]=1subtype=0 -c:v copy -c:a aac -copyts -f matroska -y rec.mkv


  


  The capture process runs without any relevant worning or error messages.

  


  However, playing the captured video with any player shows very short and evident but very annoying lags. Upon investigation I discovered that many frame PTSs have the same value. The command I used to show duplicate PTSs is as follows :

  


  ffprobe -v error -show_entries frame=pkt_pts_time -select_streams v -of csv=p=0 rec.mkv | sort | uniq -d


  


  On a recording of about 10 minutes the result of the duplicate PTS is the following :

  


  1713086493.367000
1713086493.368000
1713086493.370000
1713086493.372000
1713086543.714000
1713086558.793000
1713086558.817000
1713086558.872000
1713086561.780000
1713086564.642000
1713086564.657000
1713086564.778000
1713086565.794000
...


  


  I'm not sure if the lag problem is caused by this, however the problem does not occur with the TS container, which however I cannot use due to the PTS values being roundly 33 bit.

  


  The -vsync 0 or -vsync 2 options on input or output didn't help.

  


  This is the log using the -report option :

  


      ffmpeg started on 2024-04-15 at 09:04:38
Report written to "ffmpeg-20240415-090438.log"
Log level: 48
Command line:
ffmpeg -report -stats -hide_banner -use_wallclock_as_timestamps 1 -rtsp_transport tcp -i "rtsp://user:password1@192.168.5.21/cam/realmonitor?channel=1channel1[1]=1subtype=0" -c:v copy -c:a aac -copyts -f matroska -y rec.mkv
Splitting the commandline.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-stats' ... matched as option 'stats' (print progress report during encoding) with argument '1'.
Reading option '-hide_banner' ... matched as option 'hide_banner' (do not show program banner) with argument '1'.
Reading option '-use_wallclock_as_timestamps' ... matched as AVOption 'use_wallclock_as_timestamps' with argument '1'.
Reading option '-rtsp_transport' ... matched as AVOption 'rtsp_transport' with argument 'tcp'.
Reading option '-i' ... matched as input url with argument 'rtsp://user:password1@192.168.5.21/cam/realmonitor?channel=1channel1[1]=1subtype=0'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-c:a' ... matched as option 'c' (codec name) with argument 'aac'.
Reading option '-copyts' ... matched as option 'copyts' (copy timestamps) with argument '1'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'matroska'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option 'rec.mkv' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option report (generate a report) with argument 1.
Applying option stats (print progress report during encoding) with argument 1.
Applying option hide_banner (do not show program banner) with argument 1.
Applying option copyts (copy timestamps) with argument 1.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url rtsp://user:password1@192.168.5.21/cam/realmonitor?channel=1channel1[1]=1subtype=0.
Successfully parsed a group of options.
Opening an input file: rtsp://user:password1@192.168.5.21/cam/realmonitor?channel=1channel1[1]=1subtype=0.
[tcp @ 0x1646660] No default whitelist set
[tcp @ 0x1646660] Original list of addresses:
[tcp @ 0x1646660] Address 192.168.5.21 port 554
[tcp @ 0x1646660] Interleaved list of addresses:
[tcp @ 0x1646660] Address 192.168.5.21 port 554
[tcp @ 0x1646660] Starting connection attempt to 192.168.5.21 port 554
[tcp @ 0x1646660] Successfully connected to 192.168.5.21 port 554
[rtsp @ 0x1645e70] SDP:
v=0
o=- 2251950012 2251950012 IN IP4 0.0.0.0
s=Media Server
c=IN IP4 0.0.0.0
t=0 0
a=control:*
a=packetization-supported:DH
a=rtppayload-supported:DH
a=range:npt=now-
a=x-packetization-supported:IV
a=x-rtppayload-supported:IV
m=video 0 RTP/AVP 96
a=control:trackID=0
a=framerate:25.000000
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=4D4028;sprop-parameter-sets=Z01AKKaAeAIn5ZuAgICgAAADACAAAAZQgAA=,aO48gAA=
a=recvonly
m=audio 0 RTP/AVP 97
a=control:trackID=1
a=rtpmap:97 MPEG4-GENERIC/16000
a=fmtp:97 streamtype=5;profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1408
a=recvonly

[rtsp @ 0x1645e70] video codec set to: h264
[rtsp @ 0x1645e70] RTP Packetization Mode: 1
[rtsp @ 0x1645e70] RTP Profile IDC: 4d Profile IOP: 40 Level: 28
[rtsp @ 0x1645e70] Extradata set to 0x164af98 (size: 39)
[rtsp @ 0x1645e70] audio codec set to: aac
[rtsp @ 0x1645e70] audio samplerate set to: 16000
[rtsp @ 0x1645e70] audio channels set to: 1
[rtsp @ 0x1645e70] setting jitter buffer size to 0
[rtsp @ 0x1645e70] setting jitter buffer size to 0
[rtsp @ 0x1645e70] hello state=0
Failed to parse interval end specification ''
[h264 @ 0x164ab30] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x164ab30] Format yuvj420p chosen by get_format().
[h264 @ 0x164ab30] Reinit context to 1920x1088, pix_fmt: yuvj420p
[h264 @ 0x164ab30] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[h264 @ 0x164ab30] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 3
[rtsp @ 0x1645e70] All info found
Input #0, rtsp, from 'rtsp://user:password1@192.168.5.21/cam/realmonitor?channel=1channel1[1]=1subtype=0':
  Metadata:
    title           : Media Server
  Duration: N/A, start: 1713164678.794625, bitrate: N/A
    Stream #0:0, 22, 1/90000: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1920x1080, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1, 15, 1/16000: Audio: aac (LC), 16000 Hz, mono, fltp
Successfully opened the file.
Parsing a group of options: output url rec.mkv.
Applying option c:v (codec name) with argument copy.
Applying option c:a (codec name) with argument aac.
Applying option f (force format) with argument matroska.
Successfully parsed a group of options.
Opening an output file: rec.mkv.
[file @ 0x1699f30] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:1 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
detected 4 logical cores
[graph_0_in_0_1 @ 0x1682bb0] Setting 'time_base' to value '1/16000'
[graph_0_in_0_1 @ 0x1682bb0] Setting 'sample_rate' to value '16000'
[graph_0_in_0_1 @ 0x1682bb0] Setting 'sample_fmt' to value 'fltp'
[graph_0_in_0_1 @ 0x1682bb0] Setting 'channel_layout' to value '0x4'
[graph_0_in_0_1 @ 0x1682bb0] tb:1/16000 samplefmt:fltp samplerate:16000 chlayout:0x4
[format_out_0_1 @ 0x187f2e0] Setting 'sample_fmts' to value 'fltp'
[format_out_0_1 @ 0x187f2e0] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
[AVFilterGraph @ 0x164fd70] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed
[matroska @ 0x169c330] get_metadata_duration returned: 0
Output #0, matroska, to 'rec.mkv':
  Metadata:
    title           : Media Server
    encoder         : Lavf58.45.100
    Stream #0:0, 0, 1/1000: Video: h264 (Main) (H264 / 0x34363248), yuvj420p(pc, bt709, progressive), 1920x1080, q=2-31, 25 fps, 25 tbr, 1k tbn, 90k tbc
    Stream #0:1, 0, 1/1000: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 16000 Hz, mono, fltp, 69 kb/s
    Metadata:
      encoder         : Lavc58.91.100 aac
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:1 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[matroska @ 0x169c330] Starting new cluster with timestamp 1713164678731 at offset 770 bytes
[matroska @ 0x169c330] Writing block of size 581 with pts 1713164678731, dts 1713164678731, duration 64 at relative offset 14 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 517 with pts 1713164678795, dts 1713164678795, duration 64 at relative offset 602 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 376900 with pts 1713164678872, dts 1713164678872, duration 40 at relative offset 1126 in cluster at offset 770. TrackNumber 1, keyframe 1
[matroska @ 0x169c330] Writing block of size 8172 with pts 1713164678912, dts 1713164678912, duration 40 at relative offset 378034 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 672 with pts 1713164678912, dts 1713164678912, duration 64 at relative offset 386213 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 550 with pts 1713164679177, dts 1713164679177, duration 64 at relative offset 386892 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7654 with pts 1713164679178, dts 1713164679178, duration 40 at relative offset 387449 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 7483 with pts 1713164679213, dts 1713164679213, duration 40 at relative offset 395110 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 7703 with pts 1713164679242, dts 1713164679242, duration 40 at relative offset 402600 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 565 with pts 1713164679242, dts 1713164679242, duration 64 at relative offset 410310 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7650 with pts 1713164679271, dts 1713164679271, duration 40 at relative offset 410882 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 585 with pts 1713164679271, dts 1713164679271, duration 64 at relative offset 418539 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 8682 with pts 1713164679301, dts 1713164679301, duration 40 at relative offset 419131 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 8888 with pts 1713164679330, dts 1713164679330, duration 40 at relative offset 427820 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 506 with pts 1713164679330, dts 1713164679330, duration 64 at relative offset 436715 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 8019 with pts 1713164679360, dts 1713164679360, duration 40 at relative offset 437228 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 7919 with pts 1713164679361, dts 1713164679361, duration 40 at relative offset 445254 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 7822 with pts 1713164679361, dts 1713164679361, duration 40 at relative offset 453180 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 699 with pts 1713164679361, dts 1713164679361, duration 64 at relative offset 461009 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 619 with pts 1713164679361, dts 1713164679361, duration 64 at relative offset 461715 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7768 with pts 1713164679362, dts 1713164679362, duration 40 at relative offset 462341 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 8469 with pts 1713164679362, dts 1713164679362, duration 40 at relative offset 470116 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 601 with pts 1713164679362, dts 1713164679362, duration 64 at relative offset 478592 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 559 with pts 1713164679363, dts 1713164679363, duration 64 at relative offset 479200 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 8265 with pts 1713164679366, dts 1713164679366, duration 40 at relative offset 479766 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 7766 with pts 1713164679406, dts 1713164679406, duration 40 at relative offset 488038 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 531 with pts 1713164679415, dts 1713164679415, duration 64 at relative offset 495811 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7753 with pts 1713164679446, dts 1713164679446, duration 40 at relative offset 496349 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 8274 with pts 1713164679486, dts 1713164679486, duration 40 at relative offset 504109 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 569 with pts 1713164679496, dts 1713164679496, duration 64 at relative offset 512390 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 8445 with pts 1713164679526, dts 1713164679526, duration 40 at relative offset 512966 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 522 with pts 1713164679535, dts 1713164679535, duration 64 at relative offset 521418 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7922 with pts 1713164679566, dts 1713164679566, duration 40 at relative offset 521947 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 7954 with pts 1713164679606, dts 1713164679606, duration 40 at relative offset 529876 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 503 with pts 1713164679615, dts 1713164679615, duration 64 at relative offset 537837 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 11167 with pts 1713164679646, dts 1713164679646, duration 40 at relative offset 538347 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 503 with pts 1713164679655, dts 1713164679655, duration 64 at relative offset 549521 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 10534 with pts 1713164679686, dts 1713164679686, duration 40 at relative offset 550031 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 7607 with pts 1713164679726, dts 1713164679726, duration 40 at relative offset 560572 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 478 with pts 1713164679772, dts 1713164679772, duration 64 at relative offset 568186 in cluster at offset 770. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7842 with pts 1713164679774, dts 1713164679774, duration 40 at relative offset 568671 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 9862 with pts 1713164679806, dts 1713164679806, duration 40 at relative offset 576520 in cluster at offset 770. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Starting new cluster with timestamp 1713164679815 at offset 587166 bytes
[matroska @ 0x169c330] Writing block of size 449 with pts 1713164679815, dts 1713164679815, duration 64 at relative offset 14 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 379456 with pts 1713164679870, dts 1713164679870, duration 40 at relative offset 470 in cluster at offset 587166. TrackNumber 1, keyframe 1
[matroska @ 0x169c330] Writing block of size 415 with pts 1713164679903, dts 1713164679903, duration 64 at relative offset 379934 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7008 with pts 1713164679905, dts 1713164679905, duration 40 at relative offset 380356 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 6917 with pts 1713164679925, dts 1713164679925, duration 40 at relative offset 387371 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 513 with pts 1713164679935, dts 1713164679935, duration 64 at relative offset 394295 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7111 with pts 1713164679966, dts 1713164679966, duration 40 at relative offset 394815 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 753 with pts 1713164679975, dts 1713164679975, duration 64 at relative offset 401933 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7091 with pts 1713164680006, dts 1713164680006, duration 40 at relative offset 402693 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 7045 with pts 1713164680045, dts 1713164680045, duration 40 at relative offset 409791 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 659 with pts 1713164680055, dts 1713164680055, duration 64 at relative offset 416843 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6983 with pts 1713164680086, dts 1713164680086, duration 40 at relative offset 417509 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 6932 with pts 1713164680127, dts 1713164680127, duration 40 at relative offset 424499 in cluster at offset 587166. TrackNumber 1, keyframe 0
frame=   35 fps=0.0 q=-1.0 size=     512kB time=475879:04:40.20 bitrate=   0.0kbits/s speed=3.35e+09x    
[matroska @ 0x169c330] Writing block of size 691 with pts 1713164680135, dts 1713164680135, duration 64 at relative offset 431438 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6990 with pts 1713164680166, dts 1713164680166, duration 40 at relative offset 432136 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 651 with pts 1713164680176, dts 1713164680176, duration 64 at relative offset 439133 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7046 with pts 1713164680206, dts 1713164680206, duration 40 at relative offset 439791 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 7130 with pts 1713164680246, dts 1713164680246, duration 40 at relative offset 446844 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 601 with pts 1713164680255, dts 1713164680255, duration 64 at relative offset 453981 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 7205 with pts 1713164680286, dts 1713164680286, duration 40 at relative offset 454589 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 561 with pts 1713164680295, dts 1713164680295, duration 64 at relative offset 461801 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6936 with pts 1713164680326, dts 1713164680326, duration 40 at relative offset 462369 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 6822 with pts 1713164680366, dts 1713164680366, duration 40 at relative offset 469312 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 621 with pts 1713164680375, dts 1713164680375, duration 64 at relative offset 476141 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6845 with pts 1713164680405, dts 1713164680405, duration 40 at relative offset 476769 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 6848 with pts 1713164680445, dts 1713164680445, duration 40 at relative offset 483621 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 588 with pts 1713164680455, dts 1713164680455, duration 64 at relative offset 490476 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6828 with pts 1713164680486, dts 1713164680486, duration 40 at relative offset 491071 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 546 with pts 1713164680495, dts 1713164680495, duration 64 at relative offset 497906 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6845 with pts 1713164680526, dts 1713164680526, duration 40 at relative offset 498459 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 6924 with pts 1713164680566, dts 1713164680566, duration 40 at relative offset 505311 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 508 with pts 1713164680576, dts 1713164680576, duration 64 at relative offset 512242 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6844 with pts 1713164680606, dts 1713164680606, duration 40 at relative offset 512757 in cluster at offset 587166. TrackNumber 1, keyframe 0
frame=   48 fps= 47 q=-1.0 size=     512kB time=475879:04:40.72 bitrate=   0.0kbits/s speed=1.66e+09x    
[matroska @ 0x169c330] Writing block of size 587 with pts 1713164680615, dts 1713164680615, duration 64 at relative offset 519608 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6859 with pts 1713164680645, dts 1713164680645, duration 40 at relative offset 520202 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 6855 with pts 1713164680686, dts 1713164680686, duration 40 at relative offset 527068 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 573 with pts 1713164680695, dts 1713164680695, duration 64 at relative offset 533930 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6881 with pts 1713164680726, dts 1713164680726, duration 40 at relative offset 534510 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 10773 with pts 1713164680766, dts 1713164680766, duration 40 at relative offset 541398 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 520 with pts 1713164680775, dts 1713164680775, duration 64 at relative offset 552178 in cluster at offset 587166. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6923 with pts 1713164680805, dts 1713164680805, duration 40 at relative offset 552705 in cluster at offset 587166. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Starting new cluster with timestamp 1713164680815 at offset 1146808 bytes
[matroska @ 0x169c330] Writing block of size 580 with pts 1713164680815, dts 1713164680815, duration 64 at relative offset 14 in cluster at offset 1146808. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 380085 with pts 1713164680864, dts 1713164680864, duration 40 at relative offset 601 in cluster at offset 1146808. TrackNumber 1, keyframe 1
[matroska @ 0x169c330] Writing block of size 9916 with pts 1713164680896, dts 1713164680896, duration 40 at relative offset 380694 in cluster at offset 1146808. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 541 with pts 1713164680901, dts 1713164680901, duration 64 at relative offset 390617 in cluster at offset 1146808. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 5877 with pts 1713164680925, dts 1713164680925, duration 40 at relative offset 391165 in cluster at offset 1146808. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] Writing block of size 529 with pts 1713164680935, dts 1713164680935, duration 64 at relative offset 397049 in cluster at offset 1146808. TrackNumber 2, keyframe 1
[matroska @ 0x169c330] Writing block of size 6661 with pts 1713164680966, dts 1713164680966, duration 40 at relative offset 397585 in cluster at offset 1146808. TrackNumber 1, keyframe 0
[matroska @ 0x169c330] end duration = 1713164681006
[matroska @ 0x169c330] stream 0 end duration = 1713164681006
[matroska @ 0x169c330] stream 1 end duration = 1713164680999
frame=   54 fps= 42 q=-1.0 Lsize=    1515kB time=475879:04:40.99 bitrate=   0.0kbits/s speed=1.33e+09x    
video:1493kB audio:20kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.099897%
Input file #0 (rtsp://user:password1@192.168.5.21/cam/realmonitor?channel=1channel1[1]=1subtype=0):
  Input stream #0:0 (video): 54 packets read (1529156 bytes); 
  Input stream #0:1 (audio): 35 packets read (9268 bytes); 35 frames decoded (35840 samples); 
  Total: 89 packets (1538424 bytes) demuxed
Output file #0 (rec.mkv):
  Output stream #0:0 (video): 54 packets muxed (1529156 bytes); 
  Output stream #0:1 (audio): 35 frames encoded (35840 samples); 36 packets muxed (20446 bytes); 
  Total: 90 packets (1549602 bytes) muxed
35 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x1667620] Statistics: 2 seeks, 7 writeouts
[aac @ 0x1673880] Qavg: 142.738
Exiting normally, received signal 15.


  


  In this short 3 second capture the duplicate timestamps are 1713164679.361000 and 1713164679.362000.

  


  How can I solve this problem ? What different approach could I use to achieve this goal ?

  


  Thanks in advance.

  


• Unable to stream video file from MediaMTX media server to browser via WebRTC

  8 juin 2024, par thegreatjedi

  I took over a repository at work. It's a working demo comprising a web server which receives video and camera feeds from a media server (built from the rtsp-simple-server Docker image) via a RTSP relay server and streams the feeds to the client, all deployed via Docker Compose.

  


  I'm trying to switch over to use WebRTC instead. rtsp-simple-server has upgraded into MediaMTX since the time the demo was created 2 years ago. This is the relevant section of the updated Docker Compose configuration :

  


    media-server:
    image: bluenviron/mediamtx:latest-ffmpeg
    expose:
      - 8889
    init: true
    ports:
      - 8889:8889
    restart: unless-stopped
    volumes:
      - type: bind
        source: ./demo/vids
        target: /vids
      - type: bind
        source: ./demo/mediamtx.yml
        target: /mediamtx.yml


  


  Relevant part of the MediaMTX custom configuration in mediamtx.yml :

  


  ###############################################
# Path settings

# Settings in "paths" are applied to specific paths, and the map key
# is the name of the path.
# Any setting in "pathDefaults" can be overridden here.
# It's possible to use regular expressions by using a tilde as prefix,
# for example "~^(test1|test2)$" will match both "test1" and "test2",
# for example "~^prefix" will match all paths that start with "prefix".
paths:
  # example:
  # my_camera:
  #   source: rtsp://my_camera
  ~^demo\d+$:
    runOnDemand: ffmpeg -re -stream_loop -1 -i /vids/$MTX_PATH.mp4 -c:v libvpx -b:v 0 -crf 18 -qmin 18 -qmax 18 -f webm http://localhost:8889/$MTX_PATH/whip

  # Settings under path "all_others" are applied to all paths that
  # do not match another entry.
  all_others:


  


  I've absolutely no experience with WebRTC. This is my first time hearing of this protocol, let alone working with it. From what I understand, I need to convert my demo mp4 videos (which were successfully streaming via RTSP in the previous implementation) to a compatible video codec, so I've opted for VP8.

  


  Before trying to stream the videos into my web server, I tested the stream directly in the browser (tried with both the latest versions of Chrome and Edge). I went to http://localhost:8889/demo0 (which should convert demo0.mp4 to VP8 and then stream it over WebRTC). The video player loaded in the browser but no video data was received and nothing played. After several seconds, the screen displayed "Error : bad status code 400, retrying in some seconds". In the browser console, it showed :

  


  Failed to load resource : the server responded with a status of 400 (Bad Request)

  


  Inside the MediaMTX container's runtime logs, this is what's displayed :

  


  2024-04-02 14:53:08 ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
2024-04-02 14:53:08   built with gcc 13.2.1 (Alpine 13.2.1_git20231014) 20231014
2024-04-02 14:53:08   configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
2024-04-02 14:53:08   libavutil      58. 29.100 / 58. 29.100
2024-04-02 14:53:08   libavcodec     60. 31.102 / 60. 31.102
2024-04-02 14:53:08   libavformat    60. 16.100 / 60. 16.100
2024-04-02 14:53:08   libavdevice    60.  3.100 / 60.  3.100
2024-04-02 14:53:08   libavfilter     9. 12.100 /  9. 12.100
2024-04-02 14:53:08   libswscale      7.  5.100 /  7.  5.100
2024-04-02 14:53:08   libswresample   5.  0.100 /  5.  0.100
2024-04-02 14:53:08   libpostproc    57.  3.100 / 57.  3.100
2024-04-02 14:53:08 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/vids/demo0.mp4':
2024-04-02 14:53:08   Metadata:
2024-04-02 14:53:08     major_brand     : isom
2024-04-02 14:53:08     minor_version   : 512
2024-04-02 14:53:08     compatible_brands: isomiso2mp41
2024-04-02 14:53:08     encoder         : Lavf58.76.100
2024-04-02 14:53:08   Duration: 00:00:03.47, start: 0.000000, bitrate: 1675 kb/s
2024-04-02 14:53:08   Stream #0:0[0x1](und): Video: mpeg1video (mp4v / 0x7634706D), yuv420p(tv, progressive), 640x360 [SAR 1:1 DAR 16:9], 104857 kb/s, 30 fps, 30 tbr, 90k tbn (default)
2024-04-02 14:53:08     Metadata:
2024-04-02 14:53:08       handler_name    : VideoHandler
2024-04-02 14:53:08       vendor_id       : [0][0][0][0]
2024-04-02 14:53:08     Side data:
2024-04-02 14:53:08       cpb: bitrate max/min/avg: 0/0/0 buffer size: 49152 vbv_delay: N/A
2024-04-02 14:53:08 Stream mapping:
2024-04-02 14:53:08   Stream #0:0 -> #0:0 (mpeg1video (native) -> vp8 (libvpx))
2024-04-02 14:53:08 Press [q] to stop, [?] for help
2024-04-02 14:53:08 [libvpx @ 0x7faa8591b8c0] v1.13.1
2024-04-02 14:53:08 [libvpx @ 0x7faa8591b8c0] Bitrate not specified for constrained quality mode, using default of 256kbit/sec
2024-04-02 14:53:08 Output #0, webm, to 'http://localhost:8889/demo0/whip':
2024-04-02 14:53:08   Metadata:
2024-04-02 14:53:08     major_brand     : isom
2024-04-02 14:53:08     minor_version   : 512
2024-04-02 14:53:08     compatible_brands: isomiso2mp41
2024-04-02 14:53:08     encoder         : Lavf60.16.100
2024-04-02 14:53:08   Stream #0:0(und): Video: vp8, yuv420p(tv, progressive), 640x360 [SAR 1:1 DAR 16:9], q=2-31, 256 kb/s, 30 fps, 1k tbn (default)
2024-04-02 14:53:08     Metadata:
2024-04-02 14:53:08       handler_name    : VideoHandler
2024-04-02 14:53:08       vendor_id       : [0][0][0][0]
2024-04-02 14:53:08       encoder         : Lavc60.31.102 libvpx
2024-04-02 14:53:08     Side data:
2024-04-02 14:53:08       cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
2024-04-02 14:53:18 2024/04/02 06:53:18 INF [path demo0] runOnDemand command stopped: timed out
2024-04-02 14:53:18 2024/04/02 06:53:18 INF [WebRTC] [session 0f460c76] closed: source of path 'demo0' has timed out
[out#0/webm @ 0x7faa859487c0] video:272kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.042856%
2024-04-02 14:53:18 frame=  315 fps= 32 q=18.0 Lsize=     275kB time=00:00:10.46 bitrate= 215.1kbits/s speed=1.05x    
2024-04-02 14:53:18 Exiting normally, received signal 2.


  


  I'm not sure what this is supposed to mean ? Why isn't the server able to stream this 3-second, 709kb video even once ? The browser connected to the server and the URL successfully, but no data was being transferred.

  


  Just in case, I decided to manually convert all of my mp4 files to webm using ffmpeg, and verified with Window's media player that the webm videos work. Then, I modified MediaMTX's configuration to stream the webm videos directly :

  


  paths:
  # example:
  # my_camera:
  #   source: rtsp://my_camera
  ~^demo\d+$:
    runOnDemand: ffmpeg -re -stream_loop -1 -i /vids/$MTX_PATH.webm -c copy -f webm http://localhost:8889/$MTX_PATH/whip


  


  However, the error persists :

  


  2024-04-02 15:03:58 ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
2024-04-02 15:03:58   built with gcc 13.2.1 (Alpine 13.2.1_git20231014) 20231014
2024-04-02 15:03:58   configuration: --prefix=/usr --disable-librtmp --disable-lzma --disable-static --disable-stripping --enable-avfilter --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libmp3lame --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librist --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-lto=auto --enable-lv2 --enable-openssl --enable-pic --enable-postproc --enable-pthreads --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-vulkan --optflags=-O3 --enable-libjxl --enable-libsvtav1 --enable-libvpl
2024-04-02 15:03:58   libavutil      58. 29.100 / 58. 29.100
2024-04-02 15:03:58   libavcodec     60. 31.102 / 60. 31.102
2024-04-02 15:03:58   libavformat    60. 16.100 / 60. 16.100
2024-04-02 15:03:58   libavdevice    60.  3.100 / 60.  3.100
2024-04-02 15:03:58   libavfilter     9. 12.100 /  9. 12.100
2024-04-02 15:03:58   libswscale      7.  5.100 /  7.  5.100
2024-04-02 15:03:58   libswresample   5.  0.100 /  5.  0.100
2024-04-02 15:03:58   libpostproc    57.  3.100 / 57.  3.100
2024-04-02 15:03:58 Input #0, matroska,webm, from '/vids/demo0.webm':
2024-04-02 15:03:58   Metadata:
2024-04-02 15:03:58     COMPATIBLE_BRANDS: isomiso2mp41
2024-04-02 15:03:58     MAJOR_BRAND     : isom
2024-04-02 15:03:58     MINOR_VERSION   : 512
2024-04-02 15:03:58     ENCODER         : Lavf60.16.100
2024-04-02 15:03:58   Duration: 00:00:03.47, start: 0.000000, bitrate: 217 kb/s
2024-04-02 15:03:58   Stream #0:0: Video: vp8, yuv420p(tv, progressive), 640x360, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn (default)
2024-04-02 15:03:58     Metadata:
2024-04-02 15:03:58       HANDLER_NAME    : VideoHandler
2024-04-02 15:03:58       VENDOR_ID       : [0][0][0][0]
2024-04-02 15:03:58       ENCODER         : Lavc60.31.102 libvpx
2024-04-02 15:03:58       DURATION        : 00:00:03.466000000
2024-04-02 15:03:58 Output #0, webm, to 'http://localhost:8889/demo0/whip':
2024-04-02 15:03:58   Metadata:
2024-04-02 15:03:58     COMPATIBLE_BRANDS: isomiso2mp41
2024-04-02 15:03:58     MAJOR_BRAND     : isom
2024-04-02 15:03:58     MINOR_VERSION   : 512
2024-04-02 15:03:58     encoder         : Lavf60.16.100
2024-04-02 15:03:58   Stream #0:0: Video: vp8, yuv420p(tv, progressive), 640x360 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 1k tbn (default)
2024-04-02 15:03:58     Metadata:
2024-04-02 15:03:58       HANDLER_NAME    : VideoHandler
2024-04-02 15:03:58       VENDOR_ID       : [0][0][0][0]
2024-04-02 15:03:58       ENCODER         : Lavc60.31.102 libvpx
2024-04-02 15:03:58       DURATION        : 00:00:03.466000000
2024-04-02 15:03:58 Stream mapping:
2024-04-02 15:03:58   Stream #0:0 -> #0:0 (copy)
2024-04-02 15:03:58 Press [q] to stop, [?] for help
2024-04-02 15:04:08 2024/04/02 07:04:08 INF [path demo0] runOnDemand command stopped: timed out
2024-04-02 15:04:08 2024/04/02 07:04:08 INF [WebRTC] [session 829664cb] closed: source of path 'demo0' has timed out
[out#0/webm @ 0x7f04b00515c0] video:281kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.023511%
2024-04-02 15:04:08 size=     284kB time=00:00:10.49 bitrate= 221.3kbits/s speed=1.05x    
2024-04-02 15:04:08 Exiting normally, received signal 2.


  


  This is the same when I try to stream my other videos (demo1.mp4, demo2.mp4 etc.). What am I doing wrong ?