Recherche avancée
Autres articles (29)
-
Les tâches Cron régulières de la ferme
1er décembre 2010, parLa gestion de la ferme passe par l’exécution à intervalle régulier de plusieurs tâches répétitives dites Cron.
Le super Cron (gestion_mutu_super_cron)
Cette tâche, planifiée chaque minute, a pour simple effet d’appeler le Cron de l’ensemble des instances de la mutualisation régulièrement. Couplée avec un Cron système sur le site central de la mutualisation, cela permet de simplement générer des visites régulières sur les différents sites et éviter que les tâches des sites peu visités soient trop (...) -
HTML5 audio and video support
13 avril 2011, parMediaSPIP uses HTML5 video and audio tags to play multimedia files, taking advantage of the latest W3C innovations supported by modern browsers.
The MediaSPIP player used has been created specifically for MediaSPIP and can be easily adapted to fit in with a specific theme.
For older browsers the Flowplayer flash fallback is used.
MediaSPIP allows for media playback on major mobile platforms with the above (...) -
Sélection de projets utilisant MediaSPIP
29 avril 2011, parLes exemples cités ci-dessous sont des éléments représentatifs d’usages spécifiques de MediaSPIP pour certains projets.
Vous pensez avoir un site "remarquable" réalisé avec MediaSPIP ? Faites le nous savoir ici.
Ferme MediaSPIP @ Infini
L’Association Infini développe des activités d’accueil, de point d’accès internet, de formation, de conduite de projets innovants dans le domaine des Technologies de l’Information et de la Communication, et l’hébergement de sites. Elle joue en la matière un rôle unique (...)
Sur d’autres sites (5074)
-
Xuggle mp4 output (isom mp41 codec ?) issues
19 mai 2017, par JonSidesWe’re using Xuggler to create an mp4 from pictures, with no audio at this point. The mp4 we’re getting is playable locally, but cannot be embedded in a webpage or chromecast. I suspect the issue is that we’re getting an older ISOM mp41 codec that’s not supported by HTML5 and chromecast. That’s a guess based on this snippet of header info :
Internet media type : video/mp4
Codec ID : isom
Codec ID : isom (isom/iso2/mp41)
Codec ID/Url :
http://www.apple.com/quicktime/download/standalone.html
CodecID_Compatible : isom/iso2/mp41
Codec : MPEG-4
Codec : MPEG-4
Codec/Extensions usually used : mov mp4 m4v m4a m4b m4p 3ga 3gpa
3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4vI can embed and chromecast mp42 type files. So the first part of my question is Am I on the right track in looking at isom and mp41 ? And if so, can I get Xuggler to output using a newer codec ?
I can post the full header if that would be helpful.
The above snippet is from mediainfo. Here’s the whole thing, which is too long for a comment :
General
Count : 325
Count of stream of this kind : 1
Kind of stream : General
Kind of stream : General
Stream identifier : 0
Inform : MPEG-4 (Base Media) : 377 KiB, 12 s 301 ms
Count of video streams : 1
Video_Format_List : MPEG-4 Visual
Video_Format_WithHint_List : MPEG-4 Visual
Codecs Video : MPEG-4 Visual
Complete name : C :\Users\Jon\Desktop\vid-1.mp4
Folder name : C :\Users\Jon\Desktop
File name : vid-1
File extension : mp4
Format : MPEG-4
Format : MPEG-4
Format/Extensions usually used : mov mp4 m4v m4a m4b m4p 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v
Commercial name : MPEG-4
Format profile : Base Media
Internet media type : video/mp4
Codec ID : isom
Codec ID : isom (isom/iso2/mp41)
Codec ID/Url :
http://www.apple.com/quicktime/download/standalone.html
CodecID_Compatible : isom/iso2/mp41
Codec : MPEG-4
Codec : MPEG-4
Codec/Extensions usually used : mov mp4 m4v m4a m4b m4p 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v
File size : 386063
File size : 377 KiB
File size : 377 KiB
File size : 377 KiB
File size : 377 KiB
File size : 377.0 KiB
Duration : 12301
Duration : 12 s 301 ms
Duration : 12 s 301 ms
Duration : 12 s 301 ms
Duration : 00:00:12.301
Duration : 00:00:14:00
Duration : 00:00:12.301 (00:00:14:00)
Overall bit rate mode : CBR
Overall bit rate mode : Constant
Overall bit rate : 251077
Overall bit rate : 251 kb/s
Frame rate : 3.415
Frame rate : 3.415 FPS
Frame count : 42
Stream size : 1368
Stream size : 1.34 KiB (0%)
Stream size : 1 KiB
Stream size : 1.3 KiB
Stream size : 1.34 KiB
Stream size : 1.336 KiB
Stream size : 1.34 KiB (0%)
Proportion of this stream : 0.00354
HeaderSize : 36
DataSize : 384703
FooterSize : 1324
IsStreamable : No
File creation date : UTC 2017-05-05 19:49:25.209
File creation date (local) : 2017-05-05 15:49:25.209
File last modification date : UTC 2017-01-04 19:04:53.000
File last modification date (local) : 2017-01-04 15:04:53.000
Writing application : Lavf54.3.100
Writing application : Lavf54.3.100Video
Count : 338
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
StreamOrder : 0
Inform : 250 kb/s, 200*200 (1.000), at
3.415 FPS, MPEG-4 Visual (Simple@L1)
ID : 1
ID : 1
Format : MPEG-4 Visual
Commercial name : MPEG-4 Visual
Format profile : Simple@L1
Format settings, BVOP : No
Format settings, BVOP : No
Format settings, QPel : No
Format settings, QPel : No
Format settings, GMC : 0
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)
Format settings, Matrix : Default (H.263)
Internet media type : video/MP4V-ES
Codec ID : 20
Codec : MPEG-4V
Codec : MPEG-4 Visual
Codec/Family : MPEG-4V
Codec/CC : 20
Codec profile : Simple@L1
Codec settings, Packet bitstream : No
Codec settings, BVOP : No
Codec settings, QPel : No
Codec settings, GMC : 0
Codec settings, GMC : No warppoints
Codec settings, Matrix : Default (H.263)
Duration : 12301
Duration : 12 s 301 ms
Duration : 12 s 301 ms
Duration : 12 s 301 ms
Duration : 00:00:12.301
Duration : 00:00:14:00
Duration : 00:00:12.301 (00:00:14:00)
Bit rate mode : CBR
Bit rate mode : Constant
Bit rate : 250207
Bit rate : 250 kb/s
Width : 200
Width : 200 pixels
Height : 200
Height : 200 pixels
Sampled_Width : 200
Sampled_Height : 200
Pixel aspect ratio : 1.000
Display aspect ratio : 1.000
Display aspect ratio : 1.000
Rotation : 0.000
Frame rate mode : VFR
Frame rate mode : Variable
Frame rate : 3.415
Frame rate : 3.415 FPS
Minimum frame rate : 3.333
Minimum frame rate : 3.333 FPS
Maximum frame rate : 65535.000
Maximum frame rate : 65 535.000 FPS
Frame count : 42
Resolution : 8
Resolution : 8 bits
Colorimetry : 4:2:0
Color space : YUV
Chroma subsampling : 4:2:0
Chroma subsampling : 4:2:0
Bit depth : 8
Bit depth : 8 bits
Scan type : Progressive
Scan type : Progressive
Interlacement : PPF
Interlacement : Progressive
Compression mode : Lossy
Compression mode : Lossy
Bits/(Pixel*Frame) : 1.832
Stream size : 384695
Stream size : 376 KiB (100%)
Stream size : 376 KiB
Stream size : 376 KiB
Stream size : 376 KiB
Stream size : 375.7 KiB
Stream size : 376 KiB (100%)
Proportion of this stream : 0.99646
Writing library : Lavc54.14.101
Writing library : Lavc54.14.101 -
Could not find tag for codec h264 in stream #0 (mp4)
18 août 2019, par TabsNotSpacesI’ve been using the sickbeard_mp4_converter for a while to convert video files to mp4 by generating a script for ffmpeg. I’m not sure what I changed, but the ffmpeg script it generates no longer works and I’m having trouble debugging it. Can anyone tell from my log what the issue is ? Yes, its supposed to be an mp4 to an mp4, which is typically fine.
I’m at the point where I’m using an mp4 that worked with the same autogenerated script yesterday but it no longer is. I remember updating ffmpeg but downgrading did not resolve the issue, though I may have not downgraded enough.
ffmpeg 4.1.4
MediaInfo output :
$ mediainfo --fullscan Downloads/Dallas\ Buyers\ Club\ \(2013\).mp4.original
General
Count : 334
Count of stream of this kind : 1
Kind of stream : General
Kind of stream : General
Stream identifier : 0
Count of video streams : 1
Count of audio streams : 1
Video_Format_List : AVC
Video_Format_WithHint_List : AVC
Codecs Video : AVC
Audio_Format_List : AAC LC
Audio_Format_WithHint_List : AAC LC
Audio codecs : AAC LC
Audio_Language_List : English
Complete name : Downloads/Dallas Buyers Club (2013).mp4.original
Folder name : Downloads
File name extension : Dallas Buyers Club (2013).mp4.original
File name : Dallas Buyers Club (2013).mp4
File extension : original
Format : MPEG-4
Format : MPEG-4
Format/Extensions usually used : braw mov mp4 m4v m4a m4b m4p m4r 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v
Commercial name : MPEG-4
Format profile : Base Media
Internet media type : video/mp4
Codec ID : isom
Codec ID : isom (isom/avc1)
Codec ID/Url : http://www.apple.com/quicktime/download/standalone.html
CodecID_Compatible : isom/avc1
File size : 1987698473
File size : 1.85 GiB
File size : 2 GiB
File size : 1.9 GiB
File size : 1.85 GiB
File size : 1.851 GiB
Duration : 7017023
Duration : 1 h 56 min
Duration : 1 h 56 min 57 s 23 ms
Duration : 1 h 56 min
Duration : 01:56:57.023
Duration : 01:56:58;17
Duration : 01:56:57.023 (01:56:58;17)
Overall bit rate mode : VBR
Overall bit rate mode : Variable
Overall bit rate : 2266144
Overall bit rate : 2 266 kb/s
Frame rate : 23.976
Frame rate : 23.976 FPS
Frame count : 168239
Stream size : 3690657
Stream size : 3.52 MiB (0%)
Stream size : 4 MiB
Stream size : 3.5 MiB
Stream size : 3.52 MiB
Stream size : 3.520 MiB
Stream size : 3.52 MiB (0%)
Proportion of this stream : 0.00186
HeaderSize : 3690598
DataSize : 1984007824
FooterSize : 51
IsStreamable : Yes
Title : Dallas Buyers Club
Movie name : Dallas Buyers Club
Director : Jean-Marc Valle
Actor : Matthew McConaughey / Jennifer Garner / Jared Leto / Denis O'Hare / Steve Zahn
Screenplay by : Craig Borten / Melisa Wallack
Producer : Robbie Brenner / Rachel Winter / Kerry Barden / Rich Delia / Paul Schnee
Genre : Drama
ContentType : Unknown Type
Description : Sometimes it takes a hustler to change the world
Recorded date : 2013-11-17
Encoded date : UTC 2014-01-24 08:11:15
Tagged date : UTC 2014-01-24 08:11:15
File last modification date : UTC 2019-08-16 16:44:14
File last modification date (local) : 2019-08-16 11:44:14
Writing application : MDH:Dallas Buyers Club (2013).mp4
Writing application : MDH:Dallas Buyers Club (2013).mp4
Cover : Yes
ContentRating : mpaa|R|400|
LongDescription : Loosely based on the true-life tale of Ron Woodroof, a drug-taking, women-loving, homophobic man who in 1986 was diagnosed with HIV/AIDS and given thirty days to live.
FileExtension_Invalid : braw mov mp4 m4v m4a m4b m4p m4r 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v
Video
Count : 378
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
StreamOrder : 0
ID : 1
ID : 1
Format : AVC
Format : AVC
Format/Info : Advanced Video Codec
Format/Url : http://developers.videolan.org/x264.html
Commercial name : AVC
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, CABAC : Yes
Format settings, Reference frames : 4
Format settings, Reference frames : 4 frames
Internet media type : video/H264
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 7016967
Duration : 1 h 56 min
Duration : 1 h 56 min 56 s 967 ms
Duration : 1 h 56 min
Duration : 01:56:56.967
Duration : 01:56:58;17
Duration : 01:56:56.967 (01:56:58;17)
Bit rate : 2169000
Bit rate : 2 169 kb/s
Maximum bit rate : 12300880
Maximum bit rate : 12.3 Mb/s
Width : 1920
Width : 1 920 pixels
Height : 800
Height : 800 pixels
Sampled_Width : 1920
Sampled_Height : 800
Pixel aspect ratio : 1.000
Display aspect ratio : 2.400
Display aspect ratio : 2.40:1
Rotation : 0.000
Frame rate mode : CFR
Frame rate mode : Constant
Frame rate : 23.976
Frame rate : 23.976 (24000/1001) FPS
FrameRate_Num : 24000
FrameRate_Den : 1001
Original frame rate : 23.976
Original frame rate : 23.976 (23976/1000) FPS
FrameRate_Original_Num : 23976
FrameRate_Original_Den : 1000
Frame count : 168239
Color space : YUV
Chroma subsampling : 4:2:0
Chroma subsampling : 4:2:0
Bit depth : 8
Bit depth : 8 bits
Scan type : Progressive
Scan type : Progressive
Bits/(Pixel*Frame) : 0.059
Stream size : 1901715488
Stream size : 1.77 GiB (96%)
Stream size : 2 GiB
Stream size : 1.8 GiB
Stream size : 1.77 GiB
Stream size : 1.771 GiB
Stream size : 1.77 GiB (96%)
Proportion of this stream : 0.95674
Writing library : x264 - core 135 r2 f0c1c53
Writing library : x264 core 135 r2 f0c1c53
Encoded_Library_Name : x264
Encoded_Library_Version : core 135 r2 f0c1c53
Encoding settings : cabac=1 / ref=4 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-3 / threads=36 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=2169 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=31250 / vbv_bufsize=31250 / nal_hrd=none / ip_ratio=1.40 / aq=1:1.00
Encoded date : UTC 2014-01-24 08:11:15
Tagged date : UTC 2014-01-24 08:11:38
colour_description_present : Yes
colour_description_present_Source : Stream
Color range : Limited
colour_range_Source : Stream
Color primaries : BT.709
colour_primaries_Source : Stream
transfer_characteristics_Source : Stream
Matrix coefficients : BT.709
matrix_coefficients_Source : Stream
Codec configuration box : avcC
Audio
Count : 280
Count of stream of this kind : 1
Kind of stream : Audio
Kind of stream : Audio
Stream identifier : 0
StreamOrder : 1
ID : 2
ID : 2
Format : AAC
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Commercial name : AAC
Format settings, SBR : No (Explicit)
Format settings, SBR : No (Explicit)
Format_AdditionalFeatures : LC
Codec ID : mp4a-40-2
Duration : 7017023
Duration : 1 h 56 min
Duration : 1 h 56 min 57 s 23 ms
Duration : 1 h 56 min
Duration : 01:56:57.023
Duration : 01:56:38:17
Duration : 01:56:57.023 (01:56:38:17)
Bit rate mode : VBR
Bit rate mode : Variable
Bit rate : 93816
Bit rate : 93.8 kb/s
Maximum bit rate : 107376
Maximum bit rate : 107 kb/s
Channel(s) : 2
Channel(s) : 2 channels
Channel positions : Front: L R
Channel positions : 2/0/0
Channel layout : L R
Samples per frame : 1024
Sampling rate : 48000
Sampling rate : 48.0 kHz
Samples count : 336817104
Frame rate : 46.875
Frame rate : 46.875 FPS (1024 SPF)
Frame count : 328923
Compression mode : Lossy
Compression mode : Lossy
Stream size : 82292328
Stream size : 78.5 MiB (4%)
Stream size : 78 MiB
Stream size : 78 MiB
Stream size : 78.5 MiB
Stream size : 78.48 MiB
Stream size : 78.5 MiB (4%)
Proportion of this stream : 0.04140
Language : en
Language : English
Language : English
Language : en
Language : eng
Language : en
Encoded date : UTC 2014-01-24 08:11:37
Tagged date : UTC 2014-01-24 08:11:38Log :
$ /usr/local/bin/ffmpeg -i "/Users/Me/Downloads/Dallas Buyers Club (2013).mp4.original" -vcodec libx264 -map 0:0 -vb 2063k -c:a:0 copy -map 0:2 -metadata:s:a:0 language=eng -disposition:a:0 default -f mp4 -threads 0 -y "/Users/Me/Downloads/Dallas Buyers Club (2013).mp4"
ffmpeg version 4.1.4 Copyright (c) 2000-2019 the FFmpeg developers
built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.4_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-12.0.1.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-12.0.1.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fc38e801400] stream 0, timescale not set
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/Me/Downloads/Dallas Buyers Club (2013).mp4.original':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isomavc1
title : Dallas Buyers Club
genre : Drama
date : 2013-11-17
encoder : MDH:Dallas Buyers Club (2013).mp4
media_type : 9
hd_video : 2
description : Sometimes it takes a hustler to change the world
synopsis : Loosely based on the true-life tale of Ron Woodroof, a drug-taking, women-loving, homophobic man who in 1986 was diagnosed with HIV/AIDS and given thirty days to live.
creation_time : 2014-01-24T08:11:15.000000Z
Duration: 01:56:57.02, start: 0.000000, bitrate: 2266 kb/s
Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 500x750 [SAR 1:1 DAR 2:3], 90k tbr, 90k tbn, 90k tbc
Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/unknown), 1920x800, 2168 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
Metadata:
creation_time : 2014-01-24T08:11:15.000000Z
handler_name : video.264#trackID=1:fps=23.976 - Imported with GPAC 0.5.0-rev
Stream #0:2(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 93 kb/s (default)
Metadata:
creation_time : 2014-01-24T08:11:37.000000Z
handler_name : GPAC ISO Audio Handler
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Stream #0:2 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mp4 @ 0x7fc38e811000] Frame rate very high for a muxer not efficiently supporting it.
Please consider specifying a lower framerate, a different muxer or -vsync 2
[libx264 @ 0x7fc38e823e00] using SAR=1/1
[libx264 @ 0x7fc38e823e00] MB rate (135360000) > level limit (16711680)
[libx264 @ 0x7fc38e823e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fc38e823e00] profile High, level 6.2
[libx264 @ 0x7fc38e823e00] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2063 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[mp4 @ 0x7fc38e811000] Could not find tag for codec h264 in stream #0, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:0 --
[libx264 @ 0x7fc38e823e00] final ratefactor: 89.20
Conversion failed! -
Node.js readable maximize throughput/performance for compute intense readable - Writable doesn't pull data fast enough
31 décembre 2022, par flohallGeneral setup


I developed an application using AWS Lambda node.js 14.
I use a custom
ReadableimplementationFrameCreationStreamthat uses node-canvas to draw images, svgs and more on a canvas. This result is then extracted as a raw image buffer in BGRA. A single image buffer contains 1920 * 1080 * 4 Bytes = 8294400 Bytes 8 MB.
This is then piped tostdinof achild_processrunningffmpeg.
ThehighWaterMarkof myReadableinobjectMode:trueis set to 25 so that the internal buffer can use up to 8 MB * 25 = 200 MB.

All this works fine and also doesn't contain too much RAM. But I noticed after some time, that the performance is not ideally.


Performance not optimal


I have an example input that generates a video of 315 frames. If I set
highWaterMarkto a value above 25 the performance increases to the point, when I set to a value of 315 or above.

For some reason
ffmpegdoesn't start to pull any data untilhighWaterMarkis reached. Obviously thats not what I want.ffmpegshould always consume data if minimum 1 frame is cached in theReadableand if it has finished processing the frame before. And theReadableshould produce more frames as longhighWaterMarkisn't reached or the last frame has been reached. So ideally theReadableand theWriteableare busy all the time.

I found another way to improve the speed. If I add a timeout in the
_read()method of theReadableafter let's say every tenth frame for 100 ms. Then theffmpeg-Writablewill use this timeout to write some frames toffmpeg.

It seems like frames aren't passed to
ffmpegduring frame creation because some node.js main thread is busy ?

The fastest result I have if I increase
highWaterMarkabove the amount of frames - which doesn't work for longer videos as this would make the AWS Lambda RAM explode. And this makes the whole streaming idea useless. Using timeouts always gives me stomach pain. Also depending on the execution on different environments a good fitting timeout might differ. Any ideas ?

FrameCreationStream


import canvas from 'canvas';
import {Readable} from 'stream';
import {IMAGE_STREAM_BUFFER_SIZE, PerformanceUtil, RenderingLibraryError, VideoRendererInput} from 'vm-rendering-backend-commons';
import {AnimationAssets, BufferType, DrawingService, FullAnimationData} from 'vm-rendering-library';

/**
 * This is a proper back pressure compatible implementation of readable for a having a stream to read single frames from.
 * Whenever read() is called a new frame is created and added to the stream.
 * read() will be called internally until options.highWaterMark has been reached.
 * then calling read will be paused until one frame is read from the stream.
 */
export class FrameCreationStream extends Readable {

 drawingService: DrawingService;
 endFrameIndex: number;
 currentFrameIndex: number = 0;
 startFrameIndex: number;
 frameTimer: [number, number];
 readTimer: [number, number];
 fullAnimationData: FullAnimationData;

 constructor(animationAssets: AnimationAssets, fullAnimationData: FullAnimationData, videoRenderingInput: VideoRendererInput, frameTimer: [number, number]) {
 super({highWaterMark: IMAGE_STREAM_BUFFER_SIZE, objectMode: true});

 this.frameTimer = frameTimer;
 this.readTimer = PerformanceUtil.startTimer();

 this.fullAnimationData = fullAnimationData;

 this.startFrameIndex = Math.floor(videoRenderingInput.startFrameId);
 this.currentFrameIndex = this.startFrameIndex;
 this.endFrameIndex = Math.floor(videoRenderingInput.endFrameId);

 this.drawingService = new DrawingService(animationAssets, fullAnimationData, videoRenderingInput, canvas);
 console.time("read");
 }

 /**
 * this method is only overwritten for debugging
 * @param size
 */
 read(size?: number): string | Buffer {

 console.log("read("+size+")");
 const buffer = super.read(size);
 console.log(buffer);
 console.log(buffer?.length);
 if(buffer) {
 console.timeLog("read");
 }
 return buffer;
 }

 // _read() will be called when the stream wants to pull more data in.
 // _read() will be called again after each call to this.push(dataChunk) once the stream is ready to accept more data. https://nodejs.org/api/stream.html#readable_readsize
 // this way it is ensured, that even though this.createImageBuffer() is async, only one frame is created at a time and the order is kept
 _read(): void {
 // as frame numbers are consecutive and unique, we have to draw each frame number (also the first and the last one)
 if (this.currentFrameIndex <= this.endFrameIndex) {
 PerformanceUtil.logTimer(this.readTimer, 'WAIT -> READ\t');
 this.createImageBuffer()
 .then(buffer => this.optionalTimeout(buffer))
 // push means adding a buffered raw frame to the stream
 .then((buffer: Buffer) => {
 this.readTimer = PerformanceUtil.startTimer();
 // the following two frame numbers start with 1 as first value
 const processedFrameNumberOfScene = 1 + this.currentFrameIndex - this.startFrameIndex;
 const totalFrameNumberOfScene = 1 + this.endFrameIndex - this.startFrameIndex;
 // the overall frameId or frameIndex starts with frameId 0
 const processedFrameIndex = this.currentFrameIndex;
 this.currentFrameIndex++;
 this.push(buffer); // nothing besides logging should happen after calling this.push(buffer)
 console.log(processedFrameNumberOfScene + ' of ' + totalFrameNumberOfScene + ' processed - full video frameId: ' + processedFrameIndex + ' - buffered frames: ' + this.readableLength);
 })
 .catch(err => {
 // errors will be finally handled, when subscribing to frameCreation stream in ffmpeg service
 // this log is just generated for tracing errors and if for some reason the handling in ffmpeg service doesn't work
 console.log("createImageBuffer: ", err);
 this.emit("error", err);
 });
 } else {
 // push(null) makes clear that this stream has ended
 this.push(null);
 PerformanceUtil.logTimer(this.frameTimer, 'FRAME_STREAM');
 }
 }

 private optionalTimeout(buffer: Buffer): Promise<buffer> {
 if(this.currentFrameIndex % 10 === 0) {
 return new Promise(resolve => setTimeout(() => resolve(buffer), 140));
 }
 return Promise.resolve(buffer);
 }

 // prevent memory leaks - without this lambda memory will increase with every call
 _destroy(): void {
 this.drawingService.destroyStage();
 }

 /**
 * This creates a raw pixel buffer that contains a single frame of the video drawn by the rendering library
 *
 */
 public async createImageBuffer(): Promise<buffer> {

 const drawTimer = PerformanceUtil.startTimer();
 try {
 await this.drawingService.drawForFrame(this.currentFrameIndex);
 } catch (err: any) {
 throw new RenderingLibraryError(err);
 }

 PerformanceUtil.logTimer(drawTimer, 'DRAW -> FRAME\t');

 const bufferTimer = PerformanceUtil.startTimer();
 // Creates a raw pixel buffer, containing simple binary data
 // the exact same information (BGRA/screen ratio) has to be provided to ffmpeg, because ffmpeg cannot detect format for raw input
 const buffer = await this.drawingService.toBuffer(BufferType.RAW);
 PerformanceUtil.logTimer(bufferTimer, 'CANVAS -> BUFFER');

 return buffer;
 }
}
</buffer></buffer>

FfmpegService


import {ChildProcess, execFile} from 'child_process';
import {Readable} from 'stream';
import {FPS, StageSize} from 'vm-rendering-library';
import {
 FfmpegError,
 LOCAL_MERGE_VIDEOS_TEXT_FILE, LOCAL_SOUND_FILE_PATH,
 LOCAL_VIDEO_FILE_PATH,
 LOCAL_VIDEO_SOUNDLESS_MERGE_FILE_PATH
} from "vm-rendering-backend-commons";

/**
 * This class bundles all ffmpeg usages for rendering one scene.
 * FFmpeg is a console program which can transcode nearly all types of sounds, images and videos from one to another.
 */
export class FfmpegService {

 ffmpegPath: string = null;


 constructor(ffmpegPath: string) {
 this.ffmpegPath = ffmpegPath;
 }

 /**
 * Convert a stream of raw images into an .mp4 video using the command line program ffmpeg.
 *
 * @param inputStream an input stream containing images in raw format BGRA
 * @param stageSize the size of a single frame in pixels (minimum is 2*2)
 * @param outputPath the filepath to write the resulting video to
 */
 public imageToVideo(inputStream: Readable, stageSize: StageSize, outputPath: string): Promise<void> {
 const args: string[] = [
 '-f',
 'rawvideo',
 '-r',
 `${FPS}`,
 '-pix_fmt',
 'bgra',
 '-s',
 `${stageSize.width}x${stageSize.height}`,
 '-i',
 // input "-" means input will be passed via pipe (streamed)
 '-',
 // codec that also QuickTime player can understand
 '-vcodec',
 'libx264',
 '-pix_fmt',
 'yuv420p',
 /*
 * "-movflags faststart":
 * metadata at beginning of file
 * needs more RAM
 * file will be broken, if not finished properly
 * higher application compatibility
 * better for browser streaming
 */
 '-movflags',
 'faststart',
 // "-preset ultrafast", //use this to speed up compression, but quality/compression ratio gets worse
 // don't overwrite an existing file here,
 // but delete file in the beginning of execution index.ts
 // (this is better for local testing believe me)
 outputPath
 ];

 return this.execFfmpegPromise(args, inputStream);
 }

 private execFfmpegPromise(args: string[], inputStream?: Readable): Promise<void> {
 const ffmpegServiceSelf = this;
 return new Promise(function (resolve, reject) {
 const executionProcess: ChildProcess = execFile(ffmpegServiceSelf.ffmpegPath, args, (err) => {
 if (err) {
 reject(new FfmpegError(err));
 } else {
 console.log('ffmpeg finished');
 resolve();
 }
 });
 if (inputStream) {
 // it's important to listen on errors of input stream before piping it into the write stream
 // if we don't do this here, we get an unhandled promise exception for every issue in the input stream
 inputStream.on("error", err => {
 reject(err);
 });
 // don't reject promise here as the error will also be thrown inside execFile and will contain more debugging info
 // this log is just generated for tracing errors and if for some reason the handling in execFile doesn't work
 inputStream.pipe(executionProcess.stdin).on("error", err => console.log("pipe stream: " , err));
 }
 });
 }
}
</void></void>

Syndiquer tout le site
