Recherche avancée
Médias (2)
-
SPIP - plugins - embed code - Exemple
2 septembre 2013, par
Mis à jour : Septembre 2013
Langue : français
Type : Image
-
Publier une image simplement
13 avril 2011, par ,
Mis à jour : Février 2012
Langue : français
Type : Video
Autres articles (28)
-
Ajouter notes et légendes aux images
7 février 2011, parPour pouvoir ajouter notes et légendes aux images, la première étape est d’installer le plugin "Légendes".
Une fois le plugin activé, vous pouvez le configurer dans l’espace de configuration afin de modifier les droits de création / modification et de suppression des notes. Par défaut seuls les administrateurs du site peuvent ajouter des notes aux images.
Modification lors de l’ajout d’un média
Lors de l’ajout d’un média de type "image" un nouveau bouton apparait au dessus de la prévisualisation (...) -
Librairies et binaires spécifiques au traitement vidéo et sonore
31 janvier 2010, parLes logiciels et librairies suivantes sont utilisées par SPIPmotion d’une manière ou d’une autre.
Binaires obligatoires FFMpeg : encodeur principal, permet de transcoder presque tous les types de fichiers vidéo et sonores dans les formats lisibles sur Internet. CF ce tutoriel pour son installation ; Oggz-tools : outils d’inspection de fichiers ogg ; Mediainfo : récupération d’informations depuis la plupart des formats vidéos et sonores ;
Binaires complémentaires et facultatifs flvtool2 : (...) -
Support audio et vidéo HTML5
10 avril 2011MediaSPIP utilise les balises HTML5 video et audio pour la lecture de documents multimedia en profitant des dernières innovations du W3C supportées par les navigateurs modernes.
Pour les navigateurs plus anciens, le lecteur flash Flowplayer est utilisé.
Le lecteur HTML5 utilisé a été spécifiquement créé pour MediaSPIP : il est complètement modifiable graphiquement pour correspondre à un thème choisi.
Ces technologies permettent de distribuer vidéo et son à la fois sur des ordinateurs conventionnels (...)
Sur d’autres sites (3648)
-
mp3 decode error with ffmpeg in ios
29 novembre 2016, par LL2012I met a very strange phenomenon in decoding mp3 to pcm data in ios pad.
I have tested the code on windows 10, it works correctly.However, with the same mp3 file and the same code runing on ios pad , the pcm data become total white noise.I have compare the input compressed data(mp3) and the decoded data in memory on both system. input data is all the same(none silence frame),but the decoded data has totally different.
so, can I say that there is a bug in ffmpeg on ios platforms for decoding audio.the version I used is ffmpeg2.6
anyone meet the same things ?
-
Quick stream from file which exists
4 mai 2017, par parsaI implement live streaming from mp4 video file which exists on the path with ffmpeg under hls format.
When my video file have the quality greater than hd720 , in encoding there are so many delays for creating .ts fragments.
I mean the .ts fragments slowly produced on my path,and because of it I have so many delays in showing the stream.
Here is my ffmpeg code for streaming :ffmpeg -re -i 123.mp4 -s cga -f hls -hls_list_size 0 -c:v libx264 -c:a aac 200p/out.m3u8
-s nhd -f hls -hls_list_size 0 -c:v libx264 -c:a aac 360p/out.m3u8
-s hd480 -f hls -hls_list_size 0 -c:v libx264 -c:a aac 480p/out.m3u8
-s hd720 -f hls -hls_list_size 0 -c:v libx264 -c:a aac 720p/out.m3u8
-s hd1080 -f hls -hls_list_size 0 -c:v libx264 -c:a aac 1080p/out.m3u8And here is the console output :
ffmpeg -re -i ../files/412-887123464/video/412-887123464.mp4 -s cga -f hls -hls_list_size 0 -c:v libx264 -c:a aac ../fil
es/412-887123464/video/200p/out.m3u8 -s nhd -f hls -hls_list_size 0 -c:v libx264 -c:a aac ../files/412-887123464/video/360p/out.m3u8 -s hd480 -f hls -
hls_list_size 0 -c:v libx264 -c:a aac ../files/412-887123464/video/480p/out.m3u8 -s hd720 -f hls -hls_list_size 0 -c:v libx264 -c:a aac ../files/412-88
7123464/video/720p/out.m3u8 -s hd1080 -f hls -hls_list_size 0 -c:v libx264 -c:a aac ../files/412-887123464/video/1080p/out.m3u8
ffmpeg version N-82225-gb4e9252 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 5.4.0 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --e
nable-libebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcac
a --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-lib
opencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr
--enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwa
vpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zl
ib
libavutil 55. 35.100 / 55. 35.100
libavcodec 57. 66.101 / 57. 66.101
libavformat 57. 57.100 / 57. 57.100
libavdevice 57. 2.100 / 57. 2.100
libavfilter 6. 66.100 / 6. 66.100
libswscale 4. 3.100 / 4. 3.100
libswresample 2. 4.100 / 2. 4.100
libpostproc 54. 2.100 / 54. 2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '../files/412-887123464/video/412-887123464.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.31.100
Duration: 00:05:04.07, start: 0.000000, bitrate: 3108 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 2976 kb/s, 25 fps, 25 tbr, 90k tbn, 50
tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
[libx264 @ 0000000001c53f00] using SAR=10/9
[libx264 @ 0000000001c53f00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 0000000001c53f00] profile High, level 1.3
Output #0, hls, to '../files/412-887123464/video/200p/out.m3u8':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.57.100
Stream #0:0(und): Video: h264 (libx264), yuv420p, 320x200 [SAR 10:9 DAR 16:9], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc57.66.101 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #0:1(und): Audio: aac (LC), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
encoder : Lavc57.66.101 aac
[libx264 @ 0000000001f87180] using SAR=1/1
[libx264 @ 0000000001f87180] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 0000000001f87180] profile High, level 3.0
Output #1, hls, to '../files/412-887123464/video/360p/out.m3u8':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.57.100
Stream #1:0(und): Video: h264 (libx264), yuv420p, 640x360 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc57.66.101 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #1:1(und): Audio: aac (LC), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
encoder : Lavc57.66.101 aac
[libx264 @ 0000000001c55ba0] using SAR=640/639
[libx264 @ 0000000001c55ba0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 0000000001c55ba0] profile High, level 3.0
Output #2, hls, to '../files/412-887123464/video/480p/out.m3u8':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.57.100
Stream #2:0(und): Video: h264 (libx264), yuv420p, 852x480 [SAR 640:639 DAR 16:9], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc57.66.101 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #2:1(und): Audio: aac (LC), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
encoder : Lavc57.66.101 aac
[libx264 @ 0000000001ca0220] using SAR=1/1
[libx264 @ 0000000001ca0220] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 0000000001ca0220] profile High, level 3.1
Output #3, hls, to '../files/412-887123464/video/720p/out.m3u8':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.57.100
Stream #3:0(und): Video: h264 (libx264), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc57.66.101 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #3:1(und): Audio: aac (LC), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
encoder : Lavc57.66.101 aac
[libx264 @ 0000000001ca23e0] using SAR=1/1
[libx264 @ 0000000001ca23e0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
[libx264 @ 0000000001ca23e0] profile High, level 4.0
Output #4, hls, to '../files/412-887123464/video/1080p/out.m3u8':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.57.100
Stream #4:0(und): Video: h264 (libx264), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 90k tbn, 25 tbc (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc57.66.101 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #4:1(und): Audio: aac (LC), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : SoundHandler
encoder : Lavc57.66.101 aac
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Stream #0:0 -> #1:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #1:1 (aac (native) -> aac (native))
Stream #0:0 -> #2:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #2:1 (aac (native) -> aac (native))
Stream #0:0 -> #3:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #3:1 (aac (native) -> aac (native))
Stream #0:0 -> #4:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #4:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
frame= 7600 fps= 21 q=-1.0 Lq=-1.0 q=-1.0 q=-1.0 q=-1.0 size=N/A time=00:05:04.06 bitrate=N/A speed=0.835x
video:294980kB audio:23995kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0000000001d33f00] frame I:33 Avg QP:19.93 size: 11412
[libx264 @ 0000000001d33f00] frame P:2281 Avg QP:23.72 size: 3112
[libx264 @ 0000000001d33f00] frame B:5286 Avg QP:28.59 size: 446
[libx264 @ 0000000001d33f00] consecutive B-frames: 4.3% 6.9% 5.9% 82.8%
[libx264 @ 0000000001d33f00] mb I I16..4: 12.9% 42.9% 44.2%
[libx264 @ 0000000001d33f00] mb P I16..4: 1.3% 4.4% 2.3% P16..4: 41.4% 22.0% 15.5% 0.0% 0.0% skip:13.1%
[libx264 @ 0000000001d33f00] mb B I16..4: 0.0% 0.1% 0.0% B16..8: 35.2% 5.9% 2.1% direct: 3.0% skip:53.6% L0:35.3% L1:48.0% BI:16.8%
[libx264 @ 0000000001d33f00] 8x8 transform intra:53.4% inter:57.9%
[libx264 @ 0000000001d33f00] coded y,uvDC,uvAC intra: 70.6% 89.7% 67.6% inter: 18.1% 20.3% 7.2%
[libx264 @ 0000000001d33f00] i16 v,h,dc,p: 24% 24% 6% 45%
[libx264 @ 0000000001d33f00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 14% 19% 5% 5% 7% 5% 10% 7%
[libx264 @ 0000000001d33f00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31% 15% 15% 6% 6% 7% 6% 8% 6%
[libx264 @ 0000000001d33f00] i8c dc,h,v,p: 42% 17% 28% 14%
[libx264 @ 0000000001d33f00] Weighted P-Frames: Y:26.3% UV:12.9%
[libx264 @ 0000000001d33f00] ref P L0: 58.7% 24.7% 10.9% 4.8% 1.0%
[libx264 @ 0000000001d33f00] ref B L0: 92.6% 5.8% 1.6%
[libx264 @ 0000000001d33f00] ref B L1: 96.3% 3.7%
[libx264 @ 0000000001d33f00] kb/s:258.68
[aac @ 00000000005fc9e0] Qavg: 686.637
[libx264 @ 0000000002017180] frame I:31 Avg QP:19.77 size: 29228
[libx264 @ 0000000002017180] frame P:2040 Avg QP:23.08 size: 8765
[libx264 @ 0000000002017180] frame B:5529 Avg QP:27.28 size: 1471
[libx264 @ 0000000002017180] consecutive B-frames: 1.3% 4.2% 2.4% 92.1%
[libx264 @ 0000000002017180] mb I I16..4: 8.3% 58.8% 32.9%
[libx264 @ 0000000002017180] mb P I16..4: 3.2% 9.5% 2.6% P16..4: 41.5% 19.5% 11.4% 0.0% 0.0% skip:12.2%
[libx264 @ 0000000002017180] mb B I16..4: 0.2% 0.3% 0.1% B16..8: 34.5% 5.8% 1.7% direct: 2.7% skip:54.7% L0:35.9% L1:49.5% BI:14.6%
[libx264 @ 0000000002017180] 8x8 transform intra:61.2% inter:67.3%
[libx264 @ 0000000002017180] coded y,uvDC,uvAC intra: 55.8% 84.8% 51.3% inter: 15.6% 19.3% 3.9%
[libx264 @ 0000000002017180] i16 v,h,dc,p: 24% 29% 8% 39%
[libx264 @ 0000000002017180] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 17% 21% 5% 5% 7% 5% 8% 6%
[libx264 @ 0000000002017180] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 18% 15% 6% 7% 7% 6% 7% 5%
[libx264 @ 0000000002017180] i8c dc,h,v,p: 42% 18% 27% 13%
[libx264 @ 0000000002017180] Weighted P-Frames: Y:22.3% UV:12.2%
[libx264 @ 0000000002017180] ref P L0: 58.5% 23.4% 12.2% 5.2% 0.8%
[libx264 @ 0000000002017180] ref B L0: 93.1% 5.3% 1.6%
[libx264 @ 0000000002017180] ref B L1: 96.6% 3.4%
[libx264 @ 0000000002017180] kb/s:708.47
[aac @ 0000000001d35200] Qavg: 686.637
[libx264 @ 0000000001d35ba0] frame I:31 Avg QP:19.63 size: 41522
[libx264 @ 0000000001d35ba0] frame P:2025 Avg QP:22.86 size: 13415
[libx264 @ 0000000001d35ba0] frame B:5544 Avg QP:26.52 size: 2471
[libx264 @ 0000000001d35ba0] consecutive B-frames: 1.1% 3.9% 2.8% 92.2%
[libx264 @ 0000000001d35ba0] mb I I16..4: 10.6% 62.4% 26.9%
[libx264 @ 0000000001d35ba0] mb P I16..4: 4.7% 12.4% 2.6% P16..4: 41.1% 17.8% 9.4% 0.0% 0.0% skip:11.9%
[libx264 @ 0000000001d35ba0] mb B I16..4: 0.3% 0.5% 0.1% B16..8: 34.0% 5.5% 1.4% direct: 2.9% skip:55.3% L0:36.7% L1:50.2% BI:13.1%
[libx264 @ 0000000001d35ba0] 8x8 transform intra:61.7% inter:71.6%
[libx264 @ 0000000001d35ba0] coded y,uvDC,uvAC intra: 48.0% 81.9% 43.3% inter: 14.7% 19.7% 2.6%
[libx264 @ 0000000001d35ba0] i16 v,h,dc,p: 27% 29% 9% 35%
[libx264 @ 0000000001d35ba0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 19% 22% 5% 5% 6% 5% 7% 5%
[libx264 @ 0000000001d35ba0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 26% 25% 16% 5% 6% 7% 6% 6% 4%
[libx264 @ 0000000001d35ba0] i8c dc,h,v,p: 42% 20% 26% 12%
[libx264 @ 0000000001d35ba0] Weighted P-Frames: Y:18.1% UV:10.8%
[libx264 @ 0000000001d35ba0] ref P L0: 59.5% 22.6% 12.6% 4.8% 0.5%
[libx264 @ 0000000001d35ba0] ref B L0: 93.6% 5.0% 1.4%
[libx264 @ 0000000001d35ba0] ref B L1: 97.1% 2.9%
[libx264 @ 0000000001d35ba0] kb/s:1109.23
[aac @ 0000000001d828a0] Qavg: 686.637
[libx264 @ 0000000001d7f880] frame I:32 Avg QP:19.45 size: 64737
[libx264 @ 0000000001d7f880] frame P:2104 Avg QP:22.44 size: 23985
[libx264 @ 0000000001d7f880] frame B:5464 Avg QP:25.65 size: 4999
[libx264 @ 0000000001d7f880] consecutive B-frames: 1.2% 7.7% 3.2% 87.8%
[libx264 @ 0000000001d7f880] mb I I16..4: 12.5% 70.0% 17.5%
[libx264 @ 0000000001d7f880] mb P I16..4: 6.7% 17.1% 1.8% P16..4: 40.7% 14.9% 6.6% 0.0% 0.0% skip:12.2%
[libx264 @ 0000000001d7f880] mb B I16..4: 0.6% 0.9% 0.1% B16..8: 32.5% 4.7% 1.0% direct: 2.9% skip:57.4% L0:37.6% L1:51.6% BI:10.8%
[libx264 @ 0000000001d7f880] 8x8 transform intra:65.7% inter:77.5%
[libx264 @ 0000000001d7f880] coded y,uvDC,uvAC intra: 39.5% 76.4% 32.4% inter: 13.2% 19.8% 1.5%
[libx264 @ 0000000001d7f880] i16 v,h,dc,p: 32% 27% 10% 31%
[libx264 @ 0000000001d7f880] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 18% 23% 4% 5% 5% 4% 5% 4%
[libx264 @ 0000000001d7f880] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 20% 15% 5% 7% 7% 6% 6% 4%
[libx264 @ 0000000001d7f880] i8c dc,h,v,p: 42% 19% 27% 12%
[libx264 @ 0000000001d7f880] Weighted P-Frames: Y:10.5% UV:6.7%
[libx264 @ 0000000001d7f880] ref P L0: 62.3% 20.4% 12.7% 4.3% 0.3%
[libx264 @ 0000000001d7f880] ref B L0: 93.4% 5.2% 1.4%
[libx264 @ 0000000001d7f880] ref B L1: 97.7% 2.3%
[libx264 @ 0000000001d7f880] kb/s:2101.26
[aac @ 0000000001d81560] Qavg: 686.637
[libx264 @ 0000000001d7fd60] frame I:33 Avg QP:18.36 size: 98904
[libx264 @ 0000000001d7fd60] frame P:3180 Avg QP:21.67 size: 33377
[libx264 @ 0000000001d7fd60] frame B:4387 Avg QP:24.61 size: 7729
[libx264 @ 0000000001d7fd60] consecutive B-frames: 2.7% 55.9% 15.6% 25.9%
[libx264 @ 0000000001d7fd60] mb I I16..4: 22.4% 66.9% 10.7%
[libx264 @ 0000000001d7fd60] mb P I16..4: 9.9% 17.8% 0.7% P16..4: 40.4% 9.2% 3.0% 0.0% 0.0% skip:19.0%
[libx264 @ 0000000001d7fd60] mb B I16..4: 0.8% 1.3% 0.1% B16..8: 30.2% 2.6% 0.4% direct: 1.7% skip:63.0% L0:38.1% L1:54.9% BI: 7.0%
[libx264 @ 0000000001d7fd60] 8x8 transform intra:62.5% inter:82.9%
[libx264 @ 0000000001d7fd60] coded y,uvDC,uvAC intra: 25.4% 59.5% 11.6% inter: 9.6% 16.3% 0.7%
[libx264 @ 0000000001d7fd60] i16 v,h,dc,p: 33% 28% 15% 24%
[libx264 @ 0000000001d7fd60] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 34% 20% 29% 3% 3% 4% 3% 3% 3%
[libx264 @ 0000000001d7fd60] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 25% 16% 4% 6% 6% 5% 5% 4%
[libx264 @ 0000000001d7fd60] i8c dc,h,v,p: 39% 21% 28% 12%
[libx264 @ 0000000001d7fd60] Weighted P-Frames: Y:5.7% UV:3.6%
[libx264 @ 0000000001d7fd60] ref P L0: 70.0% 18.6% 8.3% 3.1% 0.1%
[libx264 @ 0000000001d7fd60] ref B L0: 91.4% 7.6% 1.0%
[libx264 @ 0000000001d7fd60] ref B L1: 99.0% 1.0%
[libx264 @ 0000000001d7fd60] kb/s:3771.31
[aac @ 0000000001d823e0] Qavg: 686.637But I don’t have any problem for streaming files with creating resolutions from a file which have lower orginal quality than hd1080
What I can to do ? -
ffmpeg record video plays too fast
29 avril 2023, par Kris XiaI'm a college student and I am studying FFmpeg now.



I have wrote a software that can record desktops and audio('virtual-audio-capturer') with FFmpeg.And I am now writing Audio and Video Synchronization.
I met some problems that video recording plays too fast.



When I look for audio and video synchronization help on the Internet，I find a formula for calculating PTS ：



pts = n * (（1 / timbase）/ fps)



When I use this formula,I find a phenomenon.



1.The higher frame rate is,the faster the video playback speed.



2.The slower the frame rate, the faster the video playback.



Also I find while the framerate is 10，the video playback speed will be right.



Why has this situation happened ?



I have thought this question for three days. I really hope someone can help me solve this problem.



I really appreciate the help.



#include "stdafx.h"

#ifdef __cplusplus
extern "C"
{
#endif
#include "libavcodec/avcodec.h"
#include "libavformat/avformat.h"
#include "libswscale/swscale.h"
#include "libavdevice/avdevice.h"
#include "libavutil/audio_fifo.h"

#include "libavfilter/buffersink.h"
#include "libavfilter/buffersrc.h"
#include "libavutil/imgutils.h"
#include "libavutil/mathematics.h"
#include "libavutil/samplefmt.h"
#include "libavutil/time.h"
#include "libavutil/opt.h"
#include "libavutil/pixdesc.h"
#include "libavutil/file.h"
#include "libavutil/mem.h"
#include "libavutil/frame.h"
#include "libavfilter/avfilter.h"
#include "libswresample/swresample.h"

#pragma comment(lib, "avcodec.lib")
#pragma comment(lib, "avformat.lib")
#pragma comment(lib, "avutil.lib")
#pragma comment(lib, "avdevice.lib")
#pragma comment(lib, "avfilter.lib")

#pragma comment(lib, "avfilter.lib")
#pragma comment(lib, "postproc.lib")
#pragma comment(lib, "swresample.lib")
#pragma comment(lib, "swscale.lib")
#ifdef __cplusplus
};
#endif

AVFormatContext *pFormatCtx_Video = NULL, *pFormatCtx_Audio = NULL, *pFormatCtx_Out = NULL;

AVCodecContext *outVideoCodecCtx = NULL;
AVCodecContext *outAudioCodecCtx = NULL;

AVStream *pVideoStream = NULL, *pAudioStream = NULL;

AVCodec *outAVCodec;
AVCodec *outAudioCodec;

AVCodecContext *pCodecCtx_Video;
AVCodec *pCodec_Video;
AVFifoBuffer *fifo_video = NULL;
AVAudioFifo *fifo_audio = NULL;
int VideoIndex, AudioIndex;
int codec_id;

CRITICAL_SECTION AudioSection, VideoSection;



SwsContext *img_convert_ctx;
int frame_size = 0;

uint8_t *picture_buf = NULL, *frame_buf = NULL;

bool bCap = true;

DWORD WINAPI ScreenCapThreadProc( LPVOID lpParam );
DWORD WINAPI AudioCapThreadProc( LPVOID lpParam );

int OpenVideoCapture()
{
 AVInputFormat *ifmt=av_find_input_format("gdigrab");
 AVDictionary *options = NULL;
 av_dict_set(&options, "framerate", "60", NULL);
 if(avformat_open_input(&pFormatCtx_Video, "desktop", ifmt, &options)!=0)
 {
 printf("Couldn't open input stream.（无法打开视频输入流）\n");
 return -1;
 }
 if(avformat_find_stream_info(pFormatCtx_Video,NULL)<0)
 {
 printf("Couldn't find stream information.（无法获取视频流信息）\n");
 return -1;
 }
 if (pFormatCtx_Video->streams[0]->codec->codec_type != AVMEDIA_TYPE_VIDEO)
 {
 printf("Couldn't find video stream information.（无法获取视频流信息）\n");
 return -1;
 }
 pCodecCtx_Video = pFormatCtx_Video->streams[0]->codec;
 pCodec_Video = avcodec_find_decoder(pCodecCtx_Video->codec_id);
 if(pCodec_Video == NULL)
 {
 printf("Codec not found.（没有找到解码器）\n");
 return -1;
 }
 if(avcodec_open2(pCodecCtx_Video, pCodec_Video, NULL) < 0)
 {
 printf("Could not open codec.（无法打开解码器）\n");
 return -1;
 }

 av_dump_format(pFormatCtx_Video, 0, NULL, 0);

 img_convert_ctx = sws_getContext(pCodecCtx_Video->width, pCodecCtx_Video->height, pCodecCtx_Video->pix_fmt, 
 pCodecCtx_Video->width, pCodecCtx_Video->height, PIX_FMT_YUV420P, SWS_BICUBIC, NULL, NULL, NULL); 

 frame_size = avpicture_get_size(pCodecCtx_Video->pix_fmt, pCodecCtx_Video->width, pCodecCtx_Video->height);
 fifo_video = av_fifo_alloc(30 * avpicture_get_size(AV_PIX_FMT_YUV420P, pCodecCtx_Video->width, pCodecCtx_Video->height));

 return 0;
}

static char *dup_wchar_to_utf8(wchar_t *w)
{
 char *s = NULL;
 int l = WideCharToMultiByte(CP_UTF8, 0, w, -1, 0, 0, 0, 0);
 s = (char *) av_malloc(l);
 if (s)
 WideCharToMultiByte(CP_UTF8, 0, w, -1, s, l, 0, 0);
 return s;
}

int OpenAudioCapture()
{
 AVInputFormat *pAudioInputFmt = av_find_input_format("dshow");
 char * psDevName = dup_wchar_to_utf8(L"audio=virtual-audio-capturer");

 if (avformat_open_input(&pFormatCtx_Audio, psDevName, pAudioInputFmt,NULL) < 0)
 {
 printf("Couldn't open input stream.（无法打开音频输入流）\n");
 return -1;
 }

 if(avformat_find_stream_info(pFormatCtx_Audio,NULL)<0) 
 return -1; 

 if(pFormatCtx_Audio->streams[0]->codec->codec_type != AVMEDIA_TYPE_AUDIO)
 {
 printf("Couldn't find video stream information.（无法获取音频流信息）\n");
 return -1;
 }

 AVCodec *tmpCodec = avcodec_find_decoder(pFormatCtx_Audio->streams[0]->codec->codec_id);
 if(0 > avcodec_open2(pFormatCtx_Audio->streams[0]->codec, tmpCodec, NULL))
 {
 printf("can not find or open audio decoder!\n");
 }

 av_dump_format(pFormatCtx_Audio, 0, NULL, 0);

 return 0;
}

int OpenOutPut()
{
 AVStream *pVideoStream = NULL, *pAudioStream = NULL;
 const char *outFileName = "test.mp4";
 avformat_alloc_output_context2(&pFormatCtx_Out, NULL, NULL, outFileName);

 if (pFormatCtx_Video->streams[0]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
 {
 VideoIndex = 0;
 pVideoStream = avformat_new_stream(pFormatCtx_Out, NULL);
 if (!pVideoStream)
 {
 printf("can not new stream for output!\n");
 return -1;
 }

 outVideoCodecCtx = avcodec_alloc_context3(outAVCodec);
 if ( !outVideoCodecCtx )
 {
 printf("Error : avcodec_alloc_context3()\n");
 return -1;
 }

 //set codec context param
 outVideoCodecCtx = pVideoStream->codec;
 outVideoCodecCtx->codec_id = AV_CODEC_ID_MPEG4;
 outVideoCodecCtx->width = pFormatCtx_Video->streams[0]->codec->width;
 outVideoCodecCtx->height = pFormatCtx_Video->streams[0]->codec->height;
 outVideoCodecCtx->time_base = pFormatCtx_Video->streams[0]->codec->time_base;
 outVideoCodecCtx->pix_fmt = AV_PIX_FMT_YUV420P;
 outVideoCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;

 if (codec_id == AV_CODEC_ID_H264)
 {
 av_opt_set(outVideoCodecCtx->priv_data, "preset", "slow", 0);
 }

 outAVCodec = avcodec_find_encoder(AV_CODEC_ID_MPEG4);
 if( !outAVCodec )
 {
 printf("\n\nError : avcodec_find_encoder()");
 return -1;
 }
 if (pFormatCtx_Out->oformat->flags & AVFMT_GLOBALHEADER)
 outVideoCodecCtx->flags |=CODEC_FLAG_GLOBAL_HEADER;

 if ((avcodec_open2(outVideoCodecCtx,outAVCodec, NULL)) < 0)
 {
 printf("can not open the encoder\n");
 return -1;
 }
 }

 if(pFormatCtx_Audio->streams[0]->codec->codec_type == AVMEDIA_TYPE_AUDIO)
 {
 AVCodecContext *pOutputCodecCtx;
 AudioIndex = 1;
 pAudioStream = avformat_new_stream(pFormatCtx_Out, NULL);

 pAudioStream->codec->codec = avcodec_find_encoder(pFormatCtx_Out->oformat->audio_codec);

 pOutputCodecCtx = pAudioStream->codec;

 pOutputCodecCtx->sample_rate = pFormatCtx_Audio->streams[0]->codec->sample_rate;
 pOutputCodecCtx->channel_layout = pFormatCtx_Out->streams[0]->codec->channel_layout;
 pOutputCodecCtx->channels = av_get_channel_layout_nb_channels(pAudioStream->codec->channel_layout);
 if(pOutputCodecCtx->channel_layout == 0)
 {
 pOutputCodecCtx->channel_layout = AV_CH_LAYOUT_STEREO;
 pOutputCodecCtx->channels = av_get_channel_layout_nb_channels(pOutputCodecCtx->channel_layout);

 }
 pOutputCodecCtx->sample_fmt = pAudioStream->codec->codec->sample_fmts[0];
 AVRational time_base={1, pAudioStream->codec->sample_rate};
 pAudioStream->time_base = time_base;
 //audioCodecCtx->time_base = time_base;

 pOutputCodecCtx->codec_tag = 0; 
 if (pFormatCtx_Out->oformat->flags & AVFMT_GLOBALHEADER) 
 pOutputCodecCtx->flags |= CODEC_FLAG_GLOBAL_HEADER;

 if (avcodec_open2(pOutputCodecCtx, pOutputCodecCtx->codec, 0) < 0)
 {
 printf("编码器打开失败，退出程序\n");
 return -1;
 }
 }

 if (!(pFormatCtx_Out->oformat->flags & AVFMT_NOFILE))
 {
 if(avio_open(&pFormatCtx_Out->pb, outFileName, AVIO_FLAG_WRITE) < 0)
 {
 printf("can not open output file handle!\n");
 return -1;
 }
 }

 if(avformat_write_header(pFormatCtx_Out, NULL) < 0)
 {
 printf("can not write the header of the output file!\n");
 return -1;
 }

 return 0;
}

int _tmain(int argc, _TCHAR* argv[])
{
 av_register_all();
 avdevice_register_all();
 if (OpenVideoCapture() < 0)
 {
 return -1;
 }
 if (OpenAudioCapture() < 0)
 {
 return -1;
 }
 if (OpenOutPut() < 0)
 {
 return -1;
 }
// int fps;
 /*printf("输入帧率：");
 scanf_s("%d",&fps);
 if ( NULL == fps)
 {
 fps = 10;
 }*/

 InitializeCriticalSection(&VideoSection);
 InitializeCriticalSection(&AudioSection);

 AVFrame *picture = av_frame_alloc();
 int size = avpicture_get_size(pFormatCtx_Out->streams[VideoIndex]->codec->pix_fmt, 
 pFormatCtx_Out->streams[VideoIndex]->codec->width, pFormatCtx_Out->streams[VideoIndex]->codec->height);
 picture_buf = new uint8_t[size];

 avpicture_fill((AVPicture *)picture, picture_buf, 
 pFormatCtx_Out->streams[VideoIndex]->codec->pix_fmt, 
 pFormatCtx_Out->streams[VideoIndex]->codec->width, 
 pFormatCtx_Out->streams[VideoIndex]->codec->height);



 //star cap screen thread
 CreateThread( NULL, 0, ScreenCapThreadProc, 0, 0, NULL);
 //star cap audio thread
 CreateThread( NULL, 0, AudioCapThreadProc, 0, 0, NULL);
 int64_t cur_pts_v=0,cur_pts_a=0;
 int VideoFrameIndex = 0, AudioFrameIndex = 0;

 while(1)
 {
 if (_kbhit() != 0 && bCap)
 {
 bCap = false;
 Sleep(2000);
 }
 if (fifo_audio && fifo_video)
 {
 int sizeAudio = av_audio_fifo_size(fifo_audio);
 int sizeVideo = av_fifo_size(fifo_video);
 //缓存数据写完就结束循环
 if (av_audio_fifo_size(fifo_audio) <= pFormatCtx_Out->streams[AudioIndex]->codec->frame_size && 
 av_fifo_size(fifo_video) <= frame_size && !bCap)
 {
 break;
 }
 }

 if(av_compare_ts(cur_pts_v, pFormatCtx_Out->streams[VideoIndex]->time_base, 
 cur_pts_a,pFormatCtx_Out->streams[AudioIndex]->time_base) <= 0)
 {
 if (av_fifo_size(fifo_video) < frame_size && !bCap)
 {
 cur_pts_v = 0x7fffffffffffffff;
 }
 if(av_fifo_size(fifo_video) >= size)
 {
 EnterCriticalSection(&VideoSection);
 av_fifo_generic_read(fifo_video, picture_buf, size, NULL); //将数据从avfifobuffer馈送到用户提供的回调。
 LeaveCriticalSection(&VideoSection);

 avpicture_fill((AVPicture *)picture, picture_buf,
 pFormatCtx_Out->streams[VideoIndex]->codec->pix_fmt,
 pFormatCtx_Out->streams[VideoIndex]->codec->width,
 pFormatCtx_Out->streams[VideoIndex]->codec->height); //根据指定的图像参数和提供的图像数据缓冲区设置图片字段。

 //pts = n * (（1 / timbase）/ fps);
 //picture->pts = VideoFrameIndex * ((pFormatCtx_Video->streams[0]->time_base.den / pFormatCtx_Video->streams[0]->time_base.num) / 24);
 picture->pts = VideoFrameIndex * ((outVideoCodecCtx->time_base.den * 100000 / outVideoCodecCtx->time_base.num) / 180);

 int got_picture = 0;
 AVPacket pkt;
 av_init_packet(&pkt);

 pkt.data = NULL;
 pkt.size = 0;
 //从帧中获取输入的原始视频数据
 int ret = avcodec_encode_video2(pFormatCtx_Out->streams[VideoIndex]->codec, &pkt, picture, &got_picture);
 if(ret < 0)
 {
 continue;
 }

 if (got_picture==1)
 {
 pkt.stream_index = VideoIndex;
 /*int count = 1;
 pkt.pts = pkt.dts = count * ((pFormatCtx_Video->streams[0]->time_base.den / pFormatCtx_Video->streams[0]->time_base.num) / 15);
 count++;*/

 //x = pts * (timebase1.num / timebase1.den )* (timebase2.den / timebase2.num);

 pkt.pts = av_rescale_q_rnd(pkt.pts, pFormatCtx_Video->streams[0]->time_base, 
 pFormatCtx_Out->streams[VideoIndex]->time_base, (AVRounding)(AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX)); 
 pkt.dts = av_rescale_q_rnd(pkt.dts, pFormatCtx_Video->streams[0]->time_base, 
 pFormatCtx_Out->streams[VideoIndex]->time_base, (AVRounding)(AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX)); 


 pkt.duration = ((pFormatCtx_Out->streams[0]->time_base.den / pFormatCtx_Out->streams[0]->time_base.num) / 60);
 //pkt.duration = 1000/60;
 //pkt.pts = pkt.dts = Count * (ofmt_ctx->streams[stream_index]->time_base.den) /ofmt_ctx->streams[stream_index]->time_base.num / 10;

 //Count++;


 cur_pts_v = pkt.pts;

 ret = av_interleaved_write_frame(pFormatCtx_Out, &pkt);
 //delete[] pkt.data;
 av_free_packet(&pkt);
 }
 VideoFrameIndex++;
 }
 }
 else
 {
 if (NULL == fifo_audio)
 {
 continue;//还未初始化fifo
 }
 if (av_audio_fifo_size(fifo_audio) < pFormatCtx_Out->streams[AudioIndex]->codec->frame_size && !bCap)
 {
 cur_pts_a = 0x7fffffffffffffff;
 }
 if(av_audio_fifo_size(fifo_audio) >= 
 (pFormatCtx_Out->streams[AudioIndex]->codec->frame_size > 0 ? pFormatCtx_Out->streams[AudioIndex]->codec->frame_size : 1024))
 {
 AVFrame *frame;
 frame = av_frame_alloc();
 frame->nb_samples = pFormatCtx_Out->streams[AudioIndex]->codec->frame_size>0 ? pFormatCtx_Out->streams[AudioIndex]->codec->frame_size: 1024;
 frame->channel_layout = pFormatCtx_Out->streams[AudioIndex]->codec->channel_layout;
 frame->format = pFormatCtx_Out->streams[AudioIndex]->codec->sample_fmt;
 frame->sample_rate = pFormatCtx_Out->streams[AudioIndex]->codec->sample_rate;
 av_frame_get_buffer(frame, 0);

 EnterCriticalSection(&AudioSection);
 av_audio_fifo_read(fifo_audio, (void **)frame->data, 
 (pFormatCtx_Out->streams[AudioIndex]->codec->frame_size > 0 ? pFormatCtx_Out->streams[AudioIndex]->codec->frame_size : 1024));
 LeaveCriticalSection(&AudioSection);

 AVPacket pkt_out;
 av_init_packet(&pkt_out);
 int got_picture = -1;
 pkt_out.data = NULL;
 pkt_out.size = 0;

 frame->pts = AudioFrameIndex * pFormatCtx_Out->streams[AudioIndex]->codec->frame_size;
 if (avcodec_encode_audio2(pFormatCtx_Out->streams[AudioIndex]->codec, &pkt_out, frame, &got_picture) < 0)
 {
 printf("can not decoder a frame");
 }
 av_frame_free(&frame);
 if (got_picture) 
 {
 pkt_out.stream_index = AudioIndex;
 pkt_out.pts = AudioFrameIndex * pFormatCtx_Out->streams[AudioIndex]->codec->frame_size;
 pkt_out.dts = AudioFrameIndex * pFormatCtx_Out->streams[AudioIndex]->codec->frame_size;
 pkt_out.duration = pFormatCtx_Out->streams[AudioIndex]->codec->frame_size;

 cur_pts_a = pkt_out.pts;

 int ret = av_interleaved_write_frame(pFormatCtx_Out, &pkt_out);
 av_free_packet(&pkt_out);
 }
 AudioFrameIndex++;
 }
 }
 }

 delete[] picture_buf;

 av_fifo_free(fifo_video);
 av_audio_fifo_free(fifo_audio);

 av_write_trailer(pFormatCtx_Out);

 avio_close(pFormatCtx_Out->pb);
 avformat_free_context(pFormatCtx_Out);

 if (pFormatCtx_Video != NULL)
 {
 avformat_close_input(&pFormatCtx_Video);
 pFormatCtx_Video = NULL;
 }
 if (pFormatCtx_Audio != NULL)
 {
 avformat_close_input(&pFormatCtx_Audio);
 pFormatCtx_Audio = NULL;
 }

 return 0;
}

DWORD WINAPI ScreenCapThreadProc( LPVOID lpParam )
{
 AVPacket packet;
 int got_picture;
 AVFrame *pFrame;
 pFrame=av_frame_alloc();

 AVFrame *picture = av_frame_alloc();
 int size = avpicture_get_size(pFormatCtx_Out->streams[VideoIndex]->codec->pix_fmt, 
 pFormatCtx_Out->streams[VideoIndex]->codec->width, 
 pFormatCtx_Out->streams[VideoIndex]->codec->height);

 avpicture_fill((AVPicture *)picture, picture_buf, 
 pFormatCtx_Out->streams[VideoIndex]->codec->pix_fmt, 
 pFormatCtx_Out->streams[VideoIndex]->codec->width, 
 pFormatCtx_Out->streams[VideoIndex]->codec->height);

 FILE *p = NULL;
 p = fopen("proc_test.yuv", "wb+");
 av_init_packet(&packet);
 int height = pFormatCtx_Out->streams[VideoIndex]->codec->height;
 int width = pFormatCtx_Out->streams[VideoIndex]->codec->width;
 int y_size=height*width;
 while(bCap)
 {
 packet.data = NULL;
 packet.size = 0;
 if (av_read_frame(pFormatCtx_Video, &packet) < 0)
 {
 continue;
 }
 if(packet.stream_index == 0)
 {
 if (avcodec_decode_video2(pCodecCtx_Video, pFrame, &got_picture, &packet) < 0)
 {
 printf("Decode Error.（解码错误）\n");
 continue;
 }
 if (got_picture)
 {
 sws_scale(img_convert_ctx, 
 (const uint8_t* const*)pFrame->data,
 pFrame->linesize, 
 0, 
 pFormatCtx_Out->streams[VideoIndex]->codec->height,
 picture->data,
 picture->linesize);

 if (av_fifo_space(fifo_video) >= size)
 {
 EnterCriticalSection(&VideoSection); 
 av_fifo_generic_write(fifo_video, picture->data[0], y_size, NULL);
 av_fifo_generic_write(fifo_video, picture->data[1], y_size/4, NULL);
 av_fifo_generic_write(fifo_video, picture->data[2], y_size/4, NULL);
 LeaveCriticalSection(&VideoSection);
 }
 }
 }
 av_free_packet(&packet);
 }
 av_frame_free(&pFrame);
 av_frame_free(&picture);
 return 0;
}

DWORD WINAPI AudioCapThreadProc( LPVOID lpParam )
{
 AVPacket pkt;
 AVFrame *frame;
 frame = av_frame_alloc();
 int gotframe;
 while(bCap)
 {
 pkt.data = NULL;
 pkt.size = 0;
 if(av_read_frame(pFormatCtx_Audio,&pkt) < 0)
 {
 continue;
 }

 if (avcodec_decode_audio4(pFormatCtx_Audio->streams[0]->codec, frame, &gotframe, &pkt) < 0)
 {
 av_frame_free(&frame);
 printf("can not decoder a frame");
 break;
 }
 av_free_packet(&pkt);

 if (!gotframe)
 {
 printf("没有获取到数据，继续下一次");
 continue;
 }

 if (NULL == fifo_audio)
 {
 fifo_audio = av_audio_fifo_alloc(pFormatCtx_Audio->streams[0]->codec->sample_fmt, 
 pFormatCtx_Audio->streams[0]->codec->channels, 30 * frame->nb_samples);
 }

 int buf_space = av_audio_fifo_space(fifo_audio);
 if (av_audio_fifo_space(fifo_audio) >= frame->nb_samples)
 {
 EnterCriticalSection(&AudioSection);
 av_audio_fifo_write(fifo_audio, (void **)frame->data, frame->nb_samples);
 LeaveCriticalSection(&AudioSection);
 }
 }
 av_frame_free(&frame);
 return 0;
}



Maybe there is another way to calculate PTS and DTS



I hope whatever the frame rate is,video playback speed is right.Not too fast or too slow.


Syndiquer tout le site
