Recherche avancée

Sur d’autres sites (2748)

• ffmpeg autorotate results in broken output when moving from cpu to gpu

  16 août 2023, par user2290269

  Given an input file with a Display Matrix :

  


  ffprobe -show_format -show_streams -print_format json -i rotated.mov

  


  Using CPU-transcoding is resulting in a correctly rotated output file and exitcode 0 :

  


  ffmpeg -autorotate 1 -y -i rotated.mov -f mp4 -vf scale=-2:360 -c:v libx264 -b:v 800K cpu.mp4

  


  Using GPU-transcoding is resulting in a incorretly rotated output file and exitcode 0 :

  


  ffmpeg -hwaccel cuda -hwaccel_output_format cuda -autorotate 1 -y -i rotated.mov -f mp4 -vf scale_cuda=-2:360 -c:v h264_nvenc -b:v 800K gpu.mp4

  


  See the Display Matrix rotation information.
The GPU-File plays upside down, the CPU-File plays correctly.

  


  Looks like (the default -autoroatate 1) is silently ignored for GPU-based transcoding. automatic insertion of filters fails silently. exitcode is 0 for gpu, too. but the output is actually broken, because it is upside down in playback.

  


  Same is true for input having other rotations than -180°.

  


  One (bad) way to solve, use complex filters :
When using complex filter with hwupload_cuda, the autorotate-filters get added correctly automatically.

  


  ffmpeg -y -i rotated.mov -f mp4 -vf hwupload_cuda=device=0,scale_cuda=-2:360 -c:v h264_nvenc -b:v 800K gpu-complexfilter.mp4

  


  When going into scale_cuda directly there is an error message and exitcode !=0.

  


  ffmpeg -y -loglevel debug -i rotated.mov -f mp4 -vf scale_cuda=-2:360 -c:v h264_nvenc -b:v 800K gpu-error.mp4

  


  Impossible to convert between the formats supported by the filter 'hflip' and the filter 'auto_scale_0'
[vf#0:0 @ 0000026723318540] Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while filtering: Function not implemented


  


  Using complex filters is not a good solution, because it disables automatic filter graph filter-adding and -mapping, which is very useful for e.g. -f HLS.

  


  So is there a way to keep complex filters away and get correct output or exitcode !=0 indicating a broken output for any given input-file coming as a file-upload from the internet ?

  


  e:\>ffprobe -show_format -show_streams -print_format json -i rotated.mov
ffprobe version 2023-08-14-git-c704901324-full_build-www.gyan.dev Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58. 16.101 / 58. 16.101
  libavcodec     60. 23.100 / 60. 23.100
  libavformat    60. 10.100 / 60. 10.100
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9. 11.100 /  9. 11.100
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
{
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'rotated.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf59.34.102
  Duration: 00:00:10.07, start: 0.000000, bitrate: 15190 kb/s
  Stream #0:0[0x1]: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 15003 kb/s, 30 fps, 30 tbr, 19200 tbn (default)
    Metadata:
      handler_name    : Core Media Video
      vendor_id       : FFMP
      encoder         : H.264
    Side data:
      displaymatrix: rotation of -180.00 degrees
  Stream #0:1[0x2]: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 177 kb/s (default)
    Metadata:
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
    "streams": [
        {
            "index": 0,
            "codec_name": "h264",
            "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
            "profile": "High",
            "codec_type": "video",
            "codec_tag_string": "avc1",
            "codec_tag": "0x31637661",
            "width": 1920,
            "height": 1080,
            "coded_width": 1920,
            "coded_height": 1080,
            "closed_captions": 0,
            "film_grain": 0,
            "has_b_frames": 1,
            "pix_fmt": "yuv420p",
            "level": 40,
            "color_range": "tv",
            "color_space": "bt709",
            "color_transfer": "bt709",
            "color_primaries": "bt709",
            "chroma_location": "left",
            "field_order": "progressive",
            "refs": 1,
            "is_avc": "true",
            "nal_length_size": "4",
            "id": "0x1",
            "r_frame_rate": "30/1",
            "avg_frame_rate": "30/1",
            "time_base": "1/19200",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 193280,
            "duration": "10.066667",
            "bit_rate": "15003313",
            "bits_per_raw_sample": "8",
            "nb_frames": "302",
            "extradata_size": 35,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "handler_name": "Core Media Video",
                "vendor_id": "FFMP",
                "encoder": "H.264"
            },
            "side_data_list": [
                {
                    "side_data_type": "Display Matrix",
                    "displaymatrix": "\n00000000:       -65536           0           0\n00000001:            0      -65536           0\n00000002:    125829120    70778880  1073741824\n",
                    "rotation": -180
                }
            ]
        },
        {
            "index": 1,
            "codec_name": "aac",
            "codec_long_name": "AAC (Advanced Audio Coding)",
            "profile": "LC",
            "codec_type": "audio",
            "codec_tag_string": "mp4a",
            "codec_tag": "0x6134706d",
            "sample_fmt": "fltp",
            "sample_rate": "44100",
            "channels": 2,
            "channel_layout": "stereo",
            "bits_per_sample": 0,
            "initial_padding": 0,
            "id": "0x2",
            "r_frame_rate": "0/0",
            "avg_frame_rate": "0/0",
            "time_base": "1/44100",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 441309,
            "duration": "10.007007",
            "bit_rate": "177895",
            "nb_frames": "433",
            "extradata_size": 2,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "handler_name": "Core Media Audio",
                "vendor_id": "[0][0][0][0]"
            }
        }
    ],
    "format": {
        "filename": "rotated.mov",
        "nb_streams": 2,
        "nb_programs": 0,
        "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
        "format_long_name": "QuickTime / MOV",
        "start_time": "0.000000",
        "duration": "10.066667",
        "size": "19115175",
        "bit_rate": "15190867",
        "probe_score": 100,
        "tags": {
            "major_brand": "qt  ",
            "minor_version": "512",
            "compatible_brands": "qt  ",
            "encoder": "Lavf59.34.102"
        }
    }
}


  


  E:\>ffprobe -show_format -show_streams -print_format json -i cpu.mp4
ffprobe version 2023-08-14-git-c704901324-full_build-www.gyan.dev Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  ...
{
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'cpu.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.10.100
  Duration: 00:00:10.10, start: 0.000000, bitrate: 850 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 640x360, 713 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.23.100 libx264
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
    "streams": [
        {
            "index": 0,
            "codec_name": "h264",
            "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
            "profile": "High",
            "codec_type": "video",
            "codec_tag_string": "avc1",
            "codec_tag": "0x31637661",
            "width": 640,
            "height": 360,
            "coded_width": 640,
            "coded_height": 360,
            "closed_captions": 0,
            "film_grain": 0,
            "has_b_frames": 2,
            "pix_fmt": "yuv420p",
            "level": 30,
            "color_range": "tv",
            "color_space": "bt709",
            "color_transfer": "bt709",
            "color_primaries": "bt709",
            "chroma_location": "left",
            "field_order": "progressive",
            "refs": 1,
            "is_avc": "true",
            "nal_length_size": "4",
            "id": "0x1",
            "r_frame_rate": "30/1",
            "avg_frame_rate": "30/1",
            "time_base": "1/15360",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 155136,
            "duration": "10.100000",
            "bit_rate": "713885",
            "bits_per_raw_sample": "8",
            "nb_frames": "303",
            "extradata_size": 49,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "language": "und",
                "handler_name": "Core Media Video",
                "vendor_id": "[0][0][0][0]",
                "encoder": "Lavc60.23.100 libx264"
            }
        },
        {
            "index": 1,
            "codec_name": "aac",
            "codec_long_name": "AAC (Advanced Audio Coding)",
            "profile": "LC",
            "codec_type": "audio",
            "codec_tag_string": "mp4a",
            "codec_tag": "0x6134706d",
            "sample_fmt": "fltp",
            "sample_rate": "44100",
            "channels": 2,
            "channel_layout": "stereo",
            "bits_per_sample": 0,
            "initial_padding": 0,
            "id": "0x2",
            "r_frame_rate": "0/0",
            "avg_frame_rate": "0/0",
            "time_base": "1/44100",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 441265,
            "duration": "10.006009",
            "bit_rate": "127739",
            "nb_frames": "432",
            "extradata_size": 5,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "language": "und",
                "handler_name": "Core Media Audio",
                "vendor_id": "[0][0][0][0]"
            }
        }
    ],
    "format": {
        "filename": "cpu.mp4",
        "nb_streams": 2,
        "nb_programs": 0,
        "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
        "format_long_name": "QuickTime / MOV",
        "start_time": "0.000000",
        "duration": "10.100000",
        "size": "1073806",
        "bit_rate": "850539",
        "probe_score": 100,
        "tags": {
            "major_brand": "isom",
            "minor_version": "512",
            "compatible_brands": "isomiso2avc1mp41",
            "encoder": "Lavf60.10.100"
        }
    }
}


  


  E:\>ffprobe -show_format -show_streams -print_format json -i gpu.mp4
ffprobe version 2023-08-14-git-c704901324-full_build-www.gyan.dev Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
...
{
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'gpu.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.10.100
  Duration: 00:00:10.10, start: 0.000000, bitrate: 935 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 640x360 [SAR 1:1 DAR 16:9], 799 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.23.100 h264_nvenc
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
    "streams": [
        {
            "index": 0,
            "codec_name": "h264",
            "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
            "profile": "Main",
            "codec_type": "video",
            "codec_tag_string": "avc1",
            "codec_tag": "0x31637661",
            "width": 640,
            "height": 360,
            "coded_width": 640,
            "coded_height": 360,
            "closed_captions": 0,
            "film_grain": 0,
            "has_b_frames": 2,
            "sample_aspect_ratio": "1:1",
            "display_aspect_ratio": "16:9",
            "pix_fmt": "yuv420p",
            "level": 30,
            "color_range": "tv",
            "color_space": "bt709",
            "color_transfer": "bt709",
            "color_primaries": "bt709",
            "chroma_location": "left",
            "field_order": "progressive",
            "refs": 1,
            "is_avc": "true",
            "nal_length_size": "4",
            "id": "0x1",
            "r_frame_rate": "30/1",
            "avg_frame_rate": "30/1",
            "time_base": "1/15360",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 155136,
            "duration": "10.100000",
            "bit_rate": "799210",
            "bits_per_raw_sample": "8",
            "nb_frames": "303",
            "extradata_size": 53,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "language": "und",
                "handler_name": "Core Media Video",
                "vendor_id": "[0][0][0][0]",
                "encoder": "Lavc60.23.100 h264_nvenc"
            }
        },
        {
            "index": 1,
            "codec_name": "aac",
            "codec_long_name": "AAC (Advanced Audio Coding)",
            "profile": "LC",
            "codec_type": "audio",
            "codec_tag_string": "mp4a",
            "codec_tag": "0x6134706d",
            "sample_fmt": "fltp",
            "sample_rate": "44100",
            "channels": 2,
            "channel_layout": "stereo",
            "bits_per_sample": 0,
            "initial_padding": 0,
            "id": "0x2",
            "r_frame_rate": "0/0",
            "avg_frame_rate": "0/0",
            "time_base": "1/44100",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 441265,
            "duration": "10.006009",
            "bit_rate": "127739",
            "nb_frames": "432",
            "extradata_size": 5,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "language": "und",
                "handler_name": "Core Media Audio",
                "vendor_id": "[0][0][0][0]"
            }
        }
    ],
    "format": {
        "filename": "gpu.mp4",
        "nb_streams": 2,
        "nb_programs": 0,
        "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
        "format_long_name": "QuickTime / MOV",
        "start_time": "0.000000",
        "duration": "10.100000",
        "size": "1181528",
        "bit_rate": "935863",
        "probe_score": 100,
        "tags": {
            "major_brand": "isom",
            "minor_version": "512",
            "compatible_brands": "isomiso2avc1mp41",
            "encoder": "Lavf60.10.100"
        }
    }
}


  


  E:\>ffprobe -show_format -show_streams -print_format json -i gpu-complexfilter.mp4
ffprobe version 2023-08-14-git-c704901324-full_build-www.gyan.dev Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
...
{
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'gpu-complexfilter.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.10.100
  Duration: 00:00:10.10, start: 0.000000, bitrate: 934 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 640x360 [SAR 1:1 DAR 16:9], 798 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.23.100 h264_nvenc
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
    "streams": [
        {
            "index": 0,
            "codec_name": "h264",
            "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
            "profile": "Main",
            "codec_type": "video",
            "codec_tag_string": "avc1",
            "codec_tag": "0x31637661",
            "width": 640,
            "height": 360,
            "coded_width": 640,
            "coded_height": 360,
            "closed_captions": 0,
            "film_grain": 0,
            "has_b_frames": 2,
            "sample_aspect_ratio": "1:1",
            "display_aspect_ratio": "16:9",
            "pix_fmt": "yuv420p",
            "level": 30,
            "color_range": "tv",
            "color_space": "bt709",
            "color_transfer": "bt709",
            "color_primaries": "bt709",
            "chroma_location": "left",
            "field_order": "progressive",
            "refs": 1,
            "is_avc": "true",
            "nal_length_size": "4",
            "id": "0x1",
            "r_frame_rate": "30/1",
            "avg_frame_rate": "30/1",
            "time_base": "1/15360",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 155136,
            "duration": "10.100000",
            "bit_rate": "798059",
            "bits_per_raw_sample": "8",
            "nb_frames": "303",
            "extradata_size": 53,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "language": "und",
                "handler_name": "Core Media Video",
                "vendor_id": "[0][0][0][0]",
                "encoder": "Lavc60.23.100 h264_nvenc"
            }
        },
        {
            "index": 1,
            "codec_name": "aac",
            "codec_long_name": "AAC (Advanced Audio Coding)",
            "profile": "LC",
            "codec_type": "audio",
            "codec_tag_string": "mp4a",
            "codec_tag": "0x6134706d",
            "sample_fmt": "fltp",
            "sample_rate": "44100",
            "channels": 2,
            "channel_layout": "stereo",
            "bits_per_sample": 0,
            "initial_padding": 0,
            "id": "0x2",
            "r_frame_rate": "0/0",
            "avg_frame_rate": "0/0",
            "time_base": "1/44100",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 441265,
            "duration": "10.006009",
            "bit_rate": "127739",
            "nb_frames": "432",
            "extradata_size": 5,
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0,
                "captions": 0,
                "descriptions": 0,
                "metadata": 0,
                "dependent": 0,
                "still_image": 0
            },
            "tags": {
                "language": "und",
                "handler_name": "Core Media Audio",
                "vendor_id": "[0][0][0][0]"
            }
        }
    ],
    "format": {
        "filename": "gpu-complexfilter.mp4",
        "nb_streams": 2,
        "nb_programs": 0,
        "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
        "format_long_name": "QuickTime / MOV",
        "start_time": "0.000000",
        "duration": "10.100000",
        "size": "1180075",
        "bit_rate": "934712",
        "probe_score": 100,
        "tags": {
            "major_brand": "isom",
            "minor_version": "512",
            "compatible_brands": "isomiso2avc1mp41",
            "encoder": "Lavf60.10.100"
        }
    }
}


  


• How to overlay multiple landscape regions from a single input to a new portrait video ? FFmpeg

  27 août 2023, par 3V1LXD

  I have an electron program that selects multiple regions of a landscape video and lets you rearrange them in a portrait canvas. I'm having trouble building the proper ffmpeg command to create the video. I have this somewhat working. I can export 2 layers, but i can't export if i only have 1 layer or if i have 3 or more layers selected.

  


  2 regions of video selected

  


  layers [
  { top: 0, left: 658, width: 576, height: 1080 },
  { top: 262, left: 0, width: 576, height: 324 }
]
newPositions [
  { top: 0, left: 0, width: 576, height: 1080 },
  { top: 0, left: 0, width: 576, height: 324 }
]
filtergraph [0]crop=576:1080:658:0,scale=576:1080[v0];[0]crop=576:324:0:262,scale=576:324[v1];[v0][v1]overlay=0:0:0:0[out]

No Error export successful


  


  1 region selected

  


  layers [ { top: 0, left: 650, width: 576, height: 1080 } ]
newPositions [ { top: 0, left: 0, width: 576, height: 1080 } ]
filtergraph [0]crop=576:1080:650:0,scale=576:1080[v0];[v0]overlay=0:0[out]

FFmpeg error: [fc#0 @ 000001dd3b6db0c0] Cannot find a matching stream for unlabeled input pad overlay
Error initializing complex filters: Invalid argument


  


  3 regions of video selected

  


  layers [
  { top: 0, left: 641, width: 576, height: 1080 },
  { top: 250, left: 0, width: 576, height: 324 },
  { top: 756, left: 0, width: 576, height: 324 }
]
newPositions [
  { top: 0, left: 0, width: 576, height: 1080 },
  { top: 0, left: 0, width: 576, height: 324 },
  { top: 756, left: 0, width: 576, height: 324 }
]
filtergraph [0]crop=576:1080:641:0,scale=576:1080[v0];[0]crop=576:324:0:250,scale=576:324[v1];[0]crop=576:324:0:756,scale=576:324[v2];[v0][v1][v2]overlay=0:0:0:0:0:756[out]

FFmpeg error: [AVFilterGraph @ 0000018faf2189c0] More input link labels specified for filter 'overlay' than it has inputs: 3 > 2
[AVFilterGraph @ 0000018faf2189c0] Error linking filters

FFmpeg error: Failed to set value '[0]crop=576:1080:698:0,scale=576:1080[v0];[0]crop=576:324:0:264,scale=576:324[v1];[0]crop=576:324:0:756,scale=576:324[v2];[v0][v1][v2]overlay=0:0:0:0:0:0[out]' for option 'filter_complex': Invalid argument
Error parsing global options: Invalid argument


  


  I can't figure out how to construct the proper overlay command. Here is the js code i'm using from my electron app.

  


  ipcMain.handle('export-video', async (_event, args) => {
  const { videoFile, outputName, layers, newPositions } = args;
  const ffmpegPath = path.join(__dirname, 'bin', 'ffmpeg');
  const outputDir = checkOutputDir();
  
  // use same video for each layer as input
  // crop, scale, and position each layer
  // overlay each layer on top of each other

  // export video
  console.log('layers', layers);
  console.log('newPositions', newPositions);

  let filtergraph = '';

  for (let i = 0; i < layers.length; i++) {
    const { top, left, width, height } = layers[i];
    const { width: newWidth, height: newHeight } = newPositions[i];
    const filter = `[0]crop=${width}:${height}:${left}:${top},scale=${newWidth}:${newHeight}[v${i}];`;
    filtergraph += filter;
  }

  for (let i = 0; i < layers.length; i++) {
    const filter = `[v${i}]`;
    filtergraph += filter;
  }

  filtergraph += `overlay=`;
  for (let i = 0; i < layers.length; i++) {
    const { top: newTop, left: newLeft } = newPositions[i];
    const overlay = `${newLeft}:${newTop}:`;
    filtergraph += overlay;
  }

  filtergraph = filtergraph.slice(0, -1); // remove last comma
  filtergraph += `[out]`;
  
  console.log('filtergraph', filtergraph);

  const ffmpeg = spawn(ffmpegPath, [
    '-i', videoFile,
    '-filter_complex', filtergraph,
    '-map', '[out]',
    '-c:v', 'libx264',
    '-preset', 'ultrafast',
    '-crf', '18',
    '-y',
    path.join(outputDir, `${outputName}`)
  ]);  

  ffmpeg.stdout.on('data', (data) => {
    console.log(`FFmpeg output: ${data}`);
  });

  ffmpeg.stderr.on('data', (data) => {
    console.error(`FFmpeg error: ${data}`);
  });

  ffmpeg.on('close', (code) => {
    console.log(`FFmpeg process exited with code ${code}`);
    // event.reply('ffmpeg-export-done'); // Notify the renderer process
  });
});


  


  Any advice might be helpful, The docs are confusing, Thanks.

  


  Edit
I'm getting closer with this
Output :

  


  layers [
  { top: 0, left: 677, width: 576, height: 1080 },
  { top: 240, left: 0, width: 576, height: 324 }
]
newPositions [
  { top: 0, left: 0, width: 576, height: 1080 },
  { top: 0, left: 0, width: 576, height: 324 }
]
filtergraph [0]crop=576:1080:677:0,scale=576:1080[v0];[0]crop=576:324:0:240,scale=576:324[v1];[0][v0]overlay=0:0[o0];[o0][v1]overlay=0:0[o1]


  


  ipcMain.handle('export-video', async (_event, args) => {
  const { videoFile, outputName, layers, newPositions } = args;
  const ffmpegPath = path.join(__dirname, 'bin', 'ffmpeg');
  const outputDir = checkOutputDir();
  
  // use same video for each layer as input
  // crop, scale, and position each layer
  // overlay each layer on top of each other

  // export video
  console.log('layers', layers);
  console.log('newPositions', newPositions);

  let filtergraph = '';

  for (let i = 0; i < layers.length; i++) {
    const { top, left, width, height } = layers[i];
    const { width: newWidth, height: newHeight } = newPositions[i];
    const filter = `[0]crop=${width}:${height}:${left}:${top},scale=${newWidth}:${newHeight}[v${i}];`;
    filtergraph += filter;
  }

  for (let i = 0; i < layers.length; i++) {
    if (i === 0) {
      filtergraph += `[0][v${i}]overlay=`;
    } else {
      filtergraph += `[o${i-1}][v${i}]overlay=`;
    }
    const { top: newTop, left: newLeft } = newPositions[i];
    let overlay = '';
    if (i !== layers.length - 1) {
      overlay = `${newLeft}:${newTop}[o${i}];`;
    } else {
      overlay = `${newLeft}:${newTop};`;
    }
    filtergraph += overlay;
  }

  filtergraph = filtergraph.slice(0, -1); // remove last comma
  filtergraph += `[o${layers.length-1}]`;
  
  console.log('filtergraph', filtergraph);

  const ffmpeg = spawn(ffmpegPath, [
    '-i', videoFile,
    '-filter_complex', filtergraph,
    '-map', `[o${layers.length-1}]`,
    '-c:v', 'libx264',
    '-preset', 'ultrafast',
    '-crf', '18',
    '-y',
    path.join(outputDir, `${outputName}`)
  ]);  

  ffmpeg.stdout.on('data', (data) => {
    console.log(`FFmpeg output: ${data}`);
  });

  ffmpeg.stderr.on('data', (data) => {
    console.error(`FFmpeg error: ${data}`);
  });

  ffmpeg.on('close', (code) => {
    console.log(`FFmpeg process exited with code ${code}`);
    // event.reply('ffmpeg-export-done'); // Notify the renderer process
  });
});


  


  The problem I'm having now is that its overlaying the regions over the original input and keeping the landscape dimensions instead of making a portrait video.

  


• Slideshow from .jpg

  13 octobre 2023, par DJArty

  Have an command to make slideshow from *.jpg files :

  


  ffmpeg -r 1/3 -f concat -safe 0 -i <(ls -v *.jpg | sed "s|^|file '$PWD/|") -vf "scale='min(1920,iw)':min'(1080,ih)':force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2" -c:v libx264 -pix_fmt yuv420p -r 60 -preset slow -crf 14 slideshow.mp4

  


  Need good effects from one jpg to another like morphing or just fade in fade out, not just "one picture 3 sec, next picture 3 sec, etc."

  


  need nicey slideshow