Recherche avancée

Sur d’autres sites (3139)

• 2011 In Open Source Multimedia

  5 janvier 2012, par Multimedia Mike — Open Source Multimedia

  Sometimes I think that the pace of multimedia technology is slowing down. Obviously, I’m not paying close enough attention. I thought I would do a little 2011 year-end review of what happened in the world of open source multimedia, mainly for my own benefit. Let me know in the comments what I missed.

  The Split
  The biggest deal in open source multimedia was the matter of the project split. Where once stood one project (FFmpeg) there now stands two (also Libav). Where do things stand with the projects now ? Still very separate but similar. Both projects obsessively monitor each other’s git commits and prodigiously poach each other’s work, both projects being LGPL and all. Most features that land in one code base end up in the other. Thus, I refer to FFmpeg and Libav collectively as “the projects”.

  Some philosophical reasons for the split included project stagnation and development process friction. Curiously, these problems are fond memories now and the spirit of competition has pushed development forward at a blinding pace.

  People inside the project have strong opinions about the split ; that’s understandable. People outside the project have strong opinions about the split ; that’s somewhat less understandable, but whatever. After 5 years of working for Adobe on the Flash Player (a.k.a. the most hated software in all existence if internet nerds are to be believed on the matter), I’m so over internet nerd drama.

  For my part, I just try to maintain some appearance of neutrality since I manage some shared resources for the open source multimedia community (like the wiki and samples repo) and am trying to keep them from fracturing as well.

  Apple and Open Source
  It was big news that Apple magnanimously open sourced their lossless audio codec. That sets a great example and precedent.

  New Features
  I mined the 'git log' of the projects in order to pick out some features that were added during 2011.

  First off, Apple’s ProRes video codec was reverse engineered and incorporated into the multimedia libraries. And for some weird reason, this is an item that made the rounds in the geek press. I’m not entirely sure why, but it may have something to do with inter-project conflict. Anyway, here is the decoder in action, playing a video of some wild swine, one of the few samples we have :

  
  
  

  Other new video codecs included a reverse engineered Indeo 4 decoder. Gotta catch ‘em all ! That completes our collection of Indeo codecs. But that wasn’t enough– this year, we got a completely revised Indeo 3 decoder (the previous one, while functional, exhibited a lot of code artifacts betraying a direct ASM ->C translation). Oh, and many thanks to Kostya for this gem :

  
  
  

  That’s the new Origin Xan decoder (best known for Wing Commander IV cinematics) in action, something I first started reverse engineering back in 2002. Thanks to Kostya for picking up my slack yet again.

  Continuing with the codec section, there is a decoder for Adobe Flash Screen Video 2 — big congrats on this ! One of my jobs at Adobe was documenting this format to the outside world and I was afraid I could never quite make it clear enough to build a complete re-implementation. But the team came through.

  Let’s see, there are decoders for VBLE video, Ut Video, Windows Media Image (WMVP/WMP2), Bink audio version ‘b’, H.264 4:2:2 intra frames, and MxPEG video. There is a DPX image encoder, a Cirrus Logic AccuPak video encoder, and a v410 codec.

  How about some more game stuff ? The projects saw — at long last — an SMJPEG demuxer. This will finally allow usage and testing of the SMJPEG IMA ADPCM audio decoder I added about a decade ago. Funny story behind that– I was porting all of my decoders from xine which included the SMJPEG ADPCM. I just never quite got around to writing a corresponding demuxer. Thanks to Paul Mahol for taking care of that.

  Here’s a DFA playback system for a 1995 DOS CD-ROM title called Chronomaster. No format is too obscure, nor its encoded contents too cheesy :

  
  
  

  There’s now a demuxer for a format called XMV that was (is ?) prevalent on Xbox titles. Now the projects can handle FMV files from many Xbox games, such as Thrillville.

  
  
  

  The projects also gained the ability to play BMV files. I think this surfing wizard comes from Discworld II. It’s non-computer-generated animation at a strange resolution.

  
  
  

  More demuxers : xWMA, PlayStation Portable PMP format, and CRI ADX format ; muxer for OpenMG audio and LATM muxer/demuxer.

  One more thing : an AVX-optimized fast Fourier transform (FFT). If you have a machine that supports AVX, there’s no way you’ll even notice the speed increase of a few measly FFT calls for audio coding/decoding, but that’s hardly the point. The projects always use everything on offer for any CPU.

  Please make me aware of features that I missed in the list !

  Continuous Testing
  As a result of the split, each project has its own FATE server, one for FFmpeg and one for Libav. As of the new year, FFmpeg has just over 1000 tests while Libav had 965. This is one area where I’m obviously ecstatic to see competition. Some ad-hoc measurements on my part indicate that the total code coverage via the FATEs has not appreciably increased. But that’s a total percentage. Both the test count and the code count have been steadily rising.

  Google Summer of Code and Google Code-In
  Once again, the projects were allowed to participate in the Google Summer of Code as well as Google Code-In. I confess that I didn’t keep up with these too carefully (and Code-In is still in progress as of this writing). I do know that the project split occurred after FFmpeg had already been accepted for GSoC season 2011 and the admins were gracious enough to allow FFmpeg and Libav to allow both projects to participate in the same slot as long as they could both be mature about it.

  Happy New Year
  Let’s see what we can accomplish in 2012.

• Transcoding with the GCP Transcoder API results in a time gap

  3 mars 2021, par banyan

  I've been trying GCP's Transcoder API and having trouble with the time randomly becoming shorter than specified in certain cases.

  


  The specific cases are as follows :

  


  

  1. Specifying startTimeOffset and endTimeOffset (cut off 2 seconds before and after the video)

  


  2. fMP4 is used as container

  


  3. input video is mp4 with screen recording on iPad Pro

  


  


  For example, if I don't specify startTimeOffset and endTimeOffset, the time will not be shortened. Also, there is no problem when MPEG2-TS is specified for container. There may be a problem with the video itself, but I haven't found a clue how to set it up.

  


  I'm not sure whether this is a problem with the Transcoder API or with me.

  


  The test input video : https://gofile.io/d/DUT9rr

  


  ❯ ffprobe input.mp4
ffprobe version 4.3.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.28)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3.1_8 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: isommp41mp42
    creation_time   : 2021-02-26T15:08:58.000000Z
  Duration: 00:02:51.15, start: 0.000000, bitrate: 551 kb/s
    Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 246 kb/s (default)
    Metadata:
      creation_time   : 2021-02-26T15:08:58.000000Z
      handler_name    : Core Media Audio
    Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709/bt709/iec61966-2-1), 1920x1342, 302 kb/s, 12.39 fps, 120 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      rotate          : 180
      creation_time   : 2021-02-26T15:08:58.000000Z
      handler_name    : Core Media Video
    Side data:
      displaymatrix: rotation of -180.00 degrees


  


  #!/bin/bash -eu

cat > request.json << EOF
{
  "config": {
    "inputs": [
      {
        key: "input0"
      }
    ],
    "editList": [
      {
        "key": "atom0",
        "inputs": [
          "input0"
        ],
        "startTimeOffset": "2s",
        "endTimeOffset": "169s",
      },
    ],
    "elementaryStreams": [
      {
        "videoStream": {
          "codec": "h265",
          "heightPixels": 480,
          "bitrateBps": 1200000,
          "rateControlMode": "vbr",
          "enableTwoPass": true,
          "frameRate": 30,
          "crfLevel": 31,
          "gopDuration": "3.0s",
        },
        "key": "h265-stream0"
      },
      {
        "videoStream": {
          "codec": "h265",
          "heightPixels": 720,
          "bitrateBps": 1550000,
          "rateControlMode": "vbr",
          "enableTwoPass": true,
          "frameRate": 30,
          "crfLevel": 31,
          "gopDuration": "3.0s",
        },
        "key": "h265-stream1"
      },
      {
        "videoStream": {
          "codec": "h265",
          "heightPixels": 1080,
          "bitrateBps": 2600000,
          "rateControlMode": "vbr",
          "enableTwoPass": true,
          "frameRate": 30,
          "crfLevel": 31,
          "gopDuration": "3.0s",
        },
        "key": "h265-stream2"
      },
      {
        "audioStream": {
          "codec": "aac",
          "bitrateBps": 64000,
          "channelCount": 2,
          "channelLayout": [
            "fl",
            "fr"
          ],
          "sampleRateHertz": 48000
        },
        "key": "audio-stream0"
      },
    ],
    "muxStreams": [
      {
        "key": "media-sd",
        "fileName": "media-sd.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "h265-stream0",
        ],
        "segmentSettings": {
          "individualSegments": true
        },
      },
      {
        "key": "media-hd",
        "fileName": "media-hd.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "h265-stream1",
        ],
        "segmentSettings": {
          "individualSegments": true
        },
      },
      {
        "key": "media-fhd",
        "fileName": "media-fhd.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "h265-stream2",
        ],
        "segmentSettings": {
          "individualSegments": true
        },
      },
      {
        "key": "audio-only",
        "fileName": "audio-only.m4s",
        "container": "fmp4",
        "elementaryStreams": [
          "audio-stream0"
        ],
        "segmentSettings": {
          "individualSegments": true
        },
      },
    ],
    "manifests": [
      {
        "fileName": "manifest-h265.mpd",
        "type": "DASH",
        "muxStreams": [
          "media-sd",
          "media-hd",
          "media-fhd",
          "audio-only",
        ]
      },
    ]
  }
}
EOF

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://transcoder.googleapis.com/v1beta1/projects/MY_PROJECT/locations/asia-east1/jobTemplates?jobTemplateId=test-template


  


  #!/bin/bash -eu

cat > request.json << EOF
{
  "inputUri": "gs://my-bucket/input.mp4",
  "outputUri": "gs://my-bucket/output/",
  "templateId": "test-template"
}
EOF

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://transcoder.googleapis.com/v1beta1/projects/MY_PROJECT/locations/asia-east1/jobs


  


  The following is a ffprobe of the resulting manifest file, which is 2 seconds shorter than specified. (expected : 00:02:47.00, actual : 00:02:45.00) In this case, it's a 2-second gap, but it can be 10 seconds or 30 seconds, and it varies from video to video.

  


  ❯ ffprobe manifest-h265.mpd
ffprobe version 4.3.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.28)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3.1_8 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, dash, from 'manifest-h265.mpd':
  Duration: 00:02:45.00, start: 0.000000, bitrate: 0 kb/s
  Program 0
    Stream #0:0: Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709/unknown/unknown), 686x480, 112 kb/s, 30 fps, 120 tbr, 10k tbn, 30 tbc
    Metadata:
      variant_bitrate : 113679
      id              : 113679
    Stream #0:1: Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709/unknown/unknown), 1030x720, 205 kb/s, 30 fps, 120 tbr, 10k tbn, 30 tbc
    Metadata:
      variant_bitrate : 189219
      id              : 189219
    Stream #0:2: Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709/unknown/unknown), 1544x1080, 384 kb/s, 30 fps, 120 tbr, 10k tbn, 30 tbc
    Metadata:
      variant_bitrate : 358043
      id              : 358043
    Stream #0:3: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 65 kb/s
    Metadata:
      variant_bitrate : 70245
      id              : 70245


  


  The following is the time as specified for h264 + MPEG2-TS + Apple HLS.

  


  ❯ ffprobe manifest-h264.m3u8 | pbcopy
ffprobe version 4.3.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.28)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3.1_8 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
[hls @ 0x7fe23100f200] Opening 'h264-sd-ts.m3u8' for reading
[hls @ 0x7fe23100f200] Skip ('#EXT-X-VERSION:4')
[hls @ 0x7fe23100f200] Opening 'h264-hd-ts.m3u8' for reading
[hls @ 0x7fe23100f200] Skip ('#EXT-X-VERSION:4')
[hls @ 0x7fe23100f200] Opening 'h264-fhd-ts.m3u8' for reading
[hls @ 0x7fe23100f200] Skip ('#EXT-X-VERSION:4')
[hls @ 0x7fe23100f200] Opening 'h264-sd0000000000.ts' for reading
[hls @ 0x7fe23100f200] Opening 'h264-hd0000000000.ts' for reading
[hls @ 0x7fe23100f200] Opening 'h264-fhd0000000000.ts' for reading
Input #0, hls, from 'manifest-h264.m3u8':
  Duration: 00:02:47.00, start: 0.000000, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 511576
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 686x480, 120 tbr, 90k tbn, 2000k tbc
    Metadata:
      variant_bitrate : 511576
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 511576
  Program 1
    Metadata:
      variant_bitrate : 793711
    Stream #0:2: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1030x720, 120 tbr, 90k tbn, 2000k tbc
    Metadata:
      variant_bitrate : 793711
    Stream #0:3: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 793711
  Program 2
    Metadata:
      variant_bitrate : 1305288
    Stream #0:4: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1544x1080, 120 tbr, 90k tbn, 2000k tbc
    Metadata:
      variant_bitrate : 1305288
    Stream #0:5: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 1305288


  


• ffmpeg Real-time buffer issue [rtbufsize parameter] [migrated]

  20 décembre 2016, par iNCEPTiON_

  Im trying to Stream the Audio and Video over the Network using this command

    -f gdigrab -framerate 60 -video_size 1920x1080 -i desktop -f dshow -i audio=""virtual-audio-capturer"" -vcodec libx264 -preset ultrafast -tune zerolatency-pix_fmt yuv422p -f mpegts udp://{IpAddress}:1234 

  but after 5 minutes of perfect quality streaming im getting this Error Message and the Stream suddenly stopps streaming.

  

  What am i doing wrong here ?