Recherche avancée

Sur d’autres sites (4707)

• How to decode and display real-time H264 stream using ffmpeg in Python ?

  25 mars 2022, par yiiiiiiiran

  I would like to port the live stream to ffmpeg and display it in real time using Python.

  


  Anyone knows how to port the stream to PIPE ? And in the mean time to display it after decoding ?

  


  I managed to get real-time stream from my Raspberry Pi3 to Windows PC, using RS232 connection with Baud Rate 2M.

  


  The format of the stream is in H264. The data package I get for each frame is in .
In order for the program to know when does each package ends, I've add

  


  bytes([0xcc,0xdd,0xee,0xff])


  


  to the end of package. So that my serial port will read for a package until it sees those bytes.

  


  Lets assume the stream WIDTH, HEIGHT, NUM_FRAMES, FPS = 320, 240, 90, 30

  


  I have the command for decode the h264 stream :

  


  cmd = ["C:/XXXXXX/ffmpeg.exe",
    "-probesize", "32",
    "-flags", "low_delay",
    "-f", "h264",
    "-i", "pipe:",
    "-f", "rawvideo", "-pix_fmt", "rgb24", "-s", "384x216",
    "pipe:"]

decode_process = sp.Popen(cmd, stdin=sp.PIPE, stdout=sp.PIPE)


  


  The stream package I got is

  


  while datetime.now() &lt; end_time:&#xA;    pkg = ser.read_until(expected=bytes([0xcc,0xdd,0xee,0xff])) #output <class>&#xA;    frame_len = len(pkg)-4&#xA;    frame_inBytes = pkg[0:frame_len]&#xA;    decode_process.stdin.write(frame_inBytes)&#xA;</class>

  


  I want to write the real time stream to PIPE however it shows error :

  


  [h264 @ 0000017322a3e980] missing picture in access unit with size 48
[h264 @ 0000017322a3e980] no frame!
[h264 @ 0000017322a2d240] Stream #0: not enough frames to estimate rate; consider increasing probesize
[h264 @ 0000017322a2d240] Could not find codec parameters for stream 0 (Video: h264, none): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (32) options        
Input #0, h264, from 'pipe:':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264, none, 25 tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
[h264 @ 0000017322a3f180] no frame!
Error while decoding stream #0:0: Invalid data found when processing input
Cannot determine format of input stream 0:0 after EOF
Error marking filters as finished
Conversion failed!


  


• How do I encode a video stream to multiple output formats in parallel with ffmpeg ?

  21 juillet 2022, par rgov

  I would like to use one FFmpeg process to receive video input and then pass that video to multiple separate encoder processes in order to efficiently make use of all available CPU cores.

  


  The FFmpeg wiki article on Creating multiple outputs has this note from @rogerdpack :

  


  


  Outputting and re encoding multiple times in the same FFmpeg process will typically slow down to the "slowest encoder" in your list. Some encoders (like libx264) perform their encoding "threaded and in the background" so they will effectively allow for parallel encodings, however audio encoding may be serial and become the bottleneck, etc. It seems that if you do have any encodings that are serial, it will be treated as "real serial" by FFmpeg and thus your FFmpeg may not use all available cores. One work around to this is to use multiple ffmpeg instances running in parallel, or possible piping from one ffmpeg to another to "do the second encoding" etc. Or if you can avoid the limiting encoder (ex : using a different faster one [ex : raw format] or just doing a raw stream copy) that might help.

  


  


  The article has an example of using a tee pseudo-muxer, but it uses "a single instance of FFmpeg. The example of piping from one instance of FFmpeg to another only allows one encoder process.

  


  A 10-year-old version of the same article mentions using the tee process but it was subsequently deleted :

  


  


  Another option is to output from FFmpeg to "-" then to pipe that to a "tee" command, which can send it to multiple other processes, for instance 2 different other ffmpeg processes for encoding (this may save time, as if you do different encodings, and do the encoding in 2 different simultaneous processes, it might do encoding more in parallel than elsewise). Un benchmarked, however.

  


  


  Along the same lines : Some of the example commands use the mpegts to encapsulate frames before passing them between processes. Is there any constraint that this applies to the codecs or types of metadata that can be sent to downstream processes ?

  


• Using ffmpeg on Ubuntu, how can the audio and video from an audio-video USB capture device be recorded ?

  11 avril 2022, par BlandCorporation

  I have a USB audio-video capture device, something used to digitize video cassettes. I want to record both the video and audio from the device to a video file that has dimensions 720x576 and video codec H.264 and good audio quality.

  


  I am able to record video from the device using ffmpeg and I am able to see video from the device using MPlayer. I am able also to see that audio is being delivered from the device to the computer by looking at Input tab of the Sound Preferences window or by recording the audio using Audacity, however the audio gets delivered from the device apparently only when the video is being accessed using ffmpeg or MPlayer.

  


  I have tried to get ffmpeg to record the audio and I have tried to get MPlayer to play the audio and my efforts have not been successful.

  


  The device is "Pinnacle Dazzle DVC 90/100/101" (as returned by v4l2-ctl --list-devices). The sound cards listing shows it as "DVC100" :

  


  $ cat /proc/asound/cards 
 0 [PCH            ]: HDA-Intel - HDA Intel PCH
                      HDA Intel PCH at 0x601d118000 irq 171
 1 [DVC100         ]: USB-Audio - DVC100
                      Pinnacle Systems GmbH DVC100 at usb-0000:00:14.0-4, high speed
29 [ThinkPadEC     ]: ThinkPad EC - ThinkPad Console Audio Control
                      ThinkPad Console Audio Control at EC reg 0x30, fw N2LHT33W


  


  The PulseAudio listing for the device is as follows :

  


  $ pactl list cards short
0   alsa_card.pci-0000_00_1f.3  module-alsa-card.c
14  alsa_card.usb-Pinnacle_Systems_GmbH_DVC100-01   module-alsa-card.c


  


  The following ffmpeg command successfully records video, but records severely distorted, broken and out-of-sync audio :

  


  ffmpeg -y -f rawvideo -f alsa -thread_queue_size 2048 -ar 48000 -i hw:0 \
    -c:a aac -video_size 720x576 -pixel_format uyvy422 -i /dev/video2 out.mp4


  


  The following MPlayer command successfully displays the video but does not play the audio :

  


  mplayer -tv driver=v4l2:norm=PAL:device=/dev/video2:width=720:height=576 \
    -ao alsa:device=hw=1.0 -vf pp=lb tv://


  


  Now, when the above MPlayer command is running (not the ffmpeg command) and displaying the input video in a window, Audacity can be opened and set recording audio, and it records the audio from the device clearly and in good quality. While Audacity is doing this, the input device is listed in pavucontrol as "Dazzle DVC Audio Device Analogue Stereo". Equivalently, arecord can be used also to record the audio using the following command (with output shown) :

  


  $ arecord -vv -D plughw:DVC100 -fdat out.wav
Recording WAVE 'out.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Stereo
Plug PCM: Hardware PCM card 1 'DVC100' device 0 subdevice 0
Its setup is:
  stream       : CAPTURE
  access       : RW_INTERLEAVED
  format       : S16_LE
  subformat    : STD
  channels     : 2
  rate         : 48000
  exact rate   : 48000 (48000/1)
  msbits       : 16
  buffer_size  : 24000
  period_size  : 6000
  period_time  : 125000
  tstamp_mode  : NONE
  tstamp_type  : MONOTONIC
  period_step  : 1
  avail_min    : 6000
  period_event : 0
  start_threshold  : 1
  stop_threshold   : 24000
  silence_threshold: 0
  silence_size : 0
  boundary     : 6755399441055744000
  appl_ptr     : 0
  hw_ptr       : 0


  


  Looking at the output of arecord -L, I tried a variety of audio device input names with ffmpeg and none of them seemed to work. So, for example, I tried commands like the following :

  


  ffmpeg -y -f rawvideo -f alsa -i plughw:DVC100 \
    -video_size 720x576 -pixel_format uyvy422 -i /dev/video2 out.mp4


  


  And tried the following audio device names :

  


  plughw:DVC100
plughw:CARD=DVC100,DEV=0
hw:CARD=DVC100,DEV=0
plughw:CARD=DVC100
sysdefault:CARD=DVC100
iec958:CARD=DVC100,DEV=0
dsnoop:CARD=DVC100,DEV=0


  


  So, how might I get ffmpeg to record the audio successfully to the video file ? Is there some alternative approach to this problem ?

  


  ---

  


  EDIT : The relevant output from the command pactl list sources is as follows :

  


  Source #20
    State: SUSPENDED
    Name: alsa_input.usb-Pinnacle_Systems_GmbH_DVC100-01.analog-stereo
    Description: Dazzle DVC100 Audio Device Analogue Stereo
    Driver: module-alsa-card.c
    Sample Specification: s16le 2ch 48000Hz
    Channel Map: front-left,front-right
    Owner Module: 45
    Mute: no
    Volume: front-left: 99957 / 153% / 11.00 dB,   front-right: 99957 / 153% / 11.00 dB
            balance 0.00
    Base Volume: 35466 /  54% / -16.00 dB
    Monitor of Sink: n/a
    Latency: 0 usec, configured 0 usec
    Flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY 
    Properties:
        alsa.resolution_bits = "16"
        device.api = "alsa"
        device.class = "sound"
        alsa.class = "generic"
        alsa.subclass = "generic-mix"
        alsa.name = "USB Audio"
        alsa.id = "USB Audio"
        alsa.subdevice = "0"
        alsa.subdevice_name = "subdevice #0"
        alsa.device = "0"
        alsa.card = "1"
        alsa.card_name = "DVC100"
        alsa.long_card_name = "Pinnacle Systems GmbH DVC100 at usb-0000:00:14.0-4, high speed"
        alsa.driver_name = "snd_usb_audio"
        device.bus_path = "pci-0000:00:14.0-usb-0:4:1.1"
        sysfs.path = "/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:1.1/sound/card1"
        udev.id = "usb-Pinnacle_Systems_GmbH_DVC100-01"
        device.bus = "usb"
        device.vendor.id = "2304"
        device.vendor.name = "Pinnacle Systems, Inc."
        device.product.id = "021a"
        device.product.name = "Dazzle DVC100 Audio Device"
        device.serial = "Pinnacle_Systems_GmbH_DVC100"
        device.string = "front:1"
        device.buffering.buffer_size = "352800"
        device.buffering.fragment_size = "176400"
        device.access_mode = "mmap+timer"
        device.profile.name = "analog-stereo"
        device.profile.description = "Analogue Stereo"
        device.description = "Dazzle DVC100 Audio Device Analogue Stereo"
        alsa.mixer_name = "USB Mixer"
        alsa.components = "USB2304:021a"
        module-udev-detect.discovered = "1"
        device.icon_name = "audio-card-usb"
    Ports:
        analog-input-linein: Line In (priority: 8100)
    Active Port: analog-input-linein
    Formats:
        pcm


  


  I tested the name from this with ffmpeg (version 4.3.1, compiled with -enable-libpulse) in the following way :

  


  ffmpeg -y -f video4linux2 -f pulse \
    -i alsa_input.usb-Pinnacle_Systems_GmbH_DVC100-01.analog-stereo \
    -video_size 720x576 -pixel_format uyvy422 -i /dev/video2 out.mp4


  


  Unfortunately this hasn't worked.