Recherche avancée

Sur d’autres sites (7519)

• ffmpeg produces video with misaligned frames

  28 mars 2023, par massivemoisture

  I have a WPF app that uses DeckLinkAPI to stream video from Blackmagic capture card (similar to their 'CapturePreviewCSharp' sample project, which can be found here : https://www.blackmagicdesign.com/developer/)

  


  'VideoInputFrameArrived' method is called when a video input frame arrives :

  


  void IDeckLinkInputCallback.VideoInputFrameArrived(IDeckLinkVideoInputFrame videoFrame, IDeckLinkAudioInputPacket audioPacket)
{
    if (videoFrame != null)
    {
        CurrentFrame = videoFrame;
        VideoFrameArrived?.Invoke(this, new DeckLinkDeviceInputVideoFrameEventArgs(videoFrame));
        GC.AddMemoryPressure(videoFrame.GetRowBytes() * videoFrame.GetHeight());
    }
}


  


  I want to record the stream to a file. So I open a ffmpeg process with these arguments :

  


  // My source is 1080p
string args = $".\\ffmpeg.exe -y -f rawvideo -pix_fmt bgra -s 1920x1080 -r 30 -i - -c:v libx264 -preset ultrafast -pix_fmt yuv420p -crf 23 \"{outputFilePath}\"";


  


  And I write to ffmpeg's pipe :

  


  // ffmpegInputStream = FfmpegProcess.StandardInput.BaseStream
public void WriteVideoFrameToProcessStream(Stream ffmpegInputStream)
{
    while (true)
    {
        if (isRecording == true)
        {
            if (ffmpegInputStream != null)
            {
                // Convert the frame to BGRA32 format and convert the video frame to a byte array
                byte[] frameData = ConvertVideoFrameToByteArray(CurrentFrame);
                // Write the frame data to the ffmpeg input stream
                ffmpegInputStream.Write(frameData, 0, frameData.Length);
            }
        }
    }
}


  


  But the video file output has misaligned frames. The frame looks like it has been shifted to the left. 1/5 of the frame on the right is supposed to be on the left.


  


  I tried saving the frameData byte array in WriteVideoFrameToProcessStream to an image file and it looks fine. What could be wrong here ?

  


• WebRTC Stream Freezes When Picture Complexity Increases

  12 octobre 2021, par user1259576

  I am developing an application that uses WebRTC to display a live video stream being captured from a V4L2 source. The stream originates from a Linux box that has a DVI-USB capture card, is encoded to H264 by ffmpeg and sent to RTP, received by a Janus WebRTC server which is accessed by the web interface.

  


  Here is my current ffmpeg command - pretty simple :
ffmpeg -f v4l2 -i /dev/video0 -vf "transpose=1,scale=768:1024" -vcodec libx264 -profile:v baseline -pix_fmt yuv420p -f rtp rtp ://10.116.80.86:8004

  


  I can't go into details, but the DVI source generates a portrait 768x1024 image that initially is a simple image where the only movement is a small clock near the center that increments every second. At this stage, everything appears to work great. The image is high-quality and continuous/smooth in the browser.

  


  Once I interact with the DVI source, a more complex image is generated, with some text/lines in the upper half. Still not very complex - only 2 colors involved and some basic 1px line shapes, and only the little clock is moving. At this point, the video starts to freeze frequently, and only updates once in a while for a few seconds. Bandwidth should not be an issue here, and the bitrate appears to stay high. However, many fewer frames are decoded.

  


  I have also tried scaling the video down to 480x640 from 768x1024 and with that change the issue does not occur. However, I really need the full resolution and, again, there should not be a bandwidth issue here.

  


  I have also tried capturing the output of ffmpeg to a file rather than streaming to RTP and in the file everything is good.

  


  Here is a screenshot of the WebRTC internals (in Edge) for this stream. You can clearly see when the video image changes from the simple clock to including more shapes & text (nothing is changed here other than the image from the DVI source) :

  


  

  


  In Firefox, the video just freezes whenever frames are not decoded. In Edge, the video goes black after a moment with no frames decoded.

  


  Any ideas as to what might be causing this ?

  


• how to record camera to file while encoding stream to v4l2-loopback device ?

  3 février 2021, par Jonatas

  So i have this logitech c920 camera 1920x1080 h264 capable and i would like to record the camera to file while at the same time copying/encoding the stream to a loopback device so the camera can be still used by other apps.
here is the code i got so far :

  


  ffmpeg -report -f alsa -i hw:CARD=C920,DEV=0 -r 1500 -s 1920x1080 -f v4l2 -vcodec h264 \
-i /dev/video1 -copyinkf -vcodec copy /home/jonatas/Videos/2021-02-01185658.mp4 \ 
-f v4l2 /dev/video0


  


  error :

  


  Unknown V4L2 pixel format equivalent for yuvj420p
Could not write header for output file #1 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 1:0 -- 


  


  some info on /dev/video0(loopbackdevice) :

  


  v4l2-ctl -d /dev/video0

    Device Caps      : 0x05208003
        Video Capture
        Video Output
        Video Memory-to-Memory
        Read/Write
        Streaming
        Extended Pix Format
Priority: 0
Video input : 0 (loopback: ok)
Video output: 0 (loopback in)
Format Video Output:
    Width/Height      : 1280/720
    Pixel Format      : 'YUYV' (YUYV 4:2:2)
    Field             : None
    Bytes per Line    : 2560
    Size Image        : 1843200
    Colorspace        : sRGB
    Transfer Function : Default (maps to sRGB)
    YCbCr/HSV Encoding: Default (maps to ITU-R 601)
    Quantization      : Default (maps to Limited Range)
    Flags             : 
Streaming Parameters Video Capture:
    Frames per second: 30.000 (30/1)
    Read buffers     : 8
Streaming Parameters Video Output:
    Frames per second: 30.000 (30/1)
    Write buffers    : 8

User Controls

                    keep_format 0x0098f900 (bool)   : default=0 value=0
              sustain_framerate 0x0098f901 (bool)   : default=0 value=0
                        timeout 0x0098f902 (int)    : min=0 max=100000 step=1 default=0 value=0
               timeout_image_io 0x0098f903 (bool)   : default=0 value=0


  


  some info on my camera in /dev/video1

  


  v4l2-ctl -d /dev/video1 --all
Driver Info:
    Driver name      : uvcvideo
    Card type        : HD Pro Webcam C920
    Bus info         : usb-0000:00:14.0-7.2
    Driver version   : 5.4.78
    Capabilities     : 0x84a00001
        Video Capture
        Metadata Capture
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps      : 0x04200001
        Video Capture
        Streaming
        Extended Pix Format
Media Driver Info:
    Driver name      : uvcvideo
    Model            : HD Pro Webcam C920
    Serial           : EC6C336F
    Bus info         : usb-0000:00:14.0-7.2
    Media version    : 5.4.78
    Hardware revision: 0x00000011 (17)
    Driver version   : 5.4.78
Interface Info:
    ID               : 0x03000002
    Type             : V4L Video
Entity Info:
    ID               : 0x00000001 (1)
    Name             : HD Pro Webcam C920
    Function         : V4L2 I/O
    Flags         : default
    Pad 0x01000007   : 0: Sink
      Link 0x0200001f: from remote pad 0x100000a of entity 'Processing 3': Data, Enabled, Immutable
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
    Width/Height      : 1280/720
    Pixel Format      : 'H264' (H.264)
    Field             : None
    Bytes per Line    : 2560
    Size Image        : 1843200
    Colorspace        : sRGB
    Transfer Function : Default (maps to sRGB)
    YCbCr/HSV Encoding: Default (maps to ITU-R 601)
    Quantization      : Default (maps to Full Range)
    Flags             : 
Crop Capability Video Capture:
    Bounds      : Left 0, Top 0, Width 1280, Height 720
    Default     : Left 0, Top 0, Width 1280, Height 720
    Pixel Aspect: 1/1
Selection Video Capture: crop_default, Left 0, Top 0, Width 1280, Height 720, Flags: 
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 1280, Height 720, Flags: 
Streaming Parameters Video Capture:
    Capabilities     : timeperframe
    Frames per second: 30.000 (30/1)
    Read buffers     : 0
                     brightness 0x00980900 (int)    : min=0 max=255 step=1 default=128 value=128
                       contrast 0x00980901 (int)    : min=0 max=255 step=1 default=128 value=128
                     saturation 0x00980902 (int)    : min=0 max=255 step=1 default=128 value=128
 white_balance_temperature_auto 0x0098090c (bool)   : default=1 value=1
                           gain 0x00980913 (int)    : min=0 max=255 step=1 default=0 value=255
           power_line_frequency 0x00980918 (menu)   : min=0 max=2 default=2 value=2
                0: Disabled
                1: 50 Hz
                2: 60 Hz
      white_balance_temperature 0x0098091a (int)    : min=2000 max=6500 step=1 default=4000 value=3233 flags=inactive
                      sharpness 0x0098091b (int)    : min=0 max=255 step=1 default=128 value=128
         backlight_compensation 0x0098091c (int)    : min=0 max=1 step=1 default=0 value=0
                  exposure_auto 0x009a0901 (menu)   : min=0 max=3 default=3 value=3
                1: Manual Mode
                3: Aperture Priority Mode
              exposure_absolute 0x009a0902 (int)    : min=3 max=2047 step=1 default=250 value=333 flags=inactive
         exposure_auto_priority 0x009a0903 (bool)   : default=0 value=0
                   pan_absolute 0x009a0908 (int)    : min=-36000 max=36000 step=3600 default=0 value=0
                  tilt_absolute 0x009a0909 (int)    : min=-36000 max=36000 step=3600 default=0 value=0
                 focus_absolute 0x009a090a (int)    : min=0 max=250 step=5 default=0 value=0
                     focus_auto 0x009a090c (bool)   : default=1 value=0
                  zoom_absolute 0x009a090d (int)    : min=100 max=500 step=1 default=100 value=100


  


  if i remove the 3rd line of my script the camera records the stream of video and sound to the file flawless. i tried diferent things for the 3rd line as -vcodec and something to do with pix-format flag to YUYV without success.
Is it possible to achieve this with just one ffmpeg process ?
Will the sound be made available to the loopback device also ?
How to transcode it to the proper pixel format used by loopback device ?