Recherche avancée

Sur d’autres sites (5160)

• Revision 7059 : On ne tient pas compte de modifier l’objet pour gérer les signalements ...

  3 novembre 2012, par kent1 — Log

  On ne tient pas compte de modifier l’objet pour gérer les signalements car ce n’est pas à l’auteur de les gérer, juste les admins du site
  $fluxargs ?quoi ? sur le pipeline des destinataires et pas $fluxquoi ?
  Un peu de phpdoc

• How to render a blend of two videos with alpha channel in python in real time ? [closed]

  21 décembre 2024, par Francesco Calderone

  I need to play two videos in real time in Python.
One video is a background video with no alpha channel. I am using H264 but it can be any codec.
The second video is an overlay video. This video with an alpha channel needs to be played in real-time on top of the first video. I am using Quicktime 444 with an alpha channel but again, it can be any codec.

  


  In terms of libraries, I tried a combination of cv and numpy, I tried pymovie, pyAV, ffmpeg... so far, all the results have been unsuccessful. When the videos render, the frame rate drops way below 30FPS, and the resulting stream is glitchy.

  


  I also tried rendering the video without an alpha channel and performing green screen chroma keying in real-time. Needless to say, even worse.

  


  What solution can I use ?

  


  Here's my attempted code with ffmpeg

  


  import ffmpeg
import cv2
import numpy as np

def decode_video_stream(video_path, pix_fmt, width, height, fps):
    process = (
        ffmpeg
        .input(video_path)
        .output('pipe:', format='rawvideo', pix_fmt=pix_fmt, s=f'{width}x{height}', r=fps)
        .run_async(pipe_stdout=True, pipe_stderr=True)
    )
    return process

def read_frame(process, width, height, channels):
    frame_size = width * height * channels
    raw_frame = process.stdout.read(frame_size)
    if not raw_frame:
        return None
    frame = np.frombuffer(raw_frame, np.uint8).reshape((height, width, channels))
    return frame

def play_videos_with_alpha(base_video_path, alpha_video_path, resolution=(1280, 720), fps=30):
    width, height = resolution
    frame_time = int(1000 / fps)  # Frame time in milliseconds

    # Initialize FFmpeg decoding processes
    base_process = decode_video_stream(base_video_path, 'rgb24', width, height, fps)
    alpha_process = decode_video_stream(alpha_video_path, 'rgba', width, height, fps)

    cv2.namedWindow("Blended Video", cv2.WINDOW_NORMAL)

    try:
        while True:
            # Read frames
            base_frame = read_frame(base_process, width, height, channels=3)
            alpha_frame = read_frame(alpha_process, width, height, channels=4)

            # Restart processes if end of video is reached
            if base_frame is None:
                base_process.stdout.close()
                base_process = decode_video_stream(base_video_path, 'rgb24', width, height, fps)
                base_frame = read_frame(base_process, width, height, channels=3)

            if alpha_frame is None:
                alpha_process.stdout.close()
                alpha_process = decode_video_stream(alpha_video_path, 'rgba', width, height, fps)
                alpha_frame = read_frame(alpha_process, width, height, channels=4)

            # Separate RGB and alpha channels from alpha video
            rgb_image = cv2.cvtColor(alpha_frame[:, :, :3], cv2.COLOR_RGB2BGR)
            alpha_channel = alpha_frame[:, :, 3] / 255.0  # Normalize alpha

            # Convert base frame to BGR format for blending
            base_image = cv2.cvtColor(base_frame, cv2.COLOR_RGB2BGR)

            # Blend the images
            blended_image = (base_image * (1 - alpha_channel[:, :, None]) + rgb_image * alpha_channel[:, :, None]).astype(np.uint8)

            # Display the result
            cv2.imshow("Blended Video", blended_image)

            if cv2.waitKey(frame_time) & 0xFF == ord('q'):
                break

    except Exception as e:
        print("Error during playback:", e)

    finally:
        # Clean up
        base_process.stdout.close()
        alpha_process.stdout.close()
        cv2.destroyAllWindows()

base_video_path = "test.mp4"  # Background video
alpha_video_path = "test.mov"  # Overlay video
play_videos_with_alpha(base_video_path, alpha_video_path, resolution=(1280, 720), fps=30)


  


  Which is so far the version that drops less frames. I've been thinking about threading, or using CUDA, but ideally I want something that runs pretty much on any machine. What would be the least computationally heavy operation without reducing the frame size (1920 x 1080) and without pre-rendering the blend and exporting pre-blended file ? Is there a way ? Maybe I'm getting at it all wrong. I feel lost. Please help. Thanks.

  


• Trim Overlay memory issue

  4 juillet 2019, par Костянтин Тюртюбек

  Greetings fellow FFmpeg users,

  We are experiencing a strange memory leek issue (or maybe just doing something really wrong) and need some directions on how to debug it.

  What are we trying to achieve :

  Process a conference recording that includes multiple user streams, each in its own separate file (all files are mp4/opus).

  • Make a dynamic scene from a set of recordings, based on their volume level at set point of time.
  • The scene must include two parts : smaller grid of all the participants videos, bigger grid of currently talking people. Something like Google Hangouts or Skype does in their applications.

  What went wrong :

  • Memory footprint started unpredictably skyrocketing for some reason during montage

  What are we using :

  First FFmpeg command that reads filter_complex_script from file and adds drawbox as a talking indication on each video source file, when its volume is over a set threshold.

  Second FFmpeg command that reads filter_complex_script from file and :

  • takes an input file (using 0:v),
  • trims a part of it, when the user was talking,
  • scales it according to the amount of concurrently talking users,
  • pads to that resolution (in case if user video is smaller)

  filter_complex command using SELECT :

  [0]select='between(t, 1, 2)',  scale=762:428:force_original_aspect_ratio=decrease,pad=763:429:(ow-iw)/2:(oh-ih)/2[stream-0-workspace-scale-1-1];
  
  
  
  [block-2-grid][stream-0-workspace-scale-1-1]
  
  overlay=repeatlast=1:shortest=0:x=10:y=316:enable='between(t, 1, 2)'
  
  [block-2-workspace-1]; 

  filter_complex command using TRIM :

  [input-file-tag] 
  
  trim=start=#{start}:duration=#{duration},
  
  setpts=PTS-STARTPTS,
  
  scale=#{w-1}:#{h-1}:force_original_aspect_ratio=decrease,
  
  pad=#{w}:#{h}:(ow-iw)/2:(oh-ih)/2
  
  [input-file-trimmed];
  
  
  
  [previous-block-tag] 
  
  overlay=repeatlast=1:shortest=0:x=#{x}:y=#{y}:enable='between(t, #{from}, #{to})'
  
  [next-block-tag]

  We have tried going the TRIM command way, tried the SELECT command way. Problem is, both take insane amounts of ram during execution.

  Examples and more description :

  • Lets assume that only two of the five inputs have the volume above a
    certain volume threshold from second two to five.

  • We are trying to display only them according to some overlay math.

  • Cropped commands in human readable form : https://pastebin.com/YwrnRgnA

  • Full FFmpeg command is way too long to read through, that is the reason we started using filter_complex_script and loading it from file

  • Sometimes one block of video conference may have up to 300+
    intermediate overlays, which leads to the memory issue described. We were expecting the memory footprint to be similar to the amount of input files or maybe two-to-three times higher. However, we reach 15Gb or RAM usage withing the first two minutes of montage, while the input files are no bigger than 200Mb.

  What have we done in terms of debugging :

  • We had been using split at first, but quickly figured out that split
    does in fact copy each input and load it in memory, so we had to
    ditch that approach.

  • As matter of fact we moved to using the input files themselves, so
    the problem lies elsewhere.

  • To clarify, we have split our ffmpeg command into two separate ones.
    First one overlays the talking box animation using drawbox as well as
    user avatar and name. It outputs new video files which we than use in
    the command described above as direct input files tags, like 0:v, 1:v
    etc.

  Thank you for taking time reading through our issue.
  We sure hope that you can help us narrow it down.
  Please feel free to ask for any additional information or descriptions if needed.

  Have a good day !