Recherche avancée

Sur d’autres sites (4973)

• How to Simply Remove Duplicate Frames from a Video using ffmpeg

  29 janvier 2017, par Skeeve

  First of all, I’d preface this by saying I’m NO EXPERT with video manipulation,
  although I’ve been fiddling with ffmpeg for years (in a fairly limited way). Hence, I’m not too flash with all the language folk often use... and how it affects what I’m trying to do in my manipulations... but I’ll have a go with this anyway...

  I’ve checked a few links here, for example :
  ffmpeg - remove sequentially duplicate frames

  ...but the content didn’t really help me.

  I have some hundreds of video clips that have been created under both Windows and Linux using both ffmpeg and other similar applications. However, they have some problems with times in the video where the display is ’motionless’.

  As an example, let’s say we have some web site that streams a live video into, say, a Flash video player/plugin in a web browser. In this case, we’re talking about a traffic camera video stream, for example.

  There’s an instance of ffmpeg running that is capturing a region of the (Windows) desktop into a video file, viz :-

  ffmpeg -hide_banner -y -f dshow ^
  
        -i video="screen-capture-recorder" ^
  
        -vf "setpts=1.00*PTS,crop=448:336:620:360" ^
  
        -an -r 25 -vcodec libx264 -crf 0 -qp 0 ^
  
        -preset ultrafast SAMPLE.flv

  Let’s say the actual ’display’ that is being captured looks like this :-

  123456789 XXXXX 1234567 XXXXXXXXXXX 123456789 XXXXXXX
  
  ^---a---^ ^-P-^ ^--b--^ ^----Q----^ ^---c---^ ^--R--^ 

  ...where each character position represents a (sequence of) frame(s). Owing to a poor internet connection, a "single frame" can be displayed for an extended period (the ’X’ characters being an (almost) exact copy of the immediately previous frame). So this means we have segments of the captured video where the image doesn’t change at all (to the naked eye, anyway).

  How can we deal with the duplicate frames ?... and how does our approach change if the ’duplicates’ are NOT the same to ffmpeg but LOOK more-or-less the same to the viewer ?

  If we simply remove the duplicate frames, the ’pacing’ of the video is lost, and what used to take, maybe, 5 seconds to display, now takes a fraction of a second, giving a very jerky, unnatural motion, although there are no duplicate images in the video. This seems to be achievable using ffmpeg with an ’mp_decimate’ option, viz :-

       ffmpeg -i SAMPLE.flv ^                      ... (i)
  
          -r 25 ^
  
          -vf mpdecimate,setpts=N/FRAME_RATE/TB DEC_SAMPLE.mp4

  That reference I quoted uses a command that shows which frames ’mp_decimate’ will remove when it considers them to be ’the same’, viz :-

       ffmpeg -i SAMPLE.flv ^                      ... (ii)
  
          -vf mpdecimate ^
  
          -loglevel debug -f null -

  ...but knowing that (complicated formatted) information, how can we re-organize the video without executing multiple runs of ffmpeg to extract ’slices’ of video for re-combining later ?

  In that case, I’m guessing we’d have to run something like :-

  • user specifies a ’threshold duration’ for the duplicates
    (maybe run for 1 sec only)
  • determine & save main video information (fps, etc - assuming
    constant frame rate)
  • map the (frame/time where duplicates start)->no. of
    frames/duration of duplicates
  • if the duration of duplicates is less than the user threshold,
    don’t consider this period as a ’series of duplicate frames’
    and move on
  • extract the ’non-duplicate’ video segments (a, b & c in the
    diagram above)
  • create ’new video’ (empty) with original video’s specs
  • for each video segment
    extract the last frame of the segment
    create a short video clip with repeated frames of the frame
    just extracted (duration = user spec. = 1 sec)
    append (current video segment+short clip) to ’new video’
    and repeat

  ...but in my case, a lot of the captured videos might be 30 minutes long and have hundreds of 10 sec long pauses, so the ’rebuilding’ of the videos will take a long time using this method.

  This is why I’m hoping there’s some "reliable" and "more intelligent" way to use
  ffmepg (with/without the ’mp_decimate’ filter) to do the ’decimate’ function in only a couple of passes or so... Maybe there’s a way that the required segments could even be specified (in a text file, for example) and as ffmpeg runs it will
  stop/restart it’s transcoding at specified times/frame numbers ?

  Short of this, is there another application (for use on Windows or Linux) that could do what I’m looking for, without having to manually set start/stop points,
  extracting/combining video segments manually...?

  I’ve been trying to do all this with ffmpeg N-79824-gcaee88d under Win7-SP1 and (a different version I don’t currently remember) under Puppy Linux Slacko 5.6.4.

  Thanks a heap for any clues.

• Problems with Python's azure.cognitiveservices.speech when installing together with FFmpeg in a Linux web app

  15 mai 2024, par Kakobo kakobo

  I need some help.
I'm building an web app that takes any audio format, converts into a .wav file and then passes it to 'azure.cognitiveservices.speech' for transcription.I'm building the web app via a container Dockerfile as I need to install ffmpeg to be able to convert non ".wav" audio files to ".wav" (as azure speech services only process wav files). For some odd reason, the 'speechsdk' class of 'azure.cognitiveservices.speech' fails to work when I install ffmpeg in the web app. The class works perfectly fine when I install it without ffpmeg or when i build and run the container in my machine.

  


  I have placed debug print statements in the code. I can see the class initiating, for some reason it does not buffer in the same when when running it locally in my machine. The routine simply stops without any reason.

  


  Has anybody experienced a similar issue with azure.cognitiveservices.speech conflicting with ffmpeg ?

  


  Here's my Dockerfile :

  


  # Use an official Python runtime as a parent imageFROM python:3.11-slim

#Version RunRUN echo "Version Run 1..."

Install ffmpeg

RUN apt-get update && apt-get install -y ffmpeg && # Ensure ffmpeg is executablechmod a+rx /usr/bin/ffmpeg && # Clean up the apt cache by removing /var/lib/apt/lists saves spaceapt-get clean && rm -rf /var/lib/apt/lists/*

//Set the working directory in the container

WORKDIR /app

//Copy the current directory contents into the container at /app

COPY . /app

//Install any needed packages specified in requirements.txt

RUN pip install --no-cache-dir -r requirements.txt

//Make port 80 available to the world outside this container

EXPOSE 8000

//Define environment variable

ENV NAME World

//Run main.py when the container launches

CMD ["streamlit", "run", "main.py", "--server.port", "8000", "--server.address", "0.0.0.0"]`and here's my python code:


  


  def transcribe_audio_continuous_old(temp_dir, audio_file, language):
    speech_key = azure_speech_key
    service_region = azure_speech_region

    time.sleep(5)
    print(f"DEBUG TIME BEFORE speechconfig")

    ran = generate_random_string(length=5)
    temp_file = f"transcript_key_{ran}.txt"
    output_text_file = os.path.join(temp_dir, temp_file)
    speech_recognition_language = set_language_to_speech_code(language)
    
    speech_config = speechsdk.SpeechConfig(subscription=speech_key, region=service_region)
    speech_config.speech_recognition_language = speech_recognition_language
    audio_input = speechsdk.AudioConfig(filename=os.path.join(temp_dir, audio_file))
        
    speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_input, language=speech_recognition_language)
    done = False
    transcript_contents = ""

    time.sleep(5)
    print(f"DEBUG TIME AFTER speechconfig")
    print(f"DEBUG FIle about to be passed {audio_file}")

    try:
        with open(output_text_file, "w", encoding=encoding) as file:
            def recognized_callback(evt):
                print("Start continuous recognition callback.")
                print(f"Recognized: {evt.result.text}")
                file.write(evt.result.text + "\n")
                nonlocal transcript_contents
                transcript_contents += evt.result.text + "\n"

            def stop_cb(evt):
                print("Stopping continuous recognition callback.")
                print(f"Event type: {evt}")
                speech_recognizer.stop_continuous_recognition()
                nonlocal done
                done = True
            
            def canceled_cb(evt):
                print(f"Recognition canceled: {evt.reason}")
                if evt.reason == speechsdk.CancellationReason.Error:
                    print(f"Cancellation error: {evt.error_details}")
                nonlocal done
                done = True

            speech_recognizer.recognized.connect(recognized_callback)
            speech_recognizer.session_stopped.connect(stop_cb)
            speech_recognizer.canceled.connect(canceled_cb)

            speech_recognizer.start_continuous_recognition()
            while not done:
                time.sleep(1)
                print("DEBUG LOOPING TRANSCRIPT")

    except Exception as e:
        print(f"An error occurred: {e}")

    print("DEBUG DONE TRANSCRIPT")

    return temp_file, transcript_contents


  


  The transcript this callback works fine locally, or when installed without ffmpeg in the linux web app. Not sure why it conflicts with ffmpeg when installed via container dockerfile. The code section that fails can me found on note #NOTE DEBUG"

  


• javacv and moviepy comparison for video generation

  15 septembre 2024, par Vikram

  I am trying to generate video using images, where image have some overlays text and png icons. I am using javacv library for this.
Final output video seems pixelated, i don't understand what is it since i do not have video processing domain knowledge, i am beginner to this.
I know that video bitrate and choice of video encoder are important factor which contributes to video quality and there are many more factors too.

  


  I am providing you two output for comparison, one of them is generated using javacv and another one is from moviepy library

  


  Please watch it in full screen since the problem i am talking about only gets highlighted in full screen, you will see the pixel dancing in javacv generated video, but python output seems stable

  


  https://imgur.com/a/aowNnKg - javacv generated

  


  https://imgur.com/a/eiLXrbk - Moviepy generated

  


  I am using same encoder in both of the implementation

  


  Encoder - libx264
bitrate - 
   800 Kbps for javacv 
   500 Kbps for moviepy

frame rate - 24fps for both of them

output video size -> 
    7MB (javacv)
    5MB (Moviepy)




  


  generated output size from javacv is bigger then moviepy generated video.

  


  here is my java configuration for FFmpegFrameRecorder

  


          FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(this.outputPath, 
                                           this.screensizeX, this.screensizeY);
        if(this.videoCodecName!=null && "libx264".equals(this.videoCodecName)) {
            recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
        }
        recorder.setFormat("mp4"); 
        recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420);
        recorder.setVideoBitrate(800000);
        recorder.setImageWidth(this.screensizeX);
        recorder.setFrameRate(24);



  


  and here is python configuration for writing video file

  


  Full_final_clip.write_videofile(
                            f"{video_folder_path}/{FILE_ID}_INTERMEDIATE.mp4",
                            codec="libx264",
                            audio_codec="aac",
                            temp_audiofile=f"{FILE_ID}_INTER_temp-audio.m4a",
                            remove_temp=True,
                            fps=24,
                        )



  


  as you can see i am not specifying bitrate in python, but i checked that bitrate of final output is around 500 kbps, which is lower then what i specified in java, yet java generated video quality seems poor.

  


  I have tried setting crf value also , but it seems it does not have any impact when used.

  


  increasing bitrate improve quality somewhat but at the cost of file size, still generated output seems pixelated.

  


  Can someone please highlight what might be the issue, and how python is generating better quality video, when both of the libraries use ffmpeg at the backend.

  


  Edit 1 : also, I am adding code which is being used to make zoom animation for continuous frames, As somewhere i read that this might be the cause for pixel jitter, please see and let me know if there is any improvement we can do to remove pixel jittering

  


  private Mat applyZoomEffect(Mat frame, int currentFrame, long effectFrames, int width, int height, String mode, String position, double speed) {
        long totalFrames = effectFrames;
        double i = currentFrame;
        if ("out".equals(mode)) {
            i = totalFrames - i;
        }
        double zoom = 1 + (i * ((0.1 * speed) / totalFrames));

        double originalCenterX = width/2.0;
        double originalCenterY = height/2.0;
   

        // Resize image
        //opencv_imgproc.resize(frame, resizedMat, new Size(newWidth, newHeight));

        // Determine crop region based on position
        double x = 0, y = 0;
        switch (position.toLowerCase()) {
            case "center":
                // Adjusting for center zoom
                 x = originalCenterX - originalCenterX * zoom;
                 y = originalCenterY - originalCenterY * zoom;
                 
                 x= (width-(width*zoom))/2.0;
                 y= (height-(height*zoom))/2.0;
                break;
        }

        double[][] rowData = {{zoom, 0, x},{0,zoom,y}};

        double[] flatData = flattenArray(rowData);

        // Create a DoublePointer from the flattened array
        DoublePointer doublePointer = new DoublePointer(flatData);

        // Create a Mat object with two rows and three columns
        Mat mat = new Mat(2, 3, org.bytedeco.opencv.global.opencv_core.CV_64F); // CV_64F is for double type

        // Put the data into the Mat object
        mat.data().put(doublePointer);
        Mat transformedFrame = new Mat();
        opencv_imgproc.warpAffine(frame, transformedFrame, mat, new Size(frame.cols(), frame.rows()),opencv_imgproc.INTER_LANCZOS4,0,new Scalar(0,0,0,0));
        return transformedFrame;
    }