Recherche avancée

Sur d’autres sites (2061)

• Running a py script in the Cloud

  12 janvier 2018, par Anay Bose

  I’m new to Google’s cloud & Virtual Machine(VM) instances, and I need some clarifications on a couple of points. I have a python script ; it imports a long range of functions. I need to run those functions in parallel. I’m using multiprocessing and Process, not threads. These functions are basically image and media processors, and they use many other tools like FFMPEG, imagemagick and Avisynth in addition to a wide range of python modules, including moviepy. Now, I would like to run some 50 functions in parallel assigning a CPU for each process. Images, media and avi files are stored in seperate folders. I’m on Windows7 Core-i7 machine. So, need cloud computing power.

  Now, my question can I run such a python script/app in the cloud that requires a very complicated file system and non-python tools i.e. ffmpeg, avisynth and avi files ?

  Can Google VMs emulate my local machine and empower me with more cores and memory to run such a program ? if not, then what are my options ? Is their any tutorials that I can follow ? I need your suggestions. I have given below an example script and some codes to help facilitate your understanding about my situation.

  from __future__ import unicode_literals
  
  import youtube_dl
  
  import os
  
  import time
  
  import sys
  
  reload(sys)  
  
  sys.setdefaultencoding('utf-8')
  
  from multiprocessing import Process
  
  from utils import *
  
  
  
  from clip31 import VIDEO31
  
  from clip32 import VIDEO32
  
  from clip189 import VIDEO189
  
  from clip16 import VIDEO16
  
  from clip39 import VIDEO39
  
  
  
  
  
  if __name__== '__main__':
  
  
  
      # 1. CALLING A FUNCTION
  
      folder = "bodyforce3\\16"
  
      serial = "16"
  
      images = get_filepaths("../16")
  
      videos = get_filepaths("12__media")
  
      pngs = get_filepaths("../pngs")
  
  
  
      Process(target=VIDEO192, args=(folder, serial, color1, color2, color3, images, videos)).start()
  
  
  
  
  
      # 2. CALLING A FUNCTION
  
      folder = "bodyforce3\\20"
  
      serial = "20"
  
      images = get_filepaths("../20")
  
      videos = get_filepaths("18__media")
  
  
  
      Process(target=VIDEO32, args=(folder, serial, color1, color2, color3, images, videos)).start()
  
  
  
  
  
      # 3. CALLING A FUNCTION
  
      folder = "bodyforce3\\14"
  
      serial = "14"
  
      images = get_filepaths("../14")
  
      videos = get_filepaths("16__media")
  
  
  
      Process(target=VIDEO91, args=(folder, serial, color1, color2, color3, images, videos)).start()

  I copy avi files in functions like this :

  src = "clip50_files"
  
  src_files = os.listdir(src)
  
  for file_name in src_files:
  
      full_file_name = os.path.join(src, file_name)
  
      if (os.path.isfile(full_file_name)):
  
          shutil.copy(full_file_name, folder)

  I call ffmpeg commands like this, and they are included within py functions.

  ###########################
  
  #### FFMPEG OPERATIONS ####
  
  ###########################
  
  
  
  print "Starting FFMPEG operations ..."
  
  
  
  if os.path.isfile(os.path.join(folder, "bounce-(3).avi")):
  
      os.remove(os.path.join(folder, "bounce-(3).avi"))
  
  
  
  
  
  infile = folder + "/bounce-(3).avs"
  
  outfile = folder + "/bounce-(3).avi"
  
  codec = "rawvideo"
  
  pix_fmt = "bgra"
  
  
  
  try:
  
      subprocess.call(["ffmpeg",
  
                       "-i" ,infile, 
  
                       "-c:v" ,codec, 
  
                       "-pix_fmt", pix_fmt, 
  
                       outfile],
  
                      stdout=open(os.devnull, 'w'), 
  
                      stderr=subprocess.STDOUT)
  
  except subprocess.CalledProcessError as e:   
  
      #except subprocess.CalledProcessError as e:
  
      sys.exit(e.output)
  
  except OSError as e:
  
      sys.exit(e.strerror)
  
  
  
  
  
  print "FFMPEG operations ended"

• Bash script : automate ffmpeg encoding for mpeg-dash

  13 février 2018, par Massimo Vantaggio

  I’m writing a bash file to create video encoding and concatenation for dash live streaming use,
  Basically it read an input video folder, encodes all videos into three resolution formats, after that it concatening them to create three adaption sets.

  DIAGRAM :

  

  This script checks for fps conformance,

  force/scaling resolution if the input is not 1920 x 1080p,

  Insert the channel logo png,

  Cut the end of all videos input in order to make them finish with closed gop, this to ensure that there are not videos with audio and video track with different lenght.

  ISSUE :

  Actually I’m not sure that the concatenation process respects the closed GOP alignment as it after the encoding..
  I try to cut also the end of the concatenation result in order to make it finish without decimals on a closed gop too, but im unable to erase all decimals from the total duration :

  total duration in seconds: 826.795000
  
  total duration corrected in seconds: 826

  But the real duration measured by ffprobe is

  824.044000

  I try to check keyframes alignment with mp4box has they teach without any result :

  MP4Box -info TRACK_ID source1.mp4 2>&1 | grep GOP

  This is the first time that i work with "video scripts" and probably i don’t know what input to give for TRACK_ID

  BASH SCRIPT :

  #!/bin/bash
  
  #CANCAT 0.2
  
  
  
  cd input
  
  times=()
  
  fps=()
  
  for f in *.mp4; do
  
  
  
      _t=$(ffprobe -i "$f" -show_entries format=duration -v quiet -of csv="p=0")
  
      times+=("$_t")
  
  
  
      _f=$(ffmpeg -i "$f" 2>&1 | sed -n "s/.*, \\(.*\\) fp.*/\\1/p")
  
      fps+=("$_f")
  
  done
  
  #SUM ALL DURATIONS
  
  TOTALDURATION=$( echo "${times[@]}" | sed 's/ /+/g' | bc )
  
  #DELETE DECIMAL
  
  DURROUND=$(echo "$TOTALDURATION" | cut -d'.' -f1) 
  
  #GET REST OF DIVISION BY 2 AS GOP
  
  TOTDELTA="$((DURROUND%2))"
  
  #SUBTRACT DELTA FROM TOTAL DURATION
  
  TOTDUR="$(($DURROUND-$TOTDELTA))"
  
  
  
  #GET NUMBER OF ELEMENTS IN FPS ARRAY  
  
  tLen=${#fps[@]}
  
  #CHECK FPS EQUALITY     
  
  for tLen in "${fps[@]:1}"; do
  
      if [[ $tLen != ${fps[0]} ]]; then
  
          printf "WARNING: VIDEO’S FRAME-RATE ARE NOT EQUALS, THE PROCESS CAN’T START."
  
          printf "%s\\0" "${fps[@]}" |
  
              sort -zu |
  
              xargs -0 printf " %s"
  
          printf "\\n"
  
         exit 1
  
      fi
  
  done
  
  for f in *.mp4; do
  
  #GET DURATION OF EACH VIDEO
  
  DUR="$(ffprobe -i "$f" -show_entries format=duration -v quiet -of csv="p=0")"
  
  DUR=$(echo "$DUR" | cut -d'.' -f1) # DELETE DECIMAL
  
  #GET FPS OF EACH VIDEO
  
  FPS="$(ffmpeg -i "$f" 2>&1 | sed -n "s/.*, \(.*\) fp.*/\1/p")"
  
  #ROUND FPS OF EACH VIDEO
  
  FPSC=$( echo "($FPS+0.5)/1" | bc )
  
  #REMOVE EXTENSION FROM VIDEO FILE NAME
  
  NAME=$(echo "$f" | cut -d'.' -f1)
  
  
  
  #GET GOP
  
  GOP="$((FPSC*2))"
  
  DELTADUR="$((DUR%2))"
  
  DUR="$(($DUR-$DELTADUR))"
  
  
  
  #ENCODE 1080p
  
  ffmpeg -y -i "$f" -i ../logo/logo.png -c:a aac -b:a 384k -ar 48000 -ac 2 -async 1 -c:v libx264 -x264opts keyint=$GOP:min-keyint=$GOP:no-scenecut -bf 0 -r $FPSC -b:v 4800k -maxrate 4800k -bufsize 3000k -profile:v main -crf 22 -t $DUR -filter_complex "[0:v][1:v]overlay=main_w-overlay_w-10:10,scale=1920:1080,setsar=1" ../buffer/${NAME}-1080.mp4
  
  
  
  #ENCODE 720p
  
  ffmpeg -y -i ../buffer/${NAME}-1080.mp4 -c:a aac -b:a 256k -ar 48000 -ac 2 -async 1 -c:v libx264 -x264opts keyint=$GOP:min-keyint=$GOP:no-scenecut -bf 0 -s 1280x720 -r $FPSC -b:v 2400k -maxrate 2400k -bufsize 1500k -profile:v main -crf 22 -t $DUR ../buffer/${NAME}-720.mp4
  
  
  
  #ENCODE 360p
  
  ffmpeg -y -i ../buffer/${NAME}-720.mp4 -c:a aac -b:a 128k -ar 48000 -ac 2 -async 1 -c:v libx264 -x264opts keyint=$GOP:min-keyint=$GOP:no-scenecut -bf 0 -s 640x360 -r $FPSC -b:v 800k -maxrate 800k -bufsize 500k -profile:v main -crf 22 -t $DUR ../buffer/${NAME}-360.mp4
  
  done
  
  
  
  
  
  #enter in buffer
  
  cd ..
  
  cd buffer
  
  
  
  #CONCAT 1080 SET
  
  # with a bash for loop
  
  for f in ./*1080.mp4; do echo "file '$f'" >> 1080list.txt; done
  
  
  
  ffmpeg -f concat -safe 0 -i 1080list.txt -t $TOTDUR -c copy ../output/1080set.mp4
  
  
  
  #CONCAT 720 SET
  
  # with a bash for loop
  
  for f in ./*720.mp4; do echo "file '$f'" >> 720list.txt; done
  
  
  
  ffmpeg -f concat -safe 0 -i 720list.txt -t $TOTDUR -c copy ../output/720set.mp4
  
  
  
  #CONCAT 360 SET
  
  # with a bash for loop
  
  for f in ./*360.mp4; do echo "file '$f'" >> 360list.txt; done
  
  
  
  ffmpeg -f concat -safe 0 -i 360list.txt -t $TOTDUR -c copy ../output/360set.mp4
  
  
  
  #CLEAN BUFFER
  
  rm *.mp4
  
  rm *.txt
  
  
  
  echo "CONCAT COMPLETED:"
  
  echo "frame-rate: $fps"
  
  echo "total duration in seconds: $TOTALDURATION"
  
  echo "total duration corrected in seconds: $TOTDUR"

  The full file with relative folders :

  BASH SCRIPT WITH FOLDERS

  RESULT VIDEO

  There is someone who can help me to understand why I can not eliminate the decimals of the total duration during concat ?
  And how to check overall keyframes allignment ?
  Also any impovement that i ignore is welcome !

  Thanks a lot !

  Massimo

• How to batch process entire folder with ffmpeg ? a batch script that will scan currently run folder

  8 mars 2018, par MonsterMMORPG

  I have this code but it is not working

  I want batch script to scan currently run directory and batch process the MP4 files in that directory

  But first i want it to create a new directory called as newfiles and put the processed files in that folder with the same name

  Please help me to fix this batch script ty

  md "%~dp0\newfiles"
  
  for %%a in (%~dp0\"*.mp4") do ffmpeg -i "%%a" -c:v libx264 -preset veryslow -crf 18 -c:a aac -b:a 192k -pix_fmt yuv420p "newfiles\%~dp0%%a.mp4"
  
  pause

  The code is not creating a new directory not batch process the mp4 files in that directory (the directory that is batch script executed)

  ok i have solved the question as below

  md "%~dp0\newfiles"
  
  for %%a in (%~dp0\"*.mp4") do ffmpeg -i "%%a" -c:v libx264 -preset veryslow -crf 18 -c:a aac -b:a 192k -pix_fmt yuv420p "newfiles\%%~na.mp4"
  
  pause