Recherche avancée

Sur d’autres sites (6218)

• Can I convert a django video upload from a form using ffmpeg before storing the video ?

  5 mai 2014, par GetItDone

  I’ve been stuck for weeks trying to use ffmpeg to convert user uploaded videos to flv. I use heroku to host my website, and store my static and media files on amazon S3 with s3boto. The initial video file will upload fine, however when I retrieve the video and run a celery task (in the same view where the initial video file is uploaded), the new file won’t store on S3. I’ve been trying to get this to work for over a month, with no luck, and really no good resources available for learning how to do this, so I figure maybe if I can get the ffmpeg task to run before storing the video I may be able to get it to work. Unfortunately I’m still not a very advanced at python (or django), so I don’t even know if/how this is possible. Anyone have any ideas ? I am willing to use any solution at this point no matter how ugly, as long as it successfully takes video uploads and converts to flv using ffmpeg, with the resulting file being stored on S3. It doesn’t seem that my situation is very common, because no matter where I look, I cannot find a solution that explains what I should be trying to do. Therefore I will be very appreciative of any guidance. Thanks. My relevant code follows :

  #models.py
  
  def content_file_name(instance, filename):
  
      ext = filename.split('.')[-1]
  
      new_file_name = "remove%s.%s" % (uuid.uuid4(), ext)
  
      return '/'.join(['videos', instance.teacher.username, new_file_name])
  
  
  
  class BroadcastUpload(models.Model):
  
      title = models.CharField(max_length=50, verbose_name=_('Title'))
  
      description = models.TextField(max_length=100, verbose_name=_('Description'))
  
      teacher = models.ForeignKey(User, null=True, blank=True, related_name='teacher')
  
      created_date = models.DateTimeField(auto_now_add=True)
  
      video_upload = models.FileField(upload_to=content_file_name)
  
      flvfilename = models.CharField(max_length=100, null=True, blank=True)
  
      videothumbnail = models.CharField(max_length=100, null=True, blank=True)
  
  
  
  #tasks.py
  
  @task(name='celeryfiles.tasks.convert_flv')
  
  def convert_flv(video_id):
  
      video = BroadcastUpload.objects.get(pk=video_id)
  
      print "ID: %s" % video.id
  
      id = video.id
  
      print "VIDEO NAME: %s" % video.video_upload.name
  
      teacher = video.teacher
  
      print "TEACHER: %s" % teacher
  
      filename = video.video_upload
  
      sourcefile = "%s%s" % (settings.MEDIA_URL, filename)
  
      vidfilename = "%s_%s.flv" % (teacher, video.id)
  
      targetfile = "%svideos/flv/%s" % (settings.MEDIA_URL, vidfilename)
  
      ffmpeg = "ffmpeg -i %s %s" % (sourcefile, vidfilename)
  
      try:
  
          ffmpegresult = subprocess.call(ffmpeg)
  
          #also tried separately with following line:
  
          #ffmpegresult = commands.getoutput(ffmpeg)
  
          print "---------------FFMPEG---------------"
  
          print "FFMPEGRESULT: %s" % ffmpegresult
  
      except Exception as e:
  
          ffmpegresult = None
  
          print("Failed to convert video file %s to %s" % (sourcefile, targetfile))
  
          print(traceback.format_exc())
  
      video.flvfilename = vidfilename
  
      video.save()
  
  
  
  @task(name='celeryfiles.tasks.ffmpeg_image')        
  
  def ffmpeg_image(video_id):
  
      video = BroadcastUpload.objects.get(pk=video_id)
  
      print "ID: %s" %video.id
  
      id = video.id
  
      print "VIDEO NAME: %s" % video.video_upload.name
  
      teacher = video.teacher
  
      print "TEACHER: %s" % teacher
  
      filename = video.video_upload
  
      sourcefile = "%s%s" % (settings.MEDIA_URL, filename)
  
      imagefilename = "%s_%s.png" % (teacher, video.id)
  
      thumbnailfilename = "%svideos/flv/%s" % (settings.MEDIA_URL, thumbnailfilename)
  
      grabimage = "ffmpeg -y -i %s -vframes 1 -ss 00:00:02 -an -vcodec png -f rawvideo -s 320x240 %s" % (sourcefile, thumbnailfilename)
  
      try:        
  
           videothumbnail = subprocess.call(grabimage)
  
           #also tried separately following line:
  
           #videothumbnail = commands.getoutput(grabimage)
  
           print "---------------IMAGE---------------"
  
           print "VIDEOTHUMBNAIL: %s" % videothumbnail
  
      except Exception as e:
  
           videothumbnail = None
  
           print("Failed to convert video file %s to %s" % (sourcefile, thumbnailfilename))
  
           print(traceback.format_exc())
  
      video.videothumbnail = imagefilename
  
      video.save()
  
  
  
  #views.py
  
  def upload_broadcast(request):
  
      if request.method == 'POST':
  
          form = BroadcastUploadForm(request.POST, request.FILES)
  
          if form.is_valid():
  
              upload=form.save()
  
              video_id = upload.id
  
              image_grab = ffmpeg_image.delay(video_id)
  
              video_conversion = convert_flv.delay(video_id)
  
              return HttpResponseRedirect('/current_classes/')
  
      else:
  
          form = BroadcastUploadForm(initial={'teacher': request.user,})
  
      return render_to_response('videos/create_video.html', {'form': form,}, context_instance=RequestContext(request))
  
  
  
  #settings.py
  
  DEFAULT_FILE_STORAGE = 'myapp.s3utils.MediaRootS3BotoStorage'
  
  DEFAULT_S3_PATH = "media"
  
  STATICFILES_STORAGE = 'myapp.s3utils.StaticRootS3BotoStorage'
  
  STATIC_S3_PATH = "static"
  
  AWS_STORAGE_BUCKET_NAME = 'my_bucket'
  
  CLOUDFRONT_DOMAIN = 'domain.cloudfront.net'
  
  AWS_ACCESS_KEY_ID = 'MY_KEY_ID'
  
  AWS_SECRET_ACCESS_KEY = 'MY_SECRET_KEY'
  
  MEDIA_ROOT = '/%s/' % DEFAULT_S3_PATH
  
  MEDIA_URL = 'http://%s/%s/' % (CLOUDFRONT_DOMAIN, DEFAULT_S3_PATH)
  
  ...
  
  
  
  #s3utils.py
  
  from storages.backends.s3boto import S3BotoStorage
  
  from django.utils.functional import SimpleLazyObject
  
  
  
  StaticRootS3BotoStorage = lambda: S3BotoStorage(location='static')
  
  MediaRootS3BotoStorage  = lambda: S3BotoStorage(location='media')

  I can add any other info if needed to help me solve my problem.

• Stream ffmpeg transcoding result to S3

  7 juin 2019, par mabead

  I want to transcode a large file using FFMPEG and store the result directly on AWS S3. This will be done inside of an AWS Lambda that has limited tmp space so I can’t store the transcoding result locally and then upload it to S3 in a second step. I won’t have enough tmp space. I therefore want to store the FFMPEG output directly on S3.

  I therefore created a S3 pre-signed url that allows ’PUT’ :

  var outputPath = s3Client.GetPreSignedURL(new Amazon.S3.Model.GetPreSignedUrlRequest
  
  {
  
      BucketName = "my-bucket",
  
      Expires = DateTime.UtcNow.AddMinutes(5),
  
      Key = "output.mp3",
  
      Verb = HttpVerb.PUT,
  
  });

  I then called ffmpeg with the resulting pre-signed url :

  ffmpeg -i C:\input.wav -y -vn -ar 44100 -ac 2 -ab 192k -f mp3 https://my-bucket.s3.amazonaws.com/output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550427237&Signature=%2BE8Wc%2F%2FQYrvGxzc%2FgXnsvauKnac%3D

  FFMPEG returns an exit code of 1 with the following output :

  ffmpeg version N-93120-ga84af760b8 Copyright (c) 2000-2019 the FFmpeg developers
  
    built with gcc 8.2.1 (GCC) 20190212
  
    configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  
    libavutil      56. 26.100 / 56. 26.100
  
    libavcodec     58. 47.100 / 58. 47.100
  
    libavformat    58. 26.101 / 58. 26.101
  
    libavdevice    58.  6.101 / 58.  6.101
  
    libavfilter     7. 48.100 /  7. 48.100
  
    libswscale      5.  4.100 /  5.  4.100
  
    libswresample   3.  4.100 /  3.  4.100
  
    libpostproc    55.  4.100 / 55.  4.100
  
  Guessed Channel Layout for Input Stream #0.0 : stereo
  
  Input #0, wav, from 'C:\input.wav':
  
    Duration: 00:04:16.72, bitrate: 3072 kb/s
  
      Stream #0:0: Audio: pcm_s32le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s32, 3072 kb/s
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (pcm_s32le (native) -> mp3 (libmp3lame))
  
  Press [q] to stop, [?] for help
  
  Output #0, mp3, to 'https://my-bucket.s3.amazonaws.com/output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550427237&Signature=%2BE8Wc%2F%2FQYrvGxzc%2FgXnsvauKnac%3D':
  
    Metadata:
  
      TSSE            : Lavf58.26.101
  
      Stream #0:0: Audio: mp3 (libmp3lame), 44100 Hz, stereo, s32p, 192 kb/s
  
      Metadata:
  
        encoder         : Lavc58.47.100 libmp3lame
  
  size=     577kB time=00:00:24.58 bitrate= 192.2kbits/s speed=49.1x    
  
  size=    1109kB time=00:00:47.28 bitrate= 192.1kbits/s speed=47.2x    
  
  [tls @ 000001d73d786b00] Error in the push function.
  
  av_interleaved_write_frame(): I/O error
  
  Error writing trailer of https://my-bucket.s3.amazonaws.com/output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550427237&Signature=%2BE8Wc%2F%2FQYrvGxzc%2FgXnsvauKnac%3D: I/O error
  
  size=    1143kB time=00:00:48.77 bitrate= 192.0kbits/s speed=  47x    
  
  video:0kB audio:1144kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
  
  [tls @ 000001d73d786b00] The specified session has been invalidated for some reason.
  
  [tls @ 000001d73d786b00] Error in the pull function.
  
  [https @ 000001d73d784fc0] URL read error:  -5
  
  Conversion failed!

  As you can see, I have a URL read error. This is a little surprising to me since I want to output to this url and not read it.

  Anybody know how I can store directly my FFMPEG output directly to S3 without having to store it locally first ?

  Edit 1
  I then tried to use the -method PUT parameter and use http instead of https to remove TLS from the equation. Here’s the output that I got when running ffmpeg with the -v trace option.

  ffmpeg version N-93120-ga84af760b8 Copyright (c) 2000-2019 the FFmpeg developers
  
    built with gcc 8.2.1 (GCC) 20190212
  
    configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  
    libavutil      56. 26.100 / 56. 26.100
  
    libavcodec     58. 47.100 / 58. 47.100
  
    libavformat    58. 26.101 / 58. 26.101
  
    libavdevice    58.  6.101 / 58.  6.101
  
    libavfilter     7. 48.100 /  7. 48.100
  
    libswscale      5.  4.100 /  5.  4.100
  
    libswresample   3.  4.100 /  3.  4.100
  
    libpostproc    55.  4.100 / 55.  4.100
  
  Splitting the commandline.
  
  Reading option '-i' ... matched as input url with argument 'C:\input.wav'.
  
  Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
  
  Reading option '-vn' ... matched as option 'vn' (disable video) with argument '1'.
  
  Reading option '-ar' ... matched as option 'ar' (set audio sampling rate (in Hz)) with argument '44100'.
  
  Reading option '-ac' ... matched as option 'ac' (set number of audio channels) with argument '2'.
  
  Reading option '-ab' ... matched as option 'ab' (audio bitrate (please use -b:a)) with argument '192k'.
  
  Reading option '-f' ... matched as option 'f' (force format) with argument 'mp3'.
  
  Reading option '-method' ... matched as AVOption 'method' with argument 'PUT'.
  
  Reading option '-v' ... matched as option 'v' (set logging level) with argument 'trace'.
  
  Reading option 'https://my-bucket.s3.amazonaws.com/output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550695990&Signature=dy3RVqDlX%2BlJ0INlDkl0Lm1Rqb4%3D' ... matched as output url.
  
  Finished splitting the commandline.
  
  Parsing a group of options: global .
  
  Applying option y (overwrite output files) with argument 1.
  
  Applying option v (set logging level) with argument trace.
  
  Successfully parsed a group of options.
  
  Parsing a group of options: input url C:\input.wav.
  
  Successfully parsed a group of options.
  
  Opening an input file: C:\input.wav.
  
  [NULL @ 000001fb37abb180] Opening 'C:\input.wav' for reading
  
  [file @ 000001fb37abc180] Setting default whitelist 'file,crypto'
  
  Probing wav score:99 size:2048
  
  [wav @ 000001fb37abb180] Format wav probed with size=2048 and score=99
  
  [wav @ 000001fb37abb180] Before avformat_find_stream_info() pos: 54 bytes read:65590 seeks:1 nb_streams:1
  
  [wav @ 000001fb37abb180] parser not found for codec pcm_s32le, packets or times may be invalid.
  
      Last message repeated 1 times
  
  [wav @ 000001fb37abb180] All info found
  
  [wav @ 000001fb37abb180] stream 0: start_time: -192153584101141.156 duration: 256.716
  
  [wav @ 000001fb37abb180] format: start_time: -9223372036854.775 duration: 256.716 bitrate=3072 kb/s
  
  [wav @ 000001fb37abb180] After avformat_find_stream_info() pos: 204854 bytes read:294966 seeks:1 frames:50
  
  Guessed Channel Layout for Input Stream #0.0 : stereo
  
  Input #0, wav, from 'C:\input.wav':
  
    Duration: 00:04:16.72, bitrate: 3072 kb/s
  
      Stream #0:0, 50, 1/48000: Audio: pcm_s32le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s32, 3072 kb/s
  
  Successfully opened the file.
  
  Parsing a group of options: output url https://my-bucket.s3.amazonaws.com/output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550695990&Signature=dy3RVqDlX%2BlJ0INlDkl0Lm1Rqb4%3D.
  
  Applying option vn (disable video) with argument 1.
  
  Applying option ar (set audio sampling rate (in Hz)) with argument 44100.
  
  Applying option ac (set number of audio channels) with argument 2.
  
  Applying option ab (audio bitrate (please use -b:a)) with argument 192k.
  
  Applying option f (force format) with argument mp3.
  
  Successfully parsed a group of options.
  
  Opening an output file: https://my-bucket.s3.amazonaws.com/output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550695990&Signature=dy3RVqDlX%2BlJ0INlDkl0Lm1Rqb4%3D.
  
  [http @ 000001fb37b15140] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
  
  [tcp @ 000001fb37b16c80] Original list of addresses:
  
  [tcp @ 000001fb37b16c80] Address 52.216.8.203 port 80
  
  [tcp @ 000001fb37b16c80] Interleaved list of addresses:
  
  [tcp @ 000001fb37b16c80] Address 52.216.8.203 port 80
  
  [tcp @ 000001fb37b16c80] Starting connection attempt to 52.216.8.203 port 80
  
  [tcp @ 000001fb37b16c80] Successfully connected to 52.216.8.203 port 80
  
  [http @ 000001fb37b15140] request: PUT /output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550695990&Signature=dy3RVqDlX%2BlJ0INlDkl0Lm1Rqb4%3D HTTP/1.1
  
  Transfer-Encoding: chunked
  
  User-Agent: Lavf/58.26.101
  
  Accept: */*
  
  Connection: close
  
  Host: landr-distribution-reportsdev-mb.s3.amazonaws.com
  
  Icy-MetaData: 1
  
  Successfully opened the file.
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (pcm_s32le (native) -> mp3 (libmp3lame))
  
  Press [q] to stop, [?] for help
  
  cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  detected 8 logical cores
  
  [graph_0_in_0_0 @ 000001fb37b21080] Setting 'time_base' to value '1/48000'
  
  [graph_0_in_0_0 @ 000001fb37b21080] Setting 'sample_rate' to value '48000'
  
  [graph_0_in_0_0 @ 000001fb37b21080] Setting 'sample_fmt' to value 's32'
  
  [graph_0_in_0_0 @ 000001fb37b21080] Setting 'channel_layout' to value '0x3'
  
  [graph_0_in_0_0 @ 000001fb37b21080] tb:1/48000 samplefmt:s32 samplerate:48000 chlayout:0x3
  
  [format_out_0_0 @ 000001fb37b22cc0] Setting 'sample_fmts' to value 's32p|fltp|s16p'
  
  [format_out_0_0 @ 000001fb37b22cc0] Setting 'sample_rates' to value '44100'
  
  [format_out_0_0 @ 000001fb37b22cc0] Setting 'channel_layouts' to value '0x3'
  
  [format_out_0_0 @ 000001fb37b22cc0] auto-inserting filter 'auto_resampler_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_0'
  
  [AVFilterGraph @ 000001fb37b0d940] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed
  
  [auto_resampler_0 @ 000001fb37b251c0] picking s32p out of 3 ref:s32
  
  [auto_resampler_0 @ 000001fb37b251c0] [SWR @ 000001fb37b252c0] Using fltp internally between filters
  
  [auto_resampler_0 @ 000001fb37b251c0] ch:2 chl:stereo fmt:s32 r:48000Hz -> ch:2 chl:stereo fmt:s32p r:44100Hz
  
  Output #0, mp3, to 'https://my-bucket.s3.amazonaws.com/output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550695990&Signature=dy3RVqDlX%2BlJ0INlDkl0Lm1Rqb4%3D':
  
    Metadata:
  
      TSSE            : Lavf58.26.101
  
      Stream #0:0, 0, 1/44100: Audio: mp3 (libmp3lame), 44100 Hz, stereo, s32p, delay 1105, 192 kb/s
  
      Metadata:
  
        encoder         : Lavc58.47.100 libmp3lame
  
  cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
      Last message repeated 6 times
  
  size=     649kB time=00:00:27.66 bitrate= 192.2kbits/s speed=55.3x    
  
  size=    1207kB time=00:00:51.48 bitrate= 192.1kbits/s speed=51.5x    
  
  av_interleaved_write_frame(): Unknown error
  
  No more output streams to write to, finishing.
  
  [libmp3lame @ 000001fb37b147c0] Trying to remove 47 more samples than there are in the queue
  
  Error writing trailer of https://my-bucket.s3.amazonaws.com/output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550695990&Signature=dy3RVqDlX%2BlJ0INlDkl0Lm1Rqb4%3D: Error number -10054 occurred
  
  size=    1251kB time=00:00:53.39 bitrate= 192.0kbits/s speed=51.5x    
  
  video:0kB audio:1252kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
  
  Input file #0 (C:\input.wav):
  
    Input stream #0:0 (audio): 5014 packets read (20537344 bytes); 5014 frames decoded (2567168 samples); 
  
    Total: 5014 packets (20537344 bytes) demuxed
  
  Output file #0 (https://my-bucket.s3.amazonaws.com/output.mp3?AWSAccessKeyId=AKIAJDSGJWM63VQEXHIQ&Expires=1550695990&Signature=dy3RVqDlX%2BlJ0INlDkl0Lm1Rqb4%3D):
  
    Output stream #0:0 (audio): 2047 frames encoded (2358144 samples); 2045 packets muxed (1282089 bytes); 
  
    Total: 2045 packets (1282089 bytes) muxed
  
  5014 frames successfully decoded, 0 decoding errors
  
  [AVIOContext @ 000001fb37b1f440] Statistics: 0 seeks, 2046 writeouts
  
  [http @ 000001fb37b15140] URL read error:  -10054
  
  [AVIOContext @ 000001fb37ac4400] Statistics: 20611126 bytes read, 1 seeks
  
  Conversion failed!

  So it looks like it is able to connect to my S3 pre-signed url but I still have the Error writing trailer error coupled with a URL read error.

• How to send ffmpeg based HTTP stream to CloudFront [on hold]

  5 octobre 2017, par Tarun Maheshwari

  I have a live streaming server, where media is available as HTTP / RTMP / HLS format.
  I am able to access it and play from my desktop using following links.

   rtmp://52.xx.xx.192/live/tarun1 (RTMP)
  
   http://52.xx.xx.192:8080/hls/movie.m3u8 (HLS)
  
   http://52.xx.xx.190:7090/stream (HTTP)

  I want to forward any of the above streams to CloudFront with these steps.