Recherche avancée

Sur d’autres sites (4872)

• How do you use Node.js to stream an MP4 file with ffmpeg ?

  27 avril 2023, par LaserJesus

  I've been trying to solve this problem for several days now and would really appreciate any help on the subject.

  



  I'm able to successfully stream an mp4 audio file stored on a Node.js server using fluent-ffmpeg by passing the location of the file as a string and transcoding it to mp3. If I create a file stream from the same file and pass that to fluent-ffmpeg instead it works for an mp3 input file, but not a mp4 file. In the case of the mp4 file no error is thrown and it claims the stream completed successfully, but nothing is playing in the browser. I'm guessing this has to do with the meta data being stored at the end of an mp4 file, but I don't know how to code around this. This is the exact same file that works correctly when it's location is passed to ffmpeg, rather than the stream. When I try and pass a stream to the mp4 file on s3, again no error is thrown, but nothing streams to the browser. This isn't surprising as ffmpeg won't work with the file locally as stream, so expecting it to handle the stream from s3 is wishful thinking.

  



  How can I stream the mp4 file from s3, without storing it locally as a file first ? How do I get ffmpeg to do this without transcoding the file too ? The following is the code I have at the moment which isn't working. Note that it attempts to pass the s3 file as a stream to ffmpeg and it's also transcoding it into an mp3, which I'd prefer not to do.

  



  .get(function(req,res) {
    aws.s3(s3Bucket).getFile(s3Path, function (err, result) {
        if (err) {
            return next(err);
        }
        var proc = new ffmpeg(result)
            .withAudioCodec('libmp3lame')
            .format('mp3')
            .on('error', function (err, stdout, stderr) {
                console.log('an error happened: ' + err.message);
                console.log('ffmpeg stdout: ' + stdout);
                console.log('ffmpeg stderr: ' + stderr);
            })
            .on('end', function () {
                console.log('Processing finished !');
            })
            .on('progress', function (progress) {
                console.log('Processing: ' + progress.percent + '% done');
            })
            .pipe(res, {end: true});
    });
});


  



  This is using the knox library when it calls aws.s3... I've also tried writing it using the standard aws sdk for Node.js, as shown below, but I get the same outcome as above.

  



  var AWS = require('aws-sdk');

var s3 = new AWS.S3({
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_KEY,
    region: process.env.AWS_REGION_ID
});
var fileStream = s3.getObject({
        Bucket: s3Bucket,
        Key: s3Key
    }).createReadStream();
var proc = new ffmpeg(fileStream)
    .withAudioCodec('libmp3lame')
    .format('mp3')
    .on('error', function (err, stdout, stderr) {
        console.log('an error happened: ' + err.message);
        console.log('ffmpeg stdout: ' + stdout);
        console.log('ffmpeg stderr: ' + stderr);
    })
    .on('end', function () {
        console.log('Processing finished !');
    })
    .on('progress', function (progress) {
        console.log('Processing: ' + progress.percent + '% done');
    })
    .pipe(res, {end: true});


  



  =====================================

  



  Updated

  



  I placed an mp3 file in the same s3 bucket and the code I have here worked and was able to stream the file through to the browser without storing a local copy. So the streaming issues I face have something to do with the mp4/aac container/encoder format.

  



  I'm still interested in a way to bring the m4a file down from s3 to the Node.js server in it's entirety, then pass it to ffmpeg for streaming without actually storing the file in the local file system.

  



  =====================================

  



  Updated Again

  



  I've managed to get the server streaming the file, as mp4, straight to the browser. This half answers my original question. My only issue now is that I have to download the file to a local store first, before I can stream it. I'd still like to find a way to stream from s3 without needing the temporary file.

  



  aws.s3(s3Bucket).getFile(s3Path, function(err, result){
    result.pipe(fs.createWriteStream(file_location));
    result.on('end', function() {
        console.log('File Downloaded!');
        var proc = new ffmpeg(file_location)
            .outputOptions(['-movflags isml+frag_keyframe'])
            .toFormat('mp4')
            .withAudioCodec('copy')
            .seekInput(offset)
            .on('error', function(err,stdout,stderr) {
                console.log('an error happened: ' + err.message);
                console.log('ffmpeg stdout: ' + stdout);
                console.log('ffmpeg stderr: ' + stderr);
            })
            .on('end', function() {
                console.log('Processing finished !');
            })
            .on('progress', function(progress) {
                console.log('Processing: ' + progress.percent + '% done');
            })
            .pipe(res, {end: true});
    });
});


  



  On the receiving side I just have the following javascript in an empty html page :

  



  window.AudioContext = window.AudioContext || window.webkitAudioContext;
context = new AudioContext();

function process(Data) {
    source = context.createBufferSource(); // Create Sound Source
    context.decodeAudioData(Data, function(buffer){
        source.buffer = buffer;
        source.connect(context.destination);
        source.start(context.currentTime);
    });
};

function loadSound() {
    var request = new XMLHttpRequest();
    request.open("GET", "/stream/", true);
    request.responseType = "arraybuffer";

    request.onload = function() {
        var Data = request.response;
        process(Data);
    };

    request.send();
};

loadSound()


  



  =====================================

  



  The Answer

  



  The code above under the title 'updated again' will stream an mp4 file, from s3, via a Node.js server to a browser without using flash. It does require that the file be stored temporarily on the Node.js server so that the meta data in the file is moved from the end of the file to the front. In order to stream without storing the temporary file, you need to actual modify the file on S3 first and make this meta data change. If you have changed the file in this way on S3 then you can modify the code under the title 'updated again' so that the result from S3 is piped straight into the ffmpeg constructor, rather than into a file stream on the Node.js server, then providing that file location to ffmepg, as the code does now. You can change the final 'pipe' command to 'save(location)' to get a version of the mp4 file locally with the meta data moved to the front. You can then upload that new version of the file to S3 and try out the end to end streaming. Personally I'm now going to create a task that modifies the files in this way as they are uploaded to s3 in the first place. This allows me to record and stream in mp4 without transcoding or storing a temp file on the Node.js server.

  


• Issues with video frame dropout using Accord.NET VideoFileWriter and FFMPEG

  9 janvier 2018, par David

  I am testing out writing video files using the Accord.Video library. I have a WPF project created in Visual Studio 2017, and I have installed Accord.Video.FFMPEG as well as Accord.Video.VFW using Nuget, as well as their dependencies.

  I have created a very simple video to test basic file output. However, I am running into some issues. My goal is to be able to output videos with a variable frame rate, because in the future I will be using this code to input images from a webcam device that will then be saved to a video file, and video from webcams typically has variable frame rates.

  For now, in this example, I am not inputting video from a webcam, but rather I am generating a simple "moving box" image and outputting the frames to a video file. The box changes color every 20 frames : red, green, blue, yellow, and finally white. I also set the frame rate to be 20 fps.

  When I use Accord.Video.VFW, the frame rate is correctly set, and all the frames are correctly outputted to the video file. The resulting video looks like this (see the YouTube link) : https://youtu.be/K8E9O7bJIbg

  This is just a reference, however. I don’t intend on using Accord.Video.VFW because it outputs uncompressed data to an AVI file, and it doesn’t support variable frame rates. I would like to use Accord.Video.FFMPEG because it is supposed to support variable frame rates.

  When I attempt to use the Accord.Video.FFMPEG library, however, the video does not result in how I would expect it to look. Here is a YouTube link : https://youtu.be/cW19yQFUsLI

  As you can see, in that example, the box remains the first color for a longer amount of time than the other colors. It also never reaches the final color (white). When I inspect the video file, 100 frames were not outputted to the file. There are 69 or 73 frames typically. And the expected frame rate and duration obviously do not match up.

  Here is the code that generates both these videos :

  public MainWindow()
  
  {
  
      InitializeComponent();
  
  
  
      Accord.Video.VFW.AVIWriter avi_writer = new Accord.Video.VFW.AVIWriter();
  
      avi_writer.FrameRate = 20;
  
      avi_writer.Open("test2.avi", 640, 480);
  
  
  
      Accord.Video.FFMPEG.VideoFileWriter k = new Accord.Video.FFMPEG.VideoFileWriter();
  
      k.FrameRate = 20;
  
      k.Width = 640;
  
      k.Height = 480;
  
      k.Open("test.mp4");
  
      for (int i = 0; i < 100; i++)
  
      {
  
          TimeSpan t = new TimeSpan(0, 0, 0, 0, 50 * i);
  
          var b = new System.Drawing.Bitmap(640, 480);
  
          var g = Graphics.FromImage(b);
  
          var br = System.Drawing.Brushes.Blue;
  
          if (t.TotalMilliseconds < 1000)
  
              br = System.Drawing.Brushes.Red;
  
          else if (t.TotalMilliseconds < 2000)
  
              br = System.Drawing.Brushes.Green;
  
          else if (t.TotalMilliseconds < 3000)
  
              br = System.Drawing.Brushes.Blue;
  
          else if (t.TotalMilliseconds < 4000)
  
              br = System.Drawing.Brushes.Yellow;
  
          else
  
              br = System.Drawing.Brushes.White;
  
  
  
          g.FillRectangle(br, 50 + i, 50, 100, 100);
  
          System.Console.WriteLine("Frame: " + (i + 1).ToString() + ", Millis: " + t.TotalMilliseconds.ToString());
  
  
  
          #region This is the code in question
  
  
  
          k.WriteVideoFrame(b, t);
  
          avi_writer.AddFrame(b);
  
  
  
          #endregion
  
      }
  
  
  
      avi_writer.Close();
  
      k.Close();
  
      System.Console.WriteLine("Finished writing video");
  
  }

  I have tried changing a few things under the assumption that maybe the "WriteVideoFrame" function isn’t able to finish in time, and so I need to slow down the program so it can complete itself. Under that assumption, I have replaced the "WriteVideoFrame" call with the following code :

  Task taskA = new Task(() => k.WriteVideoFrame(b, t));
  
  taskA.Start();
  
  taskA.Wait();

  And I have tried the following code :

  Task.WaitAll(
  
      Task.Run( () =>
  
      { 
  
          lock(syncObj)
  
          {
  
              k.WriteVideoFrame(b, t);
  
          }
  
      }
  
  ));

  And even just a standard call where I don’t specify a timestamp :

  k.WriteVideoFrame(b);

  None of these work. They all result in something similar.

  Any suggestions on getting the WriteVideoFrame function to work that is a part of the Accord.Video.FFMPEG.VideoFileWriter class ?

  Thanks for any and all help !

  [edits below]

  I have done some more investigating. I still haven’t found a good solution, but here is what I have found so far. After declaring my VideoFileWriter object, I have tried setting up some options for the video.

  When I use an H264 codec with the following options, it correctly saves 100 frames at a frame-rate of 20 fps, however any normal media player (both VLC and Windows Media Player) end up playing a 10-second video instead of a 5-second video. Essentially, it seems like they play it at half-speed. Here is the code that gives that result :

  k.VideoCodec = Accord.Video.FFMPEG.VideoCodec.H264;
  
  k.VideoOptions["crf"] = "18";
  
  k.VideoOptions["preset"] = "veryfast";
  
  k.VideoOptions["tune"] = "zerolatency";
  
  k.VideoOptions["x264opts"] = "no-mbtree:sliced-threads:sync-lookahead=0";

  Additionally, if I use an Mpeg4 codec, I get the same "half-speed" result :

  k.VideoCodec = Accord.Video.FFMPEG.VideoCodec.Mpeg4;

  However, if I use a WMV codec, then it correctly results in 100 frames at 20 fps, and a 5 second video that is correctly played by both media players :

  k.VideoCodec = Accord.Video.FFMPEG.VideoCodec.Wmv1;

  Although this is good news, this still doesn’t solve the problem because WMV doesn’t support variable frame rates. Also, this still doesn’t answer the question as to why the problem is happening in the first place.

  As always, any help would be appreciated !

• Museum of Multimedia Software, Part 2

  16 août 2010, par Multimedia Mike — Software Museum

  This installment includes a bunch of old, discontinued Adobe software as well as some Flash-related mutlimedia software.

  Screen Time for Flash Screen Saver Factory
  "Create High Impact Screen Savers Using Macromedia Flash."

  
  
  

  Requirements include Windows 3.1, 95 or NT 3.5.1. A 486 computer is required to play the resulting screensavers which are Flash projectors using Macromedia Flash 3.0.

  Monster Interactive Instant GUI 2
  Create eye-popping GUIs more easily for use in Flash. Usability experts would argue that this is not a good thing.

  
  
  

  Adobe Dimensions 3.0
  "The Easy Yet Powerful 3D Rendering Tool." This software was end-of-life’d in late 2004-early 2005 (depending on region).

  
  
  

  Adobe ImageStyler
  "Instantly add style to your Web site." Wikipedia claims that this product was sold from 1998 to 2000 when it was superseded by Adobe LiveMotion (see below).

  
  
  

  Google is able to excavate a link to the Latin American site for Adobe ImageStyler, a page that doesn’t seem to be replicated in any other language.

  Adobe LiveMotion
  "Professional Web graphics and animation." This is version 1, where the last version was #2, released in 2002.

  
  
  

  Adobe Streamline 4.0
  "The most powerful way to convert images into line art." This was discontinued in mid-2005.

  
  
  

  Adobe SuperATM
  "The magic that maintains the look of your documents." This is the oldest item in my collection. A close examination of the back of the box reveals an old Adobe logo. The latest copyright date on the box is 1992.