Recherche avancée

Sur d’autres sites (2425)

• Live audio using ffmpeg, javascript and nodejs

  8 novembre 2017, par klaus

  I am new to this thing. Please don’t hang me for the poor grammar. I am trying to create a proof of concept application which I will later extend. It does the following : We have a html page which asks for permission to use the microphone. We capture the microphone input and send it via websocket to a node js app.

  JS (Client) :

  var bufferSize = 4096;
  
  var socket = new WebSocket(URL);
  
  var myPCMProcessingNode = context.createScriptProcessor(bufferSize, 1, 1);
  
  myPCMProcessingNode.onaudioprocess = function(e) {
  
    var input = e.inputBuffer.getChannelData(0);
  
    socket.send(convertFloat32ToInt16(input));
  
  }
  
  
  
  function convertFloat32ToInt16(buffer) {
  
    l = buffer.length;
  
    buf = new Int16Array(l);
  
    while (l--) {
  
      buf[l] = Math.min(1, buffer[l])*0x7FFF;
  
    }
  
    return buf.buffer;
  
  }
  
  
  
  navigator.mediaDevices.getUserMedia({audio:true, video:false})
  
                                  .then(function(stream){
  
                                    var microphone = context.createMediaStreamSource(stream);
  
                                    microphone.connect(myPCMProcessingNode);
  
                                    myPCMProcessingNode.connect(context.destination);
  
                                  })
  
                                  .catch(function(e){});

  In the server we take each incoming buffer, run it through ffmpeg, and send what comes out of the std out to another device using the node js ’http’ POST. The device has a speaker. We are basically trying to create a 1 way audio link from the browser to the device.

  Node JS (Server) :

  var WebSocketServer = require('websocket').server;
  
  var http = require('http');
  
  var children = require('child_process');
  
  
  
  wsServer.on('request', function(request) {
  
    var connection = request.accept(null, request.origin);
  
    connection.on('message', function(message) {
  
      if (message.type === 'utf8') { /*NOP*/ }
  
      else if (message.type === 'binary') {
  
        ffm.stdin.write(message.binaryData);
  
      }
  
    });
  
    connection.on('close', function(reasonCode, description) {});
  
    connection.on('error', function(error) {});
  
  });
  
  
  
  var ffm = children.spawn(
  
      './ffmpeg.exe'
  
     ,'-stdin -f s16le -ar 48k -ac 2 -i pipe:0 -acodec pcm_u8 -ar 48000 -f aiff pipe:1'.split(' ')
  
  );
  
  
  
  ffm.on('exit',function(code,signal){});
  
  
  
  ffm.stdout.on('data', (data) => {
  
    req.write(data);
  
  });
  
  
  
  var options = {
  
    host: 'xxx.xxx.xxx.xxx',
  
    port: xxxx,
  
    path: '/path/to/service/on/device',
  
    method: 'POST',
  
    headers: {
  
     'Content-Type': 'application/octet-stream',
  
     'Content-Length': 0,
  
     'Authorization' : 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
  
     'Transfer-Encoding' : 'chunked',
  
     'Connection': 'keep-alive'
  
    }
  
  };
  
  
  
  var req = http.request(options, function(res) {});

  The device supports only continuous POST and only a couple of formats (ulaw, aiff, wav)

  This solution doesn’t seem to work. In the device speaker we only hear something like white noise.

  Also, I think I may have a problem with the buffer I am sending to the ffmpeg std in -> Tried to dump whatever comes out of the websocket to a .wav file then play it with VLC -> it plays everything in the record very fast -> 10 seconds of recording played in about 1 second.

  I am new to audio processing and have searched for about 3 days now for solutions on how to improve this and found nothing.

  I would ask from the community for 2 things :

  1. Is something wrong with my approach ? What more can I do to make this work ? I will post more details if required.

  2. If what I am doing is reinventing the wheel then I would like to know what other software / 3rd party service (like amazon or whatever) can accomplish the same thing.

  Thank you.

• Why does my Blink based browser play hide and seek ?

  21 janvier 2016, par Caius Jard

  We have a C# tool (that I wrote) that records online broadcasts taking place a custom written (that we wrote) flash app. (There are no DRM or copyright issues here.)

  We’ve coded up a system whereby this tool is installed on a Windows Server 2012 R2 Amazon AWS instance. After we boot the instance, the tool loads, waits for the right time to start recording, launches a browser and passes the command line argument of the URL to access the broadcast. The browser will then load the flash app and the interview audio and video will start arriving at the browser instance on AWS

  By way of a virtual audio cable driver, screen / audio capture directshow filters and ffmpeg a screen recording is taken. The C# tool calls ffmpeg and ffmpeg will record the screen reliably for the entire interview, then the tool shuts the whole thing down

  The problem I’m having is that both Chrome and Electron browser sometimes simply don’t draw themselves on the screen so all ffmpeg ends up recording is a blank desktop and the audio of the broadcast (hence, the browser IS running)

  We found this out when recordings started turning up with X hours of merely recording the windows desktop and the tool’s main window with a countdown timer.

  A screenshotting facility was built into the tool and added to its web control interface, and this way we can test whether the browser is visible - a human looks at the screenshot of every broadcast, just after recording has started (the browser is supposed to be on show by this time)

  We notice that 50% of the time, the browser isn’t drawing itself on screen. By 50% I mean that every other recording that the AWS instance carries out, will be blank : AWS starts, records ok, shuts down. AWS starts again an hour later for a different broadcast, recording is blank, shuts down.. Starts/ok/shutdown. Starts/blank/shutdown. Repeat ad infinitum

  What’s even more strange is that if I run VNCviewer on my dev machine and connect up to an instance that is having a problem, the instant that the VNC connection is up and the remote desktop is showing on my screen, the browser suddenly appears as if nothing was ever wrong. A screenshot from before the VNC connect shows blank desktop, connect VNC, take another screenshot and the browser is there. All through it the audio is fine - the browser connected to the boadcast is fine, for sure

  It’s as though Chrome/Electron thinks "you know what, noone is looking at me so I’m not going to bother drawing myself". No screen saver is set, though the power plan has the setting "turn off the display after 15 minutes".

  Perhaps Chrome/Electron have a test amounts to "if the display is off, don’t draw". I can’t explain the inconsistency though - the recorder launches at least 1 hour before it’s needed, and sits there idle until it’s time to start the browser. You’d hence imagine that the "power off the monitor after 15 mins" setting would reliably have ensured the "monitor" is "off" by the time every recording start comes around

  This behaviour doesn’t happen with any of the other browsers (but unfortunately the app doesn’t and cannot work in them because it uses some weird chrome-only technology/API).

  Can anyone suggest anything to look at to help debug this, or anything I can build into the C# tool to overcome the problem ? Coding it up to connect to itself via VNC for a few seconds after it has launched the browser.. Well that just tastes nasty.

  Naturally, as soon as I connect to the machine via VNC (rather than RDP - RDP isn’t usable because the recording context is in a logged on session for a particular user) the problem goes away, which makes it frustratingly hard to debug.

• How to write unit tests for your plugin – Introducing the Piwik Platform

  17 novembre 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to verify user permissions). This time you’ll learn how to write unit tests in Piwik. For this tutorial you will need to have basic knowledge of PHP, PHPUnit and the Piwik platform.

  When is a test a unit test ?

  There are many different opinions on this and it can be sometimes hard to decide. At Piwik we consider a test as a unit test if only a single method or class is being tested and if a test does not have a dependency to the filesystem, web, config, database or to any other plugin.

  If a test is slow it can be an indicator that it is not a unit test. “Slow” is of course a bit vague. We will cover how to write other type of tests, such as integration tests, in one of our next blog posts.

  Getting started

  In this post, we assume that you have already installed Piwik 2.9.0 or later via git, set up your development environment and created a plugin. If not, visit the Piwik Developer Zone where you’ll find the tutorial Setting up Piwik and other Guides that help you to develop a plugin.

  Let’s create a unit test

  We start by using the Piwik Console to create a new unit test :

  ./console generate:test --testtype unit

  The command will ask you to enter the name of the plugin the created test should belong to. I will use the plugin name “Insights”. Next it will ask you for the name of the test. Here you usually enter the name of the class you want to test. I will use “Widgets” in this example. There should now be a file plugins/Insights/tests/Unit/WidgetsTest.php which contains already an example to get you started easily :

  /**
   * @group Insights
   * @group WidgetsTest
   * @group Plugins
   */
  class WidgetsTest extends \PHPUnit_Framework_TestCase
  {
   
      public function testSimpleAddition()
      {
          $this->assertEquals(2, 1+1);
      }
   
  }
  Télécharger

  We don’t want to cover how you should write your unit test. This is totally up to you. If you have no experience in writing unit tests yet, we recommend to read articles on the topic, or a book, or to watch videos or anything else that will help you learn best.

  Running a test

  To run a test we will use the command tests:run which allows you to execute a test suite, a specific file or a group of tests.

  To verify whether the created test works we will run it as follows :

  ./console tests:run WidgetsTest

  This will run all tests having the group WidgetsTest. As other tests can use the same group you might want to pass the path to your test file instead :

  ./console tests:run plugins/Insights/tests/Unit/Widgets.php

  If you want to run all tests within your plugin pass the name of your plugin as an argument :

  ./console tests:run insights

  Of course you can also define multiple arguments :

  ./console tests:run insights WidgetsTest

  This will execute all tests within the insights plugin having the group WidgetsTest. If you only want to run unit tests within your plugin you can do the following :

  ./console tests:run insights unit

  Advanced features

  Isn’t it easy to create a unit test ? We never even created a file ! You can accomplish even more if you want : You can generate other type of tests, you can run tests on Amazon’s AWS and more. Unfortunately, not everything is documented yet so we recommend to discover more features by executing the commands ./console list tests and ./console help tests:run.

  If you have any feedback regarding our APIs or our guides in the Developer Zone feel free to send it to us.