Recherche avancée

Sur d’autres sites (4292)

• Adjust Opus Ogg page size using Go

  15 mars 2024, par matthew

  Using the code from Go's Pion WebRTC library play-from-disk example I'm able to create a WebRTC connection and send audio from a local Ogg file to peers.

  


  The play-from-disk example README.md details how to first convert the page size of the Ogg file to 20,000 using ffmpeg, like so :

  


  ffmpeg -i $INPUT_FILE -c:a libopus -page_duration 20000 -vn output.ogg


  


  I'd like to make this same adjustment to Ogg data natively in Go, without using ffmpeg. How can this be done ?

  


  I've tried using Pion's oggreader and oggwriter, but using these requires deep Opus file format and RTP protocol knowledge that neither I nor ChatGPT seem to have.

  


  For additional context, I'm using a Text-to-Speech (TTS) API to generate my Ogg data as follows :

  


  req, err := http.NewRequest("POST", "https://api.openai.com/v1/audio/speech", bytes.NewBuffer([]byte(fmt.Sprintf(`{
    "model": "tts-1",
    "voice": "alloy",
    "input": %q,
    "response_format": "opus",
}`, text))))

req.Header.Add("Authorization", "Bearer "+token)
req.Header.Add("Content-Type", "application/json; charset=UTF-8")

client := &http.Client{}
resp, err := client.Do(req)



  


  As I'm trying to create a real-time audio app, ideally, I'd like to pipe the response to WebRTC performing the conversion on chunks as these are received so that peers can start to listen to the audio before it has been fully received on the server.

  


• YUV420p to RGB conversion has shifted U and V values - iOS Unity3D

  28 septembre 2018, par rohit n

  I have a native plugin in Unity that decodes H264 frames to YUV420p using FFMPEG.

  To display the output image, I rearrange the YUV values into an RGBA texture and convert YUV to RGB using Unity shader (just to make it faster).

  The following is the rearrangement code in my native plugin :

  unsigned char* yStartLocation = (unsigned char*)m_pFrame->data[0];
  
  unsigned char* uStartLocation = (unsigned char*)m_pFrame->data[1];
  
  unsigned char* vStartLocation = (unsigned char*)m_pFrame->data[2];
  
  
  
  for (int y = 0; y < height; y++)
  
  {
  
      for (int x = 0; x < width; x++)
  
      {
  
  
  
          unsigned char* y = yStartLocation + ((y * width) + x);
  
          unsigned char* u = uStartLocation + ((y * (width / 4)) + (x / 2));
  
          unsigned char* v = vStartLocation + ((y * (width / 4)) + (x / 2));
  
          //REF: https://en.wikipedia.org/wiki/YUV
  
  
  
          // Write the texture pixel
  
          dst[0] = y[0]; //R
  
          dst[1] = u[0]; //G
  
          dst[2] = v[0]; //B
  
          dst[3] = 255;  //A
  
  
  
          // To next pixel
  
          dst += 4;
  
  
  
          // dst is the pointer to target texture RGBA data
  
      }
  
  }

  The shader that converts YUV to RGB works perfectly and I’ve used it in multiple projects.

  Now, I’m using the same code to decode on iOS platform. But for some reason the U and V values are now shifted :

  Y Texture

  

  U Texture

  

  Is there anything that I’m missing for iOS or OpenGL specifically ?

  Any help greatly appreciated.
  Thank You !

  Please note that I filled R=G=B = Y for the first screenshot and U for the second.(if that makes sense)

  Edit :
  Heres the output that I’m getting :
  

  Edit2 :
  Based on some research I think it may have something to do with Interlacing.

  ref : Link

  For now I’ve shifted to CPU based YUV-RGB conversion using sws_scale and it works fine.

• Is there a direct way to render/encode Vulkan output as an ffmpeg video file ?

  13 septembre 2022, par Luiz Ventura

  I'm about to generate 2D and 3D music animations and render them to video using C++. I was thinking about using OpenGL, but I've read that, unfortunately, it is being discontinued in favour of Vulkan, which seems to offer higher performance using a GPU, but is also a lower-level API, making it more difficult to learn. I still have almost no knowledge in both OpenGL and Vulkan, beginning to learn now.

  


  My question is :

  


  is there a way to encode the Vulkan render output (showing a window or not) into a video file, preferentially through FFPMEG ? If so, how could I do that ?

  


  Requisites :

  


  

  • Speed : the decrease in performance should be nearly that of encoding the video only, not much more than that (e.g. by having to save lossless frames as images first and then encoding a video from them).

  


  • Controllable FPS and resolution : the video fps and frame resolution can be freely chosen.

  


  • Reliability, reproducibility : running a code that gives a same Vulkan output twice should result in 2 equal videos independently of the system, i.e. no dropping frames, async problems (I want to sync with audio) or whatsoever. The chosen video fps should stay fixed (e.g. 60 fps), no matter if the computer can render 300 or 3 fps.

  


  


  What I found out so far :

  


  

  • An example of taking "screenshots" from Vulkan output : it writes to a ppm image at the end, which is a binary uncompressed image file.

  


  • An encoder for rendering videos from OpenGL output, which is what I want, but using OpenGL in that case.

  


  • That Khronos includes in the Vulkan API a video subset.

  


  • A video tool to decode, demux, process videos using FFMPEG and Vulkan.

  


  • That is possible to render the output into a buffer without the need of a screen to display it.