Recherche avancée

Sur d’autres sites (6474)

• DXGI Desktop Duplication : encoding frames to send them over the network

  31 août 2018, par prazuber

  I’m trying to write an app which will capture a video stream of the screen and send it to a remote client. I’ve found out that the best way to capture a screen on Windows is to use DXGI Desktop Duplication API (available since Windows 8). Microsoft provides a neat sample which streams duplicated frames to screen. Now, I’ve been wondering what is the easiest, but still relatively fast way to encode those frames and send them over the network.

  The frames come from AcquireNextFrame with a surface that contains the desktop bitmap and metadata which contains dirty and move regions that were updated. From here, I have a couple of options :

  1. Extract a bitmap from a DirectX surface and then use an external library like ffmpeg to encode series of bitmaps to H.264 and send it over RTSP. While straightforward, I fear that this method will be too slow as it isn’t taking advantage of any native Windows methods. Converting D3D texture to a ffmpeg-compatible bitmap seems like unnecessary work.
  2. From this answer : convert D3D texture to IMFSample and use MediaFoundation’s SinkWriter to encode the frame. I found this tutorial of video encoding, but I haven’t yet found a way to immediately get the encoded frame and send it instead of dumping all of them to a video file.

  Since I haven’t done anything like this before, I’m asking if I’m moving in the right direction. In the end, I want to have a simple, preferably low latency desktop capture video stream, which I can view from a remote device.

  Also, I’m wondering if I can make use of dirty and move regions provided by Desktop Duplication. Instead of encoding the frame, I can send them over the network and do the processing on the client side, but this means that my client has to have DirectX 11.1 or higher available, which is impossible if I would want to stream to a mobile platform.

• Workflow for creating animated hand-drawn videos - encoding difficulties

  8 décembre 2017, par Mircode

  I want to create YouTube videos, kind of in the style of a white-board animation.

  Tldr question : How can I encode into a lossless rgb video format with ffmpeg, including alpha channel ?

  More detailed :
  My current workflow looks like this :

  I draw the slides in Inkscape, I group all paths that are to be drawn in one go (one scene so to say) and store the slide as svg. Then I run a custom python script over that. It animates the slides as described here https://codepen.io/MyXoToD/post/howto-self-drawing-svg-animation. Each frame is exported as svg, converted to png and fed to ffmpeg for making a video from it.

  For every scene (a couple of paths being drawn, there are several scenes per slide) I create an own video file and then I also store a png file that contains the last frame of that video.

  I then use kdenlive to join it all together : A video containing the drawing of the first scene, then a png which holds the last image of the video while I talk about the drawing, then the next animated drawing, then the next still image where I continue talking and so on. I use these intermediate images because freezing the last frame is tedious in kdenlive and I have around 600 scenes. Here I do the editing, adjust the duration of the still images and render the final video.

  The background of the video is a photo of a blackboard which never changes, the strokes are paths with a filter to make it look like chalk.

  So far so good, everything almost works.

  My problem is : Whenever there is a transition between an animation and a still image, it is visible in the final result. I have tried several approaches to make this work but nothing is without flaws.

  My first approach was to encode the animations as mp4 like this :

  p = Popen(['ffmpeg', '-y', '-f', 'image2pipe', '-vcodec', 'png', '-r', str(fps), '-i', '-', '-vcodec', 'libx264', '-crf', '21', '-bf', '2', '-flags', '+cgop', '-pix_fmt', 'yuv420p', '-movflags', 'faststart', '-r', str(fps), videofile], stdin=PIPE)

  which is recommended for YouTube. But then there is a little brightness difference between video and still image.

  Then I tried mov with png codec :

  p = Popen(['ffmpeg', '-y', '-f', 'image2pipe', '-vcodec', 'png', '-r', str(fps), '-i', '-', '-vcodec', 'png', '-r', str(fps), videofile], stdin=PIPE)

  I think this encodes every frame as png in the video. It creates way bigger files since every frame is encoded separately. But it’s ok since I can use transparency for the background and just store the chalk strokes. However, sometimes I want to swipe parts of the chalk on a slide away, which I do by drawing background over it. Which would work if those overlaid, animated background chunks which are stored in the video looked exactly like the underlying png in the background. But it doesn’t. It’s slightly more blurry and I believe the color changes a tiny bit as well. Which I don’t understand since I thought the video just stores a sequence of pngs... Is there some quality setting that I miss here ?

  Then I read about ProRes4444 and tried that :

  p = Popen(['ffmpeg', '-y', '-f', 'image2pipe', '-vcodec', 'png', '-r', str(fps), '-i', '-', '-c:v', 'prores_ks', '-pix_fmt', 'yuva444p10le', '-alpha_bits', '8', '-profile:v', '4444', '-r', str(fps), videofile], stdin=PIPE)

  and this actually seems to work. However, the animation files become larger than the bunch of png files they contain, probably because this format stores 12 bit per channel. This is not thaat horrible since only intermediate videos grow big, the final result is still ok.

  But ideally there would be a lossless codec which stores in rgb colorspace with 8 bit per channel, 8 bit for alpha and considers only the difference to the previous frame (because all that changes from frame to frame is a tiny bit of chalk drawing). Is there such a thing ? Alternatively, I’d also be ok without transparency but then I have to store the background in every scene. But if only changes are stored from frame to frame within one scene, that should be manageable.

  Or should I make some fundamental changes in my workflow altogether ?

  Sorry that this is rather lengthy, I appreciate any help.

  Cheers !

• FFmpeg : How to color-key all colors except blue and white ?

  21 février 2018, par DanielNg

  I know how to colorkey a specific color from an overlay video based on the documentation from FFmpeg. Here is the code I used to colorkey the black color out of a video then overlay it with a base video keeping the audio from the overlaid video :

  "ffmpeg -i " + baseVid + " -i " + overlayVid + " -y" + ' -filter_complex "[1:v]colorkey=0x000000:0.6:0.0[ckout];[0:v][ckout]overlay=shortest=1[out]" -shortest -map 1:a -map "[out]" ' + output

  However, my problem is that the overlaid video also contains other colors such as blue, white, brown, grey, green. Even though they are only small portions compared to the black that has been removed, they make the colorkeyed overlaid video look really dirty. I want to keep only the two main colors (blue and white) and removing the rest from the overlaid video. How do I do this using FFmpeg. I tried this code but it did not work.