Recherche avancée

Sur d’autres sites (7237)

• Is 'Android+FFMpeg' friendship really available ?

  26 janvier 2016, par vold_by

  The question does not mean that I’m interested if ffmpeg code can be used on Andoid. I know that it can. I’m just asking if somebody has the real performance progress with that stuff.
  I’ve created the question after several weeks of experiments with the stuff and I’ve had enough...
  I do not want to write to branches where people even do not say what kind of video they decode (resolution, codec) and talk only about some mystical FPS. I just don’t understand what they want to do. Also I’m not going to develop application only for my phone or for Android 2.2++ phones that have some extended OpenGL features. I have quite popular phone HTC Desire so if the application does not work on it, so what’s next ?

  Well, what do I have ?

  1. FFMpeg source from the latest HEAD branch. Actually I could not buld it with NDK5 so I decided to use stolen one.

  2. Bambuser’s build script (bash) with appropriate ffmpeg source ([web] : http://bambuser.com/r/opensource/ffmpeg-4f7d2fe-android-2011-03-07.tar.gz).
     It builds well after some corrections by using NDK5.

  3. Rockplayer’s gelded ffmpeg source code with huge Android.mk in the capacity of build script ([web] : http://www.rockplayer.com/download/rockplayer_ffmpeg_git_20100418.zip).
     It builds by NDK3 and NDK5 after some corrections. Rockplayer is probably the most cool media player for Android and I supposed that I would have some perks using it’s build.

  I had suitable video for a project (is not big and is not small) : 600x360 H.264.

  Both libraries we got from clauses 2 and 3 provide us possibility to get frames from video (frame-by-frame, seek etc.). I did not try to get an audio track because I did not need one for the project. I’m not publishing my source here because I think that’s traditional and it’s easy to find.

  Well, what’s the results with video ?
  HTC Desire, Android 2.2
  600x360, H.264
  decoding and rendering are in different threads

  1. Bambuser (NDK5 buld for armv5te, RGBA8888) : 33 ms/frame average.
  2. Rockplayer (NDK3 build for neon, RGB565) : 27 ms/frame average.

  It’s not bad for the first look, but just think that these are results only to decode frames.
  If somebody has much better results with decoding time, let me know.

  The most hard thing for a video is rendering. If we have bitmap 600x360 we should scale one somehow before painting because different phones have different screen sizes and we can not expect that our video will be the same size as screen.

  What options do we have to rescale a frame to fit it to screen ?
  I was able to check (the same phone and video source) those cases :

  1. sws_scale() C function in Bambuser’s build : 70 ms/frame. Unacceptable.
  2. Stupid bitmap rescaling in Android (Bitmap.createScaledBitmap) : 65 ms/frame. Unacceptable.
  3. OpenGL rendering in ortho projection on textured quad. In this case I did not need to scale frame. I just needed to prepare texture 1024x512 (in my case it was RGBA8888) containig frame pixels and than load it in GPU (gl.glTexImage2D). Result : 220 ms/frame to render. Unacceptable. I did not expect that glTexImage2D just sucked on Snapdragon CPU.

  That’s all.
  I know that there is some way to use fragment shader to convert YUV pixels using GPU, but we will have the same glTexImage2D and 200 ms just to texture loading.

  But this is not the end. ...my only friend the end... :)
  It’s not hopeless condition.

  Trying to use RockPlayer you definitely will wonder how they do that damn frame scaling so fast. I suppose that they have really good experience in ARM achitecture. They most probably use avcodec_decode_video2 and than img_convert (as I did in RP version), but then they use some tricks (depends of ARM version) for scaling.
  Maybe they also have some "magic" buld configuration for ffmpeg decreasing decoding time but Android.mk that they published is not THE Android.mk they use. Dunno...

  So, now it looks like you can not just buld some easy JNI bridge for ffmpeg and than have real media player for Android platform. You can do this only if you have suitable video that you do not need to scale.

  Any ideas ? I hope for you ;)

• JavaFXFrameConverter consuming insane amounts of memory

  17 juillet 2023, par iexav

  The JavaFXFrameConverter (from the wrapper of the ffmpeg C library in java) convert() method is consuming an outrageous amount of memory. To elaborate a bit more, it does not happen usually. If I just make an instance of the class in my main method, grab a frame via FFMpegFrameGrabber and give it to the convert() method the memory usage is pretty much none. However, when I attempt to do pretty much the exact same in a class I made using an ExecutorService my memory usage jumps up to 8 gigabytes when convert is called. The converter and executor service are declared as member variables of my class. Namely :

  


      final JavaFXFrameConverter converter = new JavaFXFrameConverter();
    private  ExecutorService           videoExecutor;


  


  (the videoExecutor is instantiated in the constructor of the class :

  


  videoExecutor=Executors.newSingleThreadExecutor();


  


  Now, the method I am using for processing of the video frames is this :

  


      private void processVideo(){
        videoExecutor.submit(() -> {
            processingVideo.set(true);
            try{

                while(processingVideo.get() && (videoQueue.peek())!=null){

                    final Frame cloneFrame = videoQueue.poll();
                    final Image image = converter.convert(cloneFrame);
                    final long timeStampDeltaMicros = cloneFrame.timestamp - timer.elapsedMicros();
                    if (timeStampDeltaMicros > 0) {
                        final long delayMillis = timeStampDeltaMicros / 1000L;
                        try {
                            Thread.sleep(delayMillis);
                        } catch (InterruptedException e) {
                            Thread.currentThread().interrupt();
                        }
                    }

                    cloneFrame.close();
                    System.out.println("submitted image");
                videoListener.submitData(image);
                }
            }catch (NullPointerException e){
                NullPointerException ex = new NullPointerException("Error while processing video frames.");
            videoListener.notifyFailure(ex);
            }

            processingVideo.set(false);

        });
    }


  


  I sent the whole method for a bit more context but realistically speaking the only part that is of real significance is the converter.conver(cloneFrame) ; I used the intelliJ profiler and also the debugger and this is exactly where the problem occurs. When convert is called after doing some stuff it eventually ends up in this method :

  


          public <t extends="extends" buffer="buffer"> void getPixels(int x, int y, int w, int h, WritablePixelFormat<t> pixelformat, T buffer, int scanlineStride) {&#xA;            int fss = this.frame.imageStride;&#xA;            if (this.frame.imageChannels != 3) {&#xA;                throw new UnsupportedOperationException("We only support frames with imageChannels = 3 (BGR)");&#xA;            } else if (!(buffer instanceof ByteBuffer)) {&#xA;                throw new UnsupportedOperationException("We only support bytebuffers at the moment");&#xA;            } else {&#xA;                ByteBuffer bb = (ByteBuffer)buffer;&#xA;                ByteBuffer b = (ByteBuffer)this.frame.image[0];&#xA;&#xA;                for(int i = y; i &lt; y &#x2B; h; &#x2B;&#x2B;i) {&#xA;                    for(int j = x; j &lt; x &#x2B; w; &#x2B;&#x2B;j) {&#xA;                        int base = 3 * j;&#xA;                        bb.put(b.get(fss * i &#x2B; base));&#xA;                        bb.put(b.get(fss * i &#x2B; base &#x2B; 1));&#xA;                        bb.put(b.get(fss * i &#x2B; base &#x2B; 2));&#xA;                        bb.put((byte)-1);&#xA;                    }&#xA;                }&#xA;&#xA;            }&#xA;        }&#xA;</t></t>

  


  Now, everything up until this point is fine. The memory usage is at around 130mb but alas, when execution enters in these 2 for loops that's where the downright stupid memory usage occurs. Every single one of these bb.put calls is netting me around 3 more megabytes of memory usage. By the end of it you can probably guess what happens. Also all of these memory allocations do happen on the stack so I'm assuming that's why my memory usage stops at around 8-8.5 gigabytes otherwise the program would crash (that has also happened, out of memory exception thrown, but it doesn't usually happen, it kind of just lingers at those 8 gigabytes.) Frankly speaking I'm at a bit of a loss. I haven't seen virtually anyone anywhere mention this ever and I ran out of things to try to fix this so I am making this post.

  


  By the way another thing I tried is make the ExecutorService in the same class as the main method and when I submitted there I also didn't have these memory problems.

  


• Error importing VideoFileClip from moviepy : AttributeError : 'PermissionError' object has no attribute 'message'

  11 mars 2017, par Tahlil

  I’m using jupyter notebook. I have also tried from anaconda console as well.

  Tried importing with both the ways shown below

  from moviepy.editor import VideoFileClip
  
  
  
  from moviepy.video.io.VideoFileClip import VideoFileClip

  Both of them gave me same error. Full trace is below

  AttributeError                            Traceback (most recent call last)
  
   in <module>()
  
        6 import glob
  
        7 import math
  
  ----> 8 from moviepy.editor import VideoFileClip
  
        9 from moviepy.video.io.VideoFileClip import VideoFileClip
  
  
  
  C:\Program Files\Anaconda3\lib\site-packages\moviepy\editor.py in <module>()
  
       20 # Clips
  
       21 
  
  ---> 22 from .video.io.VideoFileClip import VideoFileClip
  
       23 from .video.io.ImageSequenceClip import ImageSequenceClip
  
       24 from .video.io.downloader import download_webfile
  
  
  
  C:\Program Files\Anaconda3\lib\site-packages\moviepy\video\io\VideoFileClip.py in <module>()
  
        1 import os
  
        2 
  
  ----> 3 from moviepy.video.VideoClip import VideoClip
  
        4 from moviepy.audio.io.AudioFileClip import AudioFileClip
  
        5 from moviepy.Clip import Clip
  
  
  
  C:\Program Files\Anaconda3\lib\site-packages\moviepy\video\VideoClip.py in <module>()
  
       18 
  
       19 import moviepy.audio.io as aio
  
  ---> 20 from .io.ffmpeg_writer import ffmpeg_write_image, ffmpeg_write_video
  
       21 from .io.ffmpeg_tools import ffmpeg_merge_video_audio
  
       22 from .io.gif_writers import (write_gif,
  
  
  
  C:\Program Files\Anaconda3\lib\site-packages\moviepy\video\io\ffmpeg_writer.py in <module>()
  
       13     DEVNULL = open(os.devnull, 'wb')
  
       14 
  
  ---> 15 from moviepy.config import get_setting
  
       16 from moviepy.tools import verbose_print
  
       17 
  
  
  
  C:\Program Files\Anaconda3\lib\site-packages\moviepy\config.py in <module>()
  
       49     success, err = try_cmd([FFMPEG_BINARY])
  
       50     if not success:
  
  ---> 51         raise IOError(err.message +
  
       52                  "The path specified for the ffmpeg binary might be wrong")
  
       53 
  
  
  
  AttributeError: 'PermissionError' object has no attribute 'message'
  
  </module></module></module></module></module></module>

  Python version info

  Python 3.5.2 |Anaconda custom (64-bit)| (default, Jul  5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)] on win32
  
  Type "help", "copyright", "credits" or "license" for more information.

  Running ffmpeg -version in a console gives me

  ffmpeg version N-83507-g8fa18e0 Copyright (c) 2000-2017 the FFmpeg developers
  
  built with gcc 5.4.0 (GCC)
  
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
  
  libavutil      55. 47.100 / 55. 47.100
  
  libavcodec     57. 80.100 / 57. 80.100
  
  libavformat    57. 66.102 / 57. 66.102
  
  libavdevice    57.  2.100 / 57.  2.100
  
  libavfilter     6. 73.100 /  6. 73.100
  
  libswscale      4.  3.101 /  4.  3.101
  
  libswresample   2.  4.100 /  2.  4.100
  
  libpostproc    54.  2.100 / 54.  2.100

  I’m running 64 bit version of Windows 10.

  I can’t find any solution anywhere and its driving me crazy ! Seems like its not finding the ffmpeg binary but I have put it in C :\ffmpeg\bin and added this to path environment variable. Followed the instruction from here.