Recherche avancée

Sur d’autres sites (6941)

• How to save animations with tight layout, transparency and in high quality

  23 octobre 2023, par mapf

  I am trying to implement an option in my GUI to save an image sequence displayed using matplotlib. The code looks something like this :

  



  import matplotlib.pyplot as plt
from matplotlib.backends.backend_qt5agg import \
    FigureCanvasQTAgg as FigureCanvas
from matplotlib.animation import FuncAnimation
from PIL import Image


plt.rcParams['savefig.bbox'] = 'tight' 


class Printer:
    def __init__(self, data):
        self.fig, self.ax = plt.subplots()
        self.canvas = FigureCanvas(self.fig)

        # some irrelevant color adjustment here
        #self.ax.spines['bottom'].set_color('#f9f2d7')
        #self.ax.spines['top'].set_color('#f9f2d7')
        #self.ax.spines['right'].set_color('#f9f2d7')
        #self.ax.spines['left'].set_color('#f9f2d7')
        #self.ax.tick_params(axis='both', colors='#f9f2d7')
        #self.ax.yaxis.label.set_color('#f9f2d7')
        #self.ax.xaxis.label.set_color('#f9f2d7')
        #self.fig.subplots_adjust(left=0.1, right=0.975, bottom=0.09, top=0.98)
        self.fig.patch.set_alpha(0)
        self.fig.patch.set_visible(False)
        self.canvas.setStyleSheet("background-color:transparent;")
        self.fig.set_size_inches(10, 10, True)
        self.fig.tight_layout()

        self.data = data
        self.image_artist = self.ax.imshow(data[0])

    def animate(self, i):
        self.image_artist.set_data(self.data[i])
        self.canvas.draw()


def save_animation():
    data = [
        Image.open("test000.png"),
        Image.open("test001.png"),
    ]
    file = 'test.gif'
    printer = Printer(data)

    ani = FuncAnimation(
        printer.fig, printer.animate, interval=100, frames=len(data),
    )
    # writer = animation.writers['pillow'](bitrate=1000)
    ani.save(
        file, writer='pillow', savefig_kwargs={'transparent': True, 'bbox_inches': 'tight'}
    )


save_animation()


  



  Transparency :

  



  As you can see I have already tried several different approaches as suggested elsewhere (1, 2), but didn't manage to find a solution. All of the settings and arguments patch.set_alpha(0), patch.set_visible(False), canvas.setStyleSheet("background-color:transparent;"), savefig_kwargs={'transparent': True} seem to have no effect at all on the transparency. I found this post but I didn't get the code to work (for one I had to comment out this %matplotlib inline, but then I ended up getting some error during the MovieWriter.cleanup out = TextIOWrapper(BytesIO(out)).read() TypeError: a bytes-like object is required, not 'str'). Here, it was suggested that this is actually a bug, but the proposed workaroud doesn't work for me since I would have to rely on third-party software. There also exists this bug report which was supposedly solved, so maybe it is unrelated.

  



  Tight layout

  



  I actually couldn't really find much on this, but all the things I tried (plt.rcParams['savefig.bbox'] = 'tight', fig.tight_layout(), savefig_kwargs={'bbox_inches': 'tight'}) don't have any effect or are even actively discarded in the case of the bbox_inches argument. How does this work ?

  



  High quality

  



  Since I cannot use ImageMagick and can't get ffmpeg to work (more on this below), I rely on pillow to save my animation. But the only argument in terms of quality that I can pass on seems to be the bitrate, which doesn't have any effect. The files still have the same size and the animation still looks like mush. The only way that I found to increase the resolution was to use fig.set_size_inches(10, 10, True), but this still doesn't improve the overall quality of the animation. It still looks bad. I saw that you can pass on codec and extra_args so maybe that is something that might help, but I have no idea how to use these because I couldn't find a list with allowed arguments.

  



  ffmpeg

  



  I can't get ffmpeg to work. I installed the python package from here and can import it into a python session but I don't know how I can get matplotlib to use that. I also got ffmpeg from here (Windows 64-bit version) and set the plt.rcParams['animation.ffmpeg_path'] to where I saved the files (there was no intaller to run, not sure if I did it correctly). But this didn't help either. Also this is of course also third-party software, so if somebody else were to use my code/program it wouldn't work.

  


• what is the meaning of the v4l2-ctl —list-device ?

  26 janvier 2021, par mcgregor94086

  The High level Problem :

  


  I have a custom multi-camera array that I have attached to a Raspberry Pi via some USB hubs, and I need a way to quickly identify which camera needs attention when any camera fails to respond to an image capture request.

  


  Can "v4l2-ctl —list_devices" help me ?

  


  I am trying to determine if
"v4l2-ctl —list_devices" can help me more quickly identify which one is missing. My thought was to look at which cameras are reporting, and then notice which one is NOT reporting, and investigate the missing one.

  


  My question is can I identify each reporting camera either for the v4l2-ctl output, or from "FFmpeg -list_formats"

  


  Here is the output I get from v4l2-ctl :

  


  $ v4l2-ctl --list-device            
bcm2835-codec-decode (platform:bcm2835-codec):
    /dev/video10
    /dev/video11
    /dev/video12

bcm2835-isp (platform:bcm2835-isp):
    /dev/video13
    /dev/video14
    /dev/video15
    /dev/video16

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.2.1.1):
    /dev/video4
    /dev/video5

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.2.1.2):
    /dev/video6
    /dev/video7

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.2.1.3):
    /dev/video8
    /dev/video9

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.2.1.4):
    /dev/video17
    /dev/video18

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.2.2):
    /dev/video0
    /dev/video1

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.2.3):
    /dev/video2
    /dev/video3

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.3.1.1):
    /dev/video21
    /dev/video22

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.3.1.2):
    /dev/video25
    /dev/video26

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.3.1.4):
    /dev/video29
    /dev/video30

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.3.2):
    /dev/video19
    /dev/video20

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.3.3):
    /dev/video23
    /dev/video24

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.1.3.4):
    /dev/video27
    /dev/video28

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.2.1.2):
    /dev/video35
    /dev/video36

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.2.1.3):
    /dev/video39
    /dev/video40

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.2.1.4):
    /dev/video41
    /dev/video42

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.2.2):
    /dev/video31
    /dev/video32

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.2.3):
    /dev/video33
    /dev/video34

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.2.4):
    /dev/video37
    /dev/video38

FHD Camera: FHD Camera (usb-0000:01:00.0-1.2.4.2):
    /dev/video43
    /dev/video44


  


  All the USB cameras in my array each report the same first field (in this case "FHD Camera") so I can't use the name as a unique identifier.

  


  Each "FHD Camera" is assigned two different /dev/videoNN ids (one will be assigned for the mpeg format, and the other for the UYV format). However these /dev/videoNN assignments change each time the computer reboots.

  


  The sequence that cameras are listed in the output also changes each time the v4l2-ctl command runs. So that also is of no help

  


  v4l2-ctl also reports another field, beginning "usb-0000 :" followed by a series numbers (e.g. "01:00.0-1.2.2.4").

  


  I am wondering if this number string ties in any way to the physical USB bus, and would remain permanent across reboots.

  


  The v4l2-ctl help documentation merely says that the —list-devices flag will list the video devices, but these additional fields and their meaning is not explained.

  


  Alternatively, for each camera that is responding, I can query the device with
"ffmpeg -hide_banner -f v4l2 -list_formats all -i /dev/videoNN", yielding a response like this :

  


  $ ffmpeg -hide_banner -f v4l2 -list_formats all -i /dev/video8
[video4linux2,v4l2 @ 0x1a391c0] Compressed:       mjpeg :          Motion-JPEG : 640x480 1280x720 640x360 320x240 1920x1080
[video4linux2,v4l2 @ 0x1a391c0] Raw       :     yuyv422 :           YUYV 4:2:2 : 640x480 1280x720 640x360 320x240 1920x1080
/dev/video8: Immediate exit requested


  


  This ffmpeg query shows me that the device is also registered as "0x1a391c0". I am not yet sure whether these identifiers are stable across reboots, and if they are, whether they are stable with the physical camera, or if they are just a stable identifier to the USB Hub location of the camera.

  


  My request :

  


  Can someone explain to me how the "device name", /dev/videoNN identifier, usb-0000:01:00.0-1.2.2.4 identifier, and 0x1a391c0 are assigned ?

  


  The responding order of the items listed in v4l2-ctl changes each time it runs.

  


  Is this reflecting that all cameras are polled simultaneously, but acquisition of the bus for response transmission is random ?

  


  Addendum

  


  Further investigation shows me that the "0x-------" identifiers do not seem to be stable across reboots either.

  


  The "usb-0000:01:00.0-1.2.2.2" identifier DOES seem to have a somewhat stable meaning. I have (5) 7-port usb hubs attached to my RPi 400. 4 of these are branch hubs that feed into a master hub, and the master hub feeds into the RPi.

  


  After the 0-1. in the identifier there is either 3 or 4 digits. If there are only 3, the last digit is a port identifier 2, 3, or 4.

  


  If there are 4 digits, then the 2nd to last digit will be a 1, and 1.1, 1.2, 1.3, and 1.4 will represent the last 4 physical ports on that hub.

  


  Ironically, while the same port identifiers are always used in the same order on each of my (identical brand/model) hubs, they are NOT in ascending or descending sequence in terms of their physical sequence on the hubs.

  


  I have deduced from this that each of my hubs actually uses two USB 4 port chips. So the first chip is getting the single digits, while the 1st port on that 1st chip is feeding the 2nd chip.

  


  The 2nd digit in my identifiers is identifying which branch hub the device is on, 1,2,3 or 4. The first digit is 1, which seems to be identifying that all the branches are children of the master hub.

  


  These addresses seem to be stable across reboots as long as all of the USB hubs are already connected at boot up. If not all are connected, those that are connected first will get the lowest hub number identifiers.

  


  I don't know if this tree address identifier behavior would be the same on other hubs, but perhaps these observations will be of some use to anyone else trying to find stable identifiers in their own multi hub and multi-device architecture.

  


• FFMPEG : multiplexing streams with different duration

  16 avril 2018, par Michael IV

  I am multiplexing video and audio streams. Video stream comes from generated image data. The audio stream comes from aac file. Some audio files are longer than total video time I set so my strategy to stop audio stream muxer when its time becomes larger than the total video time(the last one I control by number encoded video frames).

  I won’t put here the whole setup code, but it is similar to muxing.c example from the latest FFMPEG repo. The only difference is that I use audio stream from file,as I said, not from synthetically generated encoded frame. I am pretty sure the issue is in my wrong sync during muxer loop.Here is what I do :

  void AudioSetup(const char* audioInFileName)
  
  {
  
      AVOutputFormat* outputF = mOutputFormatContext->oformat;
  
      auto audioCodecId = outputF->audio_codec;
  
  
  
      if (audioCodecId == AV_CODEC_ID_NONE) {
  
          return false;
  
      }
  
  
  
      audio_codec = avcodec_find_encoder(audioCodecId);
  
  
  
      avformat_open_input(&mInputAudioFormatContext,
  
      audioInFileName, 0, 0);
  
      avformat_find_stream_info(mInputAudioFormatContext, 0);
  
  
  
      av_dump_format(mInputAudioFormatContext, 0, audioInFileName, 0);
  
  
  
  
  
      for (size_t i = 0; i < mInputAudioFormatContext->nb_streams; i++) {
  
          if (mInputAudioFormatContext->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {
  
              inAudioStream = mInputAudioFormatContext->streams[i];
  
  
  
              AVCodecParameters *in_codecpar = inAudioStream->codecpar;
  
              mAudioOutStream.st = avformat_new_stream(mOutputFormatContext, NULL);
  
              mAudioOutStream.st->id = mOutputFormatContext->nb_streams - 1;
  
              AVCodecContext* c = avcodec_alloc_context3(audio_codec);
  
              mAudioOutStream.enc = c;
  
              c->sample_fmt = audio_codec->sample_fmts[0];
  
              avcodec_parameters_to_context(c, inAudioStream->codecpar);
  
              //copyparams from input to autput audio stream:
  
              avcodec_parameters_copy(mAudioOutStream.st->codecpar, inAudioStream->codecpar);
  
  
  
              mAudioOutStream.st->time_base.num = 1;
  
              mAudioOutStream.st->time_base.den = c->sample_rate;
  
  
  
              c->time_base = mAudioOutStream.st->time_base;
  
  
  
              if (mOutputFormatContext->oformat->flags & AVFMT_GLOBALHEADER) {
  
                  c->flags |= CODEC_FLAG_GLOBAL_HEADER;
  
              }
  
              break;
  
          }
  
      }
  
  }
  
  
  
  void Encode()
  
  {
  
      int cc = av_compare_ts(mVideoOutStream.next_pts, mVideoOutStream.enc->time_base,
  
      mAudioOutStream.next_pts, mAudioOutStream.enc->time_base);
  
  
  
      if (mAudioOutStream.st == NULL || cc <= 0) {
  
          uint8_t* data = GetYUVFrame();//returns ready video YUV frame to work with
  
          int ret = 0;
  
          AVPacket pkt = { 0 };
  
          av_init_packet(&pkt);
  
          pkt.size = packet->dataSize;
  
          pkt.data = data;
  
          const int64_t duration = av_rescale_q(1, mVideoOutStream.enc->time_base, mVideoOutStream.st->time_base);
  
  
  
          pkt.duration = duration;
  
          pkt.pts = mVideoOutStream.next_pts;
  
          pkt.dts = mVideoOutStream.next_pts;
  
          mVideoOutStream.next_pts += duration;
  
  
  
          pkt.stream_index = mVideoOutStream.st->index;
  
          ret = av_interleaved_write_frame(mOutputFormatContext, &pkt);
  
      } else
  
      if(audio_time <  video_time) {
  
          //5 -  duration of video in seconds
  
          AVRational r = {  60, 1 };
  
  
  
          auto cmp= av_compare_ts(mAudioOutStream.next_pts, mAudioOutStream.enc->time_base, 5, r);
  
          if (cmp >= 0) {
  
              mAudioOutStream.next_pts = (int64_t)std::numeric_limits::max();
  
              return true; //don't mux audio anymore
  
          }
  
  
  
          AVPacket a_pkt = { 0 };
  
          av_init_packet(&a_pkt);
  
  
  
          int ret = 0;
  
          ret = av_read_frame(mInputAudioFormatContext, &a_pkt);
  
          //if audio file is shorter than stop muxing when at the end of the file
  
          if (ret == AVERROR_EOF) {
  
              mAudioOutStream.next_pts = (int64_t)std::numeric_limits::max(); 
  
              return true;
  
          }
  
          a_pkt.stream_index = mAudioOutStream.st->index;
  
  
  
          av_packet_rescale_ts(&a_pkt, inAudioStream->time_base, mAudioOutStream.st->time_base);
  
          mAudioOutStream.next_pts += a_pkt.pts;
  
  
  
          ret = av_interleaved_write_frame(mOutputFormatContext, &a_pkt);
  
      }
  
  }

  Now, the video part is flawless. But if the audio track is longer than video duration, I am getting total video length longer by around 5% - 20%, and it is clear that audio is contributing to that as video frames are finished exactly where there’re supposed to be.

  The closest ’hack’ I came with is this part :

  AVRational r = {  60 ,1 };
  
  auto cmp= av_compare_ts(mAudioOutStream.next_pts, mAudioOutStream.enc->time_base, 5, r);
  
  if (cmp >= 0) {
  
      mAudioOutStream.next_pts = (int64_t)std::numeric_limits::max();
  
      return true;
  
  } 

  Here I was trying to compare next_pts of the audio stream with the total time set for video file,which is 5 seconds. By setting r = {60,1} I am converting those seconds by the time_base of the audio stream. At least that’s what I believe I am doing. With this hack, I am getting very small deviation from the correct movie length when using standard AAC files,that’s sample rate of 44100,stereo. But if I test with more problematic samples,like AAC sample rate 16000,mono - then the video file adds almost a whole second to its size.
  I will appreciate if someone can point out what I am doing wrong here.

  Important note : I don’t set duration on for any of the contexts. I control the termination of the muxing session, which is based on video frames count.The audio input stream has duration, of course, but it doesn’t help me as video duration is what defines the movie length.

  UPDATE :

  This is second bounty attempt.

  UPDATE 2 :

  Actually,my audio timestamp of den,num was wrong,while 1,1 is indeed the way to go,as explained by the answer. What was preventing it from working was a bug in this line (my bad) :

       mAudioOutStream.next_pts += a_pkt.pts;

  Which must be :

       mAudioOutStream.next_pts = a_pkt.pts;

  The bug resulted in exponential increment of pts,which caused very early reach to the end of stream (in terms of pts) and therefore caused the audio stream to be terminated much earlier than it supposed to be.