Recherche avancée

Sur d’autres sites (5471)

• Best intra-frame codec for editing ? Having color issues with DNxHR and serious sync issues with Prores

  23 janvier 2020, par Raulo1985

  I have a couple of questions regarding intra-frame codecs for editing purposes (in Premiere Pro).

  A LITTLE CONTEXT :

  I ripped a HDR movie to a MP4 container some months ago, using H.264 as codec (high 10, level 5, UHD, YUV, subsampling 4:2:0). The video looks great, as it should. Now I want to edit a trailer for that movie (I edit in Adobe Premiere Pro, current version), and for fast playback I need to work with proxies or transcode the source file to an intra-frame non long GOP file and use it as the source file (hopefully as lossless as that transcoding step can be). I tried for several days to transcode the source file to DNxHR 444 10 bits (using FFmpeg and then Adobe Media Encoder), but the result was always a video with the colors messed up (sometimes very washed out, sometimes over saturated).

  FFprobe of the resulting DNxHR file said that the color space was BT709 (source file is obviously BT2020), and I don’t know why. The transcoding involved upsampling since the source file is 4:2:0 and DNxHR doesn’t support it, but I tried upsampling to 4:4:4 and also to 4:2:2, and both of those files looked exactly the same to me, and very different from the original footage (so, I don’t think upsampling is the cause of the color issue, but maybe the apparent color space change or something wrong with the metadata). The results were the same when transcoding with Adobe Media Encoder. Anyway, I seem to have given up transcoding to DNxHR and use it as the source file, unless someone has an idea of what’s causing this problem. I could have worked with the source file for exporting and DNxHR LB for proxies, but there were sync issues (between source file and proxy) that defeated all purposes while editing. Prores is out of the picture, sync issues were worse (several seconds of delay).

  For the record, the command used that didn’t work as expected (color wise) is :

  ffmpeg -channel_layout 63 -i input.mkv -map 0:0 -c:v dnxhd -vf "scale=in_range=limited:out_range=full" -color_range 2 -profile:v dnxhr_444 -pix_fmt yuv444p10le -acodec pcm_s24le -ar 48000 -ac 6 -channel_layout 63 -map 0:2 -hide_banner output.mxf

  I also tried without the commands "scale=in_range=limited:out_range=full" and "-color_range 2", with same results. Always used FFmpeg latest version, and I’m working in Windows 10 Pro, latest drivers and latest Klite codec pack. Video files were compared with Mediainfo, FFprobe, and visually with VLC.

  Well, like I said, I’m giving up using DNxHR as the source file for my project (it would have been ideal since it doesn’t have sync issues with the DNxHR proxies, and file size is not a problem for me). A user here at the forums suggested transcoding and use H.264 intra-frame as source file, which I didn’t know was an option (I didn’t know H.264 was capable of intra-frame, my bad). I’m aware that one should avoid unnecesary transcoding steps, but I can’t work with a H.264 UHD HDR source file (ultra slow playback), and the sync issues with proxies, no matter the codec, make it impossible to make accurate cuts.
  So, bottom line, I need to find a way to fix the color issue when transcoding to DNxHR, or try with an inter-frame codec that’s not DNxHR and that’s capable of preserving all the HDR info (and then see if it doesn’t have sync issues. I’m assuming that those may dissapear when using intra-frame for both source file and proxies).

  NOTE : When importing the DNxHR 444 file to Premiere Pro and looking at the Lumetri scopes tab, you can tell that the colors are clipped at 100 nits, like a regular SDR video. So apparently the color space was really reduced to BT709, and I don’t know why. The H.264 source file behaves as expected, with colors going past 100 nits.

  MY QUESTIONS :

  1) Is H.264 intra-frame a good format for editing with a good playback performance ?

  2) If H.264 intra-frame is a good option, what would be the advantages of the DNxHR codec (or Prores) over it for editing purposes ? Everybody suggests DNxHD/DNxHR or Prores as intermediate codecs, but if intra-frame H.264 has the same advantages for editing and supports HDR, what would be the reason to choose another codec like DNxHR ?

  3) Any ideas on what could be the cause of the colors not transcoding correctly from the H.264 source file to the DNxHR 444 10 bits one ? The command looks ok to me, but FFprobe output says the DNxHR video is BT709, while with the source file it says BT2020. Like I said, apparently there’s something wrong with the transcoding process regarding metadata or color space.

  4) I haven’t tried to transcode the source file to a DNxHR 444 10 bits video file, but in a MOV container. I don’t know how this works internally, but maybe the color issue has something to do with the container metadata or something. I may try this if there’s not another suggestion (transcoding this kind of files, as you know, takes time, so I’ll wait for some ideas first).

  NOTE : I tried to transcode the source file in the same way (DNxHR 444, 10 bits, etc) with Adobe Media Encoder 2020 and the result was the same, with colors messed up and FFprobe saying the video is BT709. Also tried transcoding to DNxHR HQX profile (10 bits), same result.

  Any help would be greatly appreciated.

  Thanks in advance.

• Small Time DevOps

  1er janvier 2021, par Multimedia Mike — General

  When you are a certain type of nerd who has been on the internet for long enough, you might run the risk of accumulating a lot of projects and websites. Website-wise, I have this multimedia.cx domain on which I host a bunch of ancient static multimedia documents as well as this PHP/MySQL-based blog. Further, there are 3 other PHP/MySQL-based blogs hosted on subdomains. Also, there is the wiki, another PHP/MySQL web app. A few other custom PHP- and Python-based apps are running around on the server as well.

  While things largely run on auto-pilot, I need to concern myself every now and then with their ongoing upkeep.

  If you ask N different people about the meaning of the term ‘DevOps’, you will surely get N different definitions. However, whenever I have to perform VM maintenance, I like to think I am at least dipping my toes into the DevOps domain. At the very least, the job seems to be concerned with making infrastructure setup and upgrades reliable and repeatable.

  Even if it’s not fully automated, at the very least, I have generated a lot of lists for how to make things work (I’m a big fan of Trello’s Kanban boards for this), so it gets easier every time (ideally, anyway).

  Infrastructure History

  For a solid decade, from 2004 to 2014, everything was hosted on shared, cPanel-based web hosting. In mid-2014, I moved from the shared hosting over to my own VPSs, hosted on DigitalOcean. I must have used Ubuntu 14.04 at the time, as I look down down the list of Ubuntu LTS releases. It was with much trepidation that I undertook this task (knowing that anything that might go wrong with the stack, from the OS up to the apps, would all be firmly my fault), but it turned out not to be that bad. The earliest lesson you learn for such a small-time setup is to have a frontend VPS (web server) and a backend VPS (database server). That way, a surge in HTTP requests has no chance of crashing the database server due to depleted memory.

  At the end of 2016, I decided to refresh the VMs. I brought them up to Ubuntu 16.04 at the time.

  Earlier this year, I decided it would be a good idea to refresh the VMs again since it had been more than 3 years. The VMs were getting long in the tooth. Plus, I had seen an article speculating that Azure, another notable cloud hosting environment, might be getting full. It made me feel like I should grab some resources while I still could (resource-hoarding was in this year).

  I decided to use 18.04 for these refreshed VMs, even though 20.04 was available. I think I was a little nervous about 20.04 because I heard weird things about something called snap packages being the new standard for distributing software for the platform and I wasn’t ready to take that plunge.

  Which brings me to this month’s VM refresh in which I opted to take the 20.04 plunge.

  Oh MediaWiki

  I’ve been the maintainer and caretaker of the MultimediaWiki for 15 years now (wow ! Where does the time go ?). It doesn’t see a lot of updating these days, but I know it still serves as a resource for lots of obscure technical multimedia information. I still get requests for new accounts because someone has uncovered some niche technical data and wants to make sure it gets properly documented.

  MediaWiki is quite an amazing bit of software and it undergoes constant development and improvement. According to the version history, I probably started the MultimediaWiki with the 1.5 series. As of this writing, 1.35 is the latest and therefore greatest lineage.

  This pace of development can make it a bit of a chore to keep up to date. This was particularly true in the old days of the shared hosting when you didn’t have direct shell access and so it’s something you put off for a long time.

  Honestly, to be fair, the upgrade process is pretty straightforward :

  1. Unpack a set of new files on top of the existing tree
  2. Run a PHP script to perform any database table upgrades

  Pretty straightforward, assuming that there are no hiccups along the way, right ? And the vast majority of the time, that’s the case. Until it’s not. I had an upgrade go south about a year and a half ago (I wasn’t the only MW installation to have the problem at the time, I learned). While I do have proper backups, it still threw me for a loop and I worked for about an hour to restore the previous version of the site. That experience understandably left me a bit gun-shy about upgrading the wiki.

  But upgrades must happen, especially when security notices come out. Eventually, I created a Trello template with a solid, 18-step checklist for upgrading MW as soon as a new version shows up. It’s still a chore, just not so nerve-wracking when the steps are all enumerated like that.

  As I compose the post, I think I recall my impetus for wanting to refresh from the 16.04 VM. 16.04 used PHP 7.0. I wanted to upgrade to the latest MW, but if I tried to do so, it warned me that it needed PHP 7.4. So I initialized the new 18.04 VM as described above… only to realize that PHP 7.2 is the default on 18.04. You need to go all the way to 20.04 for 7.4 standard. I’m sure it’s possible to install later versions of PHP on 16.04 or 18.04, but I appreciate going with the defaults provided by the distro.

  I figured I would just stay with MediaWiki 1.34 series and eschew 1.35 series (requiring PHP 7.4) for the time being… until I started getting emails that 1.34 would go end-of-life soon. Oh, and there are some critical security updates, but those are only for 1.35 (and also 1.31 series which is still stubbornly being maintained for some reason).

  So here I am with a fresh Ubuntu 20.04 VM running PHP 7.4 and MediaWiki 1.35 series.

  How Much Process ?

  Anyone who decides to host on VPSs vs, say, shared hosting is (or ought to be) versed on the matter that all your data is your own problem and that glitches sometimes happen and that your VM might just suddenly disappear. (Indeed, I’ve read rants about VMs disappearing and taking entire un-backed-up websites with them, and also watched as the ranters get no sympathy– “yeah, it’s a VM ; the data is your responsibility”) So I like to make sure I have enough notes so that I could bring up a new VM quickly if I ever needed to.

  But the process is a lot of manual steps. Sometimes I wonder if I need to use some automation software like Ansible in order to bring a new VM to life. Why do that if I only update the VM once every 1-3 years ? Well, perhaps I should update more frequently in order to ensure the process is solid ?

  Seems like a lot of effort for a few websites which really don’t see much traffic in the grand scheme of things. But it still might be an interesting exercise and might be good preparation for some other websites I have in mind.

  Besides, if I really wanted to go off the deep end, I would wrap everything up in containers and deploy using D-O’s managed Kubernetes solution.

  The post Small Time DevOps first appeared on Breaking Eggs And Making Omelettes.

• How to add new pages and menu items to Piwik – Introducing the Piwik Platform

  11 septembre 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a widget). This time you’ll learn how to extend Piwik by adding new pages and menu items. For this tutorial you will need to have basic knowledge of PHP and optionally of Twig which is the template engine we use.

  What can be displayed in a page ?

  To make it short : You can display any corporate related content, key metrics, news, help pages, custom reports, contact details, information about your server, forms to manage any data and anything else.

  Getting started

  In this series of posts, we assume that you have already set up your development environment. If not, visit the Piwik Developer Zone where you’ll find the tutorial Setting up Piwik.

  To summarize the things you have to do to get setup :

  • Install Piwik (for instance via git).
  • Activate the developer mode : ./console development:enable --full.
  • Generate a plugin : ./console generate:plugin --name="MyControllerPlugin". There should now be a folder plugins/MyControllerPlugin.
  • And activate the created plugin under Settings => Plugins.

  Let’s start creating a page

  We start by using the Piwik Console to create a new page :

  ./console generate:controller

  The command will ask you to enter the name of the plugin the controller should belong to. I will simply use the above chosen plugin name “MyControllerPlugin”. There should now be two files plugins/MyControllerPlugin/Controller.php and plugins/MyControllerPlugin/templates/index.twig which both already contain an example to get you started easily :

  Controller.php

  class Controller extends \Piwik\Plugin\Controller
  {
      public function index()
      {
          return $this->renderTemplate('index', array(
               'answerToLife' => 42
          ));
      }
  }
  Télécharger

  and templates/index.twig

  {% extends 'dashboard.twig' %}
   
  {% block content %}
      <strong>Hello world!</strong>
      <br/>
   
      The answer to life is {{ answerToLife }}
  {% endblock %}
  Télécharger

  Note : If you are generating the Controller before Piwik 2.7.0 the example will look slightly different.

  The controller action index assigns the view variable answerToLife to the view and renders the Twig template templates/index.twig. Any variable assigned this way can then be used in the view using for example {{ answerToLife }}.

  Using a Twig template to generate the content of your page is actually optional : instead feel free to generate any content as desired and return a string in your controller action.

  As the above template index.twig is extending the dashboard template the Logo as well as the top menu will automatically appear on top of your content which is defined within the block content.

  

  How to display the page within the admin

  If you would like to add the admin menu on the left you have to modify the following parts :

  • Extend \Piwik\Plugin\ControllerAdmin instead of \Piwik\Plugin\Controller in the file Controller.php. In a future version of Piwik this step will be no longer neccessary, see #6151
  • Extend the template admin.twig instead of dashboard.twig
  • Define a headline using an H2-element

  {% extends 'admin.twig' %}
   
  {% block content %}
      <h2>Hello world!</h2>
      <br/>
   
      The answer to life is {{ answerToLife }}
  {% endblock %}
  Télécharger

  Note : Often one needs to add a page to the admin to make a plugin configurable. We have a unified solution for this using the Settings API.

  

  How to display a blank page

  If you would like to generate a blank page that shows only your content the template should contain only your markup as follows :

  <strong>Hello world!</strong>
  <br/>
   
  The answer to life is {{ answerToLife }}
  Télécharger

  Predefined variables, UI components, security and accessing query parameters

  In this blog post we only cover the basics to get you started. We highly recommend to read the MVC guide on our developer pages which covers some of those advanced topics. For instance you might be wondering how to securely access $_GET or $_POST parameters, you might want to restrict the content of your page depending on a user role, and much more.

  If you would like to know how to make use of JavaScript, CSS and Less have a look at our Working with Piwik’s UI guide.

  Note : How to include existing UI components such as a site selector or a date selector will be covered in a future blog post. Also, there are default variables assigned to the view depending on the context. A list of those variables that may or may not be defined is unfortunately not available yet but we will catch up on this.

  Let’s add a menu item to make the page accessible

  So far you have created a page but you can still not access it. Therefore we need to add a menu item to one of the Piwik menus. We start by using the Piwik Console to create a menu template :

  ./console generate:menu

  The command will ask you to enter the name of the plugin the menu should belong to. I will use again the above chosen plugin name “MyControllerPlugin”. There should now be a file plugins/MyControllerPlugin/Menu.php which contains an example to get you started easily :

  Menu.php

  class Menu extends \Piwik\Plugin\Menu
  {
      public function configureUserMenu(MenuUser $menu)
      {
          // reuse an existing category. 
          $menu->addManageItem('My User Item', $this->urlForAction('showList'));
   
          // or create a custom category
          $menu->addItem('My Custom Category', 'My User Item', $this->urlForDefaultAction());
      }
  }
  Télécharger

  This is only a part of the generated template since all the examples of the different menus are similar. You can add items to four menus :

  • configureReportingMenu To add a new item to the reporting menu which includes all the reports like “Actions” and “Visitors”.
  • configureAdminMenu To add a new item to the admin menu which includes items like “User settings” and “Websites”.
  • configureTopMenu To add a new item to the top menu which includes items like “All Websites” and “Logout”.
  • configureUserMenu To add a new item to the user menu which is accessible when clicking on the username on the top right.

  In this blog post we will add a new item to the user menu and to do so we adjust the generated template like this :

  class Menu extends \Piwik\Plugin\Menu
  {
      public function configureUserMenu(MenuUser $menu)
      {
          $menu->addManageItem('My User Item', $this->urlForAction($method = 'index'), $orderId = 30);
      }
  }
  Télécharger

  That’s it. This will add a menu item named “My User Item” to the “Manage” section of the user menu. When a user chooses the menu item, the “index” method of your controller will be executed and your previously created page will be first rendered and then displayed. Optionally, you can define an order to influence the position of the menu item within the manage section. Following this example you can add an item to any menu for any action. I think you get the point !

  

  Note : In Piwik 2.6.0 and before the above example would look like this :

  class Menu extends \Piwik\Plugin\Menu
  {
      public function configureUserMenu(MenuUser $menu)
      {
          $menu->addManageItem('My User Item', array($module = 'MyControllerPlugin', $action = 'index'), $orderId = 30);
      }
  }
  Télécharger

  How to test a page

  After you have created your page you are surely wondering how to test it. A controller should be usually very simple as it is only the connector between model and view. Therefore, we do usually not create unit or integration test for controllers and for the view less than ever. Instead we would create a UI test that takes a screenshot of your page and compares it with an expected screenshot. Luckily, there is already a section UI tests in our Automated tests guide.

  Publishing your Plugin on the Marketplace

  In case you want to share your page with other Piwik users you can do this by pushing your plugin to a public GitHub repository and creating a tag. Easy as that. Read more about how to distribute a plugin.

  Advanced features

  Isn’t it easy to create a page ? We never even created a file ! Of course, based on our API design principle “The complexity of our API should never exceed the complexity of your use case.” you can accomplish more if you want : You can make use of Vanilla JavaScript, jQuery, AngularJS, Less and CSS, you can reuse UI components, you can access query parameters and much more.

  Would you like to know more about this ? Go to our MVC (Model-View-Controller) and Working with Piwik’s UI guides in the Piwik Developer Zone.

  If you have any feedback regarding our APIs or our guides in the Developer Zone feel free to send it to us.