Recherche avancée
Médias (1)
-
Somos millones 1
21 juillet 2014, par
Mis à jour : Juin 2015
Langue : français
Type : Video
Autres articles (50)
-
MediaSPIP Core : La Configuration
9 novembre 2010, parMediaSPIP Core fournit par défaut trois pages différentes de configuration (ces pages utilisent le plugin de configuration CFG pour fonctionner) : une page spécifique à la configuration générale du squelettes ; une page spécifique à la configuration de la page d’accueil du site ; une page spécifique à la configuration des secteurs ;
Il fournit également une page supplémentaire qui n’apparait que lorsque certains plugins sont activés permettant de contrôler l’affichage et les fonctionnalités spécifiques (...) -
Demande de création d’un canal
12 mars 2010, parEn fonction de la configuration de la plateforme, l’utilisateur peu avoir à sa disposition deux méthodes différentes de demande de création de canal. La première est au moment de son inscription, la seconde, après son inscription en remplissant un formulaire de demande.
Les deux manières demandent les mêmes choses fonctionnent à peu près de la même manière, le futur utilisateur doit remplir une série de champ de formulaire permettant tout d’abord aux administrateurs d’avoir des informations quant à (...) -
Mediabox : ouvrir les images dans l’espace maximal pour l’utilisateur
8 février 2011, parLa visualisation des images est restreinte par la largeur accordée par le design du site (dépendant du thème utilisé). Elles sont donc visibles sous un format réduit. Afin de profiter de l’ensemble de la place disponible sur l’écran de l’utilisateur, il est possible d’ajouter une fonctionnalité d’affichage de l’image dans une boite multimedia apparaissant au dessus du reste du contenu.
Pour ce faire il est nécessaire d’installer le plugin "Mediabox".
Configuration de la boite multimédia
Dès (...)
Sur d’autres sites (3019)
-
Failed to decode h264 key frame with DXVA2.0 because returned buffer is to small
16 mai 2023, par grill2010I hava a strange problem on Windows with DXVA2 h264 decoding. I recently figured out a ffmpeg decoding limitation for DXVA2 and D3D11VA on Windows and how to solve it, this solution completly fixes the problem with D3D11VA but DXVA2 still has some problems with certain keyframes. Upon further investigation it turned out that the decoding of these certain keyframes fail because the buffer returned from the IDirectXVideoDecoder_GetBuffer function was too small. FFmpeg is printing out these logs when the decoding fails :


Error: [h264 @ 0000028b2e5796c0] Buffer for type 5 was too small. size: 58752, dxva_size: 55296
Error: [h264 @ 0000028b2e5796c0] Failed to add bitstream or slice control buffer
Error: [h264 @ 0000028b2e5796c0] hardware accelerator failed to decode picture


Why is this returned buffer too low ? What kind of factors inside ffmpeg do have an effect on this buffer size or is this a limitation of DXVA2 in general ? All other decoders like Cuvid, D3D11VA or the software decoder are not affected by this problem and can decode all keyframes.


I have an example javacv project on github that can reproduce the problem. I also provide the source code of the main class here. The keyframe example data with prepended SPS and PPS in hex form can be downloaded here.


import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.layout.Pane;
import javafx.scene.layout.StackPane;
import javafx.stage.Stage;
import org.bytedeco.ffmpeg.avcodec.AVCodec;
import org.bytedeco.ffmpeg.avcodec.AVCodecContext;
import org.bytedeco.ffmpeg.avcodec.AVCodecHWConfig;
import org.bytedeco.ffmpeg.avcodec.AVPacket;
import org.bytedeco.ffmpeg.avutil.AVBufferRef;
import org.bytedeco.ffmpeg.avutil.AVDictionary;
import org.bytedeco.ffmpeg.avutil.AVFrame;
import org.bytedeco.ffmpeg.avutil.LogCallback;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.Pointer;
import org.tinylog.Logger;

import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Objects;
import java.util.function.Consumer;

import static org.bytedeco.ffmpeg.avcodec.AVCodecContext.FF_THREAD_SLICE;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avutil.*;

public class App extends Application {

 /**** decoder variables ****/

 private AVHWContextInfo hardwareContext;

 private AVCodec decoder;
 private AVCodecContext m_VideoDecoderCtx;

 private AVCodecContext.Get_format_AVCodecContext_IntPointer formatCallback;

 private final int streamResolutionX = 1920;
 private final int streamResolutionY = 1080;

 // AV_HWDEVICE_TYPE_CUDA // example works with cuda
 // AV_HWDEVICE_TYPE_DXVA2 // producing Invalid data found on keyframe
 // AV_HWDEVICE_TYPE_D3D11VA // producing Invalid data found on keyframe
 private static final int HW_DEVICE_TYPE = AV_HWDEVICE_TYPE_DXVA2;

 private static final boolean USE_HW_ACCEL = true;

 private static final boolean USE_AV_EF_EXPLODE = true;

 public static void main(final String[] args) {
 //System.setProperty("prism.order", "d3d,sw");
 System.setProperty("prism.vsync", "false");
 Application.launch(App.class);
 }

 @Override
 public void start(final Stage primaryStage) {
 final Pane dummyPane = new Pane();
 dummyPane.setStyle("-fx-background-color: black");
 final Scene scene = new Scene(dummyPane, this.streamResolutionX, this.streamResolutionY);
 primaryStage.setScene(scene);
 primaryStage.show();
 primaryStage.setMinWidth(480);
 primaryStage.setMinHeight(360);

 this.initializeFFmpeg(result -> {
 if (!result) {
 Logger.error("FFmpeg could not be initialized correctly, terminating program");
 System.exit(1);
 return;
 }
 scene.setRoot(new StackPane());
 this.performTestFramesFeeding();
 });
 }

 private void initializeFFmpeg(final Consumer<boolean> finishHandler) {
 FFmpegLogCallback.setLevel(AV_LOG_DEBUG); // Increase log level until the first frame is decoded
 FFmpegLogCallback.set();
 Pointer pointer = new Pointer((Pointer) null);
 AVCodec c;
 while ((c = av_codec_iterate(pointer)) != null) {
 if (av_codec_is_decoder(c) > 0)
 Logger.debug("{}:{} ", c.name().getString(), c.type());
 }

 this.decoder = avcodec_find_decoder(AV_CODEC_ID_H264); // usually decoder name is h264 and without hardware support it's yuv420p otherwise nv12
 if (this.decoder == null) {
 Logger.error("Unable to find decoder for format {}", "h264");
 finishHandler.accept(false);
 return;
 }
 Logger.info("Current decoder name: {}, {}", this.decoder.name().getString(), this.decoder.long_name().getString());

 if (true) {
 for (; ; ) {
 this.m_VideoDecoderCtx = avcodec_alloc_context3(this.decoder);
 if (this.m_VideoDecoderCtx == null) {
 Logger.error("Unable to find decoder for format AV_CODEC_ID_H264");
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 this.hardwareContext = null;
 }
 continue;
 }

 if (App.USE_HW_ACCEL) {
 this.hardwareContext = this.createHardwareContext();
 if (this.hardwareContext != null) {
 Logger.info("Set hwaccel support");
 this.m_VideoDecoderCtx.hw_device_ctx(this.hardwareContext.hwContext()); // comment to disable hwaccel
 }
 } else {
 Logger.info("Hwaccel manually disabled");
 }

 // Always request low delay decoding
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_LOW_DELAY);

 // Allow display of corrupt frames and frames missing references
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_OUTPUT_CORRUPT);
 this.m_VideoDecoderCtx.flags2(this.m_VideoDecoderCtx.flags2() | AV_CODEC_FLAG2_SHOW_ALL);

 if (App.USE_AV_EF_EXPLODE) {
 // Report decoding errors to allow us to request a key frame
 this.m_VideoDecoderCtx.err_recognition(this.m_VideoDecoderCtx.err_recognition() | AV_EF_EXPLODE);
 }

 // Enable slice multi-threading for software decoding
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) { // if not hw accelerated
 this.m_VideoDecoderCtx.thread_type(this.m_VideoDecoderCtx.thread_type() | FF_THREAD_SLICE);
 this.m_VideoDecoderCtx.thread_count(2/*AppUtil.getCpuCount()*/);
 } else {
 // No threading for HW decode
 this.m_VideoDecoderCtx.thread_count(1);
 }

 this.m_VideoDecoderCtx.width(this.streamResolutionX);
 this.m_VideoDecoderCtx.height(this.streamResolutionY);
 this.m_VideoDecoderCtx.pix_fmt(this.getDefaultPixelFormat());

 this.formatCallback = new AVCodecContext.Get_format_AVCodecContext_IntPointer() {
 @Override
 public int call(final AVCodecContext context, final IntPointer pixelFormats) {
 final boolean hwDecodingSupported = context.hw_device_ctx() != null && App.this.hardwareContext != null;
 final int preferredPixelFormat = hwDecodingSupported ?
 App.this.hardwareContext.hwConfig().pix_fmt() :
 context.pix_fmt();
 int i = 0;
 while (true) {
 final int currentSupportedFormat = pixelFormats.get(i++);
 System.out.println("Supported pixel formats " + currentSupportedFormat);
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) {
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == preferredPixelFormat) {
 Logger.info("[FFmpeg]: pixel format in format callback is {}", currentSupportedFormat);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for yuv
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_YUV420P) {
 Logger.info("[FFmpeg]: Not found in first match so use {}", AV_PIX_FMT_YUV420P);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for nv12
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_NV12) {
 Logger.info("[FFmpeg]: Not found in second match so use {}", AV_PIX_FMT_NV12);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 Logger.info("[FFmpeg]: pixel format in format callback is using fallback {}", AV_PIX_FMT_NONE);
 return AV_PIX_FMT_NONE;
 }
 };
 this.m_VideoDecoderCtx.get_format(this.formatCallback);

 final AVDictionary options = new AVDictionary(null);
 final int result = avcodec_open2(this.m_VideoDecoderCtx, this.decoder, options);
 if (result < 0) {
 Logger.error("avcodec_open2 was not successful");
 finishHandler.accept(false);
 return;
 }
 av_dict_free(options);
 break;
 }
 }

 if (this.decoder == null || this.m_VideoDecoderCtx == null) {
 finishHandler.accept(false);
 return;
 }
 finishHandler.accept(true);
 }

 private AVHWContextInfo createHardwareContext() {
 AVHWContextInfo result = null;
 for (int i = 0; ; i++) {
 final AVCodecHWConfig config = avcodec_get_hw_config(this.decoder, i);
 if (config == null) {
 break;
 }

 if ((config.methods() & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX) < 0) {
 continue;
 }
 final int device_type = config.device_type();
 if (device_type != App.HW_DEVICE_TYPE) {
 continue;
 }
 final AVBufferRef hw_context = av_hwdevice_ctx_alloc(device_type);
 if (hw_context == null || av_hwdevice_ctx_create(hw_context, device_type, (String) null, null, 0) < 0) {
 Logger.error("HW accel not supported for type {}", device_type);
 av_free(config);
 av_free(hw_context);
 } else {
 Logger.info("HW accel created for type {}", device_type);
 result = new AVHWContextInfo(config, hw_context);
 }
 break;
 }

 return result;
 }

 @Override
 public void stop() {
 this.releaseNativeResources();
 }

 /*****************************/
 /*** test frame processing ***/
 /*****************************/
 
 private void performTestFramesFeeding() {
 final AVPacket pkt = av_packet_alloc();
 if (pkt == null) {
 return;
 }
 try (final BytePointer bp = new BytePointer(65_535 * 15)) {


 for (int i = 0; i < 1; i++) {
 final byte[] frameData = AVTestFrames.h264KeyTestFrame;

 bp.position(0);

 bp.put(frameData);
 bp.limit(frameData.length);

 pkt.data(bp);
 pkt.capacity(bp.capacity());
 pkt.size(frameData.length);
 pkt.position(0);
 pkt.limit(frameData.length);
 //pkt.flags(AV_PKT_FLAG_KEY);
 final AVFrame avFrame = av_frame_alloc();
 System.out.println("frameData.length " + frameData.length);

 final int err = avcodec_send_packet(this.m_VideoDecoderCtx, pkt); //fill_scaling_lists
 if (err < 0) {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(err, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("Error on decoding test frame " + err + " message " + string);
 av_frame_free(avFrame);
 return;
 }

 final int result = avcodec_receive_frame(this.m_VideoDecoderCtx, avFrame);
 final AVFrame decodedFrame;
 if (result == 0) {
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) {
 decodedFrame = avFrame;
 System.out.println("SUCESS with SW decoding");
 } else {
 final AVFrame hwAvFrame = av_frame_alloc();
 if (av_hwframe_transfer_data(hwAvFrame, avFrame, 0) < 0) {
 System.out.println("Failed to transfer frame from hardware");
 av_frame_unref(hwAvFrame);
 decodedFrame = avFrame;
 } else {
 av_frame_unref(avFrame);
 decodedFrame = hwAvFrame;
 System.out.println("SUCESS with HW decoding");
 }
 }

 av_frame_unref(decodedFrame);
 } else {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(result, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("error " + result + " message " + string);
 av_frame_free(avFrame);
 }
 }
 } finally {
 if (pkt.stream_index() != -1) {
 av_packet_unref(pkt);
 }
 pkt.releaseReference();
 }
 }

 final Object releaseLock = new Object();
 private volatile boolean released = false;

 private void releaseNativeResources() {
 if (this.released) {
 return;
 }
 this.released = true;
 synchronized (this.releaseLock) {
 // Close the video codec
 if (this.m_VideoDecoderCtx != null) {
 avcodec_free_context(this.m_VideoDecoderCtx);
 this.m_VideoDecoderCtx = null;
 }

 // close the format callback
 if (this.formatCallback != null) {
 this.formatCallback.close();
 this.formatCallback = null;
 }

 // close hw context
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 }
 }
 }

 private int getDefaultPixelFormat() {
 return AV_PIX_FMT_YUV420P; // Always return yuv420p here
 }


 /*********************/
 /*** inner classes ***/
 /*********************/

 public static final class HexUtil {

 private HexUtil() {
 }

 public static byte[] unhexlify(final String argbuf) {
 final int arglen = argbuf.length();
 if (arglen % 2 != 0) {
 throw new RuntimeException("Odd-length string");
 } else {
 final byte[] retbuf = new byte[arglen / 2];

 for (int i = 0; i < arglen; i += 2) {
 final int top = Character.digit(argbuf.charAt(i), 16);
 final int bot = Character.digit(argbuf.charAt(i + 1), 16);
 if (top == -1 || bot == -1) {
 throw new RuntimeException("Non-hexadecimal digit found");
 }

 retbuf[i / 2] = (byte) ((top << 4) + bot);
 }

 return retbuf;
 }
 }
 }

 public static final class AVHWContextInfo {
 private final AVCodecHWConfig hwConfig;
 private final AVBufferRef hwContext;

 private volatile boolean freed = false;

 public AVHWContextInfo(final AVCodecHWConfig hwConfig, final AVBufferRef hwContext) {
 this.hwConfig = hwConfig;
 this.hwContext = hwContext;
 }

 public AVCodecHWConfig hwConfig() {
 return this.hwConfig;
 }

 public AVBufferRef hwContext() {
 return this.hwContext;
 }

 public void free() {
 if (this.freed) {
 return;
 }
 this.freed = true;
 av_free(this.hwConfig);
 av_free(this.hwContext);
 }


 @Override
 public boolean equals(Object o) {
 if (this == o) return true;
 if (o == null || getClass() != o.getClass()) return false;
 AVHWContextInfo that = (AVHWContextInfo) o;
 return freed == that.freed && Objects.equals(hwConfig, that.hwConfig) && Objects.equals(hwContext, that.hwContext);
 }

 @Override
 public int hashCode() {
 return Objects.hash(hwConfig, hwContext, freed);
 }

 @Override
 public String toString() {
 return "AVHWContextInfo[" +
 "hwConfig=" + this.hwConfig + ", " +
 "hwContext=" + this.hwContext + ']';
 }
 }

 public static final class AVTestFrames {

 private AVTestFrames() {

 }

 static {
 InputStream inputStream = null;
 try {
 inputStream = AVTestFrames.class.getClassLoader().getResourceAsStream("h264_test_key_frame.txt");
 final byte[] h264TestFrameBuffer = inputStream == null ? new byte[0] : inputStream.readAllBytes();
 final String h264TestFrame = new String(h264TestFrameBuffer, StandardCharsets.UTF_8);
 AVTestFrames.h264KeyTestFrame = HexUtil.unhexlify(h264TestFrame);
 } catch (final IOException e) {
 Logger.error(e, "Could not parse test frame");
 } finally {
 if (inputStream != null) {
 try {
 inputStream.close();
 } catch (final IOException e) {
 Logger.error(e, "Could not close test frame input stream");
 }
 }
 }
 }

 public static byte[] h264KeyTestFrame;
 }

 public static class FFmpegLogCallback extends LogCallback {

 private static final org.bytedeco.javacpp.tools.Logger logger = org.bytedeco.javacpp.tools.Logger.create(FFmpegLogCallback.class);

 static final FFmpegLogCallback instance = new FFmpegLogCallback().retainReference();

 public static FFmpegLogCallback getInstance() {
 return instance;
 }

 /**
 * Calls {@code avutil.setLogCallback(getInstance())}.
 */
 public static void set() {
 setLogCallback(getInstance());
 }

 /**
 * Returns {@code av_log_get_level()}.
 **/
 public static int getLevel() {
 return av_log_get_level();
 }

 /**
 * Calls {@code av_log_set_level(level)}.
 **/
 public static void setLevel(int level) {
 av_log_set_level(level);
 }

 @Override
 public void call(int level, BytePointer msg) {
 switch (level) {
 case AV_LOG_PANIC, AV_LOG_FATAL, AV_LOG_ERROR -> logger.error(msg.getString());
 case AV_LOG_WARNING -> logger.warn(msg.getString());
 case AV_LOG_INFO -> logger.info(msg.getString());
 case AV_LOG_VERBOSE, AV_LOG_DEBUG, AV_LOG_TRACE -> logger.debug(msg.getString());
 default -> {
 assert false;
 }
 }
 }
 }
}
</boolean>

-
FFmpeg invalid data found when processing input with D3D11VA and DXVA2 hw acceleration
6 mai 2023, par grill2010I'm currently porting my Android streaming app to Windows and for decoding the h264 video stream I use FFmpeg with possible hardware acceleration. Last two weeks I was reading a lot of documentation and studied a lot of examples on the internet. For my project I use JavaCV which is internally using FFmpeg 5.1.2. On Windows I support D3D11VA, DXVA2 and Cuvid for hardware acceleration (and software decoding as fallback). During testing I noticed that I get some strange artefacts when using D3D11VA or DXVA2 hw acceleration. Upon further investigation I saw that I receive a lot of


"Invalid data found when processing input"


errors when calling
avcodec_send_packet. It seems this error only occurs on certain key frames. The error is reproducable all the time. The software decoder or cuvid decoder has absolutely no problem to process and to decode such a frame, so not sure why there should be an invalid data in the frame ? I played around a lot with the decoder configuration but nothing seems to help and at that point I think this is definitely not normal behaviour.

I provided a reproducable example which can be downloaded from here. All the important part is in the App.java class. In addition an example of the code was posted below. The example is trying to decode a key frame. The keyframe data with sps and pps is read from a file in the resource folder of the project.


To run the project just perform a .\gradlew build and afterwards a .\gradlew run. If you run the example the last log message shown in the terminal should be "SUCESS with HW decoding". The hardware decoder can be changed via the HW_DEVICE_TYPE variable in the App.java class. To disable hw acceleration just set the USE_HW_ACCEL to false.


For me everything seems to be correct and I have no idea what could be wrong with the code. I looked a lot on the internet to find the root cause of the issue and I did not really found a solution but other sources which are related to (maybe) the same problem


https://www.mail-archive.com/libav-user@ffmpeg.org/...


https://stackoverflow.com/questions/67307397/ffmpeg-...


I also found another streaming app on Windows which can use D3D11VA and DXVA2 hardware acceleration called Chiaki (it requires a PS4 or a PS5) which seems to have the exact same problem. I used the build provided here. It will fail to decode certain key frames as well when hardware acceleration with D3D11VA or DXVA2 is selected (e.g. the first key frame received by the stream). Chiaki can output the seemingly faulty frame but this is also possible with my example by setting the USE_AV_EF_EXPLODE to false.


Are there any ffmpeg gurus around that can check what's the problem with D3D11VA or DXVA2 ? Anything else that needs to be done to make the D3D11VA and DXVA2 hardware decoder work ? I'm now completly out of ideas and I'm not even sure if this is fixable.


I have Windows 11 installed on my test machine, and I have the latest Nvidea drivers installed.


Edit : here is a shrinked complete example of my project (keyframe file which includes sps and pps can be downloaded from here. It's a hex string file and can be decoded with the provided HexUtil class)


import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.layout.Pane;
import javafx.stage.Stage;
import org.bytedeco.ffmpeg.avcodec.AVCodec;
import org.bytedeco.ffmpeg.avcodec.AVCodecContext;
import org.bytedeco.ffmpeg.avcodec.AVCodecHWConfig;
import org.bytedeco.ffmpeg.avcodec.AVPacket;
import org.bytedeco.ffmpeg.avutil.AVBufferRef;
import org.bytedeco.ffmpeg.avutil.AVDictionary;
import org.bytedeco.ffmpeg.avutil.AVFrame;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacv.FFmpegLogCallback;
import org.tinylog.Logger;

import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Objects;
import java.util.function.Consumer;

import static org.bytedeco.ffmpeg.avcodec.AVCodecContext.AV_EF_EXPLODE;
import static org.bytedeco.ffmpeg.avcodec.AVCodecContext.FF_THREAD_SLICE;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avutil.*;

public class App extends Application {

 /**** decoder variables ****/

 private AVHWContextInfo hardwareContext;

 private AVCodec decoder;
 private AVCodecContext m_VideoDecoderCtx;

 private AVCodecContext.Get_format_AVCodecContext_IntPointer formatCallback;
 private final int streamResolutionX = 1920;
 private final int streamResolutionY = 1080;

 // AV_HWDEVICE_TYPE_CUDA // example works with cuda
 // AV_HWDEVICE_TYPE_DXVA2 // producing Invalid data found on keyframe
 // AV_HWDEVICE_TYPE_D3D11VA // producing Invalid data found on keyframe
 private static final int HW_DEVICE_TYPE = AV_HWDEVICE_TYPE_DXVA2;

 private static final boolean USE_HW_ACCEL = true;

 private static final boolean USE_AV_EF_EXPLODE = true;

 public static void main(final String[] args) {
 //System.setProperty("prism.order", "d3d,sw");
 System.setProperty("prism.vsync", "false");
 Application.launch(App.class);
 }

 @Override
 public void start(final Stage primaryStage) {
 final Pane dummyPane = new Pane();
 dummyPane.setStyle("-fx-background-color: black");
 final Scene scene = new Scene(dummyPane, this.streamResolutionX, this.streamResolutionY);
 primaryStage.setScene(scene);
 primaryStage.show();
 primaryStage.setMinWidth(480);
 primaryStage.setMinHeight(360);

 this.initializeFFmpeg(result -> {
 if (!result) {
 Logger.error("FFmpeg could not be initialized correctly, terminating program");
 System.exit(1);
 return;
 }
 this.performTestFramesFeeding();
 });
 }

 private void initializeFFmpeg(final Consumer<boolean> finishHandler) {
 FFmpegLogCallback.setLevel(AV_LOG_DEBUG); // Increase log level until the first frame is decoded
 //FFmpegLogCallback.setLevel(AV_LOG_QUIET);
 this.decoder = avcodec_find_decoder(AV_CODEC_ID_H264); // usually decoder name is h264 and without hardware support it's yuv420p otherwise nv12

 if (this.decoder == null) {
 Logger.error("Unable to find decoder for format {}", "h264");
 finishHandler.accept(false);
 return;
 }
 Logger.info("Current decoder name: {}, {}", this.decoder.name().getString(), this.decoder.long_name().getString());

 if (true) {
 for (; ; ) {
 this.m_VideoDecoderCtx = avcodec_alloc_context3(this.decoder);
 if (this.m_VideoDecoderCtx == null) {
 Logger.error("Unable to find decoder for format AV_CODEC_ID_H264");
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 this.hardwareContext = null;
 }
 continue;
 }

 if (App.USE_HW_ACCEL) {
 this.hardwareContext = this.createHardwareContext();
 if (this.hardwareContext != null) {
 Logger.info("Set hwaccel support");
 this.m_VideoDecoderCtx.hw_device_ctx(this.hardwareContext.hwContext()); // comment to disable hwaccel
 }
 } else {
 Logger.info("Hwaccel manually disabled");
 }


 // Always request low delay decoding
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_LOW_DELAY);

 // Allow display of corrupt frames and frames missing references
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_OUTPUT_CORRUPT);
 this.m_VideoDecoderCtx.flags2(this.m_VideoDecoderCtx.flags2() | AV_CODEC_FLAG2_SHOW_ALL);

 if (App.USE_AV_EF_EXPLODE) {
 // Report decoding errors to allow us to request a key frame
 this.m_VideoDecoderCtx.err_recognition(this.m_VideoDecoderCtx.err_recognition() | AV_EF_EXPLODE);
 }

 // Enable slice multi-threading for software decoding
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) { // if not hw accelerated
 this.m_VideoDecoderCtx.thread_type(this.m_VideoDecoderCtx.thread_type() | FF_THREAD_SLICE);
 this.m_VideoDecoderCtx.thread_count(2/*AppUtil.getCpuCount()*/);
 } else {
 // No threading for HW decode
 this.m_VideoDecoderCtx.thread_count(1);
 }

 this.m_VideoDecoderCtx.width(this.streamResolutionX);
 this.m_VideoDecoderCtx.height(this.streamResolutionY);
 this.m_VideoDecoderCtx.pix_fmt(this.getDefaultPixelFormat());

 this.formatCallback = new AVCodecContext.Get_format_AVCodecContext_IntPointer() {
 @Override
 public int call(final AVCodecContext context, final IntPointer pixelFormats) {
 final boolean hwDecodingSupported = context.hw_device_ctx() != null && App.this.hardwareContext != null;
 final int preferredPixelFormat = hwDecodingSupported ?
 App.this.hardwareContext.hwConfig().pix_fmt() :
 context.pix_fmt();
 int i = 0;
 while (true) {
 final int currentSupportedFormat = pixelFormats.get(i++);
 System.out.println("Supported pixel formats " + currentSupportedFormat);
 if (currentSupportedFormat == preferredPixelFormat) {
 Logger.info("[FFmpeg]: pixel format in format callback is {}", currentSupportedFormat);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for yuv
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_YUV420P) {
 Logger.info("[FFmpeg]: Not found in first match so use {}", AV_PIX_FMT_YUV420P);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for nv12
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_NV12) {
 Logger.info("[FFmpeg]: Not found in second match so use {}", AV_PIX_FMT_NV12);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 Logger.info("[FFmpeg]: pixel format in format callback is using fallback {}", AV_PIX_FMT_NONE);
 return AV_PIX_FMT_NONE;
 }
 };
 this.m_VideoDecoderCtx.get_format(this.formatCallback);

 final AVDictionary options = new AVDictionary(null);
 final int result = avcodec_open2(this.m_VideoDecoderCtx, this.decoder, options);
 if (result < 0) {
 Logger.error("avcodec_open2 was not successful");
 finishHandler.accept(false);
 return;
 }
 av_dict_free(options);
 break;
 }
 }

 if (this.decoder == null || this.m_VideoDecoderCtx == null) {
 finishHandler.accept(false);
 return;
 }
 finishHandler.accept(true);
 }

 private AVHWContextInfo createHardwareContext() {
 AVHWContextInfo result = null;
 for (int i = 0; ; i++) {
 final AVCodecHWConfig config = avcodec_get_hw_config(this.decoder, i);
 if (config == null) {
 break;
 }

 if ((config.methods() & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX) < 0) {
 continue;
 }
 final int device_type = config.device_type();
 if (device_type != App.HW_DEVICE_TYPE) {
 continue;
 }
 final AVBufferRef hw_context = av_hwdevice_ctx_alloc(device_type);
 if (hw_context == null || av_hwdevice_ctx_create(hw_context, device_type, (String) null, null, 0) < 0) {
 Logger.error("HW accel not supported for type {}", device_type);
 av_free(config);
 av_free(hw_context);
 } else {
 Logger.info("HW accel created for type {}", device_type);
 result = new AVHWContextInfo(config, hw_context);
 }
 break;
 }

 return result;
 }

 @Override
 public void stop() {
 this.releaseNativeResources();
 }

 /************************/
 /*** video processing ***/
 /************************/


 private void performTestFramesFeeding() {
 final AVPacket pkt = av_packet_alloc();
 if (pkt == null) {
 return;
 }
 try (final BytePointer bp = new BytePointer(65_535 * 4)) {
 final byte[] frameData = AVTestFrames.h264KeyTestFrame;


 bp.position(0);

 bp.put(frameData);
 bp.limit(frameData.length);

 pkt.data(bp);
 pkt.capacity(bp.capacity());
 pkt.size(frameData.length);
 pkt.position(0);
 pkt.limit(frameData.length);
 final AVFrame avFrame = av_frame_alloc();

 final int err = avcodec_send_packet(this.m_VideoDecoderCtx, pkt); // this will fail with D3D11VA and DXVA2
 if (err < 0) {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(err, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("Error on decoding test frame " + err + " message " + string);
 av_frame_free(avFrame);
 return;
 }

 final int result = avcodec_receive_frame(this.m_VideoDecoderCtx, avFrame);
 final AVFrame decodedFrame;
 if (result == 0) {
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) {
 decodedFrame = avFrame;
 av_frame_unref(decodedFrame);
 System.out.println("SUCESS with SW decoding");
 } else {
 final AVFrame hwAvFrame = av_frame_alloc();
 if (av_hwframe_transfer_data(hwAvFrame, avFrame, 0) < 0) {
 System.out.println("Failed to transfer frame from hardware");
 av_frame_unref(hwAvFrame);
 decodedFrame = avFrame;
 } else {
 av_frame_unref(avFrame);
 decodedFrame = hwAvFrame;
 System.out.println("SUCESS with HW decoding");
 }
 av_frame_unref(decodedFrame);
 }
 } else {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(result, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("error " + result + " message " + string);
 av_frame_free(avFrame);
 }
 } finally {
 if (pkt.stream_index() != -1) {
 av_packet_unref(pkt);
 }
 pkt.releaseReference();
 }
 }

 final Object releaseLock = new Object();
 private volatile boolean released = false;

 private void releaseNativeResources() {
 if (this.released) {
 return;
 }
 this.released = true;
 synchronized (this.releaseLock) {
 // Close the video codec
 if (this.m_VideoDecoderCtx != null) {
 avcodec_free_context(this.m_VideoDecoderCtx);
 this.m_VideoDecoderCtx = null;
 }

 // close the format callback
 if (this.formatCallback != null) {
 this.formatCallback.close();
 this.formatCallback = null;
 }

 // close hw context
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 }
 }
 }

 private int getDefaultPixelFormat() {
 return AV_PIX_FMT_YUV420P; // Always return yuv420p here
 }

 public static final class HexUtil {
 private static final char[] hexArray = "0123456789ABCDEF".toCharArray();

 private HexUtil() {
 }

 public static String hexlify(final byte[] bytes) {
 final char[] hexChars = new char[bytes.length * 2];

 for (int j = 0; j < bytes.length; ++j) {
 final int v = bytes[j] & 255;
 hexChars[j * 2] = HexUtil.hexArray[v >>> 4];
 hexChars[j * 2 + 1] = HexUtil.hexArray[v & 15];
 }

 return new String(hexChars);
 }

 public static byte[] unhexlify(final String argbuf) {
 final int arglen = argbuf.length();
 if (arglen % 2 != 0) {
 throw new RuntimeException("Odd-length string");
 } else {
 final byte[] retbuf = new byte[arglen / 2];

 for (int i = 0; i < arglen; i += 2) {
 final int top = Character.digit(argbuf.charAt(i), 16);
 final int bot = Character.digit(argbuf.charAt(i + 1), 16);
 if (top == -1 || bot == -1) {
 throw new RuntimeException("Non-hexadecimal digit found");
 }

 retbuf[i / 2] = (byte) ((top << 4) + bot);
 }

 return retbuf;
 }
 }
 }

 public static final class AVHWContextInfo {
 private final AVCodecHWConfig hwConfig;
 private final AVBufferRef hwContext;

 private volatile boolean freed = false;

 public AVHWContextInfo(final AVCodecHWConfig hwConfig, final AVBufferRef hwContext) {
 this.hwConfig = hwConfig;
 this.hwContext = hwContext;
 }

 public AVCodecHWConfig hwConfig() {
 return this.hwConfig;
 }

 public AVBufferRef hwContext() {
 return this.hwContext;
 }

 public void free() {
 if (this.freed) {
 return;
 }
 this.freed = true;
 av_free(this.hwConfig);
 av_free(this.hwContext);
 }


 @Override
 public boolean equals(Object o) {
 if (this == o) return true;
 if (o == null || getClass() != o.getClass()) return false;
 AVHWContextInfo that = (AVHWContextInfo) o;
 return freed == that.freed && Objects.equals(hwConfig, that.hwConfig) && Objects.equals(hwContext, that.hwContext);
 }

 @Override
 public int hashCode() {
 return Objects.hash(hwConfig, hwContext, freed);
 }

 @Override
 public String toString() {
 return "AVHWContextInfo[" +
 "hwConfig=" + this.hwConfig + ", " +
 "hwContext=" + this.hwContext + ']';
 }

 }

 public static final class AVTestFrames {

 private AVTestFrames() {

 }

 static {
 InputStream inputStream = null;
 try {
 inputStream = AVTestFrames.class.getClassLoader().getResourceAsStream("h264_test_key_frame.txt");
 final byte[] h264TestFrameBuffer = inputStream == null ? new byte[0] : inputStream.readAllBytes();
 final String h264TestFrame = new String(h264TestFrameBuffer, StandardCharsets.UTF_8);
 AVTestFrames.h264KeyTestFrame = HexUtil.unhexlify(h264TestFrame);
 } catch (final IOException e) {
 Logger.error(e, "Could not parse test frame");
 } finally {
 if (inputStream != null) {
 try {
 inputStream.close();
 inputStream = null;
 } catch (final IOException e) {
 Logger.error(e, "Could not close test frame input stream");
 }
 }
 }
 }

 public static byte[] h264KeyTestFrame;
 }
}
</boolean>

The build gradle of the project looks like this


plugins {
 id 'application'
 id 'org.openjfx.javafxplugin' version '0.0.13'
}

group 'com.test.example'
version '1.0.0'

repositories {
 mavenCentral()
 mavenLocal()
 maven { url 'https://jitpack.io' }
}

dependencies {
 implementation group: 'org.bytedeco', name: 'javacv-platform', version: '1.5.8'
 implementation group: 'com.github.oshi', name: 'oshi-core', version: '3.4.3'
 implementation 'org.tinylog:tinylog-api:2.1.0'
 implementation 'org.tinylog:tinylog-impl:2.1.0'
 implementation 'org.jcodec:jcodec:0.2.5'
}

test {
 useJUnitPlatform()
}

javafx {
 version = '17.0.6'
 modules = ['javafx.graphics', 'javafx.controls', 'javafx.fxml', 'javafx.base']
}

mainClassName = 'com.test.example.App'


-
VLC Player shows broken HLS stream with 4k HDR10 mkv
8 avril 2023, par goodkid38I am trying to convert a 4k mkv to an HLS stream but I am not having any luck. I have tried a few ffmpeg commands to try and fix the issue but none have worked. Here are the commands I have tried.


- 

- Basic copy command :




ffmpeg -i "video.mkv" -c copy -f hls "plexTemp/out.m3u8"

- 

- Command used to see if it was an HDR color issue :




ffmpeg -i "video.mkv" -c copy -pix_fmt yuv420p10le -f hls "plexTemp/out.m3u8"
3. Command used to revert to 8 bit color :

ffmpeg -i "video.mkv" -c copy -pix_fmt yuv420p -f hls "plexTemp/out.m3u8"

- 

- I tried removing extra streams and just focusing on audio and video :




ffmpeg -i "out.mkv" -map 0:v:0 -map 0:a:1 -c copy -pix_fmt yuv420p10le -f hls "plexTemp/out.m3u8"

I also saw these warnings when running each command.
Stream HEVC is not hvc1, you should use tag:v hvc1 to set it.
And this

[matroska,webm @ 000001d7921803c0] Stream #12: not enough frames to estimate rate; consider increasing probesize
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 6 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 7 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 8 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 9 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 10 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 11 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options


So I tried increasing the
analyzedurationandprobesizeand adding the tag like so :
ffmpeg -analyzeduration 10000000 -probesize 10000000 -i "out.mkv" -c copy -tag:v hvc1 -f hls "plexTemp/out.m3u8"

To no avail. Here is what the output looks like on VLC. It's mostly black with a few lines of color that randomly change.


On my TV I see this :




One thing I see that stands out in ffmpegs output is this :


[hls @ 00000207239a9ec0] Opening 'plexTemp/out0.ts' for writing7 bitrate= -0.0kbits/s speed=N/A
[hls @ 00000207239a9ec0] Opening 'plexTemp/out.m3u8.tmp' for writing


Here is the audio and video info on the mkv :


General
Unique ID : 92280908398971492516286250889389584022 (0x456CA80EF29B1357B572719D6EC4AE96)
Complete name : I:\video.mkv
Format : Matroska
Format version : Version 2
File size : 49.4 GiB
Duration : 1 h 39 min
Overall bit rate mode : Variable
Overall bit rate : 71.2 Mb/s
Frame rate : 23.976 FPS
Movie name : video
Encoded date : 2023-04-06 22:39:53 UTC
Writing application : MakeMKV v1.16.7 win(x64-release)
Writing library : libmakemkv v1.16.7 (1.3.10/1.5.2) win(x64-release)
Cover : Yes
Attachments : cover.jpg

Video
ID : 1
ID in the original source medium : 4113 (0x1011)
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
HDR format : SMPTE ST 2086, HDR10 compatible
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1 h 39 min
Bit rate : 63.6 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.320
Stream size : 44.1 GiB (89%)
Writing library : ATEME Titan File 3.9.6 (4.9.6.2) 
Language : English
Default : No
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 1000 cd/m2
Maximum Frame-Average Light Level : 140 cd/m2
Original source medium : Blu-ray

Audio #1
ID : 2
ID in the original source medium : 4352 (0x1100)
Format : DTS XLL X
Format/Info : Digital Theater Systems
Commercial name : DTS:X
Codec ID : A_DTS
Duration : 1 h 39 min
Bit rate mode : Variable
Bit rate : 4 174 kb/s
Channel(s) : 8 channels
Channel layout : C L R LFE Lb Rb Lss Rss
Sampling rate : 48.0 kHz
Frame rate : 93.750 FPS (512 SPF)
Bit depth : 24 bits
Stream size : 2.89 GiB (6%)
Title : Surround 7.1
Language : English
Default : Yes
Forced : No
Original source medium : Blu-ray
Here is information on the HLS output:
General
Complete name : I:\out.m3u8
Format : HLS
Format profile : Media
File size : 67.4 MiB
Duration : 8 s 138 ms
Overall bit rate mode : Variable
Overall bit rate : 69.4 Mb/s
Frame rate : 23.976 FPS


And my
Here is the output of my HLS stream :


Video
ID : 256 (0x100)
Menu ID : 1 (0x1)
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
HDR format : SMPTE ST 2086, HDR10 compatible
Muxing mode : MPEG-TS
Codec ID : 36
Duration : 8 s 49 ms
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Writing library : ATEME Titan File 3.9.6 (4.9.6.2) 
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 1000 cd/m2
Maximum Frame-Average Light Level : 140 cd/m2
Source : out92.ts

Audio
ID : 257 (0x101)
Menu ID : 1 (0x1)
Format : DTS XLL X
Format/Info : Digital Theater Systems
Commercial name : DTS:X
Muxing mode : MPEG-TS
Codec ID : 130
Duration : 8 s 138 ms
Bit rate mode : Variable
Channel(s) : 8 channels
Channel layout : C L R LFE Lb Rb Lss Rss
Sampling rate : 48.0 kHz
Frame rate : 93.750 FPS (512 SPF)
Bit depth : 24 bits
Delay relative to video : -125 ms
Language : English
Source : out92.ts


Syndiquer tout le site
