Recherche avancée
Autres articles (63)
-
Le profil des utilisateurs
12 avril 2011, parChaque utilisateur dispose d’une page de profil lui permettant de modifier ses informations personnelle. Dans le menu de haut de page par défaut, un élément de menu est automatiquement créé à l’initialisation de MediaSPIP, visible uniquement si le visiteur est identifié sur le site.
L’utilisateur a accès à la modification de profil depuis sa page auteur, un lien dans la navigation "Modifier votre profil" est (...) -
Configurer la prise en compte des langues
15 novembre 2010, parAccéder à la configuration et ajouter des langues prises en compte
Afin de configurer la prise en compte de nouvelles langues, il est nécessaire de se rendre dans la partie "Administrer" du site.
De là, dans le menu de navigation, vous pouvez accéder à une partie "Gestion des langues" permettant d’activer la prise en compte de nouvelles langues.
Chaque nouvelle langue ajoutée reste désactivable tant qu’aucun objet n’est créé dans cette langue. Dans ce cas, elle devient grisée dans la configuration et (...) -
XMP PHP
13 mai 2011, parDixit Wikipedia, XMP signifie :
Extensible Metadata Platform ou XMP est un format de métadonnées basé sur XML utilisé dans les applications PDF, de photographie et de graphisme. Il a été lancé par Adobe Systems en avril 2001 en étant intégré à la version 5.0 d’Adobe Acrobat.
Étant basé sur XML, il gère un ensemble de tags dynamiques pour l’utilisation dans le cadre du Web sémantique.
XMP permet d’enregistrer sous forme d’un document XML des informations relatives à un fichier : titre, auteur, historique (...)
Sur d’autres sites (7745)
-
Code can not read property 1 of undefined [closed]
25 mai 2023, par Jesse CopasI'm a very new programmer and am working on a Tdarr plugin in JS.
Everything works fine until a 4k file tries to get transcoded and it fails with this log


2023-05-24T19:09:54.906Z ZoBKWMMKG:Node\[hidden-hog\]:Worker\[tall-tuna\]:{"pluginInputs":{"BitRate":"4000","ResolutionSelection":"1080p","Container":"mkv","AudioType":"AAC","FrameRate":"24"}}

2023-05-24T19:09:54.907Z ZoBKWMMKG:Node\[hidden-hog\]:Worker\[tall-tuna\]:Error TypeError: Cannot read property '1' of undefined


It's saying that it's unable to read property 1 of undefined and I'm looked and looked and looked and can't find what it is referring to. Hoping to get another set of eyes on it
The plugin Code is here


/* eslint no-plusplus: ["error", { "allowForLoopAfterthoughts": true }] */
/* eslint-disable no-restricted-globals */
const details = () => ({
 id: 'Tdarr_Plugin_Jeso_AV1_HandBrake_Transcode',
 Stage: 'Pre-processing',
 Name: 'AV1 HandBrake Transcoder',
 Type: 'Video',
 Operation: 'Transcode',
 Description: 'Transcodes to AV1 at the selected Bitrate. This is best used with Remux Files.',
 Version: '2.1.3',
 Tags: 'HandBrake,configurable',
 Inputs: [
 {
 name: 'BitRate',
 type: 'string',
 defaultValue: '4000',
 inputUI: {
 type: 'text',
 },
 tooltip: `
 ~ Requested Bitrate ~ \\n
 Put in the Bitrate you want to process to in Kbps. For example 4000Kbps is 4Mbps. `,
 },
 {
 name: 'ResolutionSelection',
 type: 'string',
 defaultValue: '1080p',
 inputUI: {
 type: 'dropdown',
 options: [
 '8KUHD',
 '4KUHD',
 '1080p',
 '720p',
 '480p',
 ],
 },
 // eslint-disable-next-line max-len
 tooltip: 'Any Resolution larger than this will become this Resolution same as the bitrate if the Res is lower than the selected it will use the res of the file as to not cause bloating of file size.',
 },
 {
 name: 'Container',
 type: 'string',
 defaultValue: 'mkv',
 inputUI: {
 type: 'dropdown',
 options: [
 'mp4',
 'mkv',
 ],
 },
 tooltip: ` Container Type \\n\\n
 mkv or mp4.\\n`,
 },
 {
 name: 'AudioType',
 type: 'string',
 defaultValue: 'AAC',
 inputUI: {
 type: 'dropdown',
 options: [
 'AAC',
 'EAC3',
 'MP3',
 'Vorbis',
 'Flac16',
 'Flac24',
 ],
 },
 // eslint-disable-next-line max-len
 tooltip: 'Set Audio container type that you want to use',
 },
 {
 name: 'FrameRate',
 type: 'string',
 defaultValue: '24',
 inputUI: {
 type: 'text',
 },
 // eslint-disable-next-line max-len
 tooltip: 'If the files framerate is higher than 24 and you want to maintain that framerate you can do so here',
 },
 ],
});
const MediaInfo = {
 videoHeight: '',
 videoWidth: '',
 videoFPS: '',
 videoBR: '',
 videoBitDepth: '',
 overallBR: '',
 videoResolution: '',
}; // var MediaInfo
// Easier for our functions if response has global scope.
const response = {
 processFile: false,
 preset: '',
 container: '.mkv',
 handBrakeMode: true,
 FFmpegMode: false,
 reQueueAfter: true,
 infoLog: '',
}; // var response
// Finds the first video stream and populates some useful variables
function getMediaInfo(file) {
 let videoIdx = -1;
 for (let i = 0; i < file.ffProbeData.streams.length; i += 1) {
 const strstreamType = file.ffProbeData.streams[i].codec_type.toLowerCase();
 // Looking For Video
 // Check if stream is a video.
 if (videoIdx === -1 && strstreamType === 'video') {
 videoIdx = i;
 // get video streams resolution
 MediaInfo.videoResolution = `${file.ffProbeData.streams[i].height}x${file.ffProbeData.streams[i].width}`;
 MediaInfo.videoHeight = Number(file.ffProbeData.streams[i].height);
 MediaInfo.videoWidth = Number(file.ffProbeData.streams[i].width);
 MediaInfo.videoFPS = Number(file.mediaInfo.track[i + 1].FrameRate) || 25;
 // calulate bitrate from dimensions and fps of file
 MediaInfo.videoBR = (MediaInfo.videoHeight * MediaInfo.videoWidth * MediaInfo.videoFPS * 0.08).toFixed(0);
 }
 }
} // end getMediaInfo()
// define resolution order from ResolutionSelection from biggest to smallest
const resolutionOrder = ['8KUHD', '4KUHD', '1080p', '720p', '480p'];
// define the width and height of each resolution from the resolution order
const resolutionsdimensions = {
 '8KUHD': '--width 7680 --height 4320',
 '4KUHD': '--width 3840 --height 2160',
 '1080p': '--width 1920 --height 1080',
 '720p': '--width 1280 --height 720',
 '480p': '--width 640 --height 480',
};
// eslint-disable-next-line no-unused-vars
const plugin = (file, librarySettings, inputs) => {
 // eslint-disable-next-line no-unused-vars
 const importFresh = require('import-fresh');
 // eslint-disable-next-line no-unused-vars
 const library = importFresh('../methods/library.js');
 // eslint-disable-next-line no-unused-vars
 const lib = require('../methods/lib')();
 // Get the selected resolution from the 'ResolutionSelection' variable
 const selectedResolution = inputs.ResolutionSelection;
 getMediaInfo(file);
 // use mediainfo to match height and width to a resolution on resolutiondimensions
 let dimensions = resolutionsdimensions[selectedResolution];
 // if the file is smaller than the selected resolution then use the file resolution
 if (MediaInfo.videoHeight < dimensions.split(' ')[3] || MediaInfo.videoWidth < dimensions.split(' ')[1]) {
 dimensions = `--width ${MediaInfo.videoWidth} --height ${MediaInfo.videoHeight}`;
 // eslint-disable-next-line brace-style
 }
 // read the bitrate of the video stream
 let videoBitRate = MediaInfo.videoBR;
 // if videoBitrate is over 1000000 devide by 100 to get the bitrate in Kbps
 if (videoBitRate > 1000000) {
 videoBitRate /= 100;
 } else { videoBitRate /= 1000; }
 // if VideoBitrate is smaller than selected bitrate then use the videoBitrate
 if (videoBitRate < inputs.BitRate) {
 // eslint-disable-next-line no-param-reassign
 inputs.BitRate = videoBitRate;
 // eslint-disable-next-line brace-style
 }
 // if VideoBitrate is larger than selected bitrate then use the selected bitrate
 else {
 // eslint-disable-next-line no-self-assign, no-param-reassign
 inputs.BitRate = inputs.BitRate;
 }

 //Skip Transcoding if File is already AV1
 if (file.ffProbeData.streams[0].codec_name === 'av1') {
 response.processFile = false;
 response.infoLog += 'File is already AV1 \n';
 return response;
 }
 // eslint-disable-next-line no-constant-condition
 if ((true) || file.forceProcessing === true) {
 // eslint-disable-next-line max-len
 response.preset = `--encoder svt_av1 -b ${inputs.BitRate} -r ${inputs.FrameRate} -E ${inputs.AudioType} -f ${inputs.Container} --no-optimize ${dimensions} --crop 0:0:0:0`;
 response.container = `.${inputs.Container}`;
 response.handbrakeMode = true;
 response.ffmpegMode = false;
 response.processFile = true;
 response.infoLog += `File is being transcoded at ${inputs.BitRate} Kbps to ${dimensions} as ${inputs.Container} \n`;
 return response;
 }
 response.infoLog += 'File is being transcoded using custom arguments \n';
 return response;
};
 };

module.exports.details = details;
module.exports.plugin = plugin;


Tried transcoding 4k files down to 1080p but it fails due to that undefined error. All Res 1080p and lower that I have tried work correctly


EDIT : I used Console.log and got this back


[2023-05-24T23:29:51.001] [ERROR] Tdarr_Server - Error running MediaInfo 1
[2023-05-24T23:29:51.004] [ERROR] Tdarr_Server - RangeError: Maximum call stack size exceeded
 at x (<anonymous>:wasm-function[381]:0x15c4d)
 at <anonymous>:wasm-function[46]:0x5dc0
 at <anonymous>:wasm-function[652]:0x21cb9
 at <anonymous>:wasm-function[1023]:0x47018
 at <anonymous>:wasm-function[853]:0x37827
 at <anonymous>:wasm-function[3684]:0xf4884
 at <anonymous>:wasm-function[3516]:0xeb5b7
 at <anonymous>:wasm-function[1061]:0x487c9
 at <anonymous>:wasm-function[795]:0x3006d
 at <anonymous>:wasm-function[3628]:0xf01cc
[2023-05-24T23:29:51.006] [ERROR] Tdarr_Server - Error running MediaInfo 2
[2023-05-24T23:29:51.006] [ERROR] Tdarr_Server - RangeError: Maximum call stack size exceeded
 at x (<anonymous>:wasm-function[381]:0x15c4d)
 at <anonymous>:wasm-function[46]:0x5dc0
 at <anonymous>:wasm-function[652]:0x21cb9
 at <anonymous>:wasm-function[1023]:0x47018
 at <anonymous>:wasm-function[853]:0x37827
 at <anonymous>:wasm-function[3684]:0xf4884
 at <anonymous>:wasm-function[3516]:0xeb5b7
 at <anonymous>:wasm-function[1061]:0x487c9
 at <anonymous>:wasm-function[795]:0x3006d
 at <anonymous>:wasm-function[3628]:0xf01cc
[2023-05-24T23:29:58.220] [ERROR] Tdarr_Server - Error running MediaInfo 1
[2023-05-24T23:29:58.223] [ERROR] Tdarr_Server - RangeError: Maximum call stack size exceeded
 at x (<anonymous>:wasm-function[381]:0x15c4d)
 at <anonymous>:wasm-function[46]:0x5dc0
 at <anonymous>:wasm-function[652]:0x21cb9
 at <anonymous>:wasm-function[1023]:0x47018
 at <anonymous>:wasm-function[853]:0x37827
 at <anonymous>:wasm-function[3684]:0xf4884
 at <anonymous>:wasm-function[3516]:0xeb5b7
 at <anonymous>:wasm-function[1061]:0x487c9
 at <anonymous>:wasm-function[795]:0x3006d
 at <anonymous>:wasm-function[3628]:0xf01cc
[2023-05-24T23:29:58.224] [ERROR] Tdarr_Server - Error running MediaInfo 2
[2023-05-24T23:29:58.224] [ERROR] Tdarr_Server - RangeError: Maximum call stack size exceeded
 at x (<anonymous>:wasm-function[381]:0x15c4d)
 at <anonymous>:wasm-function[46]:0x5dc0
 at <anonymous>:wasm-function[652]:0x21cb9
 at <anonymous>:wasm-function[1023]:0x47018
 at <anonymous>:wasm-function[853]:0x37827
 at <anonymous>:wasm-function[3684]:0xf4884
 at <anonymous>:wasm-function[3516]:0xeb5b7
 at <anonymous>:wasm-function[1061]:0x487c9
 at <anonymous>:wasm-function[795]:0x3006d
 at <anonymous>:wasm-function[3628]:0xf01cc
</anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous>

-
Failed to decode h264 key frame with DXVA2.0 because returned buffer is to small
16 mai 2023, par grill2010I hava a strange problem on Windows with DXVA2 h264 decoding. I recently figured out a ffmpeg decoding limitation for DXVA2 and D3D11VA on Windows and how to solve it, this solution completly fixes the problem with D3D11VA but DXVA2 still has some problems with certain keyframes. Upon further investigation it turned out that the decoding of these certain keyframes fail because the buffer returned from the IDirectXVideoDecoder_GetBuffer function was too small. FFmpeg is printing out these logs when the decoding fails :


Error: [h264 @ 0000028b2e5796c0] Buffer for type 5 was too small. size: 58752, dxva_size: 55296
Error: [h264 @ 0000028b2e5796c0] Failed to add bitstream or slice control buffer
Error: [h264 @ 0000028b2e5796c0] hardware accelerator failed to decode picture


Why is this returned buffer too low ? What kind of factors inside ffmpeg do have an effect on this buffer size or is this a limitation of DXVA2 in general ? All other decoders like Cuvid, D3D11VA or the software decoder are not affected by this problem and can decode all keyframes.


I have an example javacv project on github that can reproduce the problem. I also provide the source code of the main class here. The keyframe example data with prepended SPS and PPS in hex form can be downloaded here.


import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.layout.Pane;
import javafx.scene.layout.StackPane;
import javafx.stage.Stage;
import org.bytedeco.ffmpeg.avcodec.AVCodec;
import org.bytedeco.ffmpeg.avcodec.AVCodecContext;
import org.bytedeco.ffmpeg.avcodec.AVCodecHWConfig;
import org.bytedeco.ffmpeg.avcodec.AVPacket;
import org.bytedeco.ffmpeg.avutil.AVBufferRef;
import org.bytedeco.ffmpeg.avutil.AVDictionary;
import org.bytedeco.ffmpeg.avutil.AVFrame;
import org.bytedeco.ffmpeg.avutil.LogCallback;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.Pointer;
import org.tinylog.Logger;

import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Objects;
import java.util.function.Consumer;

import static org.bytedeco.ffmpeg.avcodec.AVCodecContext.FF_THREAD_SLICE;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avutil.*;

public class App extends Application {

 /**** decoder variables ****/

 private AVHWContextInfo hardwareContext;

 private AVCodec decoder;
 private AVCodecContext m_VideoDecoderCtx;

 private AVCodecContext.Get_format_AVCodecContext_IntPointer formatCallback;

 private final int streamResolutionX = 1920;
 private final int streamResolutionY = 1080;

 // AV_HWDEVICE_TYPE_CUDA // example works with cuda
 // AV_HWDEVICE_TYPE_DXVA2 // producing Invalid data found on keyframe
 // AV_HWDEVICE_TYPE_D3D11VA // producing Invalid data found on keyframe
 private static final int HW_DEVICE_TYPE = AV_HWDEVICE_TYPE_DXVA2;

 private static final boolean USE_HW_ACCEL = true;

 private static final boolean USE_AV_EF_EXPLODE = true;

 public static void main(final String[] args) {
 //System.setProperty("prism.order", "d3d,sw");
 System.setProperty("prism.vsync", "false");
 Application.launch(App.class);
 }

 @Override
 public void start(final Stage primaryStage) {
 final Pane dummyPane = new Pane();
 dummyPane.setStyle("-fx-background-color: black");
 final Scene scene = new Scene(dummyPane, this.streamResolutionX, this.streamResolutionY);
 primaryStage.setScene(scene);
 primaryStage.show();
 primaryStage.setMinWidth(480);
 primaryStage.setMinHeight(360);

 this.initializeFFmpeg(result -> {
 if (!result) {
 Logger.error("FFmpeg could not be initialized correctly, terminating program");
 System.exit(1);
 return;
 }
 scene.setRoot(new StackPane());
 this.performTestFramesFeeding();
 });
 }

 private void initializeFFmpeg(final Consumer<boolean> finishHandler) {
 FFmpegLogCallback.setLevel(AV_LOG_DEBUG); // Increase log level until the first frame is decoded
 FFmpegLogCallback.set();
 Pointer pointer = new Pointer((Pointer) null);
 AVCodec c;
 while ((c = av_codec_iterate(pointer)) != null) {
 if (av_codec_is_decoder(c) > 0)
 Logger.debug("{}:{} ", c.name().getString(), c.type());
 }

 this.decoder = avcodec_find_decoder(AV_CODEC_ID_H264); // usually decoder name is h264 and without hardware support it's yuv420p otherwise nv12
 if (this.decoder == null) {
 Logger.error("Unable to find decoder for format {}", "h264");
 finishHandler.accept(false);
 return;
 }
 Logger.info("Current decoder name: {}, {}", this.decoder.name().getString(), this.decoder.long_name().getString());

 if (true) {
 for (; ; ) {
 this.m_VideoDecoderCtx = avcodec_alloc_context3(this.decoder);
 if (this.m_VideoDecoderCtx == null) {
 Logger.error("Unable to find decoder for format AV_CODEC_ID_H264");
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 this.hardwareContext = null;
 }
 continue;
 }

 if (App.USE_HW_ACCEL) {
 this.hardwareContext = this.createHardwareContext();
 if (this.hardwareContext != null) {
 Logger.info("Set hwaccel support");
 this.m_VideoDecoderCtx.hw_device_ctx(this.hardwareContext.hwContext()); // comment to disable hwaccel
 }
 } else {
 Logger.info("Hwaccel manually disabled");
 }

 // Always request low delay decoding
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_LOW_DELAY);

 // Allow display of corrupt frames and frames missing references
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_OUTPUT_CORRUPT);
 this.m_VideoDecoderCtx.flags2(this.m_VideoDecoderCtx.flags2() | AV_CODEC_FLAG2_SHOW_ALL);

 if (App.USE_AV_EF_EXPLODE) {
 // Report decoding errors to allow us to request a key frame
 this.m_VideoDecoderCtx.err_recognition(this.m_VideoDecoderCtx.err_recognition() | AV_EF_EXPLODE);
 }

 // Enable slice multi-threading for software decoding
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) { // if not hw accelerated
 this.m_VideoDecoderCtx.thread_type(this.m_VideoDecoderCtx.thread_type() | FF_THREAD_SLICE);
 this.m_VideoDecoderCtx.thread_count(2/*AppUtil.getCpuCount()*/);
 } else {
 // No threading for HW decode
 this.m_VideoDecoderCtx.thread_count(1);
 }

 this.m_VideoDecoderCtx.width(this.streamResolutionX);
 this.m_VideoDecoderCtx.height(this.streamResolutionY);
 this.m_VideoDecoderCtx.pix_fmt(this.getDefaultPixelFormat());

 this.formatCallback = new AVCodecContext.Get_format_AVCodecContext_IntPointer() {
 @Override
 public int call(final AVCodecContext context, final IntPointer pixelFormats) {
 final boolean hwDecodingSupported = context.hw_device_ctx() != null && App.this.hardwareContext != null;
 final int preferredPixelFormat = hwDecodingSupported ?
 App.this.hardwareContext.hwConfig().pix_fmt() :
 context.pix_fmt();
 int i = 0;
 while (true) {
 final int currentSupportedFormat = pixelFormats.get(i++);
 System.out.println("Supported pixel formats " + currentSupportedFormat);
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) {
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == preferredPixelFormat) {
 Logger.info("[FFmpeg]: pixel format in format callback is {}", currentSupportedFormat);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for yuv
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_YUV420P) {
 Logger.info("[FFmpeg]: Not found in first match so use {}", AV_PIX_FMT_YUV420P);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for nv12
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_NV12) {
 Logger.info("[FFmpeg]: Not found in second match so use {}", AV_PIX_FMT_NV12);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 Logger.info("[FFmpeg]: pixel format in format callback is using fallback {}", AV_PIX_FMT_NONE);
 return AV_PIX_FMT_NONE;
 }
 };
 this.m_VideoDecoderCtx.get_format(this.formatCallback);

 final AVDictionary options = new AVDictionary(null);
 final int result = avcodec_open2(this.m_VideoDecoderCtx, this.decoder, options);
 if (result < 0) {
 Logger.error("avcodec_open2 was not successful");
 finishHandler.accept(false);
 return;
 }
 av_dict_free(options);
 break;
 }
 }

 if (this.decoder == null || this.m_VideoDecoderCtx == null) {
 finishHandler.accept(false);
 return;
 }
 finishHandler.accept(true);
 }

 private AVHWContextInfo createHardwareContext() {
 AVHWContextInfo result = null;
 for (int i = 0; ; i++) {
 final AVCodecHWConfig config = avcodec_get_hw_config(this.decoder, i);
 if (config == null) {
 break;
 }

 if ((config.methods() & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX) < 0) {
 continue;
 }
 final int device_type = config.device_type();
 if (device_type != App.HW_DEVICE_TYPE) {
 continue;
 }
 final AVBufferRef hw_context = av_hwdevice_ctx_alloc(device_type);
 if (hw_context == null || av_hwdevice_ctx_create(hw_context, device_type, (String) null, null, 0) < 0) {
 Logger.error("HW accel not supported for type {}", device_type);
 av_free(config);
 av_free(hw_context);
 } else {
 Logger.info("HW accel created for type {}", device_type);
 result = new AVHWContextInfo(config, hw_context);
 }
 break;
 }

 return result;
 }

 @Override
 public void stop() {
 this.releaseNativeResources();
 }

 /*****************************/
 /*** test frame processing ***/
 /*****************************/
 
 private void performTestFramesFeeding() {
 final AVPacket pkt = av_packet_alloc();
 if (pkt == null) {
 return;
 }
 try (final BytePointer bp = new BytePointer(65_535 * 15)) {


 for (int i = 0; i < 1; i++) {
 final byte[] frameData = AVTestFrames.h264KeyTestFrame;

 bp.position(0);

 bp.put(frameData);
 bp.limit(frameData.length);

 pkt.data(bp);
 pkt.capacity(bp.capacity());
 pkt.size(frameData.length);
 pkt.position(0);
 pkt.limit(frameData.length);
 //pkt.flags(AV_PKT_FLAG_KEY);
 final AVFrame avFrame = av_frame_alloc();
 System.out.println("frameData.length " + frameData.length);

 final int err = avcodec_send_packet(this.m_VideoDecoderCtx, pkt); //fill_scaling_lists
 if (err < 0) {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(err, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("Error on decoding test frame " + err + " message " + string);
 av_frame_free(avFrame);
 return;
 }

 final int result = avcodec_receive_frame(this.m_VideoDecoderCtx, avFrame);
 final AVFrame decodedFrame;
 if (result == 0) {
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) {
 decodedFrame = avFrame;
 System.out.println("SUCESS with SW decoding");
 } else {
 final AVFrame hwAvFrame = av_frame_alloc();
 if (av_hwframe_transfer_data(hwAvFrame, avFrame, 0) < 0) {
 System.out.println("Failed to transfer frame from hardware");
 av_frame_unref(hwAvFrame);
 decodedFrame = avFrame;
 } else {
 av_frame_unref(avFrame);
 decodedFrame = hwAvFrame;
 System.out.println("SUCESS with HW decoding");
 }
 }

 av_frame_unref(decodedFrame);
 } else {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(result, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("error " + result + " message " + string);
 av_frame_free(avFrame);
 }
 }
 } finally {
 if (pkt.stream_index() != -1) {
 av_packet_unref(pkt);
 }
 pkt.releaseReference();
 }
 }

 final Object releaseLock = new Object();
 private volatile boolean released = false;

 private void releaseNativeResources() {
 if (this.released) {
 return;
 }
 this.released = true;
 synchronized (this.releaseLock) {
 // Close the video codec
 if (this.m_VideoDecoderCtx != null) {
 avcodec_free_context(this.m_VideoDecoderCtx);
 this.m_VideoDecoderCtx = null;
 }

 // close the format callback
 if (this.formatCallback != null) {
 this.formatCallback.close();
 this.formatCallback = null;
 }

 // close hw context
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 }
 }
 }

 private int getDefaultPixelFormat() {
 return AV_PIX_FMT_YUV420P; // Always return yuv420p here
 }


 /*********************/
 /*** inner classes ***/
 /*********************/

 public static final class HexUtil {

 private HexUtil() {
 }

 public static byte[] unhexlify(final String argbuf) {
 final int arglen = argbuf.length();
 if (arglen % 2 != 0) {
 throw new RuntimeException("Odd-length string");
 } else {
 final byte[] retbuf = new byte[arglen / 2];

 for (int i = 0; i < arglen; i += 2) {
 final int top = Character.digit(argbuf.charAt(i), 16);
 final int bot = Character.digit(argbuf.charAt(i + 1), 16);
 if (top == -1 || bot == -1) {
 throw new RuntimeException("Non-hexadecimal digit found");
 }

 retbuf[i / 2] = (byte) ((top << 4) + bot);
 }

 return retbuf;
 }
 }
 }

 public static final class AVHWContextInfo {
 private final AVCodecHWConfig hwConfig;
 private final AVBufferRef hwContext;

 private volatile boolean freed = false;

 public AVHWContextInfo(final AVCodecHWConfig hwConfig, final AVBufferRef hwContext) {
 this.hwConfig = hwConfig;
 this.hwContext = hwContext;
 }

 public AVCodecHWConfig hwConfig() {
 return this.hwConfig;
 }

 public AVBufferRef hwContext() {
 return this.hwContext;
 }

 public void free() {
 if (this.freed) {
 return;
 }
 this.freed = true;
 av_free(this.hwConfig);
 av_free(this.hwContext);
 }


 @Override
 public boolean equals(Object o) {
 if (this == o) return true;
 if (o == null || getClass() != o.getClass()) return false;
 AVHWContextInfo that = (AVHWContextInfo) o;
 return freed == that.freed && Objects.equals(hwConfig, that.hwConfig) && Objects.equals(hwContext, that.hwContext);
 }

 @Override
 public int hashCode() {
 return Objects.hash(hwConfig, hwContext, freed);
 }

 @Override
 public String toString() {
 return "AVHWContextInfo[" +
 "hwConfig=" + this.hwConfig + ", " +
 "hwContext=" + this.hwContext + ']';
 }
 }

 public static final class AVTestFrames {

 private AVTestFrames() {

 }

 static {
 InputStream inputStream = null;
 try {
 inputStream = AVTestFrames.class.getClassLoader().getResourceAsStream("h264_test_key_frame.txt");
 final byte[] h264TestFrameBuffer = inputStream == null ? new byte[0] : inputStream.readAllBytes();
 final String h264TestFrame = new String(h264TestFrameBuffer, StandardCharsets.UTF_8);
 AVTestFrames.h264KeyTestFrame = HexUtil.unhexlify(h264TestFrame);
 } catch (final IOException e) {
 Logger.error(e, "Could not parse test frame");
 } finally {
 if (inputStream != null) {
 try {
 inputStream.close();
 } catch (final IOException e) {
 Logger.error(e, "Could not close test frame input stream");
 }
 }
 }
 }

 public static byte[] h264KeyTestFrame;
 }

 public static class FFmpegLogCallback extends LogCallback {

 private static final org.bytedeco.javacpp.tools.Logger logger = org.bytedeco.javacpp.tools.Logger.create(FFmpegLogCallback.class);

 static final FFmpegLogCallback instance = new FFmpegLogCallback().retainReference();

 public static FFmpegLogCallback getInstance() {
 return instance;
 }

 /**
 * Calls {@code avutil.setLogCallback(getInstance())}.
 */
 public static void set() {
 setLogCallback(getInstance());
 }

 /**
 * Returns {@code av_log_get_level()}.
 **/
 public static int getLevel() {
 return av_log_get_level();
 }

 /**
 * Calls {@code av_log_set_level(level)}.
 **/
 public static void setLevel(int level) {
 av_log_set_level(level);
 }

 @Override
 public void call(int level, BytePointer msg) {
 switch (level) {
 case AV_LOG_PANIC, AV_LOG_FATAL, AV_LOG_ERROR -> logger.error(msg.getString());
 case AV_LOG_WARNING -> logger.warn(msg.getString());
 case AV_LOG_INFO -> logger.info(msg.getString());
 case AV_LOG_VERBOSE, AV_LOG_DEBUG, AV_LOG_TRACE -> logger.debug(msg.getString());
 default -> {
 assert false;
 }
 }
 }
 }
}
</boolean>

-
Overlaying a text stream on a video stream with ffmpeg in Node.js
16 mai 2023, par TchouneI am creating a streaming system with Node.js that uses ffmpeg to send video and text streams to a local RTMP server, then combines those streams and sends them to Twitch.


I'm using canvas to create a text image with a transparent background, and I need to change that text every time a new video in the playlist starts.


Currently in stream I see only the video stream of my video and not the text. But if I go via VLC to see each more separate, I see them


However, I'm running into a problem where the text stream doesn't appear in the final video stream on Twitch. In addition, I get the following error message :


Combine stderr: [NULL @ 0x1407069f0] Unable to find a suitable output format for 'rtmp://live.twitch.tv/app/streamKey'
rtmp://live.twitch.tv/app/streamKey: Invalid argument


Here is my current Node.js code :



const createTextImage = (runner) => {
 return new Promise((resolve, reject) => {
 const canvas = createCanvas(1920, 1080);
 const context = canvas.getContext('2d');

 // Fill the background with transparency
 context.fillStyle = 'rgba(0,0,0,0)';
 context.fillRect(0, 0, canvas.width, canvas.height);

 // Set the text options
 context.fillStyle = '#ffffff';
 context.font = '24px Arial';
 context.textAlign = 'start';
 context.textBaseline = 'middle';

 // Draw the text
 context.fillText(`Speedrun by ${runner}`, canvas.width / 2, canvas.height / 2);

 // Define the images directory
 const imagesDir = path.join(__dirname, 'images', 'runners');

 // Ensure the images directory exists
 fs.mkdirSync(imagesDir, { recursive: true });

 // Define the file path
 const filePath = path.join(imagesDir, runner + '.png');

 // Create the write stream
 const out = fs.createWriteStream(filePath);

 // Create the PNG stream
 const stream = canvas.createPNGStream();

 // Pipe the PNG stream to the write stream
 stream.pipe(out);

 out.on('finish', () => {
 console.log('The PNG file was created.');
 resolve();
 });

 out.on('error', reject);
 });
}
const streamVideo = (video) => {
 ffmpegLibrary.ffprobe(video.video, function (err, metadata) {
 if (err) {
 console.error(err);
 return;
 }
 currentVideoDuration = metadata.format.duration;

 // Annulez le délai précédent avant d'en créer un nouveau
 if (nextVideoTimeoutId) {
 clearTimeout(nextVideoTimeoutId);
 }

 // Déplacez votre appel setTimeout ici
 nextVideoTimeoutId = setTimeout(() => {
 console.log('Fin de la vidéo, passage à la suivante...');
 nextVideo();
 }, currentVideoDuration * 1000 + 10000);
 })


 ffmpegVideo = childProcess.spawn('ffmpeg', [
 '-nostdin', '-re', '-f', 'concat', '-safe', '0', '-i', 'playlist.txt',
 '-vcodec', 'libx264',
 '-s', '1920x1080',
 '-r', '30',
 '-b:v', '5000k',
 '-acodec', 'aac',
 '-preset', 'veryfast',
 '-f', 'flv',
 `rtmp://localhost:1935/live/video` // envoie le flux vidéo au serveur rtmp local
 ]);

 createTextImage(video.runner).then(() => {
 ffmpegText = childProcess.spawn('ffmpeg', [
 '-nostdin', '-re',
 '-loop', '1', '-i', `images/runners/${video.runner}.png`, // Utilise l'image créée par Puppeteer
 '-vcodec', 'libx264rgb', // Utilise le codec PNG pour conserver la transparence
 '-s', '1920x1080',
 '-r', '30',
 '-b:v', '5000k',
 '-acodec', 'aac',
 '-preset', 'veryfast',
 '-f', 'flv',
 `rtmp://localhost:1935/live/text` // envoie le flux de texte au serveur rtmp local
 ]);

 ffmpegText.stdout.on('data', (data) => {
 console.log(`text stdout: ${data}`);
 });

 ffmpegText.stderr.on('data', (data) => {
 console.error(`text stderr: ${data}`);
 });
 }).catch(error => {
 console.error(`Erreur lors de la création de l'image de texte: ${error}`);
 });

 ffmpegCombine = childProcess.spawn('ffmpeg', [
 '-i', 'rtmp://localhost:1935/live/video',
 '-i', 'rtmp://localhost:1935/live/text',
 '-filter_complex', '[0:v][1:v]overlay=main_w-overlay_w:0',
 '-s', '1920x1080',
 '-r', '30',
 '-vcodec', 'libx264',
 '-b:v', '5000k',
 '-acodec', 'aac',
 '-preset', 'veryfast',
 '-f', 'flv',
 `rtmp://live.twitch.tv/app/${twitchStreamKey}` // envoie le flux combiné à Twitch
 ]);

 ffmpegVideo.stdout.on('data', (data) => {
 console.log(`video stdout: ${data}`);
 });

 ffmpegVideo.stderr.on('data', (data) => {
 console.error(`video stderr: ${data}`);
 });

 ffmpegCombine.stdout.on('data', (data) => {
 console.log(`Combine stdout: ${data}`);
 });

 ffmpegCombine.stderr.on('data', (data) => {
 console.error(`Combine stderr: ${data}`);
 });

 ffmpegCombine.on('close', (code) => {
 console.log(`ffmpeg exited with code ${code}`);
 if (currentIndex >= playlist.length) {
 console.log('End of playlist');
 currentIndex = 0;
 }
 });
}



Locally I use nginx with rtmp module to manage multi-streams and combined into one to send to twitch


In NGINX it's my nginx.conf for module :


rtmp {
 server {
 listen 1935; # le port pour le protocole RTMP
 
 application live {
 live on; # active le streaming en direct
 record off; # désactive l'enregistrement du flux
 
 # définit l'endroit où les flux doivent être envoyés
 push rtmp://live.twitch.tv/app/liveKey;
 }
 
 application text {
 live on; # active le streaming en direct
 record off; # désactive l'enregistrement du flux
 }
 }
}


I have checked that the codecs, resolution and frame rate are the same for both streams. I am also overlaying the text stream on top of the video stream with the -filter_complex command, but I am not sure if it works correctly.


Does each stream have to have the same parameters ?


I would like to know if anyone has any idea what could be causing this problem and how to fix it. Should I use a different format for the output stream to Twitch ? Or is there another approach I should consider for layering a dynamic text stream over a video stream ?


Also, I'm wondering if I'm handling updating the text stream correctly when the video changes. Currently, I create a new text image with Canvas every time the video changes, then create a new ffmpeg process for the text stream. Is this the right approach, or is there a better way to handle this ?


Thanks in advance for any help or advice.


Syndiquer tout le site
