Recherche avancée
Autres articles (34)
-
Publier sur MédiaSpip
13 juin 2013Puis-je poster des contenus à partir d’une tablette Ipad ?
Oui, si votre Médiaspip installé est à la version 0.2 ou supérieure. Contacter au besoin l’administrateur de votre MédiaSpip pour le savoir -
Submit enhancements and plugins
13 avril 2011If you have developed a new extension to add one or more useful features to MediaSPIP, let us know and its integration into the core MedisSPIP functionality will be considered.
You can use the development discussion list to request for help with creating a plugin. As MediaSPIP is based on SPIP - or you can use the SPIP discussion list SPIP-Zone. -
D’autres logiciels intéressants
12 avril 2011, parOn ne revendique pas d’être les seuls à faire ce que l’on fait ... et on ne revendique surtout pas d’être les meilleurs non plus ... Ce que l’on fait, on essaie juste de le faire bien, et de mieux en mieux...
La liste suivante correspond à des logiciels qui tendent peu ou prou à faire comme MediaSPIP ou que MediaSPIP tente peu ou prou à faire pareil, peu importe ...
On ne les connais pas, on ne les a pas essayé, mais vous pouvez peut être y jeter un coup d’oeil.
Videopress
Site Internet : (...)
Sur d’autres sites (4935)
-
audio issues when merging a video with an audio file using python ffmpeg and moviepy
29 février 2024, par Stevenb123I’m trying to create a code that syncs between an audio file and a background file in terms of duration.
When creating the merged video I hear a cut or a loop sound of the last sentence for like 0.2 seconds.
I have tried to solve it in many different ways, listed below.


Has anyone solved this issue ? I saw many people having a similar problem.
I’m using Ubuntu version 20.04 and ffmpeg version 4.2.7


This is my code :


def merge_videos_with_subs(
 background_path, audio_path, subs, output_directory, output_filename
):
 try:
 # Load background video and audio
 background_clip = VideoFileClip(background_path)
 background_clip = background_clip.without_audio()
 audio_clip = AudioFileClip(audio_path)

 # Adjust video duration to match audio duration
 audio_duration = audio_clip.duration

 # If the background video is longer, trim it to match the audio duration
 if background_clip.duration > audio_duration:
 background_clip = background_clip.subclip(0, audio_duration)
 # If the audio is longer, loop the background video
 else:
 background_clip = background_clip.loop(duration=audio_duration)

 # Set audio of the background clip
 background_clip = background_clip.set_audio(audio_clip)

 # Overlay subtitles on the video
 final_clip = CompositeVideoClip(
 [background_clip, subtitles.set_pos(("center", "bottom"))]
 )

 # Ensure the output directory exists
 os.makedirs(output_directory, exist_ok=True)

 # Define the output path
 output_path = os.path.join(output_directory, output_filename)

 # Write the merged video with subtitles
 final_clip.write_videofile(
 output_path, codec="libx264", audio_codec="aac", threads=4, fps=24
 )

 # Close the clips
 final_clip.close()
 background_clip.close()
 audio_clip.close()

 print(f"Merged video with subtitles saved to: {output_path}")
 except Exception as e:
 print(f"Error merging videos: {e}")


I’ve tried changing the codec, tried cutting 0.2 seconds of the audio before and after the merge or silencing it, nothing seems to help.

When I run my code without the background subclip to match the audio it worked flawlessy.
If I let the background run until its full duration or make it loop there were no audio issues.

Looks like the issue is on the cutting part.

-
FFMPEG API - Recording video and audio - Performance issues
24 juillet 2015, par SolidusI’m developing an app which is able to record video from a webcam and audio from a microphone. I also need to process user input at the same time as they can, for example, draw on top of the video, in real time. I’ve been using QT but unfortunately the camera module does not work on windows which led me to use ffmpeg to record the video/audio.
My Camera module is now working well besides a slight problem with syncing (which I asked for a solution in another thread). The problem is that when I start recording the performance drops (around 50-60% cpu usage). I’m not sure where or if I can improve on this. I figure the problem comes from the running threads. I have followed the dranger tutorial to help me develop my code and I am use 3 threads in total. One for capturing the video and audio packets, adding them to a queue, one to process the video packets and one to process the audio packets. While processing the video packets I also have to convert the frame (which comes raw from the camera feed) to a RGB format so that QT is able to show it. The frame also has to be converted to YUV420P to be saved to the mp4 file. This could also be hindering performance.
Every time I try to get a packet from the queue I verify if it is empty and, if it is, I tell the thread to sleep until more data is available as this helps saving CPU usage. The problem is sometimes the threads don’t wake up on time and the queue starts filling up with packages adding a cumulative delay, which never stops.
Bellow is a part of the code I am using for the 3 threads.
One thread is capturing both audio and video packets and adding them on a queue :
void FFCapture::grabFrames(int videoStream, int audioStream) {
grabActive = true;
videoQueue.clear();
audioQueue.clear();
AVPacket pkt;
while (av_read_frame(iFormatContext, &pkt) >= 0 && active) {
if(pkt.stream_index == videoStream) {
enqueueVideoPacket(pkt);
}
else if(pkt.stream_index == audioStream) {
enqueueAudioPacket(pkt);
}
else {
av_free_packet(&pkt);
}
//QThread::msleep(20);
}
//Wake up threads that might be sleeping
videoWait.wakeOne();
audioWait.wakeOne();
grabActive = false;
cleanupAll();
}And then I have one thread for each stream (video and audio) :
void FFCapture::captureVideoFrames() {
videoActive = true;
outStream.videoPTS = 2;
int min = 0;
if(cacheMs > 0) {
QThread::msleep(cacheMs);
min = getVideoQueueSize();
}
while(active || (!active && (getVideoQueueSize() > min))) {
qDebug() << "Video:" << videoQueue.size() << min;
AVPacket pkt;
if(dequeueVideoPacket(pkt, min) >= 0) {
if(processVideoPacket(&pkt) < 0) {
av_free_packet(&pkt);
break;
}
av_free_packet(&pkt);
}
}
videoActive = false;
cleanupAll();
}
void FFCapture::captureAudioFrames() {
audioActive = true;
outStream.audioPTS = 0;
int min = 0;
if(cacheMs > 0) {
QThread::msleep(cacheMs);
min = getAudioQueueSize();
}
while(active || (!active && (getAudioQueueSize() > min))) {
qDebug() << "Audio:" << audioQueue.size() << min;
AVPacket pkt;
if(dequeueAudioPacket(pkt, min) >= 0) {
if(recording) {
if(processAudioPacket(&pkt) < 0) break;
}
else av_free_packet(&pkt);
}
}
audioActive = false;
cleanupAll();
}When I remove a packet from the queue I verify if it is empty and if it is I tell the thread to wait for more data. The code is as follows :
void FFCapture::enqueueVideoPacket(const AVPacket &pkt) {
QMutexLocker locker(&videoQueueMutex);
videoQueue.enqueue(pkt);
videoWait.wakeOne();
}
int FFCapture::dequeueVideoPacket(AVPacket &pkt, int sizeConstraint) {
QMutexLocker locker(&videoQueueMutex);
while(1) {
if(videoQueue.size() > sizeConstraint) {
pkt = videoQueue.dequeue();
return 0;
}
else if(!active) {
return -1;
}
else {
videoWait.wait(&videoQueueMutex);
}
}
return -2; //Should never happen. Just to avoid compile error.
} -
Issues with Video Recording Duration and Smooth Playback when Using v4l2 to MP4 (FFmpeg)
9 décembre 2024, par ReenaI'm trying to record a video from a USB device with v4l2 framework and save it in MP4 format using FFmpeg. My sample code successfully captures and saves the video, but I'm running into some issues :


The recorded video duration is shorter than expected. For instance :


When recording a 1-minute video at 1280x720, the output file only has 58 or 59 seconds.
For 1920x1080, the duration is even more off — only about 28 or 30 seconds, instead of the expected 1 minute.
Additionally, the video is not smooth. There are noticeable drops in frames and playback inconsistencies.


My setup :


Using a USB device with v4l2 framework
Saving the video in MP4 format
Tested with different resolutions (1280x720, 1920x1080)
I've attached my sample code below. Could someone help me figure out why I'm experiencing these issues with video duration and smooth playback ?


Any advice, fixes, or suggestions would be greatly appreciated !


#include 
#include 
#include 
#include 
#include <sys></sys>ioctl.h>
#include <sys></sys>mman.h>
#include <linux></linux>videodev2.h>
#include <libavformat></libavformat>avformat.h>
#include <libavcodec></libavcodec>avcodec.h>
#include <libavutil></libavutil>imgutils.h>
#include <libavutil></libavutil>opt.h>
#include <libswscale></libswscale>swscale.h>
#include 
#include <sys></sys>time.h>
#include 

#define WIDTH 1280
#define HEIGHT 720
#define FPS 30
#define DURATION 10 // Recording duration in seconds
#define BUFFER_COUNT 4 // Number of buffers

struct buffer {
 void *start;
 size_t length;
};

struct buffer *buffers;

void open_device(int *fd, const char *device) {
 *fd = open(device, O_RDWR | O_NONBLOCK);
 if (*fd < 0) {
 perror("Cannot open video device");
 exit(1);
 }
}

void init_mmap(int fd) {
 struct v4l2_requestbuffers req;
 memset(&req, 0, sizeof(req));
 req.count = BUFFER_COUNT;
 req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 req.memory = V4L2_MEMORY_MMAP;

 if (ioctl(fd, VIDIOC_REQBUFS, &req) < 0) {
 perror("Requesting buffer");
 exit(1);
 }

 buffers = calloc(req.count, sizeof(*buffers));
 for (size_t i = 0; i < req.count; ++i) {
 struct v4l2_buffer buf;
 memset(&buf, 0, sizeof(buf));
 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 buf.memory = V4L2_MEMORY_MMAP;
 buf.index = i;

 if (ioctl(fd, VIDIOC_QUERYBUF, &buf) < 0) {
 perror("Querying buffer");
 exit(1);
 }

 buffers[i].length = buf.length;
 buffers[i].start = mmap(NULL, buf.length, PROT_READ | PROT_WRITE, MAP_SHARED, fd, buf.m.offset);

 if (MAP_FAILED == buffers[i].start) {
 perror("mmap");
 exit(1);
 }
 }
}

void start_capturing(int fd) {
 for (size_t i = 0; i < BUFFER_COUNT; ++i) {
 struct v4l2_buffer buf;
 memset(&buf, 0, sizeof(buf));
 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 buf.memory = V4L2_MEMORY_MMAP;
 buf.index = i;

 if (ioctl(fd, VIDIOC_QBUF, &buf) < 0) {
 perror("Queue buffer");
 exit(1);
 }
 }

 enum v4l2_buf_type type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 if (ioctl(fd, VIDIOC_STREAMON, &type) < 0) {
 perror("Start capture");
 exit(1);
 }
}

void stop_capturing(int fd) {
 enum v4l2_buf_type type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

 if (ioctl(fd, VIDIOC_STREAMOFF, &type) < 0) {
 perror("Stop capture");
 exit(1);
 }

 printf("Video capture stopped.\n");
}

void unmap_buffers() {
 for (size_t i = 0; i < BUFFER_COUNT; ++i) {
 if (munmap(buffers[i].start, buffers[i].length) < 0) {
 perror("munmap");
 exit(1);
 }
 }

 free(buffers);
}

void initialize_ffmpeg(AVFormatContext **fmt_ctx, AVCodecContext **codec_ctx, AVStream **video_stream, const char *filename) {
 av_register_all();

 AVOutputFormat *fmt = av_guess_format(NULL, filename, NULL);
 if (!fmt) {
 fprintf(stderr, "Could not determine output format\n");
 exit(1);
 }

 if (avformat_alloc_output_context2(fmt_ctx, fmt, NULL, filename) < 0) {
 fprintf(stderr, "Could not allocate format context\n");
 exit(1);
 }

 AVCodec *codec = avcodec_find_encoder(AV_CODEC_ID_H264);
 if (!codec) {
 fprintf(stderr, "Codec not found\n");
 exit(1);
 }

 *video_stream = avformat_new_stream(*fmt_ctx, NULL);
 if (!*video_stream) {
 fprintf(stderr, "Could not create stream\n");
 exit(1);
 }

 *codec_ctx = avcodec_alloc_context3(codec);
 if (!*codec_ctx) {
 fprintf(stderr, "Could not allocate codec context\n");
 exit(1);
 }

 (*codec_ctx)->codec_type = AVMEDIA_TYPE_VIDEO;
 (*codec_ctx)->width = WIDTH;
 (*codec_ctx)->height = HEIGHT;
 (*codec_ctx)->time_base = (AVRational){1, FPS};
 (*codec_ctx)->framerate = (AVRational){FPS, 1};
 (*codec_ctx)->pix_fmt = AV_PIX_FMT_YUV420P;
 (*codec_ctx)->gop_size = 10;
 (*codec_ctx)->max_b_frames = 1;

 av_opt_set(*codec_ctx, "preset", "fast", 0);
 av_opt_set_int(*codec_ctx, "crf", 23, 0);

 (*video_stream)->time_base = (*codec_ctx)->time_base;
 (*video_stream)->codecpar->codec_id = fmt->video_codec;
 (*video_stream)->codecpar->codec_type = AVMEDIA_TYPE_VIDEO;
 (*video_stream)->codecpar->width = (*codec_ctx)->width;
 (*video_stream)->codecpar->height = (*codec_ctx)->height;
 (*video_stream)->codecpar->format = (*codec_ctx)->pix_fmt;

 if ((*fmt_ctx)->oformat->flags & AVFMT_GLOBALHEADER) {
 (*codec_ctx)->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
 }

 if (avcodec_open2(*codec_ctx, codec, NULL) < 0) {
 fprintf(stderr, "Could not open codec\n");
 exit(1);
 }

 if (avcodec_parameters_from_context((*video_stream)->codecpar, *codec_ctx) < 0) {
 fprintf(stderr, "Could not copy codec parameters\n");
 exit(1);
 }

 if (!(fmt->flags & AVFMT_NOFILE)) {
 if (avio_open(&(*fmt_ctx)->pb, filename, AVIO_FLAG_WRITE) < 0) {
 fprintf(stderr, "Could not open output file\n");
 exit(1);
 }
 }

 if (avformat_write_header(*fmt_ctx, NULL) < 0) {
 fprintf(stderr, "Could not write header\n");
 exit(1);
 }
}

void capture_and_encode(int fd, AVFormatContext *fmt_ctx, AVCodecContext *codec_ctx, struct SwsContext *sws_ctx, AVStream *video_stream, int duration) {
 struct v4l2_buffer buffer;
 AVFrame *frame = av_frame_alloc();
 AVPacket packet;
 av_init_packet(&packet);

 frame->format = codec_ctx->pix_fmt;
 frame->width = codec_ctx->width;
 frame->height = codec_ctx->height;
 av_image_alloc(frame->data, frame->linesize, codec_ctx->width, codec_ctx->height, codec_ctx->pix_fmt, 32);

 struct timespec start_time;
 clock_gettime(CLOCK_MONOTONIC, &start_time);
 double elapsed_time = 0;
 int64_t pts_counter = 0;
 int frame_count = 0;

 while (elapsed_time < duration) {
 fd_set fds;
 struct timeval tv;
 int r;

 FD_ZERO(&fds);
 FD_SET(fd, &fds);

 tv.tv_sec = 2;
 tv.tv_usec = 0;

 r = select(fd + 1, &fds, NULL, NULL, &tv);
 if (r == -1) {
 perror("select");
 exit(1);
 }

 if (r == 0) {
 fprintf(stderr, "select timeout\n");
 exit(1);
 }

 memset(&buffer, 0, sizeof(buffer));
 buffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 buffer.memory = V4L2_MEMORY_MMAP;

 if (ioctl(fd, VIDIOC_DQBUF, &buffer) < 0) {
 if (errno == EAGAIN) continue;
 perror("Could not dequeue buffer");
 exit(1);
 }

 uint8_t *src_slices[1] = {buffers[buffer.index].start};
 int src_stride[1] = {WIDTH * 2}; // UYVY is 2 bytes per pixel

 sws_scale(sws_ctx, src_slices, src_stride, 0, HEIGHT, frame->data, frame->linesize);

 frame->pts = pts_counter;
 pts_counter += av_rescale_q(1, (AVRational){1, FPS}, codec_ctx->time_base);

 if (avcodec_send_frame(codec_ctx, frame) < 0) {
 fprintf(stderr, "Error sending frame\n");
 exit(1);
 }

 while (avcodec_receive_packet(codec_ctx, &packet) == 0) {
 av_packet_rescale_ts(&packet, codec_ctx->time_base, video_stream->time_base);
 packet.stream_index = video_stream->index;

 if (av_interleaved_write_frame(fmt_ctx, &packet) < 0) {
 fprintf(stderr, "Error writing frame\n");
 exit(1);
 }

 av_packet_unref(&packet);
 }
 printf("Processed frame %d\n", frame_count);

 if (ioctl(fd, VIDIOC_QBUF, &buffer) < 0) {
 perror("Could not requeue buffer");
 exit(1);
 }
 frame_count++;
 struct timespec current_time;
 clock_gettime(CLOCK_MONOTONIC, &current_time);
 elapsed_time = (current_time.tv_sec - start_time.tv_sec) + (current_time.tv_nsec - start_time.tv_nsec) / 1e9;
 printf("Elapsed time: %f seconds\n", elapsed_time);
 }

 av_freep(&frame->data[0]);
 av_frame_free(&frame);
 printf("Total frames processed: %d\n", frame_count);
}

int main(int argc, char *argv[]) {
 if (argc != 2) {
 fprintf(stderr, "Usage: %s <output file="file">\n", argv[0]);
 exit(1);
 }

 const char *output_file = argv[1];
 int fd;
 open_device(&fd, "/dev/video2");

 struct v4l2_format fmt;
 memset(&fmt, 0, sizeof(fmt));
 fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 fmt.fmt.pix.width = WIDTH;
 fmt.fmt.pix.height = HEIGHT;
 fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_UYVY;
 fmt.fmt.pix.field = V4L2_FIELD_NONE;

 if (ioctl(fd, VIDIOC_S_FMT, &fmt) < 0) {
 perror("Setting pixel format");
 exit(1);
 }

 if (fmt.fmt.pix.pixelformat != V4L2_PIX_FMT_UYVY) {
 fprintf(stderr, "Device does not support UYVY format\n");
 exit(1);
 }

 init_mmap(fd);
 start_capturing(fd);

 AVFormatContext *fmt_ctx = NULL;
 AVCodecContext *codec_ctx = NULL;
 AVStream *video_stream = NULL;

 initialize_ffmpeg(&fmt_ctx, &codec_ctx, &video_stream, output_file);

 struct SwsContext *sws_ctx = sws_getContext(WIDTH, HEIGHT, AV_PIX_FMT_UYVY422,
 WIDTH, HEIGHT, AV_PIX_FMT_YUV420P,
 SWS_BICUBIC, NULL, NULL, NULL);

 if (!sws_ctx) {
 fprintf(stderr, "Could not initialize SwsContext\n");
 exit(1);
 }

 capture_and_encode(fd, fmt_ctx, codec_ctx, sws_ctx, video_stream, DURATION);

 sws_freeContext(sws_ctx);
 av_write_trailer(fmt_ctx);
 avcodec_free_context(&codec_ctx);
 avformat_free_context(fmt_ctx);
 stop_capturing(fd);
 unmap_buffers();
 close(fd);

 return 0;
}
</output>

Thank you !


Syndiquer tout le site
