Recherche avancée
Médias (91)
-
MediaSPIP Simple : futur thème graphique par défaut ?
26 septembre 2013, par
Mis à jour : Octobre 2013
Langue : français
Type : Video
-
avec chosen
13 septembre 2013, par
Mis à jour : Septembre 2013
Langue : français
Type : Image
-
sans chosen
13 septembre 2013, par
Mis à jour : Septembre 2013
Langue : français
Type : Image
-
config chosen
13 septembre 2013, par
Mis à jour : Septembre 2013
Langue : français
Type : Image
-
SPIP - plugins - embed code - Exemple
2 septembre 2013, par
Mis à jour : Septembre 2013
Langue : français
Type : Image
-
GetID3 - Bloc informations de fichiers
9 avril 2013, par
Mis à jour : Mai 2013
Langue : français
Type : Image
Autres articles (38)
-
MediaSPIP Core : La Configuration
9 novembre 2010, parMediaSPIP Core fournit par défaut trois pages différentes de configuration (ces pages utilisent le plugin de configuration CFG pour fonctionner) : une page spécifique à la configuration générale du squelettes ; une page spécifique à la configuration de la page d’accueil du site ; une page spécifique à la configuration des secteurs ;
Il fournit également une page supplémentaire qui n’apparait que lorsque certains plugins sont activés permettant de contrôler l’affichage et les fonctionnalités spécifiques (...) -
Encodage et transformation en formats lisibles sur Internet
10 avril 2011MediaSPIP transforme et ré-encode les documents mis en ligne afin de les rendre lisibles sur Internet et automatiquement utilisables sans intervention du créateur de contenu.
Les vidéos sont automatiquement encodées dans les formats supportés par HTML5 : MP4, Ogv et WebM. La version "MP4" est également utilisée pour le lecteur flash de secours nécessaire aux anciens navigateurs.
Les documents audios sont également ré-encodés dans les deux formats utilisables par HTML5 :MP3 et Ogg. La version "MP3" (...) -
Support de tous types de médias
10 avril 2011Contrairement à beaucoup de logiciels et autres plate-formes modernes de partage de documents, MediaSPIP a l’ambition de gérer un maximum de formats de documents différents qu’ils soient de type : images (png, gif, jpg, bmp et autres...) ; audio (MP3, Ogg, Wav et autres...) ; vidéo (Avi, MP4, Ogv, mpg, mov, wmv et autres...) ; contenu textuel, code ou autres (open office, microsoft office (tableur, présentation), web (html, css), LaTeX, Google Earth) (...)
Sur d’autres sites (4972)
-
Revision 37301 : une image qui n’est plus nécessaire
15 avril 2010, par kent1@… — Logune image qui n’est plus nécessaire
-
FFMpeg CUDA yuvj420p frame conversion to cv::Mat layers shifted
26 février 2023, par AcidTonicI am trying to retrieve hardware decoded H264 frames from the cuda backend of ffmpeg and display them as a cv::Mat. I got decently far and was able to get color images but it seems the conversion is not quite right as the image I get has a green bar at the top and if you look closely the blue parts of the image are offset down and to the right a little bit making everything look a little wonky.


Correct Image as shown by ffplay using the same driver


Image I am getting


Here is the full source code in the hopes someone can help me to get the correct image here...


#include 

#include 

#include 

#include 

#include 

#include 

#include <iostream>

#include <fstream>

#include <cstdlib>

#include <chrono>

#include <cstring>

extern "C" {

 //Linker errors if not inside extern. FFMPEG headers are not C++ aware
 #include <libavcodec></libavcodec>avcodec.h>

 #include <libavformat></libavformat>avformat.h>

 #include <libavutil></libavutil>pixdesc.h>

 #include <libavutil></libavutil>hwcontext.h>

 #include <libavutil></libavutil>opt.h>

 #include <libavutil></libavutil>avassert.h>

 #include <libavutil></libavutil>imgutils.h>

}

#include <iomanip>

#include <string>

#include <sstream>

#include <opencv2></opencv2>opencv.hpp>

#ifdef __cplusplus
extern "C" {
 #endif // __cplusplus
 #include <libavdevice></libavdevice>avdevice.h>

 #include <libavfilter></libavfilter>avfilter.h>

 #include <libavformat></libavformat>avio.h>

 #include <libavutil></libavutil>avutil.h>

 #include <libpostproc></libpostproc>postprocess.h>

 #include <libswresample></libswresample>swresample.h>

 #include <libswscale></libswscale>swscale.h>

 #ifdef __cplusplus
} // end extern "C".
#endif // __cplusplus

static AVBufferRef * hw_device_ctx = NULL;
static enum AVPixelFormat hw_pix_fmt;
static FILE * output_file_fd = NULL;
cv::Mat output_mat;
int bgr_size;

static int hw_decoder_init(AVCodecContext * ctx,
 const enum AVHWDeviceType type) {
 int err = 0;

 if ((err = av_hwdevice_ctx_create( & hw_device_ctx, type,
 NULL, NULL, 0)) < 0) {
 fprintf(stderr, "Failed to create specified HW device.\n");
 return err;
 }
 ctx -> hw_device_ctx = av_buffer_ref(hw_device_ctx);

 return err;
}

static enum AVPixelFormat get_hw_format(AVCodecContext * ctx,
 const enum AVPixelFormat * pix_fmts) {
 const enum AVPixelFormat * p;

 for (p = pix_fmts;* p != -1; p++) {
 if ( * p == hw_pix_fmt)
 return * p;
 }

 fprintf(stderr, "Failed to get HW surface format.\n");
 return AV_PIX_FMT_NONE;
}

static int decode_write(AVCodecContext * avctx, AVPacket * packet) {
 AVFrame * frame = NULL, * sw_frame = NULL;
 AVFrame * tmp_frame = NULL;
 uint8_t * buffer = NULL;
 int size;
 int ret = 0;

 ret = avcodec_send_packet(avctx, packet);
 if (ret < 0) {
 fprintf(stderr, "Error during decoding\n");
 return ret;
 }

 while (1) {
 if (!(frame = av_frame_alloc()) || !(sw_frame = av_frame_alloc())) {
 fprintf(stderr, "Can not alloc frame\n");
 ret = AVERROR(ENOMEM);
 av_frame_free( & frame);
 av_frame_free( & sw_frame);
 av_freep( & buffer);
 if (ret < 0) {
 return ret;
 }

 }

 ret = avcodec_receive_frame(avctx, frame);
 if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
 av_frame_free( & frame);
 av_frame_free( & sw_frame);
 return 0;
 } else if (ret < 0) {
 fprintf(stderr, "Error while decoding\n");
 av_frame_free( & frame);
 av_frame_free( & sw_frame);
 av_freep( & buffer);
 if (ret < 0) {
 return ret;
 }

 }

 if (frame -> format == hw_pix_fmt) {
 /* retrieve data from GPU to CPU */
 if ((ret = av_hwframe_transfer_data(sw_frame, frame, 0)) < 0) {
 fprintf(stderr, "Error transferring the data to system memory\n");
 av_frame_free( & frame);
 av_frame_free( & sw_frame);
 av_freep( & buffer);
 if (ret < 0) {
 return ret;
 }

 }
 tmp_frame = sw_frame;
 } else {
 tmp_frame = frame;
 }

 AVPixelFormat format_to_use = AV_PIX_FMT_YUVJ420P;
 cv::Mat mat_src = cv::Mat(sw_frame -> height + (sw_frame -> height / 2), sw_frame -> width, CV_8UC1, sw_frame -> data[0]);
 cv::Mat out_mat;
 cv::cvtColor(mat_src, out_mat, cv::COLOR_YUV2RGB_NV21);

 output_mat = out_mat;

 if (output_mat.empty() == false) {
 cv::imshow("image", output_mat);
 cv::waitKey(1);
 }

 av_frame_free( & frame);
 av_frame_free( & sw_frame);
 av_freep( & buffer);
 return ret;
 }
}

TEST_CASE("CUDAH264", "Tests hardware h264 decoding") {

 AVFormatContext * input_ctx = NULL;
 int video_stream, ret;
 AVStream * video = NULL;
 AVCodecContext * decoder_ctx = NULL;
 AVCodec * decoder = NULL;
 AVPacket * packet = NULL;
 enum AVHWDeviceType type;
 int i;

 std::string device_type = "cuda";
 std::string input_file = "rtsp://10.100.2.152"; //My H264 network stream here...

 /* The stream data is below...
 Input #0, rtsp, from 'rtsp://10.100.2.152':
 Metadata:
 title : VCP IPC Realtime stream
 Duration: N/A, start: 0.000000, bitrate: N/A
 Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1920x1080, 10 fps, 10 tbr, 90k tbn, 20 tbc
 */

 type = av_hwdevice_find_type_by_name(device_type.c_str());
 if (type == AV_HWDEVICE_TYPE_NONE) {
 fprintf(stderr, "Device type %s is not supported.\n", device_type.c_str());
 fprintf(stderr, "Available device types:");
 while ((type = av_hwdevice_iterate_types(type)) != AV_HWDEVICE_TYPE_NONE)
 fprintf(stderr, " %s", av_hwdevice_get_type_name(type));
 fprintf(stderr, "\n");
 throw std::runtime_error("Error");
 }

 packet = av_packet_alloc();
 if (!packet) {
 fprintf(stderr, "Failed to allocate AVPacket\n");
 throw std::runtime_error("Error");
 }

 /* open the input file */
 if (avformat_open_input( & input_ctx, input_file.c_str(), NULL, NULL) != 0) {
 fprintf(stderr, "Cannot open input file '%s'\n", input_file.c_str());
 throw std::runtime_error("Error");
 }

 if (avformat_find_stream_info(input_ctx, NULL) < 0) {
 fprintf(stderr, "Cannot find input stream information.\n");
 throw std::runtime_error("Error");
 }

 av_dump_format(input_ctx, 0, input_file.c_str(), 0);

 for (int i = 0; i < input_ctx -> nb_streams; i++) {
 auto pCodec = avcodec_find_decoder(input_ctx -> streams[i] -> codecpar -> codec_id);
 auto pCodecCtx = avcodec_alloc_context3(pCodec);
 avcodec_parameters_to_context(pCodecCtx, input_ctx -> streams[i] -> codecpar);

 printf("Found Video stream with ID: %d\n", input_ctx -> streams[i] -> id);
 printf("\t Stream Index: %d\n", input_ctx -> streams[i] -> index);

 AVCodecParameters * codecpar = input_ctx -> streams[i] -> codecpar;
 printf("\t Codec Type: %s\n", av_get_media_type_string(codecpar -> codec_type));
 printf("\t Side data count: %d\n", input_ctx -> streams[i] -> nb_side_data);
 printf("\t Pixel format: %i\n", input_ctx -> streams[i] -> codecpar -> format);
 printf("\t Pixel Format Name: %s\n", av_get_pix_fmt_name((AVPixelFormat) input_ctx -> streams[i] -> codecpar -> format));
 printf("\t Metadata count: %d\n", av_dict_count(input_ctx -> streams[i] -> metadata));
 }

 /* find the video stream information */
 ret = av_find_best_stream(input_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, & decoder, 0);
 if (ret < 0) {
 fprintf(stderr, "Cannot find a video stream in the input file\n");
 throw std::runtime_error("Error");
 }

 video_stream = ret;

 for (i = 0;; i++) {
 const AVCodecHWConfig * config = avcodec_get_hw_config(decoder, i);
 if (!config) {
 fprintf(stderr, "Decoder %s does not support device type %s.\n",
 decoder -> name, av_hwdevice_get_type_name(type));
 throw std::runtime_error("Error");
 }
 if (config -> methods & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX &&
 config -> device_type == type) {
 hw_pix_fmt = config -> pix_fmt;
 break;
 }
 }

 if (!(decoder_ctx = avcodec_alloc_context3(decoder))) {
 throw std::runtime_error("NO MEMORY");
 }

 video = input_ctx -> streams[video_stream];
 if (avcodec_parameters_to_context(decoder_ctx, video -> codecpar) < 0) {
 throw std::runtime_error("Error");
 }

 decoder_ctx -> get_format = get_hw_format;

 if (hw_decoder_init(decoder_ctx, type) < 0) {
 throw std::runtime_error("Error");
 }

 if ((ret = avcodec_open2(decoder_ctx, decoder, NULL)) < 0) {
 fprintf(stderr, "Failed to open codec for stream #%u\n", video_stream);
 throw std::runtime_error("Error");
 }

 /* actual decoding and dump the raw data */
 while (ret >= 0) {
 if ((ret = av_read_frame(input_ctx, packet)) < 0)
 break;

 if (video_stream == packet -> stream_index)
 ret = decode_write(decoder_ctx, packet);

 av_packet_unref(packet);
 }

 /* flush the decoder */
 ret = decode_write(decoder_ctx, NULL);

 if (output_file_fd) {
 fclose(output_file_fd);
 }
 av_packet_free( & packet);
 avcodec_free_context( & decoder_ctx);
 avformat_close_input( & input_ctx);
 av_buffer_unref( & hw_device_ctx);

}
</sstream></string></iomanip></cstring></chrono></cstdlib></fstream></iostream>

-
The 11th Hour RoQ Variation
12 avril 2012, par Multimedia Mike — Game Hacking, dreamroq, Reverse Engineering, roq, Vector QuantizationI have been looking at the RoQ file format almost as long as I have been doing practical multimedia hacking. However, I have never figured out how the RoQ format works on The 11th Hour, which was the game for which the RoQ format was initially developed. When I procured the game years ago, I remember finding what appeared to be RoQ files and shoving them through the open source decoders but not getting the right images out.
I decided to dust off that old copy of The 11th Hour and have another go at it.
Baseline
The game consists of 4 CD-ROMs. Each disc has a media/ directory that has a series of files bearing the extension .gjd, likely the initials of one Graeme J. Devine. These are resource files which are merely headerless concatenations of other files. Thus, at first glance, one file might appear to be a single RoQ file. So that’s the source of some of the difficulty : Sending an apparent RoQ .gjd file through a RoQ player will often cause the program to complain when it encounters the header of another RoQ file.I have uploaded some samples to the usual place.
However, even the frames that a player can decode (before encountering a file boundary within the resource file) look wrong.
Investigating Codebooks Using dreamroq
I wrote dreamroq last year– an independent RoQ playback library targeted towards embedded systems. I aimed it at a gjd file and quickly hit a codebook error.RoQ is a vector quantizer video codec that maintains a codebook of 256 2×2 pixel vectors. In the Quake III and later RoQ files, these are transported using a YUV 4:2:0 colorspace– 4 Y samples, a U sample, and a V sample to represent 4 pixels. This totals 6 bytes per vector. A RoQ codebook chunk contains a field that indicates the number of 2×2 vectors as well as the number of 4×4 vectors. The latter vectors are each comprised of 4 2×2 vectors.
Thus, the total size of a codebook chunk ought to be (# of 2×2 vectors) * 6 + (# of 4×4 vectors) * 4.
However, this is not the case with The 11th Hour RoQ files.
Longer Codebooks And Mystery Colorspace
Juggling the numbers for a few of the codebook chunks, I empirically determined that the 2×2 vectors are represented by 10 bytes instead of 6. Now I need to determine what exactly these 10 bytes represent.I should note that I suspect that everything else about these files lines up with successive generations of the format. For example if a file has 640×320 resolution, that amounts to 40×20 macroblocks. dreamroq iterates through 40×20 8×8 blocks and precisely exhausts the VQ bitstream. So that all looks valid. I’m just puzzled on the codebook format.
Here is an example codebook dump :
ID 0x1002, len = 0x0000014C, args = 0x1C0D 0 : 00 00 00 00 00 00 00 00 80 80 1 : 08 07 00 00 1F 5B 00 00 7E 81 2 : 00 00 15 0F 00 00 40 3B 7F 84 3 : 00 00 00 00 3A 5F 18 13 7E 84 4 : 00 00 00 00 3B 63 1B 17 7E 85 5 : 18 13 00 00 3C 63 00 00 7E 88 6 : 00 00 00 00 00 00 59 3B 7F 81 7 : 00 00 56 23 00 00 61 2B 80 80 8 : 00 00 2F 13 00 00 79 63 81 83 9 : 00 00 00 00 5E 3F AC 9B 7E 81 10 : 1B 17 00 00 B6 EF 77 AB 7E 85 11 : 2E 43 00 00 C1 F7 75 AF 7D 88 12 : 6A AB 28 5F B6 B3 8C B3 80 8A 13 : 86 BF 0A 03 D5 FF 3A 5F 7C 8C 14 : 00 00 9E 6B AB 97 F5 EF 7F 80 15 : 86 73 C8 CB B6 B7 B7 B7 85 8B 16 : 31 17 84 6B E7 EF FF FF 7E 81 17 : 79 AF 3B 5F FC FF E2 FF 7D 87 18 : DC FF AE EF B3 B3 B8 B3 85 8B 19 : EF FF F5 FF BA B7 B6 B7 88 8B 20 : F8 FF F7 FF B3 B7 B7 B7 88 8B 21 : FB FF FB FF B8 B3 B4 B3 85 88 22 : F7 FF F7 FF B7 B7 B9 B7 87 8B 23 : FD FF FE FF B9 B7 BB B7 85 8A 24 : E4 FF B7 EF FF FF FF FF 7F 83 25 : FF FF AC EB FF FF FC FF 7F 83 26 : CC C7 F7 FF FF FF FF FF 7F 81 27 : FF FF FE FF FF FF FF FF 80 80
Note that 0x14C (the chunk size) = 332, 0x1C and 0x0D (the chunk arguments — count of 2×2 and 4×4 vectors, respectively) are 28 and 13. 28 * 10 + 13 * 4 = 332, so the numbers check out.
Do you see any patterns in the codebook ? Here are some things I tried :
- Treating the last 2 bytes as U & V and treating the first 4 as the 4 Y samples :
- Treating the last 2 bytes as U & V and treating the first 8 as 4 16-bit little-endian Y samples :
- Disregarding the final 2 bytes and treating the first 8 bytes as 4 RGB565 pixels (both little- and big-endian, respectively, shown here) :
- Based on the type of data I’m seeing in these movies (which appears to be intended as overlays), I figured that some of these bits might indicate transparency ; here is 15-bit big-endian RGB which disregards the top bit of each pixel :
These images are taken from the uploaded sample bdpuz.gjd, apparently a component of the puzzle represented in this screenshot.
Unseen Types
It has long been rumored that early RoQ files could contain JPEG images. I finally found one such specimen. One of the files bundled early in the uploaded fhpuz.gjd sample contains a JPEG frame. It’s a standard JFIF file and can easily be decoded after separating the bytes from the resource using ‘dd’. JPEGs serve as intraframes in the coding scheme, with successive RoQ frames moving objects on top.However, a new chunk type showed up as well, one identified by 0×1030. I have never encountered this type. Where could I possibly find data about this ? Fortunately, iD Games recently posted all of their open sourced games at Github. Reading through the code for their official RoQ decoder, I see that this is called a RoQ_PACKET. The name and the code behind it are both supremely unhelpful. The code is basically a no-op. The payloads of the various RoQ_PACKETs from one sample are observed to be either 8784, 14752, or 14760 bytes in length. It’s very likely that this serves the same purpose as the JPEG intraframes.
Other Tidbits
I read through the readme.txt on the first game disc and found this nugget :g) Animations displayed normally or in SPOOKY MODESPOOKY MODE is blue-tinted grayscale with color cursors, puzzle
and game pieces. It is the preferred display setting of the
developers at Trilobyte. Just for fun, try out the SPOOKY
MODE.The MobyGames screenshot page has a number of screenshots labeled as being captured in spooky mode. Color tricks ?
Meanwhile, another twist arose as I kept tweaking dreamroq to deal with more RoQ weirdness : After modifying my dreamroq code to handle these 10-byte vectors, it eventually chokes on another codebook. These codebooks happen to have 6-byte vectors again ! Fortunately, I was already working on a scheme to automatically detect which codebook is in play (plugging the numbers into a formula and seeing which vector size checks out).
- Treating the last 2 bytes as U & V and treating the first 4 as the 4 Y samples :
Syndiquer tout le site
