Recherche avancée

Sur d’autres sites (3877)

• How should I write my privacy notice for Matomo Analytics under GDPR ?

  24 avril 2018, par InnoCraft

  Important note : this blog post has been written by digital analysts, not lawyers. The purpose of this article is to show you an example of a privacy notice for Matomo under GDPR. This work comes from our interpretation of the UK privacy commission : ICO. It cannot be considered as professional legal advice. So as GDPR, this information is subject to change. We strongly advise you to have a look at the different privacy authorities in order to have up to date information.

  A basic rule of thumb is that if you are not processing personal data, then you do not need to show any privacy notice. But if you are doing so, such as processing full IP addresses, then a privacy notice is required at the time of the data collection. Please note that personal data may also be hidden, for example, in page titles or page URLs.

  In this blog post, we will define what a privacy notice is according to GDPR and how to write it if you are using Matomo and you are processing personal data.

  What is a privacy notice under GDPR ?

  One of the most important rights that a data subject has under GDPR, is the right to be informed about the collection and use of their personal data.

  Here is what ICO is saying about the privacy notice :

  “You must provide individuals with information including : your purposes for processing their personal data, your retention periods for that personal data, and who it will be shared with. We call this ‘privacy information’.”

  “When you collect personal data from the individual it relates to, you must provide them with privacy information at the time you obtain their data.”

  Note that a privacy notice is different from a privacy policy.

  The privacy notice has to include :

  • the reasons why you are processing the personal data
  • for how long
  • who the different parties you are going to share them with are

  So whatever lawful basis you are using (explicit consent or legitimate interest), you need to have a privacy notice if you collect personal data.

  What does this privacy notice look like ?

  ICO is providing best practices in order to display the information :

  • a layered approach
  • dashboards
  • just-in-time notices
  • icons
  • mobile and smart device functionalities

  Once more, it really depends on the data you are processing with Matomo. If you wish to track personal data on the entire website, you will probably have an upper or footer privacy notice such as :

  

  If you wish to process specific data, you could also insert just-in-time notices such as :

  

  What is the information you need to disclose to the final user ?

  To us, there are two things to distinguish between the privacy notice and the privacy policy.

  According to ICO, the privacy notice needs to include the 3 following elements :

  • the reasons why you are processing the personal data
  • for how long
  • who are the different parties you are going to share them with

  But you also need to inform them about :

  • The name and contact details of your organisation.
  • The name and contact details of your representative (if applicable).
  • The contact details of your data protection officer (if applicable).
  • The purposes of the processing.
  • The lawful basis for the processing.
  • The legitimate interests for the processing (if applicable).
  • The categories of personal data obtained (if the personal data is not obtained from the individual it relates to).
  • The recipients or categories of recipients of the personal data.
  • The details of transfers of the personal data to any third countries or international organisations (if applicable).
  • The retention periods for the personal data.
  • The rights available to individuals in respect of the processing.
  • The right to withdraw consent (if applicable).
  • The right to lodge a complaint with a supervisory authority.
  • The source of the personal data (if the personal data is not obtained from the individual it relates to).
  • The details of whether individuals are under a statutory or contractual obligation to provide the personal data (if applicable, and if the personal data is collected from the individual it relates to).
  • The details of the existence of automated decision-making, including profiling (if applicable).

  Pretty long, don’t you think ? In order to reduce it, you can either adopt a layered approach where your “pop-up” window will act as a drop down menu. Or from what we understood, page 5 of this document provided by ICO, a privacy notice can link to a more detailed document, such as a privacy policy page.

  Examples

  Let’s take the example of a website which tracks the non-anonymised full IP address, and using User ID functionality to keep track of logged-in users. Under GDPR, the owner of the website will have to choose either to process personal data based on “Legitimate interests” or on “Consent”. Here is how it will look like :

  Example of a privacy notice under GDPR Legitimate interests

  This site uses Matomo to analyze traffic and help us to improve your user experience.

  We process your email address and IP address and cookies are stored on your browser for 13 months. This data is only processed by us and our web hosting platform. Please read our Privacy Policy to learn more.

  Example of a privacy notice under GDPR Consent

  This site uses Matomo to analyze traffic and help us to improve your user experience.

  We process your email address and IP address and cookies are stored on your browser for 13 months. This data is only processed by us and our web hosting platform.

  [Accept] or [Opt-out]

  Please read our Privacy Policy to learn more.

  Once that information is provided to the user, you can then link it to your privacy policy where you will provide more details about it. Soon we will issue a blog post dealing with how to write a privacy policy page for Matomo.

  The post How should I write my privacy notice for Matomo Analytics under GDPR ? appeared first on Analytics Platform - Matomo.

• send h264 video to nginx-rtmp server using ffmpeg API

  11 décembre 2019, par Glen

  I have C++ code that grabs frames from a GigE camera and writes them out to a file. I’m using the libx264 codec and ffmpeg version 4.0.

  Writing to the file works fine, however I would also like to send the video to nginx configured with the nginx-rtmp plug-in to make the video available live via HLS.

  I can use the ffmpeg command line program to stream one of my previously captured files to my nginx server and rebroadcast as HLS, however if I try to stream from my C++ code the nginx server closes the connection after one or two frames are sent.

  To test further, I used the ffmpeg command line program to receive a rtmp stream and write it out to a file. I am able to send video to ffmpeg from my C++ program with rtmp, however every frame generates a warning like this :

  [avi @ 0x1b6b6f0] Non-monotonous DTS in output stream 0:0; previous: 1771, current: 53; changing to 1772. This may result in incorrect timestamps in the output file.
  
  [avi @ 0x1b6b6f0] Non-monotonous DTS in output stream 0:0; previous: 1772, current: 53; changing to 1773. This may result in incorrect timestamps in the output file.
  
  [avi @ 0x1b6b6f0] Non-monotonous DTS in output stream 0:0; previous: 1773, current: 53; changing to 1774. This may result in incorrect timestamps in the output file.
  
  [avi @ 0x1b6b6f0] Non-monotonous DTS in output stream 0:0; previous: 1774, current: 53; changing to 1775. This may result in incorrect timestamps in the output file.
  
  [avi @ 0x1b6b6f0] Non-monotonous DTS in output stream 0:0; previous: 1775, current: 53; changing to 1776. This may result in incorrect timestamps in the output file.
  
  [avi @ 0x1b6b6f0] Non-monotonous DTS in output stream 0:0; previous: 1776, current: 53; changing to 1777. This may result in incorrect timestamps in the output file.
  
  [avi @ 0x1b6b6f0] Non-monotonous DTS in output stream 0:0; previous: 1777, current: 53; changing to 1778. This may result in incorrect timestamps in the output file.
  
  [avi @ 0x1b6b6f0] Non-monotonous DTS in output stream 0:0; previous: 1778, current: 53; changing to 1779. This may result in incorrect timestamps in the output file.
  
  [avi @ 0x1b6b6f0] Non-monotonous DTS in output stream 0:0; previous: 1779, current: 53; changing to 1780. This may result in incorrect timestamps in the output file.

  I printed PTS and DTS for my packet before writing it, and the numbers were monotonous (for example, in this last frame the pts and dts printed from my code were 1780, not the ’current : 53’ that ffmpeg reports>

  also, unless I tell ffmpeg what the output framerate should be I end up with a file that plays 2x speed.

  After ffmpeg receives the rtmp stream and writes it to the file, I am then able to successfully send that file to my nginx server using ffmpeg.

  here is some relevant code :

  //configuring the codec context
  
  // make sure that config.codec is something we support
  
  // for now we are only supporting LIBX264
  
  if (config.codec() != codecs::LIBX264) {
  
      throw std::invalid_argument("currently only libx264 codec is supported");
  
  }
  
  
  
  // lookup specified codec
  
  ffcodec_ = avcodec_find_encoder_by_name(config.codec().c_str());
  
  if (!ffcodec_) {
  
      throw std::invalid_argument("unable to get codec " + config.codec());
  
  }
  
  
  
  // unique_ptr to manage the codec_context
  
  codec_context_ = av_pointer::codec_context(avcodec_alloc_context3(ffcodec_));
  
  
  
  if (!codec_context_) {
  
      throw std::runtime_error("unable to initialize AVCodecContext");
  
  }
  
  
  
  // setup codec_context_
  
  codec_context_->width = frame_width;
  
  codec_context_->height = frame_height;
  
  codec_context_->time_base = (AVRational){1, config.target_fps()};
  
  codec_context_->framerate = (AVRational){config.target_fps(), 1};
  
  codec_context_->global_quality = 0;
  
  codec_context_->compression_level = 0;
  
  codec_context_->bits_per_raw_sample = 8;
  
  codec_context_->gop_size = 1;
  
  codec_context_->max_b_frames = 1;
  
  codec_context_->pix_fmt = AV_PIX_FMT_YUV420P;
  
  
  
  // x264 only settings
  
  if (config.codec() == codecs::LIBX264) {
  
      av_opt_set(codec_context_->priv_data, "preset", config.compression_target().c_str(), 0);
  
      av_opt_set(codec_context_->priv_data, "crf", std::to_string(config.crf()).c_str(), 0);
  
  }
  
  
  
  // Open up the codec
  
  if (avcodec_open2(codec_context_.get(), ffcodec_, NULL) < 0) {
  
      throw std::runtime_error("unable to open ffmpeg codec");
  
  }
  
  
  
  // setup the output format context and stream for RTMP
  
  AVFormatContext *tmp_f_context;
  
  avformat_alloc_output_context2(&tmp_f_context, NULL, "flv", uri.c_str());
  
  rtmp_format_context_ = av_pointer::format_context(tmp_f_context);
  
  rtmp_stream_ = avformat_new_stream(rtmp_format_context_.get(), ffcodec_);
  
  avcodec_parameters_from_context(rtmp_stream_->codecpar, codec_context_.get());
  
  rtmp_stream_->time_base = codec_context_->time_base;
  
  rtmp_stream_->r_frame_rate = codec_context_->framerate;
  
  
  
  /* open the output file */
  
  if (!(rtmp_format_context_->flags & AVFMT_NOFILE)) {
  
      int r = avio_open(&rtmp_format_context_->pb, uri.c_str(), AVIO_FLAG_WRITE);
  
      if (r < 0) {
  
          throw std::runtime_error("unable to open " + uri + " : " + av_err2str(r));
  
      }
  
  }
  
  
  
  if (avformat_write_header(rtmp_format_context_.get(), NULL) < 0) {
  
      throw std::runtime_error("unable to write header");
  
  }
  
  
  
  av_dump_format(rtmp_format_context_.get(), 0,uri.c_str() , 1);

  at this point the av_dump_format produces this output :

  Output #0, flv, to 'rtmp://[MY URI]':
  
    Metadata:
  
      encoder         : Lavf58.12.100
  
      Stream #0:0, 0, 1/1000: Video: h264 (libx264), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p, 800x800 (0x0), 0/1, q=-1--1, 30 tbr, 1k tbn

  encoding and writing the frame :

  // send the frame to the encoder, filtering first if necessary
  
  void VideoWriter::Encode(AVFrame *frame)
  
  {
  
      int rval;
  
      if (!apply_filter_) {
  
          //send frame to encoder
  
          rval = avcodec_send_frame(codec_context_.get(), frame);
  
          if (rval < 0) {
  
              throw std::runtime_error("error sending frame for encoding");
  
          }
  
      } else {
  
          // push frame to filter 
  
          // REMOVED, currently testing without filtering
  
      }
  
  
  
      // get packets from encoder
  
      while (rval >= 0) {
  
          // create smart pointer to allocated packet
  
          av_pointer::packet pkt(av_packet_alloc());
  
          if (!pkt) {
  
              throw std::runtime_error("unable to allocate packet");
  
          }
  
  
  
          rval = avcodec_receive_packet(codec_context_.get(), pkt.get());
  
          if (rval == AVERROR(EAGAIN) || rval == AVERROR_EOF) {
  
              return;
  
          } else if (rval < 0) {
  
              throw std::runtime_error("error during encoding");
  
          }
  
  
  
          // if I print pkt->pts and pkt->dts here, I see sequential numbers
  
  
  
          // write packet
  
          rval = av_interleaved_write_frame(rtmp_format_context_.get(), pkt.get());
  
          if (rval < 0 ) {
  
              std::cerr << av_err2str(rval) << std::endl;
  
          }
  
      }
  
  }

  Since I am able to send video from a previously recorded file to nginx with the ffmpeg command line program, I believe the problem is in my code and not my nginx configuration.

  EDIT : I think it may have to do with SPS/PPS as I see a bunch of these error messages in the nginx log before it closes the stream

  2019/12/11 11:11:31 [error] 10180#0: *4 hls: failed to read 5 byte(s), client: XXX, server: 0.0.0.0:1935
  
  2019/12/11 11:11:31 [error] 10180#0: *4 hls: error appenging SPS/PPS NALs, client: XXX, server: 0.0.0.0:1935

  As I mentioned, this code works fine if I set it up to write to an avi file rather stream to rtmp, and I can stream to ffmpeg listening for rtmp but with lots of warnings about the DTS but if I try to send to nginx, it closes the connection almost immediately. My first thought was that there was something wrong with the frame timestamps, but when I print pts and dts prior to writing the packet to the stream they look okay to me.

  My end goal is to capture video to a file, and also be able to turn on the rtmp stream on demand — but for now I’m just trying to get the rtmp stream working continuously (without writing to a file)

  Thanks for any insights.

• ffmpeg failed to load audio file

  14 avril 2024, par Vaishnav Ghenge

  Failed to load audio: ffmpeg version 5.1.4-0+deb12u1 Copyright (c) Failed to load audio: ffmpeg version 5.1.4-0+deb12u1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr --extra-version=0+deb12u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
/tmp/tmpjlchcpdm.wav: Invalid data found when processing input


  


  backend :

  


  
@app.route("/transcribe", methods=["POST"])
def transcribe():
    # Check if audio file is present in the request
    if 'audio_file' not in request.files:
        return jsonify({"error": "No file part"}), 400
    
    audio_file = request.files.get('audio_file')

    # Check if audio_file is sent in files
    if not audio_file:
        return jsonify({"error": "`audio_file` is missing in request.files"}), 400

    # Check if the file is present
    if audio_file.filename == '':
        return jsonify({"error": "No selected file"}), 400

    # Save the file with a unique name
    filename = secure_filename(audio_file.filename)
    unique_filename = os.path.join("uploads", str(uuid.uuid4()) + '_' + filename)
    # audio_file.save(unique_filename)
    
    # Read the contents of the audio file
    contents = audio_file.read()

    max_file_size = 500 * 1024 * 1024
    if len(contents) > max_file_size:
        return jsonify({"error": "File is too large"}), 400

    # Check if the file extension suggests it's a WAV file
    if not filename.lower().endswith('.wav'):
        # Delete the file if it's not a WAV file
        os.remove(unique_filename)
        return jsonify({"error": "Only WAV files are supported"}), 400

    print(f"\033[92m{filename}\033[0m")

    # Call Celery task asynchronously
    result = transcribe_audio.delay(contents)

    return jsonify({
        "task_id": result.id,
        "status": "pending"
    })


@celery_app.task
def transcribe_audio(contents):
    # Transcribe the audio
    try:
        # Create a temporary file to save the audio data
        with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as temp_audio:
            temp_path = temp_audio.name
            temp_audio.write(contents)

            print(f"\033[92mFile temporary path: {temp_path}\033[0m")
            transcribe_start_time = time.time()

            # Transcribe the audio
            transcription = transcribe_with_whisper(temp_path)
            
            transcribe_end_time = time.time()
            print(f"\033[92mTranscripted text: {transcription}\033[0m")

            return transcription, transcribe_end_time - transcribe_start_time

    except Exception as e:
        print(f"\033[92mError: {e}\033[0m")
        return str(e)


  


  frontend :

  


      useEffect(() => {
        const init = () => {
            navigator.mediaDevices.getUserMedia({audio: true})
                .then((audioStream) => {
                    const recorder = new MediaRecorder(audioStream);

                    recorder.ondataavailable = e => {
                        if (e.data.size > 0) {
                            setChunks(prevChunks => [...prevChunks, e.data]);
                        }
                    };

                    recorder.onerror = (e) => {
                        console.log("error: ", e);
                    }

                    recorder.onstart = () => {
                        console.log("started");
                    }

                    recorder.start();

                    setStream(audioStream);
                    setRecorder(recorder);
                });
        }

        init();

        return () => {
            if (recorder && recorder.state === 'recording') {
                recorder.stop();
            }

            if (stream) {
                stream.getTracks().forEach(track => track.stop());
            }
        }
    }, []);

    useEffect(() => {
        // Send chunks of audio data to the backend at regular intervals
        const intervalId = setInterval(() => {
            if (recorder && recorder.state === 'recording') {
                recorder.requestData(); // Trigger data available event
            }
        }, 8000); // Adjust the interval as needed


        return () => {
            if (intervalId) {
                console.log("Interval cleared");
                clearInterval(intervalId);
            }
        };
    }, [recorder]);

    useEffect(() => {
        const processAudio = async () => {
            if (chunks.length > 0) {
                // Send the latest chunk to the server for transcription
                const latestChunk = chunks[chunks.length - 1];

                const audioBlob = new Blob([latestChunk]);
                convertBlobToAudioFile(audioBlob);
            }
        };

        void processAudio();
    }, [chunks]);

    const convertBlobToAudioFile = useCallback((blob: Blob) => {
        // Convert Blob to audio file (e.g., WAV)
        // This conversion may require using a third-party library or service
        // For example, you can use the MediaRecorder API to record audio in WAV format directly
        // Alternatively, you can use a library like recorderjs to perform the conversion
        // Here's a simplified example using recorderjs:

        const reader = new FileReader();
        reader.onload = () => {
            const audioBuffer = reader.result; // ArrayBuffer containing audio data

            // Send audioBuffer to Flask server or perform further processing
            sendAudioToFlask(audioBuffer as ArrayBuffer);
        };

        reader.readAsArrayBuffer(blob);
    }, []);

    const sendAudioToFlask = useCallback((audioBuffer: ArrayBuffer) => {
        const formData = new FormData();
        formData.append('audio_file', new Blob([audioBuffer]), `speech_audio.wav`);

        console.log(formData.get("audio_file"));

        fetch('http://34.87.75.138:8000/transcribe', {
            method: 'POST',
            body: formData
        })
            .then(response => response.json())
            .then((data: { task_id: string, status: string }) => {
                pendingTaskIdsRef.current.push(data.task_id);
            })
            .catch(error => {
                console.error('Error sending audio to Flask server:', error);
            });
    }, []);


  


  I was trying to pass the audio from frontend to whisper model which is in flask app