Recherche avancée

Sur d’autres sites (5806)

• runtime/cgo : pthread_create failed : Resource temporarily unavailable

  15 février 2020, par nadermx

  I currently have this function, that pipes multiple youtubedl commands thru ffmpeg, and then pipes the output of ffmpeg to an HTTP client.

  func pipeThruFfmpegToMp4(vi *VideoInfo, rw web.ResponseWriter) error {
  
      var ffmpeg *exec.Cmd
  
      ffmpeg = exec.Command(
  
          "ffmpeg",
  
          "-i", "-",
  
          "-i", "pipe:3",
  
          "-c:v", "copy", "-c:a", "copy",
  
          "-preset", "veryfast",
  
          "-metadata", fmt.Sprintf(`title=%s`, vi.GetTitle()),
  
          "-movflags", "frag_keyframe+empty_moov",
  
          "-f", "mp4",
  
          "-")
  
  
  
  
  
      youtubevideo := exec.Command(YoutubeDLPath, "-c", "-f", fmt.Sprintf("%s/bestvideo[ext=mp4]/bestvideo/best", vi.GetFormat()), "--no-cache-dir", "--restrict-filenames", "--hls-prefer-native", "-o", "-", fmt.Sprintf("%s", vi.GetVideoUrl()))
  
      fmt.Println(youtubevideo)
  
  
  
      youtube := exec.Command(YoutubeDLPath, "-c", "-f", "bestaudio[ext=m4a]/bestaudio/best", "--no-cache-dir", "--restrict-filenames", "--hls-prefer-native", "-o", "-", fmt.Sprintf("%s", vi.GetVideoUrl()))
  
      fmt.Println(youtube)
  
  
  
      var ytvbuf, ytbuf, ffbuf bytes.Buffer
  
      youtubevideo.Stderr = &ytvbuf
  
      youtube.Stderr = &ytbuf
  
      ffmpeg.Stderr = &ffbuf
  
  
  
      video, err := youtubevideo.StdoutPipe()
  
      if err != nil {
  
          log.Printf("pipeThruFfmpegToMp4: %v\n", err)
  
          return err
  
      }
  
  
  
      pipe3, err := youtube.StdoutPipe()
  
      if err != nil {
  
          log.Printf("pipeThruFfmpegToMp4: %v\n", err)
  
          return err
  
      }
  
  
  
      ffmpeg.Stdin = video
  
      ffmpeg.ExtraFiles = []*os.File{pipe3.(*os.File)}
  
      ffmpeg.Stdout = rw
  
  
  
      // Headers sent, no turning back now
  
      rw.Header().Set("Content-Type", "video/mp4")
  
      rw.Header().Set("Content-Disposition", fmt.Sprintf("attachment;filename=\"%s.mp4\"", vi.GetSlug()))
  
      rw.Flush()
  
  
  
      ffmpeg.Start()
  
      youtubevideo.Start()
  
      youtube.Start()
  
      ffmpeg.Wait()
  
      youtubevideo.Wait()
  
      youtube.Wait()
  
  
  
      // check ytvbuf, ytbuf, ffbuf for stderr errors
  
  
  
      if ffbuf.Len() != 0 {
  
          rollbar.Error(rollbar.ERR, err, &rollbar.Field{"stderr", ffbuf.String()})
  
          log.Printf("pipeThruFfmpegToMp4: %v\n", ffbuf.String())
  
      }
  
      if ytvbuf.Len() != 0 {
  
          rollbar.Error(rollbar.ERR, err, &rollbar.Field{"stderr", ytvbuf.String()})
  
          log.Printf("pipeThruYouTubevDLToMp4: %v\n", ytvbuf.String())
  
      }
  
      if ytbuf.Len() != 0 {
  
          rollbar.Error(rollbar.ERR, err, &rollbar.Field{"stderr", ytbuf.String()})
  
          log.Printf("pipeThruYouTubeDLToMp4: %v\n", ytbuf.String())
  
      }
  
  
  
      return nil
  
  }

  The problem is that everything runs fine, but after a while the ram starts filling up on the server, till it gets to the point that it errors out with runtime/cgo: pthread_create failed: Resource temporarily unavailable

  I’m unsure if it’s a memory leak, or if either instance of youtube-dl is not closing right or if ffmpeg isn’t closing right and just consuming more and more ram as the program runs more, until the program crashes with this error

  runtime/cgo: pthread_create failed: Resource temporarily unavailable
  
  SIGABRT: abort
  
  PC=0x7f083501fe97 m=128 sigcode=18446744073709551610
  
  
  
  goroutine 0 [idle]:
  
  runtime: unknown pc 0x7f083501fe97
  
  stack: frame={sp:0x7f05ff7fd7d0, fp:0x0} stack=[0x7f05feffe288,0x7f05ff7fde88)
  
  00007f05ff7fd6d0:  00007f05ff7fd700  0000ffff00001fa0
  
  00007f05ff7fd6e0:  00007f05ff7fdbe0  00007f05f8000da0
  
  00007f05ff7fd6f0:  0000000000000000  000000000093032c
  
  00007f05ff7fd700:  0000000000000000  00007f0835600ec3
  
  00007f05ff7fd710:  0000000000000005  0000000000000000
  
  00007f05ff7fd720:  000000c0000ce120  00007f0834ff1ce0
  
  00007f05ff7fd730:  00007f05ff7fdaf0  00007f083560870a
  
  00007f05ff7fd740:  0000000000000000  0000000000000000
  
  00007f05ff7fd750:  0000000000000000  00007f05ff7fdbe0
  
  00007f05ff7fd760:  2525252525252525  2525252525252525
  
  00007f05ff7fd770:  000000ffffffffff  0000000000000000
  
  00007f05ff7fd780:  000000ffffffffff  0000000000000000
  
  00007f05ff7fd790:  000000c00010d1a0  000000c000953740
  
  00007f05ff7fd7a0:  000000c0000ce120  000000c000cf2300
  
  00007f05ff7fd7b0:  000000c00010d260  000000c001f4e180
  
  00007f05ff7fd7c0:  000000c001f4e000  000000c00169f680
  
  00007f05ff7fd7d0: <0000000000000000  000000c001f34180
  
  00007f05ff7fd7e0:  6e75720000000000  6f67632f656d6974
  
  00007f05ff7fd7f0:  0000000000000000  0000000000000000
  
  00007f05ff7fd800:  000000c000cf2300  000000c00010d260
  
  00007f05ff7fd810:  000000c001f4e180  000000c001f4e000
  
  00007f05ff7fd820:  000000c00169f680  000000c00169f500
  
  00007f05ff7fd830:  000000c001f34180  000000c001f34000
  
  00007f05ff7fd840:  000000c000c92780  000000c001ec2600
  
  00007f05ff7fd850:  fffffffe7fffffff  ffffffffffffffff
  
  00007f05ff7fd860:  ffffffffffffffff  ffffffffffffffff
  
  00007f05ff7fd870:  ffffffffffffffff  ffffffffffffffff
  
  00007f05ff7fd880:  ffffffffffffffff  ffffffffffffffff
  
  00007f05ff7fd890:  ffffffffffffffff  ffffffffffffffff
  
  00007f05ff7fd8a0:  ffffffffffffffff  ffffffffffffffff
  
  00007f05ff7fd8b0:  ffffffffffffffff  ffffffffffffffff
  
  00007f05ff7fd8c0:  ffffffffffffffff  ffffffffffffffff
  
  runtime: unknown pc 0x7f083501fe97
  
  stack: frame={sp:0x7f05ff7fd7d0, fp:0x0} stack=[0x7f05feffe288,0x7f05ff7fde88)
  
  00007f05ff7fd6d0:  00007f05ff7fd700  0000ffff00001fa0
  
  00007f05ff7fd6e0:  00007f05ff7fdbe0  00007f05f8000da0
  
  00007f05ff7fd6f0:  0000000000000000  000000000093032c
  
  00007f05ff7fd700:  0000000000000000  00007f0835600ec3
  
  00007f05ff7fd710:  0000000000000005  0000000000000000

  I’ve also tried building the binary with CGO_ENABLED=0 even though I don’t even have a import "c" but that also ends up erroring out as well with

  runtime: failed to create new OS thread (have 21 already; errno=11)
  
  runtime: may need to increase max user processes (ulimit -u)
  
  fatal error: newosproc

  my limit’s are already very generouse, which maybe could also be a problem ? Or do I maybe have to increase the pipe size ?

  $ ulimit -a
  
  core file size          (blocks, -c) 0
  
  data seg size           (kbytes, -d) unlimited
  
  scheduling priority             (-e) 0
  
  file size               (blocks, -f) unlimited
  
  pending signals                 (-i) 192907
  
  max locked memory       (kbytes, -l) 16384
  
  max memory size         (kbytes, -m) unlimited
  
  open files                      (-n) 100000
  
  pipe size            (512 bytes, -p) 8
  
  POSIX message queues     (bytes, -q) 819200
  
  real-time priority              (-r) 0
  
  stack size              (kbytes, -s) 65536
  
  cpu time               (seconds, -t) unlimited
  
  max user processes              (-u) 63883
  
  virtual memory          (kbytes, -v) unlimited

  Either way, I apprecate any and all the help I can get on this. Here is a link to the entire program and here is a thread in the golang-nuts forum regarding this questions

• Usage with FFMPEG on a vServer ?

  28 février 2020, par MaxChri

  Heyo !

  I’m just setting up a video platform for a small project. So far a user is able to upload a video in any common format. The video will be converted using FFMPEG.

  ffmpeg -i input.mp4 -vcodec h264 -acodec aac -strict -2 output.mp4

  At the moment I’m testing all the stuff on my 3€ testserver which is actually pretty slow. But imagine, a 10 seconds 60fps full hd video takes like 6 minutes of converting. I think I’m not able to run this configurations on a live version. :D

  My current test setup :
  vCPU 1x (Events per second : 175)

  So my question is, which cpu should I use ? I searched for a server on hetzner. The setup with dedicated vCPU is especially declared for video converting. But the cpu benchmark is not much better with 600 events per second while the price is around 20€.

  What do I need to convert a 10 min video (full hd 60fps) in just 1-2 minutes ? My budget should be around 30€/m.

• FFMpeg amix filter volume changes. Explanation request

  22 janvier 2020, par klodoma

  First, the question :
  IS THERE somewhere documented or explained how ffmpeg amix filter changes the volume in the mixed audio files ? I couldn’t find any good reference.

  There are several threads regarding this "issue" that the amix filter changes the volume of the input files. I played around with these and I just cannot explain what is going on. It makes a difference if you "mix" files like concatenate them at specific times or you overlap audio tracks and volume changes just doesn’t make any sense.

  Firstly, I don’t understand WHY the volumes need to be changed.

  Some related refs :

  • FFMPEG amix filter volume issue with inputs of different duration
  • FFMPEG set volume in amix