High-speed video recording - codecs

[HarrietFeldman]

Hi all, I need some help with high-speed video recording.

I have four cameras recording at 200-400fps, onto one computer. The SSD drive can write plenty fast enough as recording without a codec goes fine with no skipped frames. Unfortunately it fills up 1TB of storage after ~10 minutes, and I need to be able to record at least 1 hour footage before moving it off the computer.

I am using the VideoWriter codec from Vision. I have tried using the FFmpeg codec but unfortunately this gives non-deterministic errors saying that it cannot find the pipe, which can only be solved by restarting the computer (restarting Bonsai is not enough).

I had to start recording in a hurry so I used the DIVX codec. This works fine when recording 700fps but moving up to 1000fps fills up the RAM after about 10 minutes. However, the CPU does not get above 20-25%.

So it seems that I have enough disc write speed and a fast enough processor, I just need to find a way to compress using more of the CPU so I don't back up the RAM? Either that or install more RAM (although this feels overkill...)?

I'd ideally like to move to a lossless compression too. Any suggestions about how to proceed?

[bruno-f-cruz]

Can you provide more details as to what codecs you have tried so far? Also, what are your hardware specs? Do you have access to GPU acceleration? If so, how many cores?

This is largely not an issue with Bonsai. Provided you can find a ffmpeg codec you are happy with, we will likely be able to run it in bonsai. Have you tried to play with ffmpeg directly to find something you are happy with?

[HarrietFeldman]

Hi Bruno,

Thanks for your reply, and sorry it's taken a while to get back to me. I have tried FFV1, XVID and DIVX. FFV1 caused some problems at higher frame rates but I would prefer to use it.

The computer specs are as follows:

• Intel® Core™ i5 14th Gen 14600K (24 MB cache, 14 cores, 20 threads,
  3.5 GHz to 5.3 GHz, 125W)
• NVIDIA GeForce RTX 4070, 12GB GDDR6X, HDMI, 3 DP
• RAM 32GB: 2 x 16 GB, DDR5, 4400 MT/s, non-ECC

I have not tried with FFmpeg directly - would you suggest just running the four cameras and then running four FFmpeg pipelines in four terminals? Any ideas about the pipe error?

[bruno-f-cruz]

I would definitely use ffmpeg given your requirements and available hardware. I would start with a vanilla x264 codec and go from there. Take a look at the FFMPEG operator available in bonsai. It is nearly a drop-in replacement for the OpenCV video writer.

[HarrietFeldman]

I did have a go with ffmpeg video writer - but ended up with the non-deterministic errors when it failed to open the pipe, that could only be resolved by restarting the computer.

[bruno-f-cruz]

Can you share your workflow?

[HarrietFeldman]

moseq.txt

Uploaded as .txt as Github wouldn't let me upload with a .bonsai extension.

I don't have access to the cameras right now but I will try to run them next week and see if I can replicate the ffmpeg error.

[bruno-f-cruz]

I dont see any VideoWriter(Ffmpeg) in your workflow. Anyway here is a minimal workflow that uses ffmpeg and either x264 or x264_nvenc as codecs for online encoding. Note this is just a starting point, and you will likely have to tweak parameters depending on your images, quality and performance requirements:

<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.9.0"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:cv="clr-namespace:Bonsai.Vision;assembly=Bonsai.Vision"
                 xmlns:ffmpeg="clr-namespace:Bonsai.FFmpeg;assembly=Bonsai.FFmpeg"
                 xmlns="https://bonsai-rx.org/2018/workflow">
  <Workflow>
    <Nodes>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="cv:CameraCapture">
          <cv:Index>0</cv:Index>
          <cv:CaptureProperties />
        </Combinator>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="ffmpeg:VideoWriter">
          <ffmpeg:FileName>test.mp4</ffmpeg:FileName>
          <ffmpeg:Suffix>None</ffmpeg:Suffix>
          <ffmpeg:Overwrite>true</ffmpeg:Overwrite>
          <ffmpeg:FrameRate>30</ffmpeg:FrameRate>
          <ffmpeg:OutputArguments>-c:v libx264 -pix_fmt yuv420p -crf 23</ffmpeg:OutputArguments>
        </Combinator>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="cv:CameraCapture">
          <cv:Index>0</cv:Index>
          <cv:CaptureProperties />
        </Combinator>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="ffmpeg:VideoWriter">
          <ffmpeg:FileName>test2.mp4</ffmpeg:FileName>
          <ffmpeg:Suffix>None</ffmpeg:Suffix>
          <ffmpeg:Overwrite>true</ffmpeg:Overwrite>
          <ffmpeg:FrameRate>30</ffmpeg:FrameRate>
          <ffmpeg:OutputArguments>-c:v h264_nvenc -rc vbr -cq 23 -pix_fmt yuv420p</ffmpeg:OutputArguments>
        </Combinator>
      </Expression>
    </Nodes>
    <Edges>
      <Edge From="0" To="1" Label="Source1" />
      <Edge From="2" To="3" Label="Source1" />
    </Edges>
  </Workflow>
</WorkflowBuilder>