Golang loop through video frames and pixels?

Question

Is it possible in Golang to loop through an .mp4/.mov video file frame by and modify each frame's pixels?

I know this is a complicated answer and there are better methods of doing this in libraries for other languages such as processing for Java, but I am just wondering if Golang has this capability.

Answer 1

I can't think of any project using Golang to edit videos frames by itself.

Most of other higher level languages also do not do that, simply because they are not fast enough to do that. Even considering the speed and concurrency strategies of Golang. This type of task is typically done in C.

Probably, the best approach to solve this problem is what most projects do: using FFmpeg, either by calling it as a command line tool, through a Golang binding or embedding FFmpeg itself to your project and calling it using CGO

The first linked project creates an API to do simple tasks to a video (like adding an image) and may serve as a good reference.

The second linked project, called joy4, does stream related tasks, among other interesting and complex tasks.

Of course, it is possible to do this directly in Golang, but it is not easily feasible due to the lack of libraries to do the hard work of decoding, encoding and editing different formats of video.

Answer 2

I made an FFmpeg wrapper for Go that handles video I/O. All you need is to install the package using go get github.com/AlexEidt/Vidio and download FFmpeg and FFprobe and add them to your system path as environment variables.

The example below will loop through every frame in input.mp4 (where you can modify the frames as you need to) and then stores them in output.mp4.

video, err := vidio.NewVideo("input.mp4")
// Error handling...

options := vidio.Options{
    FPS: video.FPS(),
    Bitrate: video.Bitrate(),
    // Only include "Audio" if you'd like to copy audio from the
    // input to the output.
    Audio: "input.mp4",
}

writer, err := vidio.NewVideoWriter(
    "output.mp4",
    video.Width(),
    video.Height(),
    &options,
)
// Error handling...

defer writer.Close()

for video.Read() {
    frame := video.FrameBuffer()
    // "frame" is a byte array storing the frame data in row-major order.
    // Each pixel is stored as 3 sequential bytes in RGB format.
    err := writer.Write(frame)
    // Error handling...
}

The project source: https://github.com/AlexEidt/Vidio

FFmpeg Download: https://ffmpeg.org/download.html

Hopefully it helps!