Pipe opencv images to ffmpeg using python

Question

How can I pipe openCV images to ffmpeg (running ffmpeg as a subprocess)? (I am using spyder/anaconda)

I am reading frames from a video file and do some processing on each frame.

import cv2   
cap = cv2.VideoCapture(self.avi_path)
img = cap.read()
gray = cv2.cvtColor(img[1], cv2.COLOR_BGR2GRAY)
bgDiv=gray/vidMed #background division

then, to pipe the processed frame to ffmpeg, I found this command in a related question:

sys.stdout.write( bgDiv.tostring() )

next, I am trying to run ffmpeg as a subprocess:

cmd='ffmpeg.exe -f rawvideo -pix_fmt gray -s 2048x2048 -r 30 -i - -an -f avi -r 30 foo.avi'
sp.call(cmd,shell=True)

(this also from the mentioned post) However, this fills my IPython console with cryptic hieroglyphs and then crashes it. any advice?

ultimately, I would like to pipe out 4 streams and have ffmpeg encode those 4 streams in parallel.

Answer 1

I had similar problem once. I opened an issue on Github, turns out it may be a platform issue.

Related to your question, you can as well pipe OpenCV images to FFMPEG. Here's a sample code:

# This script copies the video frame by frame
import cv2
import subprocess as sp

input_file = 'input_file_name.mp4'
output_file = 'output_file_name.mp4'

cap = cv2.VideoCapture(input_file)
ret, frame = cap.read()
height, width, ch = frame.shape

ffmpeg = 'FFMPEG'
dimension = '{}x{}'.format(width, height)
f_format = 'bgr24' # remember OpenCV uses bgr format
fps = str(cap.get(cv2.CAP_PROP_FPS))

command = [ffmpeg,
        '-y',
        '-f', 'rawvideo',
        '-vcodec','rawvideo',
        '-s', dimension,
        '-pix_fmt', 'bgr24',
        '-r', fps,
        '-i', '-',
        '-an',
        '-vcodec', 'mpeg4',
        '-b:v', '5000k',
        output_file ]

proc = sp.Popen(command, stdin=sp.PIPE, stderr=sp.PIPE)

while True:
    ret, frame = cap.read()
    if not ret:
        break
    proc.stdin.write(frame.tostring())

cap.release()
proc.stdin.close()
proc.stderr.close()
proc.wait()

Answer 2

I'm Kind of late, But my powerful VidGear Python Library automates the process of pipelining OpenCV frames into FFmpeg on any platform with its WriteGear API's Compression Mode. OP, You can implement your answer as follows:

# import libraries
from vidgear.gears import WriteGear
import cv2

output_params = {"-s":"2048x2048", "-r":30} #define FFmpeg tweak parameters for writer

stream = cv2.VideoCapture(0) #Open live webcam video stream on first index(i.e. 0) device

writer = WriteGear(output_filename = 'Output.mp4', compression_mode = True, logging = True, **output_params) #Define writer with output filename 'Output.mp4' 

# infinite loop
while True:
    
    (grabbed, frame) = stream.read()
    # read frames

    # check if frame empty
    if not is grabbed:
        #if True break the infinite loop
        break
    

    # {do something with frame here}
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # write a modified frame to writer
    writer.write(gray) 
       
    # Show output window
    cv2.imshow("Output Frame", frame)

    key = cv2.waitKey(1) & 0xFF
    # check for 'q' key-press
    if key == ord("q"):
        #if 'q' key-pressed break out
        break

cv2.destroyAllWindows()
# close output window

stream.release()
# safely close video stream
writer.close()
# safely close writer

Source: https://abhitronix.github.io/vidgear/latest/gears/writegear/compression/usage/#using-compression-mode-with-opencv

You can check out VidGear Docs for more advanced applications and features.

Hope that helps!

Answer 3

You can use this pkg. ffmpegcv has Reader and Writer in ffmpeg backbone, similar to cv2.

#!pip install ffmpegcv
import ffmpegcv
vfile_in = 'A.mp4'
vfile_out = 'A_h264.mp4'
vidin = ffmpegcv.VideoCapture(vfile_in)
w, h = vidin.width, vidin.height
vidout = ffmpegcv.VideoWriter(vfile_out, 'h264_nvenc', vidin.fps, (w, h))

for frame in vidin:
    vidout.write(frame)

vidin.release()
vidout.release()