'Is there a way to extract frame by frame Both the Depth and RGB from an MP4 captured using ARCore API?
I tried creating a scanner through Recording and Playback API from ARCore in Android Studio, right now I can save the mp4 file format. I assumed that the mp4 file had the extra data in capturing both the RGB and Depth of the file. I wanted to get those data frame by frame where the image extracted has both the depth and RGB data on it, but I actually don't have any idea how to do it. I tried searching for related problems but nothing was found. P.S I am kinda new to this type of project and I will use any help you guys can give.
Solution 1:[1]
In ARCore the depth information is actually another set of video frames, i.e. a separate video track with a depth frame matching every 'real' frame in the main video track.
So depth information is not stored as metadata but as an extra video track in the MP4 container.
The online documentation gives an example of a video frame and the correspond depth frame, which makes this much easier to visualise:
You can see that the color corresponds to the depth of the pixel - red is closer and blue further away.
More info (at this time - the online documentation can change quite regularly) here: https://developers.google.com/ar/develop/depth
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Mick |