Is there a way to get RGB value from a live camera feed in reactJs?

Question

I was wondering in ReactJS/ReactApp if it is possible to capture the RGB color from Center Pixel of a Live Camera Feed displayed on a website (honestly any pixel can do fine or we could average out the entire video frame - whatever simpler)?

All this camera does is displays a feed (of the devices rear camera) which is designed by the website (no need for pictures or video taken).

function Camera() {
    const videoRef = useRef(null);

    useEffect(() => {
        getVideo();
    }, [videoRef]);

    const getVideo = () => {
        navigator.mediaDevices
            .getUserMedia({ video: {facingMode: 'environment', width: 600 , height: 400}})
            .then(stream => {
                let video = videoRef.current;
                video.srcObject = stream;
                video.play();
            })
            .catch(err => {
                console.error("error:", err);
            });
    };

        return (
            <div>
                <video ref={videoRef} />
            </div>
        )
}

export default Camera;

As well, this camera Implementation was Modified from: https://itnext.io/accessing-the-webcam-with-javascript-and-react-33cbe92f49cb

Answer 1

You can use your videoRef and do something like this.

  const {video} = videoRef.current;
  const canvas = document.createElement('canvas'); 
  canvas.width = 600;
  canvas.height = 400;
  const ctx = canvas.getContext('2d');
  ctx.drawImage(video, 0, 0, 600, 400);
  const rgbaData = ctx.getImageData(0,0,600,400).data;

The above code grabs the video stream from your videoRef then you create a 2d canvas and draw your stream onto it.

From there you just do getImage fill out the parameters on what part of the image you want the pixels from and then .data is a single dimensional array of rgba values containing all the pixels in the image.

(note: this also doesn't require drawing the canvas onto the screen)

https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/getImageData

And then to get the average pixel color you can look at this post:

Get average color of image via Javascript

But if you're looking to find pixel brightness you can use this

Formula to determine perceived brightness of RGB color

And lastly for contrast of pixels

How to programmatically calculate the contrast ratio between two colors?

Note you will also probably want to chunk together the single dimensional array into [[rgba],[rgba]] chunks rather than the [r,g,b,a,r,g,b,a] structure the array already comes in.

const chunk = (a, n) => [...Array(Math.ceil(a.length / n))].map((_, i) => a.slice(n * i, n + n * i));

a - being the array

n - being the chunk size (in this case 4 since rgba)