How to fix an extensive echoing while recording audio using navigator.mediaDevices.getUserMedia?

Question

So, I am working on a small electron desktop app that captures desktop screen and records video and audio. When I am trying to add audio to the stream it starts echoing really badly and I am not sure why.

I am using:

• Windows 10 PRO 18362.778
• Chrome 81.0.4044.113
• Electron 8.2.3

Here is some code.

I create these constraints when I want to capture and record video only:

const constraints = {
        audio: false,
        video: {
            mandatory: {
                chromeMediaSource: 'desktop',
                chromeMediaSourceId: source.id
            }
        }
    }

Then I pass is to the stream like that:

const stream = await navigator.mediaDevices.getUserMedia(constraints)

It works like a charm. However when I start adding audio it gives me echo:

const constraints = {
        audio: {
            mandatory: {
                chromeMediaSource: 'desktop',
            }
        },
        video: {
            mandatory: {
                chromeMediaSource: 'desktop',
            }
        }
    }

Also, I can't just set the audio to true. It then gives me this error:

Uncaught (in promise) DOMException: Error starting screen capture

An interesting fact. When I go to Mozilla documentation page on audio constraints and use the demo button it gives me echo too. I tried doing it on Edge and the result was better, but still had echo. So can it be the audio codec?

Here it says that echoCancellation constraint is supported and on by default starting Chrome version 62.

Here is the branch on the Github where I tried to find solution, but failed.

Here is my git repo if you want to look at it more closely.

PS: this is my first post here. Let me know if I did something wrong here and can improve the post. Thank you!

Answer 1

The problem comes from attempting to initiate a single stream of your microphone audio and computer screen video at the same time. To fix this issue, you will need to create an audio stream first, then separately create a video stream that captures your computer screen, and finally, combine the streams into one.

// create audio and video constraints
const constraintsVideo = {
    audio: false,
    video: {
        mandatory: {
            chromeMediaSource: 'desktop',
            chromeMediaSourceId: source.id
        }
    }
}
const constraintsAudio = {audio: true}

// create audio and video streams separately
const audioStream = await navigator.mediaDevices.getUserMedia(constraintsAudio)
const videoStream = await navigator.mediaDevices.getUserMedia(constraintsVideo)

// combine the streams 
const combinedStream = new MediaStream([...videoStream.getVideoTracks(), ...audioStream.getAudioTracks()])

After you combine the streams, you can use combinedStream as if it originated as a single stream.

Answer 2

I think the easy fix here would be to add a muted element to the playback on your page.

 // Preview the source in a video element
    videoElement.srcObject = stream
    videoElement.muted = true
    videoElement.play()

That will work in all browsers and you'll still record the audio.

Answer 3

A very late answer I know, but I was trying to understand your issue, because I'm currently dealing with echo cancelling issues via WebRTC in regards to desktop capturing and I was wondering why your initial approach (adding chromeMediaSource: 'desktop' as a mandatory constraint to video and audio) causes echo, because this is the correct usage, as described in the Electron docs. I don't think that post which is marked as the correct answer, really solves the problem. Instead the second answer is correct in this case. The problem is not related to the usage of the constraints / streams. The reason you are hearing echo is, that you are adding the captured audio to your speaker output implicitly again, by assigning it to the <video> element. This creates the echoing loop. To avoid this, the correct solution is to mute the <video> element like you did in your repo. I tried your code and recording works fine (also with the muted <video> element), when re-enabling the code section which captures video/audio at once (via the constraint mentioned above).