ARFaceTrackingConfiguration – Depth map not aligned

Question

I'm working currently of getting depth map from an ARFaceTrackingConfiguration but I have some trouble with this, because it's like the depth map drift a bit from RGB one (see the picture).

So I declared my configuration as ARFaceTrackingConfiguration and setup my session. And I use a delegate to wait the depth map when is ready to be captured

func session(_ session: ARSession, didUpdate frame: ARFrame) {
   if (isRecording) {
      if (frame.capturedDepthData? != nil){
          let capturedFrame = frame
          captureImages(frame: capturedFrame)
      }
   }    
}

And I captured the camera, depthmap and rgb image from the frame

func captureImages(frame: ARFrame){
   let camera = frame.camera
   let pixelBuffer = frame.capturedImage
   guard let depthBuffer = frame.capturedDepthData else {
      print("Dropped, no depthBuffer")
      return
   }
}

After that I saved both depth and rbg as png to diplay them.

Thanks

Answer 1

The discrepancy between the edges of the Depth channel and RGB channels is due to the fact that the Depth channel in ARKit 3 is 15 fps instead of 60 fps, and in ARKit 4 and RealityKit 2 it is 60 fps, but these 60 frames are the same interpolated ones out of 15. Also, Depth channel has an aliased edge, because antialiasing feature is turned off for depth channel.

For more info look at this SO post.