AR camera distance measurement

Question

I have a question about AR(Augmented Reality).

I want to know how to show the distance information(like centermeter...) between AR camera and target object. (Using Smartphone)

Can I do that in Unity ? Should I use AR Foundation? and with ARcore? How to write code?

I tried finding some relative code(below), but it seems just like Printing information between object and object, nothing about "AR camera"...

var other : Transform;
if (other) {
    var dist = Vector3.Distance(other.position, transform.position);
    print ("Distance to other: " + dist);
}
 

Thank again!

Answer 1

Here is how to do it Unity and AR Foundation 4.1. This example script prints the depth in meters at the depth texture's center and works both with ARCore and ARKit:

using System;
using System.Collections;
using UnityEngine;
using UnityEngine.Assertions;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;


public class GetDepthOfCenterPixel : MonoBehaviour {
    // assign this field in inspector
    [SerializeField] AROcclusionManager manager = null;
    
    
    IEnumerator Start() {
        while (ARSession.state < ARSessionState.SessionInitializing) {
            // manager.descriptor.supportsEnvironmentDepthImage will return a correct value if ARSession.state >= ARSessionState.SessionInitializing 
            yield return null;
        }
        
        if (!manager.descriptor.supportsEnvironmentDepthImage) {
            Debug.LogError("!manager.descriptor.supportsEnvironmentDepthImage");
            yield break;
        }
        
        while (true) {
            if (manager.TryAcquireEnvironmentDepthCpuImage(out var cpuImage) && cpuImage.valid) {
                using (cpuImage) {
                    Assert.IsTrue(cpuImage.planeCount == 1);
                    var plane = cpuImage.GetPlane(0);
                    var dataLength = plane.data.Length;
                    var pixelStride = plane.pixelStride;
                    var rowStride = plane.rowStride;
                    Assert.AreEqual(0, dataLength % rowStride, "dataLength should be divisible by rowStride without a remainder");
                    Assert.AreEqual(0, rowStride % pixelStride, "rowStride should be divisible by pixelStride without a remainder");

                    var numOfRows = dataLength / rowStride;
                    var centerRowIndex = numOfRows / 2;
                    var centerPixelIndex = rowStride / (pixelStride * 2);
                    var centerPixelData = plane.data.GetSubArray(centerRowIndex * rowStride + centerPixelIndex * pixelStride, pixelStride);
                    var depthInMeters = convertPixelDataToDistanceInMeters(centerPixelData.ToArray(), cpuImage.format);
                    print($"depth texture size: ({cpuImage.width},{cpuImage.height}), pixelStride: {pixelStride}, rowStride: {rowStride}, pixel pos: ({centerPixelIndex}, {centerRowIndex}), depthInMeters of the center pixel: {depthInMeters}");
                }
            }
            
            yield return null;
        }
    }

    float convertPixelDataToDistanceInMeters(byte[] data, XRCpuImage.Format format) {
        switch (format) {
            case XRCpuImage.Format.DepthUint16:
                return BitConverter.ToUInt16(data, 0) / 1000f;
            case XRCpuImage.Format.DepthFloat32:
                return BitConverter.ToSingle(data, 0);
            default:
                throw new Exception($"Format not supported: {format}");
        }
    }
}

Answer 2

I'm working on AR depth image as well and the basic idea is:

1. Acquire an image using API, normally it's in format Depth16;
2. Split the image into shortbuffers, as Depth16 means each pixel is 16 bits;
3. Get the distance value, which is stored in the lower 13 bits of each shortbuffer, you can do this by doing (shortbuffer & 0x1ff), then you can have the distance for each pixel, normally it's in millimeters.

By doing this through all the pixels, you can create a depth image and store it as jpg or other formats, here's the sample code of using AR Engine to get the distance:

try (Image depthImage = arFrame.acquireDepthImage()) {
        int imwidth = depthImage.getWidth();
        int imheight = depthImage.getHeight();
        Image.Plane plane = depthImage.getPlanes()[0];
        ShortBuffer shortDepthBuffer = plane.getBuffer().asShortBuffer();
        File sdCardFile = Environment.getExternalStorageDirectory();
        Log.i(TAG, "The storage path is " + sdCardFile);
        File file = new File(sdCardFile, "RawdepthImage.jpg");

        Bitmap disBitmap = Bitmap.createBitmap(imwidth, imheight, Bitmap.Config.RGB_565);
        for (int i = 0; i < imheight; i++) {
            for (int j = 0; j < imwidth; j++) {
                int index = (i * imwidth + j) ;
                shortDepthBuffer.position(index);
                short depthSample = shortDepthBuffer.get();
                short depthRange = (short) (depthSample & 0x1FFF);
                //If you only want the distance value, here it is
                byte value = (byte) depthRange;
          byte value = (byte) depthRange ;
                disBitmap.setPixel(j, i, Color.rgb(value, value, value));
            }
        }
        //I rotate the image for a better view
        Matrix matrix = new Matrix();
        matrix.setRotate(90);
        Bitmap rotatedBitmap = Bitmap.createBitmap(disBitmap, 0, 0, imwidth, imheight, matrix, true);

        try {
            FileOutputStream out = new FileOutputStream(file);
            rotatedBitmap.compress(Bitmap.CompressFormat.JPEG, 90, out);
            out.flush();
            out.close();
            MainActivity.num++;
        } catch (Exception e) {
            e.printStackTrace();
        }
    } catch (Exception e) {
        e.printStackTrace();
    }
}

Answer 3

While the answers are great, they may be too complicated and advanced for this question, which is about the distance between the ARCamera and another object, and not about the depth of pixels and their occlusion.

transform.position gives you the position of whatever game object you attach the script to in the hierarchy. So attach the script to the ARCamera object. And obviously, other should be the target object.

Alternately, you can get references to the two game objects using inspector variables or GetComponent

Answer 4

/raycasting should be in update/ Ray ray = new Ray(cam.transform.position, cam.transform.forward); if (Physics.Raycast(ray, out info, 50f, layerMaskAR))//50 meters detection range bcs of 50f { distanca.text = string.Format("{0}: {1:N2}m", info.collider.name, info.distance, 2); }

This is func that does it what u need with this is ofc on UI txt element and layer assigne to object/prefab. int layerMaskAR = 1 << 6; (here u see 6 bcs 6th is my custom layer ,,layerMaskAR,,) This is ray cating on to objects in only this layer rest object are ignored(if u dont want to ignore anything remove layerMask from raycast and it will print out name of anything with collider).