RealityKit – Difference between loading model using `.rcproject` vs `.usdz`

Question

I'm building a simple app that adds a hat on top of the user's face. I've seen examples of 2 different approaches:

1. Adding the object as a scene to Experience.rcproject
2. Reading the object from the bundle directly as a .usdz file

Approach #1

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
        arView = ARView(frame: .zero)
        arView.automaticallyConfigureSession = false
        return arView
    }
    
    func updateUIView(_ uiView: ARView, context: Context) {
        let arConfiguration = ARFaceTrackingConfiguration()
        
        uiView.session.run(arConfiguration,
                           options:[.resetTracking, .removeExistingAnchors])
        
        let arAnchor = try! Experience.loadHat()
        
        uiView.scene.anchors.append(arAnchor)
    }
}

Approach #2

struct ARViewContainer: UIViewRepresentable {
    
    func makeUIView(context: Context) -> ARView {
        let arView = ARView(frame: .zero)
        let modelEntity = try! ModelEntity.load(named: "hat.usdz")
        
        modelEntity.position = SIMD3(0, 0, -8)
        modelEntity.orientation = simd_quatf.init(angle: 0, axis: SIMD3(-90, 0, 0))
        modelEntity.scale = SIMD3(0.02, 0.02, 0.02)
        
        arView.session.run(ARFaceTrackingConfiguration())
        
        let anchor = AnchorEntity(.face)
        anchor.position.y += 0.25
        anchor.addChild(modelEntity)
        arView.scene.addAnchor(anchor)
        return arView
    }
    
    func updateUIView(_ uiView: ARView, context: Context) {
        let arConfiguration = ARFaceTrackingConfiguration()
        
        uiView.session.run(arConfiguration,
                           options:[.resetTracking, .removeExistingAnchors])
        
        let fileName = "hat.usdz"
        let modelEntity = try! ModelEntity.loadModel(named: fileName)
        
        modelEntity.position = SIMD3(0, 0, -8)
        modelEntity.orientation = simd_quatf.init(angle: 0, axis: SIMD3(-90, 0, 0))
        modelEntity.scale = SIMD3(0.02, 0.02, 0.02)
        
        let arAnchor = AnchorEntity(.face)
        arAnchor.addChild(modelEntity)
                
        uiView.scene.anchors.append(arAnchor)
    }
}

What is the main difference between these approaches? Approach #1 works, but the issue is that approach #2 doesn't even work for me - the object simply doesn't load into the scene. Could anyone explain a bit?

Thanks!

Answer 1

For approach number 2, try removing the the position for the modelEntity. You provided position as 0, -4.9 and 11.8. Those positions are in meters. So try to remove it and see if appears.

Answer 2

The difference between .rcproject and .usdz is quite obvious: the Reality Composer file already has an anchor for the model (and it's at the top of the hierarchy). When you prototype in Reality Composer, you have the ability to visually control the scale of your models. .usdz models very often have a huge scale, which you need to reduce by 100 times.

As a rule, .usdz model doesn't have a floor, while .rcproject has a floor by default and this floor acts as a shadow catcher. Also, note that the .rcproject file is larger than the .usdz file.

let scene = try! Experience.loadHat()
arView.scene.anchors.append(scene)

print(scene)

When loading .usdz into a scene, you have to programmatically create an anchor (either swiftly or pythonically). It also makes sense to use .reality files as they are optimized for faster loading.

let model = try! ModelEntity.load(named: "hat.usdz")
let anchor = AnchorEntity(.face)
anchor.addChild(model)
arView.scene.anchors.append(anchor)

print(model)

Also, put a face tracking config inside makeUIView method:

import SwiftUI
import RealityKit
import ARKit 

func makeUIView(context: Context) -> ARView {

    let arView = ARView(frame: .zero)
    let model = try! ModelEntity.load(named: "hat.usdz")
    arView.session.run(ARFaceTrackingConfiguration())
    
    let anchor = AnchorEntity(.face)
    anchor.position.y += 0.25
    anchor.addChild(model)
    arView.scene.addAnchor(anchor)
    return arView
}

Also, check if the following render options are disabled.

arView.renderOptions = [.disableFaceMesh, .disablePersonOcclusion]

And check a position of pivot point in hat model.