Building Augmented Reality Apps in RealityKit & ARKit

Understand reality kit concepts by using the air view to present the camera, and place themed anchors and entities on horizontal and vertical planes, faces, or images.

Create your first augmented reality project by launching Xcode, selecting the AR template with RealityKit, and running on a real device to place a reflective cube upon detected planes.

Create an anchor on a horizontal plane and attach a blue, metallic box as a model entity to display your first AR object when the plane is found.

Explore adding multiple model entities in a RealityKit AR app, creating a box, a yellow metallic spear, and a plane, and position them in 3D space.

Display virtual text in an AR app using the two text constructors, configure depth, font, container, alignment, blue metallic material, then anchor the text in the scene and test.

Learn to attach a box to an anchor and respond to tap gestures on virtual objects in RealityKit and ARKit, changing color via a coordinator and gesture recognizers.

Use ray casting to detect horizontal planes and tap to place a box as a model entity via ARAnchor and anchor entity, adding objects to the scene.

Replace the air anchor with an anchor entity and use ray casting results to place the model entity, then add the anchor to the scene.

Master scale, rotate, and move gestures for virtual objects in RealityKit and ARKit by generating collision shapes and applying gesture components with just a few lines of code.

Import usdz models into RealityKit apps by loading a shoe model on a plane using an anchor entity; place models with taps and use async loading to avoid glitches.

Learn to asynchronously load USDZ models in RealityKit by using the load async function, handling completion and errors, and attaching entities with anchors while keeping the main thread responsive.

Learn to load multiple models in RealityKit, such as a teapot and a shoe, collect entities, and place them on an anchor with distinct x, y, z coordinates.

Load a lunar rover animated model from a reality file into a scene via an anchor, and ensure only one rover appears by guarding taps.

Explore how occlusion works in augmented reality by watching a real table hide a virtual object, then learn how to create occlusion in AR apps.

Implement occlusion in RealityKit by using an occlusion material on a movable box anchored to a horizontal plane, so the box hides parts of the robot model.

Load a video from the bundle and apply it as video material on a plane in a reality kit AR scene, anchoring to a detected surface and enabling spatial audio.

Explore unlit material in RealityKit by applying it to a red 0.3-meter box and observe how lighting no longer affects its flat appearance.

Display images as textures on a 3D box by loading a texture resource, applying an unlit material, and mapping the image to all faces in RealityKit.

Learn to apply different images to every face of a 3d box by loading textures asynchronously, enabling per-face materials with split faces in RealityKit and ARKit.

Explore building physics-enabled augmented reality apps by adding a static plane anchor and a dynamic box with a physics body, generate collision shapes, and plane detection to create gravity-driven interactions.

Learn how to implement collision detection in RealityKit by setting a trigger collision component, subscribing to begin and end collision events, and changing a box material during contact.

Explore collision detection in RealityKit and ARKit by building an environment with boxes and spheres, enabling collision shapes, gestures, and real-time color changes when collisions begin or end.

Define collision groups and masks to control interactions, creating a box group and a sphere group with subtracting filters so boxes collide only with boxes and spheres only with spheres.

Add physics to collision detection by creating dynamic physics bodies for boxes and spears, place a floor plane, and observe gravity and collisions in RealityKit.

Refactor your AR project by introducing a movable entity that encapsulates mesh, materials, physics, and collision, enabling quick creation of boxes and spears with configurable shape and color.

Explore how the AR coaching overlay guides users through plane detection and environment tracking with tutorials before detection, and use coaching visuals for horizontal and vertical planes to guide placement.

Implement an AR coaching overlay by adding a horizontal plane anchor and AR view extension; trigger the overlay delegate to add a green box after the plane is found.

Create an augmented reality measuring app with RealityKit and ARKit that lets you place two points, view their distance, and reset to measure again in horizontal and vertical orientations.

Set up the initial AR project by adding a coaching overlay and a coordinator, configure world tracking, and create a UI with measurement and reset buttons to guide surface discovery.

Add two virtual anchors on real-world planes, place simple objects, and calculate the distance between them using RealityKit and ARKit, with a reset to start over.

Explore image detection in augmented reality by replacing a real image or card with a virtual object, such as a wristwatch, and trigger animations for diverse uses.

Explore image detection by registering a reference image as an AR resource, creating an anchor, and displaying a 3D model on the detected image with RealityKit and ARKit.

Detect a poster image and display a video on a RealityKit plane anchored to the image, using ARKit and loading the video from the bundle.

Explore how to create a point light in RealityKit by tapping the screen to place a colored, luminous light that casts reflections on the background.

Create a directional light in RealityKit, attach it to an anchor, set color and intensity, enable real world shadow casting, and adjust shadow distance and depth bias.

Create a spotlight in RealityKit by anchoring a light, setting inner and outer cone angles, and a 10-meter radius. Observe shadows and adjust colors and intensity to see the effect.