Unity VR Multiplayer Development (Meta XR SDK & Fusion 1)

Kickstart your journey into the realm of Extended Reality (XR) with our first lesson focusing on Unity, a versatile, cross-platform game engine pivotal for creating interactive 3D and 2D experiences in various fields including gaming, film, architecture, and engineering. We'll guide you through selecting the right Unity version, installing it, and adding the essential modules for "Meta Quest" development. Unity stands out for its ease of use, extensive community support, and ability to deploy on multiple platforms, making it a preferred choice for both novice and seasoned XR developers. By the end of this lesson, you'll be well-equipped to delve deeper into creating immersive experiences blending the physical and digital worlds, and ready to explore project setup and collaboration using GitHub in upcoming lessons.

Download the finished and complete project below!

(If you have any issues with the project below, make sure if all the dependencies like Fusion SDK, Voice SDK, Avatar SDK and Oculus Integration (and also the project assets from lesson 2.7 if necessary) are installed or install them first to not get any compilation errors!

Dive deeper into project setup and collaboration with this lesson on using "GitHub Desktop" in our Unity course. Learn the essentials of installing Git, Git Large File Storage (LFS), and managing a GitHub repository, laying a solid foundation for version control in Unity projects. This lesson is designed to equip you with the skills to efficiently handle large assets, create and merge branches, and revert changes, all crucial for professional XR development. Get ready to explore more of Unity and embark on your first XR project in the upcoming lessons!

In this lesson, we transition our Unity project into an XR project, focusing on setting up the Android platform and utilizing XR-Plugin Management to integrate various virtual and augmented reality technologies seamlessly. We guide you through installing and activating the Oculus plugin for both PC and Android, ensuring optimal settings like “Multi-view” on Android and “Single Pass Instanced” on PC for efficient stereo rendering. This setup is crucial for creating immersive XR experiences with Unity. Stay tuned as we continue to explore importing SDKs and delve deeper into Oculus integration in the upcoming lessons!

Imoportant: When using the new Meta XR SDK, make sure to also install the Interaction SDK OVR Samples!

In this lesson, we delve into the installation of an interaction framework, focusing on the Meta XR SDK, a popular choice for implementing Meta Avatars. We guide you through the process of adding and importing the Oculus Integration from the Unity Asset Store, ensuring optimal performance improvements for Meta Quest devices. Note that development for Meta Avatars is currently only supported on Windows PC. Additionally, we address the installation of the Meta Avatar SDK, a separate component not included in the Oculus Integration. After successfully setting up, don’t forget to commit your changes, especially when working with large integrations like Oculus. Stay tuned as we advance our setup in the next lesson, introducing the essential multiplayer components for a captivating multiplayer experience!

In this lesson we continue the meticulous setup of our Unity project, focusing on integrating Photon’s Fusion and Voice APIs for enhanced multiplayer features. The lesson guides you through creating a Photon account, generating Fusion and Voice apps, and acquiring the necessary App-IDs. We ensure the Unity project meets the requirements for Fusion and proceed with the installation of the Fusion SDK and Photon Voice SDK, paying close attention to import instructions and component selections. This comprehensive setup is pivotal for the seamless functioning of multiplayer components in your VR experience. Looking ahead, we will explore the course assets and scenes, providing insights into their utilization. Stay engaged for more valuable information on efficient app testing and running in upcoming lessons!

In this concise lesson, participants are walked through the process of importing pre-prepared assets essential for the project, including two pivotal scenes: “home” and "collab space." These scenes act as templates, laying the groundwork for crafting social VR experiences and implementing a variety of advanced, networked features. The lesson structure offers flexibility, allowing learners to focus on the elements most pertinent to their projects. Additionally, valuable insights into optimizing the development process for the Meta Quest device are shared.

In this lesson, learners are guided on how to test apps through the Unity Editor on their Meta Quest, focusing on utilizing the “Oculus Link” app, exclusive to Windows users, to facilitate this process. The lesson covers the installation and setup of the Oculus Link app, connecting the Meta Quest device to the PC, and launching experiences directly from the Unity editor. For Mac users, alternative methods involving the “Build and run” function in Unity are explored, ensuring they can effectively test and visualize their projects on the Meta Quest. The upcoming lesson promises insights into the Meta Quest Developer Hub, a crucial tool for XR developers working with Meta Quest devices.

In this final lesson of the project setup, participants will explore the functionalities of the Meta Quest Developer Hub, a versatile tool available for both Mac and Windows. The lesson guides users through the installation process, connecting the Meta Quest device, and enabling developer mode, offering insights into both cable and Wi-Fi connections. Learners will discover how to manage and launch APKs directly on their device using the Device Manager. With the completion of this lesson, participants are equipped with the knowledge to work efficiently on a Unity project, and are prepared to delve into setting up the OVR rig and adding interactive elements in the upcoming chapter.

In Chapter Three, participants will delve into setting up the OVR Camera Rig, a foundational element for interactions and avatar movement. The lesson meticulously guides through configuring the OVR Manager and optimizing settings for the targeted device, with a focus on enabling and refining hand tracking features. Learners will integrate Meta Avatars, adjust hand tracking frequency, and ensure synchronization with the latest hand tracking version. The session progresses with adding logic for hand movements, setting up OVR Interaction, and incorporating both hands and controllers for versatile user experience. Additionally, the lesson addresses potential visual issues and concludes with the successful setup of the Oculus VR Rig, paving the way for exploring locomotion in the upcoming lesson.

In this lesson, we continue building on our Oculus VR Rig by focusing on enabling interactions with game-objects such as grabbables and poke interactables. We start by creating a primitive cube, configuring it with components like “rigid-body”, collider, “grabbable”, and “hand grab interactable” from the Oculus Integration. Adjustments are made to the physics layers to prevent player collision with interactable objects. The lesson introduces the “Hand Grab Interactor” and “Hand Poke Interactor”, explaining their roles and how to integrate them into the VR Rig. We also explore the functionalities of “Poke Interactable” and “Interactable Unity Event Wrapper” components, demonstrating how they enhance user interactions with objects and UI. By the end of the lesson, students will have a comprehensive understanding of setting up interactors and interactables, laying a solid foundation for more advanced VR experiences.

If you are using the Meta XR SDK instead of the Oculus Integration, the SimpleCapsuleStickMovement script will be missing. You can find it in the resources of this lesson and simply import it into your project. It will still work like before!

In v57 or later the LocomotionHandInteractorGroup prefab does NOT have the ActiveStateFingerVisual or InteractorActiveState (They're just totally unused). It may also be worth noting that the prefab now has the BestSelectInteractorGroup on it now instead of the BestHoverInteractorGroup component. If you are on a version later than v56 just skip this part of the setup and the rest should still work like intended!

In this lesson, enhancements are made to the OVR Rig to enable movement within the VR environment, focusing on continuous movement and teleportation. A Rigid-body is attached to the OVR Camera Rig, set to be non-kinematic with gravity enabled, and a Capsule Collider is added for realistic interactions with the virtual world. The Simple Capsule With Stick Movement component is configured to facilitate avatar rotation and movement. Teleport locomotion is set up using the Locomotion Hand Interactor Group, and the Finger Feature State Provider is configured for hand pose recognition, allowing for actions like teleporting and turning. The lesson also covers the creation of teleport areas, ensuring a seamless navigation experience. Lastly, controller setup and sound enhancements are incorporated to enrich the user experience.

In Chapter Four, we delve into the intricacies of setting up Meta Avatars, a revolutionary step towards achieving highly expressive and customizable avatars, enhancing the social presence in your VR applications. This SDK, available to all Unity developers on Meta Quest, Oculus Rift, and Windows-based VR platforms, unfortunately, excludes Mac device compatibility. The focus will be on exploring the capabilities of the Meta Avatars SDK, its integration into VR applications, and the enhancement of user experiences through authentic representation. Additionally, we’ll uncover the setup of audio input functionalities for lip-synchronization and discuss the potential limitations and additional requirements for integrating with other VR platforms or technologies. This chapter lays the groundwork for Chapter Five, where the networking of avatars for smooth animations across applications using Photon Fusion will be detailed. Get ready to immerse yourself in the fascinating and highly personalized world of Meta Avatars!

this lesson, participants will delve into implementing Meta Avatars using the Avatar SDK Manager Horizon prefab, focusing on the Meta Horizon style, a popular avatar style. The lesson breaks down the four major components of this prefab: the OVR Avatar Manager, which manages tracking and animations; the Avatar LOD Manager, ensuring render and animation processes stay within performance boundaries; the GPU Skinning Configuration, for setting skinning quality options; and the Sample Input Manager, responsible for sending input and Inverse Kinematics transforms. Participants will learn to add a random default avatar to the scene using the Sample Avatar Entity component and explore the customization options available with the 32 pre-customized avatars provided by Meta. The lesson also covers essential settings adjustments, such as disabling “Load User From CDN” and removing visuals for hands and controllers, setting the stage for exploring personal Meta Avatar setups in subsequent lessons.

In this lesson, learners will explore the process of loading custom Meta Avatars in Unity, requiring the setup of two new apps on the Oculus Developer Dashboard for testing in the Unity Editor and running on the Meta Quest headset. The lesson guides through the steps of creating apps, selecting platforms, and requesting access to specific features like User ID, User Profile, and Avatars. Participants will learn to navigate through the dashboard, submit requests, and integrate the acquired App IDs into Unity’s Oculus Platform Settings. The lesson concludes with testing the setup in the Unity Editor, ensuring the visibility of the custom avatar, and sets the stage for the upcoming chapter focusing on multiplayer basics and further Meta Avatar synchronization across the network.

In this fifth chapter, participants are introduced to the Photon Fusion networking library, a pivotal tool developed by Exit Games for creating synchronized multiplayer gaming experiences. Fusion is distinguished by its efficiency and advanced data-compression techniques, ensuring optimal performance, especially in bandwidth-intensive VR games. The library offers a high-level API and is celebrated for its scalability and versatility, with a focus on Shared-Mode for VR applications. A comparative overview is provided, positioning Fusion against other networking solutions like Unity's Netcode and Normcore SDK, highlighting Fusion's superior balance of efficiency, usability, and reliability. The chapter promises an in-depth exploration of Fusion’s features and a step-by-step guide to crafting a multiplayer VR game. The upcoming lesson teases hands-on coding in Unity, introducing the Fusion Connection Manager, Network-Runner Events, and scene management with Fusion, setting the stage for more practical learning.

In lesson two of chapter five, learners explore connecting runtime to Fusion servers, starting game-mode, and spawning avatar prefab in Unity. The lesson guides through adding networking functionality to the scene, introducing Prototype Network Start and Prototype Runner game-objects, and creating a custom script named Connection Manager for better control. The Connection Manager script is meticulously crafted to manage connections, register Network Objects, and handle server connections. Practical application is emphasized, with steps to test the Connection Manager in Unity, ensuring objects are properly registered and synchronized with Fusion. The lesson concludes with a successful connection to Fusion servers and a teaser for upcoming topics, including loading different scenes with Fusion and synchronizing Meta Avatars across the network. The engaging tutorial sets the stage for more advanced networking concepts and hands-on learning in future lessons.

In this comprehensive lesson, users are guided through the intricate process of synchronizing Meta Avatars across a network using Unity, Oculus, and Photon Fusion. The tutorial begins with setting up applications on the Oculus Developer Dashboard and delves into creating scripts for User Entitlement checks and Avatar State Synchronization. A pivotal part of the lesson involves configuring Network Avatar Spawner and Avatar Entity State scripts to manage avatar spawning and state. The lesson also emphasizes the importance of handling Networked Variables and State Authority to ensure seamless avatar interactions across different clients. Towards the end, users are introduced to various testing options, including adding members to the organization, creating Test Users, and utilizing App Lab for user invitations. The tutorial concludes with a teaser for the next lesson, which promises to enhance the virtual experience by incorporating Voice Chat and avatar lip animations.

In this lesson, participants elevate their immersive interaction by integrating voice chat and lip-sync functionalities using the Photon Voice SDK. The process begins by adding a new child game-object, named “Voice”, to the network-avatar and attaching essential components like Voice Network Object, Recorder, and Speaker. The Recorder captures player audio, while the Speaker plays back the recorded audio, enhancing the avatar's realism. To synchronize avatar lip movements with audio, the OVR Avatar Lip Sync Context script from Oculus Integration is employed. Adjustments in the Avatar Entity State script ensure that only the avatar with state authority can utilize lip-sync, preventing local and remote avatars from overriding each other’s lip movements. The lesson concludes with the addition of the Fusion Voice Client component for managing connections and a glimpse into the upcoming chapter, promising insights into using Photon Fusion for various applications like scene-loading and object interaction.

In chapter six, participants learn to create a Scene-Loading system using Fusion by developing a Scriptable Object, a versatile data-container named "Scene Data". This object aids in managing scene-related information and visuals, enhancing project organization and reusability. The lesson progresses with the creation of the Scene Loading Manager script, incorporating UI elements and Poke Interactables for user interaction, and implementing a validation process to ensure the legitimacy of scenes. The system developed allows for dynamic scene transitions, with the UI reflecting current scene selections and facilitating scene loading through user interactions. The lesson concludes with the successful integration of the Scene-Loading system in Unity, showcasing the ability to load different scenes with corresponding names and preview-images, and a teaser for the upcoming lesson on object interaction across clients.

In this lesson, the focus is on resolving synchronization issues related to object grabbing in a multiplayer environment using Fusion. A new script, Network Grabbable, is introduced, inheriting from Network Behavior and coordinating with the existing Grabbable script to maintain synchronized events across all objects that can be grabbed. The script utilizes a Network Bool property named Is Kinematic to synchronize the “Rigid-body’s” state across clients, ensuring a consistent experience for all participants. The lesson delves into managing Pointer Events, requesting State Authority, and implementing a workaround for releasing objects using Oculus Integration. Upon returning to Unity, participants set up objects for grabbing, adjust settings, and test the newly implemented feature, witnessing real-time interaction and object grabbing synchronization. The lesson concludes with a preview of the next topic on synchronizing color changes of game-objects across the network.

In this tutorial, XR developers are guided through creating a Synced Color booth at Station one, focusing on two methods of changing object colors while maintaining synchronization between clients. A new script, Color Sync, is introduced, inheriting from Network Behavior and utilizing networked variables and callbacks. The script incorporates an array of Mesh Renderers and a Networked Color to manage color alterations. Developers are taught to create callback methods, such as On Networked Color Changed and Color Update, to handle color changes and maintain code readability. The tutorial emphasizes the importance of State Authority and introduces Remote Procedure Calls (RPCs) to manage color changes across different clients. Practical applications include setting custom colors through the inspector and randomizing colors, with the script ensuring synchronization. The lesson concludes with setting up Unity, assigning Mesh Renderers, configuring poke interactables, and testing the feature, demonstrating successful color changes. The tutorial hints at future applications, such as avatar color changes, and teases the next lesson on timers and particle effects.

At the end we never mention that you should also add a new field on the Poke Interactable's "When Select"  and add the "ToggleTimer" method there!  We will update the video soon!

In lesson four of chapter six, learners are guided to create a synchronized timer feature using a new script named Timer Sync, attached to the Timer Station parent object. The script incorporates several elements including a Text Mesh Pro U Gui for time display, a Particle System for visual effects, and a Networked Timer property with callback functionality. The lesson emphasizes the importance of State Authority in managing timer activity and ensuring synchronization across clients. The Photon Fusion framework is utilized, operating at 60 ticks per second, to facilitate real-time updates and interactions. Upon successful implementation in Unity, learners can observe the timer start, stop, and reset, with all clients witnessing the same visual effects, such as confetti firing. The lesson concludes with a preview of the next feature, focusing on creating a counter with auditory feedback.

In this lesson, participants are guided to create an interactive Counter Sync script in Unity, which is assigned to the Counter Station parent-object. This script incorporates two essential serialized fields: counter Text, a Text Mesh Pro U-GUI for visual representation, and Audio Source for auditory notifications. The tutorial emphasizes the creation and utilization of a networked property of type Integer with callback functionality, ensuring the counter's value remains synchronized across different clients. The methods Increase Counter and Decrease Counter are introduced, employing Remote Procedure Calls (RPCs) to maintain consistency by delegating changes to the state authority. Learners are instructed to assign references and configure settings within Unity, leading to the development of a fully synchronized and interactive counter with real-time auditory feedback. The lesson concludes with an enticing glimpse into the upcoming feature, a controllable slide-deck presentation, keeping learners engaged for future tutorials.

In this lesson, participants are introduced to the process of synchronizing slides for a presentation using a new script named Slide Sync, assigned to the Slideshow parent-object. The script, inheriting from Network Behaviour, utilizes Fusion's network capabilities and incorporates two private serialized fields: Slide Image for displaying slides and Slide Sprites array for storing different slides. A networked property, Networked Image Index, is created to manage transitions between slides, with Next Slide and Previous Slide methods ensuring synchronization and state authority verification. The lesson emphasizes the importance of callbacks, specifically On Networked Image Index Changed, and the Update Slide method for achieving a unified viewing experience across various clients. Upon completion, participants can test the feature in Unity, ensuring that all clients view the same slide, with late joiners also receiving updates. The lesson concludes with a preview of creating a synchronized step-by-step workflow in the upcoming lesson.