Mixed Reality Development Fundamentals

In this lecture, let me introduce the Meta Quest Developer Hub, or MQDH for short, to you. Setting it up with the Meta Quest headset is quick and easy. It’s a cross-platform desktop tool, which runs on Mac and Windows. It allows us to easily connect and manage all our Meta Quest devices, monitor our performance, record videos and screenshots of our gameplay, and most importantly, to easily install APKs on our devices, as well as publish them.

In this lecture we will learn how to use the Meta Quest Link app, or Oculus Link app, as it is still commonly called! This app lets us directly test our scene through our Unity Editor on our Meta Quest device. Keep in mind that this app only exists for Windows PCs at the time of recording! Setting this app up for testing can save us a lot of time in our development process.

If you are having issues testing you scene in the Editor with the Meta Link app try deploying it to your device. Capsense is not yet fully supported by the Link app!

In the last two lectures, we learned how to implement controller and hand tracking. In this video we are going to look at the new improvements that version 62 of Meta’s XR SDK brought to our hand tracking. We are going to look at Capsense (for showing hand and controller visuals at the same time) and Multimodal (to track hands and controllers at the same time).

OVRInput exposes a unified input API for multiple controller types.

It is used to query virtual or raw controller state, such as buttons, thumbsticks, triggers, and capacitive touch data. It supports the Meta Quest Touch controllers.

In addition to traditional gamepad buttons, the controllers feature capacitive-sensitive control surfaces which detect when the user’s fingers or thumbs make physical contact (Touch), as well as when they are in close proximity (NearTouch). This allows for detecting several distinct states of a user’s interaction with a specific control surface.

Hand tracking enables the use of hands as an input method for the Meta Quest headsets. To build a rich experience when using hands as input modality, you need to incorporate multiple interactions considering how your object is placed. Near-field objects are within arm’s reach. Direct interaction such as poking or pinching works well with these objects. Far-field objects are beyond arm’s reach and require raycasting, which directs a raycast at objects at a far distance. It is very similar to Touch controller interaction.

Poking and pinching are real time gestures and very intuitive for any user to perform basic tasks such as setting focus, selecting, or manipulating an object in space. To standardize interactions across apps, OVR Hand provides access to the filtered pointer pose and detection for pinch gestures to ensure your app conforms to the same interaction models as Oculus system apps. Simple apps that only require point and click interactions can use the pointer pose to treat hands as a simple pointing device, with the pinch gesture acting as the click action.