After speaking of haptics design and Interaction Building, let’s talk about one of the newest and most powerful technology: hand tracking for VR/AR/MR and its input mechanisms.
Hand tracking for VR & MR is bringing a new level of immersion and user experience into XR applications. Indeed, users see a button, reach out and click with their fingertips. Everything is easy there! Super user experience, little usage friction and excellent immersion.
This feature is so compelling that almost every major headset maker is implementing hand tracking for VR and MR for their headsets.
Hand tracking for VR and MR works undoubtedly well because the user doesn’t have to learn anything while interacting with a virtual object. The user sees an object and reacts exactly as they would in real life. A ball on the table? Reach out and grab it!
Hand tracking for VR and MR: the GRASP Taxonomy
It can be argued that the development of great intelligence as human species is tied to the ability to create and manipulate new tools and objects. When we look in detail into how we, as humans, grasp and manipulate objects we come out with a map explained in The GRASP Taxonomy of Human Grasp Types 1 illustrated in Figure 2.
This subdivision is split into two major categories: Power and Precision manipulation, as a function of how we control the position of the object. Moreover, the finger pad is for precision manipulation and the hand palm is related to power manipulation. This is fundamental for hand tracking for extended reality.
Indeed, what we see in this division is the radically different way how we manipulate different objects. More precisely, how the finger is arranged and how the pre-manipulation action is made to accommodate the final position. We implemented these aspects into hand tracking for VR and MR, read the paragraphs below to discover how.
Hand tracking for extended reality: our experience
During the last several years working on customers’ projects using hand tracking for VR and MR, we realized that a unique pinching and grabbing input were not responding to the necessities of a smooth user experience. On the contrary, they generated a significant amount of frustration in the final user. For instance, pinching to move a large object is not expected at all by the user!
In some cases, we ended up avoiding the immersive characteristic of the environment because of the inconsistency of the user’s input actions. Therefore, users were not able to use the product. That means that we had a non-consistent user expectation across the product. Some interactions were realistic, and some of them had a high level of abstraction. This choice was not driven by the user preference, but by an inconsistent response of the hand tracking for VR and MR not taking into consideration human factors.
The mission of Interhaptics is to give consistent, realistic, and immersive development tools for XR creators to increase the value of the final product. For this reason, in partnership with several research experts, we developed grabbing algorithms that inspired the human behavior for hand tracking for VR and MR. The objective of these algorithms is to give the best UX possible while using hand tracking for extended reality.
We segmented the real hand behavior, extracted the core parameters, and transcribed it in virtual reality to create a consistent tool for hand tracking for VR/AR/MR! This will be available for any hand tracking software supported by Interhaptics: Oculus, Vive, Ultra Leap, and the others planned in the future.
In the next paragraphs you can see some examples:
Precision Grab
Power Grab with Thumb
Power Grab without Thumb
Extreme cases
As you can see, hand tracking for extended reality is a fundamental aspect of it. For us, its development is the result of a long reflection to improve each characteristic of its features.
By observing the way each of us grabs or pulls something, we built the most realistic technology in order to maintain and optimize your own XR use. But this process is still open to your feedback, which enriches the development.
Click here to see our entire video dedicated to hand tracking for VR & MR. These features are all available in our Hand Tracking for VR & MR package.
Check out all our articles here to read about how haptics keep you immersed in your VR experiences.
Comments are closed.