Interhaptics is a member of the Haptics Industry Forum and co-author of the original article. Interhaptics plan to support all haptic standards as soon as accepted. To learn more about the submission, click here.
Current state of the haptic ecosystem
Human-computer interaction is increasingly utilizing multiple modes of simultaneous sensory feedback that includes video and animation for the eyes, audio for the ears, and haptics for the sense of touch.
The increasing use of haptics has led to growing awareness and demand for haptic technology and content. However, haptics faces unique challenges. There exists no common media format or tooling to enable content creators, no standard platform APIs to enable developers to support haptic hardware, and no standard mechanical configurations for OEMs to drop into their devices.
Because these problems are interdependent, most products with haptics are vertically integrated with components that cannot interoperate with other systems.
For experience creators who are used to content being platform-neutral, the haptic landscape is highly fragmented and inconsistent, which impedes creative exploration of the use of haptics to make novel user experiences.
Our vision for haptic standards
- Standards for haptic representation and coding enable content creators to create haptic content and monetize it across a wide range of end-user devices. Content creators can expect that the end-user experience will preserve the creative intent behind the haptics.
- Standards for device manufacturers allow them to showcase rich haptic experiences on their devices and rely on a robust pipeline of compelling content to support device sales.
- Haptic standards enable broader adoption of haptics in the mass-market overall, which grows the ecosystem and the commercial opportunities of the entire market.
This vision translates to these 3 agenda Items detailed later in the article.
- Agenda item 1: Haptic experience encoding.
- Agenda item 2: Haptic platform API
- Agenda item 3: Device protocol
In the domains of audio and video, standards have been developed that simultaneously enforce baseline fidelity levels and define a design space for creative exploration. Such standards exist for media coding, platform APIs, and physical components.
These standards enabled a rich ecosystem of content creators who could rely on a large enough content market to justify the cost of production. The existence of this market, in turn, triggered investment in design tools, content libraries, streaming technologies, and so on, and has resulted in a robust standards-based media ecosystem.
Agenda item 1: Haptic experience encoding
At the founding meeting of the Haptics Industry Forum in December of 2019, the assembled industry experts aligned on a common need to provide a standard mechanism for abstracting devices and platforms from content.
Since that meeting, Immersion has led the development of content standards in the context of MPEG, with these results:
- Haptics is being considered as a first-order media type, along with video and audio. This means that MPEG files could contain just haptics, haptics, and audio, or any other combination of first-order media tracks. This activity is on the ballot and may become formally adopted as early as October of 2021.
- MPEG has issued a call for proposals for a coded representation of haptic signals. This will become a component of the MPEG-Immersive multi-year program.
- MPEG has mandated an ad-hoc group to develop requirements and eventually a call for proposals for coded representation of interactive haptic experiences. This will become a component of the MPEG-Immersive multi-year program.
- Haptics Industry Forum is seeking to become a Liaison to the International Standards Organization (ISO) to facilitate contributions from HIF members to MPEG. All MPEG ad-hoc group activities are open to participation from any people with expertise in haptics, regardless of their membership status in ISO or MPEG.
This standard will enable interoperable content-encoding which will allow the ecosystem to develop standardized haptic libraries and creation tools that can be used by designers in multiple markets. This will reduce the barriers to entry for haptic design and enable more and more impactful haptic experiences.
Interhaptics Engagement: Interhaptics is a proponent for the MPEG phase 1 and 2 for the haptics encoding standard without Haptic Material encoding. To date, the only proponent covering both phase 1 and phase 2 requirements is Interhaptics.
Agenda item 2: Haptic platform API
There is no standard platform API that allows developers to access the functionality of a broad range of haptic hardware without writing device-specific code.
The development of a broadly applicable device protocol would encourage abstraction between end-user experiences and the physical hardware that renders them. The haptics ecosystem needs to define common APIs with standard functionality for applications running on Android, iOS, Windows, and other platforms.
Haptics Industry Forum has recognized the need to drive these APIs and is engaged in a Liaison relationship with Khronos Group, specifically targeting the OpenXR and glTF working groups. The goal of entering into this relationship is to ensure OpenXR has sufficient API functionality and abstraction to support a wide range of hardware capabilities. HIF will also engage the IEEE P2861.3 group working on mobile platform API and presentation haptic standards.
This standard will reduce the artificial barriers between different products by providing a consistent API that developers and toolmakers can rely upon.
Interhaptics Engagement: Interhaptics is a representative for the HIF-Kronos liaison working group to help OpenXR to structure haptics standards.
Agenda item 3: Device protocol
On a haptic peripheral, the communication between the microcontroller and the overall system commonly uses USB-defined HID (human interface device) protocols. Although there is an HID protocol for haptics that was ratified in 2016, it is primarily intended for single haptic effects that are triggered by a single event; it does not contemplate streaming of haptic effects, multiple channels of haptics, synchronization of haptics with other modalities, high-definition effects, kinesthetic effects, and so on.
An extensible device protocol is needed to support legacy devices such as game controllers with rumble features, haptic mice, and so on, as well as near-future devices that can provide an endpoint for platform APIs and rich MPEG content.
The existence of such a protocol will enable 3rd party peripheral vendors to compete and innovate to generate amazing new peripherals that can immediately access experiences already developed for the current best-of-breed peripherals.
Haptics Industry Forum has recognized this need and formed an HID Working Group in April of 2021. This group is expected to register a new haptic usage page for HID before June 30th, 2021.
This standard will enable plug-and-play compatibility of disparate haptic peripherals and reduce the cost of implementing advanced haptic functionality for device makers. This will enable innovative and competitive hardware to come to market that meaningfully leverages existing content but also provides new experience opportunities for content creators.
Interhaptics Engagement: Interhaptics is the only software pure player contributing to the working group for the USB-HID Haptics Protocol. Interhaptics expects to support this protocol from day 1.
Interhaptics Strategy and execution: Interhaptics fully believes that an interoperable ecosystem is a way to grow the opportunities for content creators and device manufacturers alike to create a vibrant market.
Interhaptics is already acting today to enact this vision
Thanks to Interhaptics, creators can create Haptics experiences for VR in a unique way for several haptics devices thanks to our Haptic Composer and Unity Plugin (Click here to start).
Currently Interhaptics supports: all VR controllers, Senseglove Exoskeleton, Manus VR prime 2 gloves, and several others.
Thanks to Interhaptics mobile game developers can create Haptics experiences for both iOS and all the different way Haptics is managed in Android. (Click here to start)
We are actively collaborating with several haptics technology companies to enable a vibrant ecosystem around our platform. Some of them are:
If you are interested to share your vision and to participate to these activities, contact ev@interhaptics.com
This article was adapted from https://hapticsif.org/standards/
Eric Vezzoli, Ph.D.
