About HoloLens® Air Grab Navigation

HoloLens® 2 is the latest augmented reality headset from Microsoft. It comes with holographic display and advanced tracking and gesture recognition capabilities. This device lets you modify the position and orientation (pose) of all the holograms in the physical space through hand gestures. For more information, see https://docs.microsoft.com/en-us/hololens/.

When air grab navigation is enabled, you can modify the positions and orientations of all the holograms displayed through the HoloLens® visor by executing intuitive gestures.

Important: For a satisfying experience, you need:
  • A computer with a high-end AR/VR-compatible graphics card.
  • Fast and reliable Wi-Fi network to stream video from your computer to HoloLens®. Performance quality directly impacts display stability in HoloLens®, and the tracking quality.

This page discusses:

Air Grab

The air grab navigation feature is based on tap gestures (index and thumb) or grab/grasp gestures (whole hand).

Depending on the gestures, holograms are either translated or rotated around the vertical axis (in the physical space).

Important:
  • Air grab does not distinguish tap from grab and handle both gestures as the same input.
  • You cannot scale holograms through air grab navigation.

Gestures

Gestures are hand movements or stances recognized by the HoloLens® 2 tracking system.

For more information on:

There may be a visual feedback in the form of a ring hologram around each of the user’s wrist.

Note:
  • There is no ring when the user hand is not tracked.
  • There is a dashed ring when no gesture is being performed but the hand is tracked.

  • There is a plain ring when the user performs an air tap or a grab gesture.

Navigation with One Hand

This kind of navigation lets you perform translations and rotations in the horizontal plane.

Below is an example:

  1. Tap or grab with any of your hands and hold.

    The ring associated with this hand becomes plain: air grab is activated in one-hand translation mode.

  2. Move your hand (tap or grab is still performed) before you or walk.

    The whole virtual world is translated. The perceived result is that for any hologram:

    • (1) its position in the physical world relative to the user hand is unchanged.
    • (2) its orientation in the physical world remains unchanged.

  3. Strongly rotate your body to the left or right (tap or grab is still performed): air grab switches to one-hand translation and rotation mode.
    Note: This mode is active until air grab is stopped.
  4. With one-hand translation and rotation mode active, move your hand (tap or grab is still performed) before you or walk.

    The whole virtual world is translated or rotated. The perceived result is that for any hologram:

    • (1) its position in the physical world relative to the user hand is unchanged.
    • (2) its orientation in the physical world relative to the direction (user shoulder, user hand) in the horizontal plane remains unchanged.

  5. End the gesture: the ring associated with the hand becomes dashed and air grab navigation stops.

Navigation with Two Hands

This kind of navigation lets you perform translations and general rotations.

Below is an example:

  1. Tap or grab with both hands and hold.

    The rings associated with your hand become plain: air grab is activated in two-hand translation and rotation mode.

  2. With two-hand translation+rotation mode active, move your hand (tap or grab are still performed) before you or walk.

    The whole virtual world is translated or rotated. The perceived result is that for any hologram:

    • (1) its position in the physical world relative to the barycenter of the user hands remains unchanged.
    • (2) its orientation in the physical world relative to the direction (left hand, right hand) remains unchanged.

    More precisely, its orientation is the orientation when two-hand navigation started, updated by applying the following rotation:

    • rotation axis: (L, R)^(L’, R’)
    • rotation angle: angular distance ((L, R), (L’, R’))

    Note: L is the left-hand position when two-hand navigation started; L’ is the current left-hand position

  3. End the gesture: the rings associated with the hands become dashed and air grab stops.