visionOS Custom Gestures

CustomGestures

The built-in gestures did not let me snap objects onto surfaces, and visionOS volumes did not give enough control over orientation, so I built CustomGestures. It is a RealityKit component for visionOS 2 that handles drag, rotate, and scale on an entity. This allows much greater control over position, orientation and scale, and allows users to quickly and easily snap objects on and off of surfaces.

Features

• Surface snapping against a collision-group mask, with animated attach and release.

• Drag, scale, and rotate, each independently toggleable through GestureCapabilities

• Drag pivot modes: keep orientation, face the camera, or follow the gesture

• Rotation configurable to a set of: yaw, pitch, roll. Plus a gesture threshold.

  • Developers could lock rotation to yaw-only, for example.

• Short move() smoothing so drag updates do not jump between gesture samples

• RealityView.installGestures() wires DragGesture, MagnifyGesture, and RotateGesture3D to any entity

The package targets visionOS 2 and is distributed as a Swift package. It uses RealityKit and SwiftUI, and it depends on LookAt so a dragged entity can turn to face the camera.

Rebuilding the Interface

visionOS volumes did not give me the control I wanted, so I rebuilt part of the familiar window chrome as a RealityKit entity. The handle is a small white rounded bar, flattened so it reads as a thin grab surface. A view-direction based highlight effect brightens the Entity as you look at it, and the highlight is kept on its own group so it does not spill onto the content behind the handle.