Interaction in AR

With smart AR glasses or similar, there is no display to interact with, so touchless gestures become a necessity. Especially when interaction with virtual object is needed. Also voice control and buttons in combination with gestures is an option forming a multimodal interaction experience for the user.

Voice works well for text input while gestures work excellent for more complex and dynamic interaction like zooming, scrolling, panorating, drawing and manipulation in general of with virtual 3D objects. The situation and interaction need decides which method that is most efficient.

How to interact in AR

Depending on the type of AR hardware and application the interaction can be either smartphone-like in 2D or through interaction with virtual objects in 3D.

If you want to work in 3D with virtual objects there is no real alternative to gestures.

Gesture control vs. voice control

Both gesture and voice control are developing rapidly and solve interaction problems in different situations. Both alternatives work very well and is preferably used in a multimodal interaction solution. Since camera sensors are developing rapidly and special processing units for neural nets (NPU) become more commonly used in devices, it is expected that gesture control will evolve even faster moving forward and Crunchfish is actively driving that market.

Read more