Body, Device and Instrument
Across a number of works, the interface is not simply a screen placed between a person and software. It is a relationship between body, device, sensing, mapping and the changing result.
In i Touch the Moon, movement and touch on an iPod touch become performance data carried by OSC into Max/MSP and Csound. In Memory of Orchestra, iPhones operate as instruments inside an ensemble, connecting touchscreens, three-axis motion sensing and coordinated bodily action to real-time sound. In Light Object, an iPhone provides a live interface for shaping video, sound and stage lighting during performance.
LH XYZ — Motion as OSC
LH XYZ turns the iPhone into a wireless motion interface. Accelerometer and gyroscope data are streamed as OSC messages to compatible hosts, allowing movement to control musical, visual and other real-time parameters.
The important shift is from a fixed on-screen control to continuous physical movement. Orientation, acceleration and gesture become data that can be mapped elsewhere in the system.
Recognising Motion
Version 2.0 of LH XYZ extends the interface from continuous sensor streaming into motion recognition. A neural-network Human Activity Recognition system identifies tagged movement patterns from smartphone sensor data and sends the predicted motion tags to OSC receivers.
This adds another layer between physical action and artistic control: the system can respond not only to raw acceleration and rotation values, but also to recognised categories of movement.
Touch as Sound Interface
Motion is only one part of this line. Touch-based instrument design is equally important. LH Grain Pad is an iPad granular synthesizer organised around direct touch control, with independent granular channels, integrated effects and OSC communication.
Here the interface does not merely expose synthesis parameters. The touch surface becomes part of how granular sound is performed and shaped in real time.
Interface as a Compositional Layer
Across these works and systems, interface design determines what kinds of actions can enter the work and what those actions can become.
A tilt may become timbral change. A sweep may alter a larger musical process. Environmental values can enter the same system as bodily movement. A touch surface can become an instrument. Motion-recognition tags can become higher-level control signals.
The interface therefore sits between physical action and computational process, but it is not neutral. It selects, transforms and structures the relationship between body, software, sound and image.
Interface Paths
Body → Mobile Sensors → OSC
Accelerometers and gyroscopes turn phone movement into continuous control data that can be mapped to sound, image and other real-time parameters.
Data → Mapping → Recognition
Max/MSP, custom mappings and motion-recognition systems interpret incoming values and relate them to musical or visual behaviour.
Touch → Instrument → Form
Touchscreens and custom software interfaces shape how synthesis, performance and larger compositional processes are accessed in real time.
Current Project Line
- Title
- Motion & Interfaces
- Period represented
- 2010–
- Artist
- Liu Hao / 刘昊
- Core concerns
- Bodily movement, touch, sensing, interaction, interface design and real-time mapping
- Motion technologies
- Accelerometer · gyroscope · mobile-device sensing · motion recognition
- Communication
- OSC · Open Sound Control
- Software / systems represented
- Max/MSP · Csound · iOS · LH XYZ · LH Grain Pad
- Related works
- i Touch the Moon · Light Object · Memory of Orchestra
- Status
- Continuing project line; additional interfaces and motion-based systems can be added later