4.3 KiB
Gibalna rokavička
Opis
Wavey Wind, the Modul-air Mitt
Using an ESP32 microcontroller with wifi and bluetooth capability, a gyroscope/accelerometer module and keyboards switches, I will be building an experimental low cost motion "glove" controller with a button for each finger.
The idea stems from an earlier abstract visualisation project which was intended to visualise music. After building the initial prototype and hooking it up with a touch user interface of sliders that modulate the visuals, I asked myself why should I make the program interpret music? We already do that when we listen and (can) react by dancing; using a motion sensor, the dancing can be "aplified" by turning bodily motions into shapes and colours projected on a surface.
Using the motion mitt, the operator of visuals doesn't have to get locked into a clumsy little touchscreen but can immerse in the experience of sound and video while co-creating it. A workshop will be held to build a number of gloves that can connect in an ad-hoc network, so that group of people could collaborate with them.
The glove(s) will send events via the open sound control protocol, opening the possibility to using using them for audio synthesis/modulation, or maybe even as a general interface.
"What Can a Body Do?"
Gibalna rokavička ("Wavey Wind, the Modul-air Mitt")
Z ESP32 mikrokontrolerjem, ki omogoča wifi in bluetooth povezljivost, modulom z žiroskopom in pospeškomerjem ter tipkovničnimi stikali bom izdelal eksperimentalni nizkocenovni gibalni vmesnik z gumbom za vsak prst.
Ideja izhaja iz zgodnješega projekta abstraktne vizualizacije, ki naj bi odsevala glasbo. Po izgradnji zgodnjega prototipa z vmesnikom na drsnikov na dotik, ki modulirajo prikaz, sem se vprašal zakaj bi program interpretiral glasbo? To namreč počnemo že mi, na poslušanje (lahko) odreagiramo s plesom, gibalni senzor pa bi lahko telesne gibe "ojačal" v oblike in barve, projecirane na površino.
Z gibalno rokvičko se opravljalka_ec vizualij lahko izogne ujetosti v neroden zaslon na dotitk in potopi raje v izkušnjo zvoka in videa med soustvarjanjem le-te. Organizirana bo tudi delavnica za izgradnjo večih rokavičk, ki se lahko med sabo povežejo v ad-hoc mrežo, kar skupini ljudi omogoči sodelovanje.
Rokavica oz. rokavice bodo pošiljale podatke preko OSC (open sound control) protokola, kar odpira tudi možnost rabe za zvočno sintezo/modulacijo ali celo kot splošni vmesnik.
"Kaj zmore telo?"
Shematika
ESP32 <-> MPU-6050 (gyro/accel)
- 21 - SDA (data)
- 22 - SCL (clock)
- 3V3 - VCC (strom)
- GND - GND (ZEMLA)
key switches
- one pin of each to GND
- other pin of each to 16, 17, 5, 18
Arduino sketch
dependencies (libraries)
- OSC (filter by "open sound control")
- MPU6050
calibration
- IMU_Zero
testing, prototypes
- kbdtest
- keypadtest (kbd matrix test, didn't work)
- proto32, proto32plot, osc32, osc32h, osc32hh
Final sketch
- osc32final
3d models
- proto1.blend first prototype
- proto2.blend second prototype
visual
https://git.kompot.si/g1smo/wavey-wind
start with `npm i; npm start`
localhost:6676/ localhost:6676/test
older prototypes
history
The original animation, modularized with sliders, was first "exhibited" on october 6th 2017 in Modri kot, a beautiful space for performance that used to be on the side of the autonomous factory Rog in Ljubljana, which was evicted by force on january 19th 2021 by the municipality to pave way for future gentrification. Special thanks to Zala and Lio for hosting me during early development and Tisa Neža for curating the first performance.
TODO
- fix bad rotation
- better documentation
- use accelerometer
- experiment with sound (suppercollider via OSC?)
- wireless (wifi or bluetooth?)
- ad-hoc connection to use multiple controllers
- better starting parameters
- ergonomic control
- dashboard to see parameter values
- live coding interface