SonoChroma Keyboard
Interactive Software Prototype | Accessible Design Innovation | Sonochromatic Expression | Synaesthetics
#AffectiveDesign #Accessibility #InteractiveDesign #SoundDesign #ColorTheory #Innovation #WebApplication
The Concept
What if typing became music? What if words had colour?
SonoChroma Keyboard is an interactive web-based application that transforms typed language into sonic and colored 'sonochromatic' expression. Each keystroke becomes both a character and a note—an act of real-time audiovisual composition. Words emerge as melodies and chords, with punctuation marks acting as percussive elements.
This isn't just a creative experiment—it's a research prototype exploring how multi-sensory interaction can expand accessibility and create new forms of expressive communication.
The Challenge
Traditional accessibility design often treats sensory differences as deficits to compensate for. I ask: what if we flipped that model? What if users who navigate primarily through visual-sense or sound only weren't accommodated as an afterthought, but instead became power users of entirely new interaction paradigms? Combining the two as both discretely and independently important but also as novel design interdependencies positions the SonoChroma app accessibility as a generative design principle—not a compliance checkbox.
How It Works
Real-Time Audiovisual Composition
Each letter triggers a specific musical note and color combination
Words become melodic sequences with chromatic accompaniment
Punctuation adds rhythmic elements (drum sounds, pauses, accents)
Typing speed and rhythm influence the musical phrasing
The result: Writing becomes performing—every message is a composition
Accessibility Through Synesthesia
The system explores what happens when affective resonance guides meaning-making:
Users with visual impairments navigate through sound feedback
Users with hearing differences navigate through color cues
Neurodivergent users can express mood and tone through compositional choice
All users gain a multi-sensory dimension to text-based communication
Design Philosophy
Disability as Social Model
SonoChroma challenges the idea that "disability" exists in the user rather than in the design. By creating an interface where sound and color are primary, not supplementary, it asks:
What might users with sensory expertise teach us about interface design?
Could multimodal interaction become the default, not the accommodation?
How do we design for emotional expression, not just information transmission?
Progressive Design Thinking
Rather than retrofitting accessibility features, SonoChroma builds them into the core interaction model. This approach:
Centers diverse users in the design process
Explores novel interaction paradigms rather than replicating existing ones
Values affective communication alongside semantic meaning
Treats play and exploration as legitimate forms of interaction.
Technical Innovation
Web-Based Architecture
Built as accessible web application (no proprietary platforms or downloads)
Responsive across devices (desktop, tablet, mobile)
Low bandwidth requirements for accessibility in varied contexts
Sound Design System
Real-time audio synthesis engine
Customizable instrument and scale options
Spatial audio capabilities for immersive experience
Adaptive dynamics based on typing patterns
Colour Mapping Framework
Research-based color-emotion associations
HSL/RGB spectrum integration
Customizable palettes for user preference
High contrast modes for visual accessibility
Research Questions Explored
Interaction Design:
How does multi-sensory feedback change typing behavior and emotional expression?
Can sonochromatic mapping create more emotionally nuanced digital communication?
What new forms of creative expression emerge from expanded input modalities?
Accessibility:
How do users with different sensory capacities experience and adapt the system?
Can "expert users" (those with sensory differences) guide future development?
What universal design principles emerge from accessibility-first prototyping?
Affective Design:
Does audiovisual feedback increase emotional awareness during communication?
How does the playful, musical quality affect user engagement and wellbeing?
Can affective interfaces reduce cognitive load while increasing expressiveness?
Future Development (Beyond Prototype)
Phase 1: Expanded User Testing (Current)
Conducting studies with diverse user groups
Gathering feedback from accessibility community
Iterating based on real-world usage patterns
Phase 2: Customisation & Personalisation
User-defined key mappings (notes, colours, sounds)
Preset "mood palettes" for different emotional contexts
Save and share compositions/settings
Phase 3: Communication Integration
Plugin versions for messaging platforms
Collaborative composition features (multi-user)
Integration with assistive technologies
Impact & Insights
Design Learnings
Accessibility can drive innovation, not constrain it
Play is a legitimate research methodology—exploration reveals unexpected insights
Affective feedback transforms functional interactions into meaningful experiences
Multi-sensory design benefits all users, not just those with accessibility needs
User Feedback
Early testing reveals that SonoChroma:
Increases mindfulness during typing—users become more aware of rhythm, pace, and emotional tone
Creates joyful interactions—the musical quality adds delight to mundane communication
Reveals new expressive possibilities—users discover they can "compose" mood through writing style
Democratises creative expression—no musical training required to create sonic experiences
Try It Yourself (Desktop Prototype Only)
https://sonochroma.vercel.app/
Key Takeaways
Accessibility-first design that treats sensory diversity as creative opportunity
Interactive prototype demonstrating real-time sonochromatic expression
Research-driven approach exploring multimodal communication and affective feedback
Progressive design thinking challenging traditional accessibility models
Web-based and open—designed for broad access and iterative development