Signie
/01The Idea
Learning through doing, not only watching.
Signie began with a basic communication and learning problem: ASL learners need to observe a sign, try it with their own hands, and receive feedback within the same experience. The first spatial prototype placed a virtual instructor and learner in a shared MR environment so practice could become embodied rather than passive.
Early ShapeXR prototype testing the spatial relationship between learner, instructor, and interface.
/02V1
Building the Core Experience
The initial concept became a working MR experience built around a virtual instructor, a spatial learning environment, and hand-based interaction. The learner could watch the instructor, mirror the demonstrated movement, and move through the early learning flow inside the headset.
- Virtual instructor
- Demonstrates signs within the learner’s view.
- Hand interaction
- Turns observation into embodied practice.
- Working build
- Connects the scene, spatial interface, and interaction logic in Unity.
Primary MR Experience
Unity MR prototype connecting the virtual instructor, spatial UI, and hand interaction.
Word Recall Exercise
A learner-facing word-recall exercise extending the first working MR flow.
/03V2
From a Demo to a Learning System
V2 expanded the working demo into a structured learning loop. Learners first followed a guided sign, then practiced from recall, reviewed the motion when needed, and applied it in a rhythm-based game.
Learn
Copy a static pose to unlock the full motion, visualized with movement bubbles.
/04V3
Exploring AI Glasses
V3 carried the same communication idea into a lighter wearable context. Micro-gestures provided hands-free control, while speech was converted to text through Wit.ai and routed through an animation state machine for signed output.
Micro-Gesture Interaction
Micro-gesture input for hands-free system control.
Wit.ai / Speech Input
Speech-to-text and animation-state workflow for live sign output.
/05Process
From Design to Build
The learning flow moved from sketches to motion, in-headset gesture recording, and working Unity prototypes—bringing the virtual instructor, spatial UI, and hand interaction into one MR experience.
Virtual Guide Tool
A custom in-headset authoring tool for recording tutor and two-hand gestures used in guided learning sequences.
Interaction / Production Planning
Mapped the learning flow, spatial interface, and demo narrative.
Motion Capture
Prepared selected ASL sentences for animation through motion capture.
Unity / MR Prototyping
Brought the scene, spatial interface, and interaction logic together in a working build.
/06Role
My Contribution
Signie was a team project. My contribution connected the project’s structure and learning experience to the practical work required to build and test it.
- Project ownership
- Meeting structure, task breakdown, production planning, and coordination across the team.
- Experience direction
- Owned the learning flow, spatial interface, and interaction behavior from concept through testing.
- XR implementation
- Built scenes, implemented interactions, and tested the working mixed-reality experience.
- Motion / assets
- Prepared production assets, animated characters, and integrated them into the Unity build.
/07Outcome
2× XRDC AWARD WINNER
Signie received the Contextual AI and Community Impact awards. Across three iterations, the project developed from an MR learning concept into a playable learning system and a wearable live-translation exploration.
Supporting evidence from the award outcome.
