Our Story

This work started because I got something wrong.

I was watching tennis with a friend who is blind. The ball was small, fast, and blurry, so I started describing the point to her. I told her where the ball was going and when it crossed the net.

Then the crowd started clapping.

She asked me why.

I realized I had described the movement, but I had not helped her experience the point. I had left out the tension, the surprise, and the reason everyone around us was reacting. She did not just want to know where the ball was. She wanted to be part of the match.

With her guidance, I started building Feel the Ball, a system that lets someone feel the movement of the ball through a force-feedback joystick. I wanted it to communicate more than location. I wanted it to carry some of the speed and energy of the game.

While working with my friend, I saw the same gap appear outside of sports.

Reaching for something on a table could still require another person’s help. A tool might say that a cup was nearby, but that was not enough. It did not explain where her hand should move, whether something was in the way, or when she needed to change direction.

The problem was not that she did not know what she wanted to do. The problem was that the technology stopped at describing the scene instead of helping her act within it.

That led to my next project: a system that guides a person’s hand toward an object in real time. It checks the path, warns about possible collisions, and gives corrections during the movement. The goal is not to control the person’s hand. It is to provide enough information for them to complete the movement themselves.

Working on that project made me pay closer attention to the amount of effort an interface asks from its user.

That became especially clear when I began studying gaze keyboards used by people with ALS. To communicate a simple thought, a person may need to find each letter, hold their gaze on it, move to the next one, correct mistakes, and repeat the process until the sentence is finished.

The person already knows what they want to say. The difficult part is translating that thought into a long sequence of eye movements.

That is where my gaze communication project began.

Instead of treating the eyes like a mouse controlling a keyboard, the system starts with what the person is looking at in the real world. Looking at a cup, a phone, a door, or another person can carry meaning. The system uses the scene and the situation to identify what the user may want to express. When it is unsure, it presents a small number of choices so the user can clarify with another glance.

These projects are different, but each one started with a moment when the available technology was technically providing access while still leaving something important out.

For tennis, it left out the feeling of the game.

For reaching, it left out the information needed to act.

For gaze communication, it left too much work between a thought and its expression.

I am still learning what these systems get right and what they get wrong. That is why I am sharing them here. I want the people who might actually use these technologies to try them, challenge them, and help shape what they become.