Learn with your eyes. How an eye tracker helps the immobilized
Hi, I’m Ivan Bakaidov.
You know that for many years I have been making software that helps people with severe disabilities communicate with the world. I want to tell you a little about what I have been doing for the last year, and I also need help from browser game developers.
Over the years, I have come to use Tobii gaze trackers to communicate with severe disabilities.
Last year I started working with the Close Others Foundation. We had an interesting experience when several dozen families learned how to work with an eye tracker using my open-source programs.
This year, we decided to scale this experience and for this to make a free methodological course that any family can use on their own.
Yura Dud tries LINKa look, for the development of which he allocated money.
Communication is the basis of human life,
Many, due to various circumstances, are deprived of the possibility of familiar communications.
I, Ivan Bakaidov, apply modern technologies and develop interfaces to help people remove these restrictions:
For people with speech problems – make a user-friendly interface with a voice synthesizer
People with limited motor abilities – write text quickly.
People with severe motor impairments – use an alternative input, using an eye tracker, forming a statement with the eyes.
I know firsthand about the problems of non-speaking people, because I have cerebral palsy and I speak very incomprehensibly.
Many times I found myself in situations in which the inability to respond to people put me in the position of a “third person”, a person not with whom, but about whom they talk.
This is a terrible situation.
That all changed when I developed LINKa, a program that reads written text. I stopped holding on to people who understand my lowing, and was able to travel, work, meet new people.
This is an amazing path to life and I want to give it to everyone who has speech problems.
Therefore, my task is to invent interfaces that will help tell everyone.
During the year of the project’s work, the fund bought equipment for more than 20 families. In the course of the “Learn with Eyes” project, we established the need not only for the purchase of equipment, but also for the work of a defectologist with families. A defectologist helps to establish and develop communication skills with the help of an eye tracker in families with children with developmental disabilities.
The set of equipment includes a laptop and an eye tracker, a device that tells the computer the point of view on the screen. Read more about the tracker here.
In the course of work, the idea of the LINKa program has proven itself perfectly. The simplest system, which allows a parent or specialist to create a “board” of cards, turned out to be very flexible and made it possible to construct scenarios from the simplest (a choice of two cards) to playing a memo.
About the course.
Having received an extremely interesting and positive experience, we proceeded to institutionalize it in the form of a course. The course will consist of online lectures and video master classes for parents and methodologists, as well as specialized software for conducting classes with the eye tracker: games – to develop gaze control skills, and a collection of sets of cards – to communicate with the world. Including for communication in primary school.
We have begun work on specialized games designed for various aspects of developing the skill of controlling a computer with the help of a glance:
– awareness of the process of interaction with the screen (demonstration of the direction of the gaze, keeping the gaze on the object to select it);
– training of gaze control (games for speed, vigilance and attention);
– pleasure (arkanoid, wolf and eggs).
The main purpose of the post is to advise on the choice of technologies.
The first version of the games was written in Unity, but we came to the conclusion that development on this stack is too expensive.
I wrote a shell for running electron games, with support for the tobii eye tracker driver, as well as some simple games using paper.js to work with Canvas.
Example game Wolf and eggs…
We also wrote technical tasks for games…
I have an important technical question for the community:
What is better to write games on paper.js, phaser.io, construct.net or something else?
How you can help the project
We can be helped by advising technology, writing a game, or drawing graphics for the game.
Or by making a donation