LINKa look. The system of choosing cards with the help of an IT tracker and not only

Remember who I am?

Hi, Habr. I AM Ivan Bakaidov. Many years ago I wrote about my school project of a program that allows people with severe physical disabilities to print with one button. Here is that article.

Since then, a lot has changed the girl diedFor which I wrote this program, the project became known as LINKa. I have done many programs for other forms of motor impairment. I realized that I lost the wave of volunteers that arose during the past habrohaip, since Open source projects need to be maintained.

But I did not leave the topic of developing programs for communication. And for money from “Grant Dude” developed a new communicator that allows you to select cards with a look. And I need help adding some features. Stack: C #, WPF.

A look? o_O! Like this? Does your head twitch?

Actually, for a long time in the comments of the habr they wrote to me about the technology of aitracking, and I skipped these comments, because I thought it was expensive and could not work with a shaking head. I was wrong!

Since 2016, Tobii (aka. Apple from the world of IT tracking) began developing low-cost devices for the gamer market. These devices are 10 times cheaper than devices for “disabled people” (the law of the sticker is Special device). For a couple of years they reached a beautiful model Tobii 4c, which you can buy in a regular electronics hypermarket and plug into USB 2.0.

One of the funds threw me this device with the words “Try it, man.” I put it on the shelf with the words “Aitracking and cerebral palsy – it won’t work, then I’ll watch it somehow.” It only works with Windows, and I have a Mac. But foundations are such great organizations that require reports.

Bootcamp installed, tuned, and realized that the IT trackers of this generation learned to track the position of the head, and from it the position of the eyes. At a very high frequency. And everything works, shake your head as much as you like.

The main software for this tracker is gaming, in Dock 2 enemies to look. All built-in demos show gaming features. But there is an open API for working in third-party programs.

Open space for creating eye-driven interfaces. Such interfaces are designed very simply: they consist of a grid of buttons. When you look at the button, a click occurs. And here many have already distinguished themselves. Windows 10 itself has a built-in keyboard and mouse emulator for an IT tracker (True, there are no letters “X” and “B” in the localization. Do not even ask for Bread!).

No way!

There is quite sensible BB2K keyboard, the developer wrote about it on a geek magazine I personally really like OptikeyThis is a well-localized keyboard from a British developer. It has several options, including with pictograms for non-writing children.

Well, if you already have optics, why write your own?

Natural question. As you remember from previous articles, it is very important for specialists who work with people from speech to be able to create content themselves chosen by the ward, and not to work with a finished brick, in which even very good cards are sewn. This is due to the fact that often people without speech have yet to learn how to write, or simply understand that the picture of the mug denotes the very cup in the kitchen from which they drink it. And in the same Optica you need to create a set of cards via PowerPoint.

That is why Link is based on the idea that I learned well when I studied at the school for children with disabilities – “Everything must be customizable”. Based on the idea, a very simple task arose: Make a grid of cards from which cards can be selected with your eyes, but at the same time, the choice method, cards and the grid should be easily customizable. I designed all this in TK and found a C # programmer who understood in WPF (better than me). At this time, even Jura Dud transferred money and in general everything turned out.

What is the result?

It turned out a program that shows a set of cards from a .linka file in the grid and allows you to select them using: Tobii IT tracker, mouse, keyboard and game joystick. There is also a mode in which a card is selected with a glance, and confirmation with a button.

The program contains a .linka file editor.

→ You can download from here for free: linka.su/looks

So, what do you need to finish?

Despite the simplicity and conciseness of the program, I hope its flexibility will help to establish communication in a variety of cases.

I have a few small ideas that I would like to implement the program and I would be very happy if you could help with this. These tasks are described in this issue, but I will bring some of them here and explain what I mean.

• Modify the speech synthesizer, make a choice of Yandex votes and add a fallback so that in the absence of an Internet connection.

  To voice the sets in the link, the Yandex TTS API is used. Link look designed so that the voice of the cards is set during the creation of the set. By creating a card, you can either use Yandex voice acting or select a file from a computer. That is, pre-voiced dialing can work offline. But the watch link supports sets consisting not of whole words, but of letters, for example, and in this case the program reads text from the output field using Yandex in real time. And I would like for the program to use a shitty Windows synthesizer when there is no Internet. It would also be good to add a choice of Yandex votes.

• Make a one-click card selection system.

  Despite the magic of the IT tracker, I met comrades with whom he did not work. And I would like to add to transfer the card selection algorithm for them using one button from a good old link, click (and stop supporting the latter).

• Make it possible to print in third-party programs.
• Localization in the languages ​​of the CIS countries.

I will also be glad if you just test the program and write your ideas.

→ Github

→ Patreon

Thank!