Helping Many: An Android App for People with Special Needs
Greetings to all! I am Begletsov Gleb, I am in 11th grade, I finished the program last summer “Samsung IT School” in St. Petersburg at the FML 239 site under the leadership of Mikhail Konstantinovich Levin. For my graduation work, I developed an application that I called “Parus”. This is my first big project for Android, and it brought me the GRAND PRIX of the finals of the All-Russian IT School Samsung competition (roller). I want to share the history of the creation of this project.
I sometimes heard a similar phrase among my classmates at Samsung IT School: “I don’t know what application to write to me”. I had no doubts. My application should help people with special needs fully communicate with the world, because this problem is relevant both in our country and in the world. And she personally concerns me. There are many standalone applications: speech synthesis, artificial vision, reminders, etc. I wanted to combine everything I needed in one application.
I started studying at the Samsung IT School in September, and at the beginning of November I already knew what project I would develop. But this is a classic, put it off until the last moment! As a result, April and May were very tense. At the same time, it was a very exciting time. I would wake up, code intermittently for meals, go to bed, wake up and code again if I was not in school. This was the daily routine during the final two months of training. I learned something new every day. Several times it happened: I understood that what I had written was not good and rewrote half of the work already done. Then, as you know, it was not very fun.
By early April, I had mastered the basics of Java and Android. And over the next two months, he added speech synthesis, speech recognition, computer vision, heart rate tracking, creating daily reminders (not todo list), linking accounts of a person with special needs and his assistant, chat and data transfer between them. Further details about each function.
Speak. Speech synthesis
This feature helps people with speech impairments communicate. Speech synthesis itself is the easiest task of all. I connected TextToSpeech and you’re done. But this functionality is not very user-friendly, so I decided to make a system of frequent phrases and thematic collections. The first problems have already begun with this.
While creating a system of collections and frequent phrases, I did not know about the existence of Snapshots (real-time data update), so I implemented data updating, as I later realized, through very shaky crutches. I read the data from the database every time, when there, hypothetically, some fields could change. And such a system was implemented EVERYWHERE… This was one of those moments when, having learned about something new and useful, I was delighted, and then began to rewrite the entire code.
Computer vision should help people with low vision to recognize text or an object in a photo. But not everything is as perfect as we would like. Firebase Ml Vision is convenient and practical, but the free version does not recognize Russian, and the results of object recognition sometimes look at least strange.
In general, Firebase, a great tool, helped me a lot when implementing many tasks. For computer vision, I first tried using Tesseract, because it was possible to connect a file with the Russian alphabet to it to recognize the Cyrillic alphabet. It turned out in vain. Having spent about a week only on its installation in the project, the text with Russian letters was still not recognized. So I started looking for other options. And then Firebase Ml Kit saved me, which I set up 5 times faster! Firebase gives a lot of goodies, and without it I would hardly have done everything. Out of inexperience, of course, not everything worked out right away, but I figured it out. For the current task, I’m using the API from Firebase for now. Maybe in the future I will find something more suitable.
Just speech recognition using Google’s SpeechRecognizer API and displaying the result on the screen.
Pulse. Google Fit vs Samsung Health SDK
Heart rate tracking is a useful feature for people with heart disease. With the help of it, the user can monitor his heartbeat. During data processing, if the heart rate is outside the normal interval (55
But this task also had to pant. First, I figured out how to read data from Google Fit. Then I read about Samsung Health SDK and decided to switch to it. The S Health SDK has a very handy add-on – Data Viewer. With it, you can record data in S Health without any trackers. This helps a lot when testing the application, in which Samsung’s SDK beats GoogleFit. Also, you do not need to link to your account and connect QAuth from Google, also a plus.
But if anyone can integrate Google Fit into their project, then in order to use Samsung Health, the application must receive the status of a partner. Without this status, the application will work fully only in developer mode. Unfortunately, at the moment the acceptance of applications for the status of an affiliate application has been suspended due to the renewal of the affiliate program. So I had to go back to GoogleFit.
I know that there are many todo list applications, but as they say, this is different … First, the user can set several hours (12:50, 13:10 and 18:30, for example) for one task. Secondly, it is possible to set the reminder time interval and repetition: from 12:00 to 20:00 every hour, for example. Reminders reboot every day and start all over again, like a regular alarm clock. This can be useful for people who need to do something important every day. For example, people with diabetes can set a time for insulin shots.
It goes without saying that a person can set many alarms that will perform the same task. My program just simplifies this. For reminders from 9:00 to 21:00 with a regularity of half an hour, you must set either 24 regular alarms, or 1 reminder with an interval in Parus. I think the second option is easier.
Reminders do not waste much battery power, because they are implemented at the expense of WorkManagers. I thought for a long time between Alarm and WorkManager, because the first loads the phone and the battery too much (+ is limited on some phones), and the second has one significant drawback – due to over-optimization, the reminder may not be displayed if the phone was in sleep mode for a long time. I tested this case about 100 times, and 3-5 times the reminder did not work until I exited sleep mode. I read on the Android developers site that Google is working on this, so I hope to get it fixed soon.
Communication with the assistant
This is one of the main functions of my application. A person with disabilities does not always manage to cope with all life situations on their own, so sometimes there is a person who plays the role of an assistant. There is a lot of communication with him, and therefore I think it is very important to simplify this interaction.
The assistant will be calmer if he has information about the heart rate of his ward and his location. In addition, the assistant can edit the disabled person’s reminders in the app. You can also always see when a linked user last logged into Parus.
When linking accounts, a person with special needs will have a quick button “Call an assistant”. When you click on it, a notification is sent to the assistant on the device.
Sometimes it happens that your assistant is not a family member or close person, but a volunteer or social worker. In order not to communicate with him in various messengers, you can communicate with him separately in Parus’ chat.
Samsung Android Bootcamp. Conclusion
Above, I described the main features of my application at the time of participation in the competition of graduation projects “Samsung IT School”. I didn’t mention the application architecture or the patterns that I used at all. Do you know why? Because this was not the case at the time of the final of the competition. The project consisted of classes organized into packages. And that’s all. I suspected it was terrible, but did not know how to fix it. And then we were told that immediately after the competition there will be a summer Android Bootcamp for the graduates of “Samsung IT School”. I decided that I was clearly there.
During two weeks of intensive I learned a lot of useful things. For example, I got hooked on MVVM + Data Binding + Material Design. It became clear that Parus could not be produced in this form. Work to be done: architecture first, then design, after which you can go to Google Play. In August, there was no longer such a fuse as in late spring, by mid-September I almost redid everything for MVVM architecture … and that’s it …
From October to January, I never opened Android Studio once. I didn’t want to program or complete my project at all. In short, all enthusiasm was gone. I began to think that in fact the project was about nothing, that I was not able to make a decent application. I don’t know why, I’m probably tired and realized that you can endlessly bring the project to an ideal state.
The New Year has passed. Parus is still not on Google Play. I opened Android Studio and suddenly felt I had the strength to complete the project. I made a promise to myself that in February Parus will appear in the Play Market. In two weeks I completed everything I had planned, and on February 4, I uploaded the project to Google play…
This is the story of my first big project. Yes, Parus is not perfect, but I managed to bring it to a finished state. Hope it is useful to someone. I would be glad if you download the application and write your suggestions. Especially if you are my target audience – a person with special needs or care about such a person.
Thank you for reading the article to the end.
Grade 11 student, school “Ozerki”