The pet project that hasn’t died yet
Imagine that you have decided to start a small pet project. Or rather, it wasn’t even you who decided, but mother asked you to help with one routine task that arises in her browser game. You saw a lot of things that can be automated and optimized in a process that can take up to 6 hours of continuous and tedious work a day, and so the project was born.
The further fate of the project, of course, depends on many factors and most likely you will abandon it (like the rest of your 150 pet projects that no one will see), but even in the very birth of this project, many architectural and religious issues need to be resolved, and with your experience I decided to share their solutions with you so that you do not make such mistakes or point out to me those errors that I may not have noticed yet (even punctuation).
0. The essence of the problem
The essence of the problem: there is a browser game in which you need to build your city, attack monsters and other players, rob cows and develop. The game has a very developed system of clans and social interaction in general. Each time a player attacks another crypt (citadel and other ziggurats), in addition to receiving a reward, he brings a chest to all his clanmates. And other players in a special tab in the game can collect this chest and get resources. And this mechanic is so important that players are ready to spend 6 hours a day counting how many chests they brought, what norms and plans they fulfilled, and what quality the chests were. I won’t dive deep, but the players who count chests have a whole unofficial position in the clan (treasurers) and without automation tools the game turns into a full-fledged and very routine job.
1. A question of motivation
A lot has been said about motivation by smart people, but here I would like to analyze a few very obvious things, without which your project will die before it starts due to your laziness, lack of time, procrastination, unwashed dishes and “what will I live on.”
If you work alone, then most likely the project will die. History knows cases of successful success of loners who solo make games or operating systems, but we are not like that, we have 35 repositories in github with blanks of brilliant breakthrough projects and 350 ideas in our heads that were lost among dead neural connections and unspoken thoughts. The solution to this problem was my mother, who once every couple of weeks or once a month turned to me with a request to correct the pivot table or deal with the missing chests, which in one way or another helped me remember about the project and add something to it.
Try to make mvp as soon as possible and show it to someone. This will bring at least one client, and if he starts using your project on an ongoing basis, then it will probably be writing and motivating you to continue the project. It took me a couple of weeks to create the very first version of mvp (on March 3, I decided to make a project, on March 7, the beginning and first testing, on March 15, a site with registration, on March 20, mvp was ready). Here everything is like with letters to relatives, gifts for the new year – at least something is better than nothing.
Delegate some of the work. I was lucky that my wife got interested in design and helped me with the visual part of the site (I myself am a complete 0 in design, I can only teach it). Yes, this will not save the project from death, even about the lost money and time it is sometimes easier to forget than pulling a stillborn pet, but it will facilitate the work and speed up the creation of mvp. I also delegated the most boring and unpleasant part of the development to my intern, who, for a very modest and conditional monetary reward, as well as invaluable experience, put things in order in my creative mess (shit code) and included types in the typescript.
Comfort yourself with the illusion that just about there is just a little bit left and millions will fall on your head. Yes, over time this is disappointing, it comes to understanding that up to millions as before Blu-ray dial-up, but it allows you to make that important feature without which users cannot be allowed in, but with it you definitely can. So I have already reworked the report functionality several times and changed the framework for tables twice (these are not the most pleasant tasks, but millions cannot be seen without this)
Stickers, tasks, task trackers, a world board and explain to someone what you are planning to do and why it is important and cool. I lead, and not because it is necessary, but because otherwise it is not realistic at all.
2. A question of architecture
The very first architectural solution to the problem was proposed by a huckster from a neighboring clan. For 5,000 rubles, he sets up a “program for counting chests.” In fact, it was an autoclicker that takes a screenshot of the chest (player name and chest type) and saves the whole thing in a folder. The user of this “program” should simply open the game client, launch the autoclicker and wait until all the chests open. The opening speed depends on the power of the user’s computer, and it also blocks the use of the computer. Then these screenshots are translated into text using OCR (the “programmer” carefully threw off the link to the pirated version of ocr), the text is inserted into the table, the VBA script is divided into 2 columns, and a pivot table is formed in the adjacent table with formulas …
By the way, you need to open chests every day, and reports are needed once a week. The number of chests depends on the size of the clan and the activity of the players, in the clan my mother had about 5000 chests a week and because of this approach, about 100 chests were lost. For various reasons: either the window will move and the place of the screenshot will go astray, then the necessary screenshots will be mechanically deleted, something will be read wrong (the player’s name, for example), and the search and correction of such errors is dreary, then the player’s name in the pivot table does not match the name from screenshot…
But even such a crutch solution worked and saved a lot of time. 6 hours of manual labor turned into an hour of time daily and a couple of hours at the end of the week.
I was burned out by the fact that for such a “work” a person asked for real money. And after all, my mother is not the first who bought this solution from him! But I didn’t have many options. The study of the source code of the page with the game, the analysis of http requests and websockets did not give any results. I’m not so advanced a hacker to decrypt the still encrypted server responses, the game itself is drawn in the canvas tag, so it doesn’t even have html elements (I have experience in automating such games, if you’re interested – comment, I’ll tell you). You really need to run a real browser and take screenshots.
Question – where? It would be possible on the client’s machine, but in such a way that it does not interfere with using the computer (sufficiently), but you also need a stable Internet connection and a “boxed solution” that can be easily installed on the client’s computer. It is also possible on the server, this solves the problem of accessing the Internet and using a computer, but imposes restrictions on the server’s capacity and the security of transferring authorization data from the game to “some left site”.
For me, as a web developer, this question did not even arise. I immediately decided that this would be a web service where the user would simply press one button, turn off the browser and leave to drink tea while the game was running on my server, open chests and save everything to the database.
3. The question of programming language
I have always adhered to the idea “it doesn’t matter on what, the main thing is how”, and I believed that the choice of programming language is not so important. Especially when you need to do something quickly. It is better to choose a convenient tool that you already know how to work with than something cool, but which will take a long time to figure out. But the experience of developing this project has shown the opposite, and now I will tell you why.
Server.
Back in the now seemingly so distant and carefree 2018, when I was just learning the basics of programming and mastering the professions of a data scientist and data analyst, in the first real project, my team and I used the lightweight Flask web framework. I immediately fell in love with this framework, and the book by Miguel Grinberg and its translation on Habré “mega-tutorial Flask” became a “desktop” for me. It’s a cool tool and I know how to work with it, many successful projects have been launched on it, it’s great for prototyping. But problems arise when deploying the application and suddenly it turns out that using the built-in flask server is not recommended for production! For a long time, the server just hung for no apparent reason and error, and hung if I pressed ctrl + c once in the console and continued to work as if nothing had happened. Yes, there are all sorts of wsgi, and in general tutorials on how to translate, run, how to wrap it in a container, set up gunicorn … but I didn’t sign up for devops! In view of the fact that the server is currently running on my home computer (on windows 10, I also play on it sometimes), the server is started using waitress. And then, when I needed to use web sockets, it turned out that wsgi does not support them 🙂
Given that I actually only use flask as an api and don’t use jinja at all, I might as well write a fastApi server, and now I’m seriously thinking about rewriting it.
Conclusions: don’t use web frameworks that pass for prototypes if there is a chance that you will ever deploy it. Judging by the number of bugs on Instagram, I’m not the first to step on this rake (I remind you that Instagram is written in django – the “big brother” of flask)
Client
For the client side, a fresh vue3 was chosen. At the time of the start of the project, I already had a year of experience with this framework of the second version and it suited me with everything. Satisfied so far, the only problem is finding compatible libraries, because of this, for example, wave-ui was first chosen as a component library. From it I needed only a table, other libraries like vuetify were not ready for vue3 or had terrible tables. Then I realized that the minimalistic view of the tables does not suit me for the report and I transferred the report table to the tabulator.
4. The question of protection from bots (and other unsolvable problems)
Somewhere in a couple of months, work on the project got up. My selenium couldn’t login because the game’s developers have enabled a recaptcha that blocks suspicious login attempts. And I, as you know, are a very suspicious type, I go incognito in a headless browser, from one ip address, pull the mouse very sharply and without delay enter my login and password, clicking on the same pixel right in the center of the element with clear movements.
And in general, I think that in every project there is such a killer problem. A block that cannot be bypassed in any way. Searching the forums, stackoverflow, some of my ideas and research on the topic of bypassing the latest version of recaptcha did not yield any results. I got frustrated and began to think about how to transfer all this to the client side now, stop using a computer at the time of opening … in short, I make an autoclicker, screenshots and ocr, I return to the roots, I will take 5000 rubles each …
Subsequently, it turned out that I was not the first to come up with the idea of a boxed solution on the client side. Mom gave my contacts to another player-treasurer, and he said that he was launching a certain program that also calculates and opens everything itself, but on the client side. At the time of the discussion of my project, I dug it out, shook off the dust, launched it to show how it does not work and … everything worked. It turned out that protection against the bot works in 50-70% of cases, and it’s enough just to reload the page a couple of times to let me in.
I am aware that this is the thinnest part of my project. At any time, developers can close the hole completely and everything will go to dust. I have some ideas how it would be possible to radically solve the issue with the entrance, but I would not want to bring it to this.
5. The issue of monetization
How much money do you spend on games and entertainment? Everyone will answer for himself, but according to rumors, there are people who are ready to fork out large sums of money every month for donated clothes, loot boxes and other pixels. I decided for myself that at the mvp stage I would happily meet users, warn them that this is a beta test, and open their chests for free. And when I feel that my home computer can’t cope, and the project turns into a product, I will take a little money and not be impudent.
