We write our own mini-DBMS in Golang (Part 1)
Requirements
Let’s start by defining the requirements that we want to put forward to our future DBMS (further for posturing convenience we will call it repaDB). There are many stages here, but in short these are just a few questions:
Why are we working on this?
What do we do?
How will we do it?
What are our deadlines?
Let’s go in order. Why are we working on this? Of course for three hundred bucks invaluable experience or school/student projectbut naturally, for big uncles, it will be “to solve some business problem and, if possible, replenish your wallet with a couple of three million.”
In general, at this stage, it is usually determined whether it is necessary to deal with the project at all and spend resources on it, and also analyze whether existing alternatives can be dispensed with. So, below is the answer to the question that sounded in letters:
After analyzing the market and realizing that, oh my God, such projects
so manydoes not yet exist on the market, we need to create our own open database that stores information on a key-value basis, which will urgently fill the existing niche and receive 100,500 stars on GitHub.
And now to the next question – What are we going to do? Here you need to understand what exactly we (or our big boss) want to see as a result of the project. Our goal is to store data, in memory, on a key-value basis with a certain TTL (ak Time to Live, ak time to live). Any goal can be achieved in different ways and our case is no exception. For example, we can use a ready-made solution, or ask someone to write for a pack of doshik or *here is a synonym for the word to steal *. But we are intelligent people, so we will write our own solution. So the answer to the second question would be:
Creating your own database is the best and most optimal solution, because. this will allow us to strengthen the community around our products and have a flexible and lightweight tool,
and also potentially, someday, get an offer to a cool company
Having decided on what and why, let’s move on to the fundamental question – how to do it at all? The question is truly unique and contains many subsections, but we will do everything in an easy way and will not go deep into the jungle of design. Let’s go straight to an example:
Our DBMS should be able to: allow you to connect via TCP, store data in memory, read it safely, update it, delete data and check the existence of some keys, and also have your own library for ease of development (because without it it will be difficult to interact with our database, if at all possible). All of the above is business requirements.
It follows from business requirements that users should have all of the above capabilities, which is custom requirement (i.e. requirements describing user actions).
It turns out that our IT system should have all of the above functionality, and this is functional requirement, i.e. requirement describing the behavior of the system.
Also, we will store all the data in JSON format on disk as a file, and also communicate with the database via TCP using a client, or a library for Go – that’s all. not functional requirements.
Well, as a result, we do not need to store complex structures such as arrays, maps, etc., and therefore only data types can be used as a key value string, bool And int. This restrictions our system.
Of course, there is the last question – what are the deadlines? But we all understand that it will be necessary to do everything before “yesterday” and refactor sometime later. So we will not stop here for a long time and move on to the next section.
Project Structure
Having finished with the definition of requirements for our future must-have solution, and having lost somewhere along the way the stage with the creation of technical specifications, we move on to the structure of our future project. This will be a multiple repository, i.e. all parts of the project will be in different folders – modules (did you think? we have everything here for adults!). Each of the modules will be based on the standard principles for building the structure of a goroutine application described right here.
Below is a list with a description of each module:
repa-db – here is the main code of our database
repa-cli – and here is the cli-client code for our repaDB
repa-client – well, and here, respectively, the library code for Golang
Description of the structure of modules separately, we will consider in the relevant articles devoted to a particular module.
Conclusion
That’s all for now. This is my first article from the DBMS creation cycle in which I briefly reviewed the creation of requirements for the project and also briefly defined its structure. If you have comments or want to share ideas? I will be glad to read! More details and code examples will be available later.
