MinIo for the little ones

MinIO is a great solution when you need to easily and easily organize object storage. Basic customization, multiple platforms and good performance have done their job in the field of people’s love. So we had no other way than to declare compatibility a month ago Veeam Backup & Replication and MinIO… Including such an important feature as Immutability. In fact, MinIO has a whole section For more information, see our integration documentation.

Therefore, today we will talk about how:

  • Setting up MinIO is very fast.
  • Setting up MinIO is a little less fast, but much better.
  • Use it as the Archive Tier for the Veeam SOBR scalable repository.


What are you?

A short introduction for those who have not encountered MinIO. It is an open source object storage compatible with the Amazon S3 API. It is released under the Apache v2 license and adheres to the philosophy of Spartan minimalism.

That is, it does not have a spreading GUI with dashboards, charts and numerous menus. MinIO simply starts its server with one command, where you can simply store data using the full power of the S3 API. But it should be noted that this simplicity can be deceiving when it comes to the resources used. RAM and CPU are absorbed perfectly well, but the reasons will be below. And, by the way, such combines as FreeNAS and TrueNAS use MinIO under the hood.

This introduction can be completed.

Setting up MinIO is very fast

Setting it up is so quick that we’ll cover it for Windows and Linux. There are options for docker, and for kubernetis, and even for makosi, but the meaning will be the same everywhere.

So, in the case of Windows, we go to the official website https://min.io/download#/windows and download the latest version. In the same place, we observe the launch instructions:

 minio.exe server F:Data

And there is also a link to a slightly more detailed Quick start guide… There is no point in not believing the instructions, so we run and get something like this.

That’s all! The storage is working and you can start working with it. I was not joking when I said that MinIO is minimalism and just works. If you follow the link suggested when starting up, then the maximum of the functions available there is to create a bucket. And you can start writing data.

For Linux lovers, everything remains no less simple. The simplest instruction:


wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
./minio server /data

The result will be indistinguishable from what was seen earlier.

Setting up MinIO is a little more meaningful

As we understand it, the previous paragraph is mischief for testing purposes. And, frankly, it is for testing MinIO that we use very widely, which we are not at all ashamed to admit. Of course, it works, but it’s a shame to endure such a thing beyond test benches. Therefore, we take a file in our hands and begin to bring it to mind.

HTTPS

The first mandatory step on the road to production is encryption. There are already a million and a thousand manuals on the network for adding certificates to MiniIO, but their general plan is as follows:

  • Create a certificate
  • In the case of Windows, put it in C: Users % User% . Minio certs
  • In case of Linux in $ {HOME} /. Minio / certs
  • We restart the server

Banal Let’s Encrypt is boring and described everywhere, so our path is the path of a samurai, so in the case of Windows, download Cygwin, and in the case of Linux, just check that we have openssl installed. And do some console magic:

  • Create keys: openssl ecparam -genkey -name prime256v1 | openssl ec -out private.key
  • Create a certificate by key: openssl req -new -x509 -days 3650 -key private.key -out public.crt
  • Copy private.key and public.crt to the above folder
  • Restart MinIO

If everything went as it should, then the following lines will appear in the status.

Turn on MinIO Erasure Coding

First, a few words about the subject. In a nutshell: it is software-based data protection against damage and loss. Like a raid, only much more reliable. While classic RAID6 can afford to lose two drives, MinIO is comfortable losing half. The technology is described in more detail in official guide… But if you take the very essence, then this is the implementation of Reed-Solomon codes: all information is stored in the form of data blocks, to which there are parity blocks. And it seems that all this has already been done many times, only there is an important “but”: we can explicitly indicate the ratio of parity blocks to data blocks for stored objects.
Want 1: 1? You are welcome!
Want 5: 2? No problem!

A very important function if you use several nodes at once and want to find your own balance between the maximum data security and the expended resources. Out of the box, MinIO uses the N / 2 formula (where N is the total number of disks), i.e. divides your data between N / 2 data disks and N / 2 parity disks. Translating into a human: you can lose half of the disks and recover data. This ratio is given through Storage class, allowing you to independently choose which is more important: reliability or capacity.

The guide gives an example: suppose you have an installation on 16 disks and you need to save a file of 100 MB. If the default settings are used (8 disks for data, 8 for parity blocks), then the file will eventually take up almost double the size, i.e. 200 MB. If the drive ratio is 10/6, then 160 MB is needed. 14/2 – 114 Mb.

Another important difference from raids: in case of dropping disks, MinIO will work at the object level, restoring one after another, without stopping the entire system. While a normal raid will be forced to restore the entire volume, which will take an unpredictable amount of time. In the memory of the author, the disk shelf, which, after the loss of two disks, was recalculated for a week and a half. It was very unpleasant.

This all sounds great, but how difficult is the setup going to be? Let’s get a look. We take the command to start and simply list the disks on which you need to create storage. If everything is done correctly, then in the report we will see the number of used disks. And advice that it is useless to add half of the disks to one host at once, because this is a loss of data.

c:minio>minio.exe server F: G: H: I: J: K:

Further, to manage and configure the MinIO server, we need an agent, which you can download everything in the same place from the official site.

In order not to erase your fingers, each time typing the address and access keys (and it’s unsafe), it is convenient at the first start to immediately create an alias using the formula mc alias set [YOUR-ACCESS-KEY] [YOUR-SECRET-KEY]

mc alias set veeamS3 https://172.17.32.52:9000 YOURS3ACCESSKEY YOURSECERTKE

Or you can add your host right away:

mc config host add minio-veeam https://minio.jorgedelacruz.es YOURS3ACCESSKEY YOURSECERTKEY

And then we will create an immutable bucket with a beautiful team

mc mb --debug -l veeamS3/immutable 

mc: <DEBUG> PUT /immutable/ HTTP/1.1
Host: 172.17.32.52:9000
User-Agent: MinIO (windows; amd64) minio-go/v7.0.5 mc/2020-08-08T02:33:58Z
Content-Length: 0
Authorization: AWS4-HMAC-SHA256 Credential=minioadmin/20200819/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-bucket-object-lock-enabled;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Bucket-Object-Lock-Enabled: true
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20200819T092241Z
Accept-Encoding: gzip
mc: <DEBUG> HTTP/1.1 200 OK
Content-Length: 0
Accept-Ranges: bytes
Content-Security-Policy: block-all-mixed-content
Date: Wed, 19 Aug 2020 09:22:42 GMT
Location: /immutable
Server: MinIO/RELEASE.2020-08-16T18-39-38Z
Vary: Origin
X-Amz-Request-Id: 162CA0F9A3A3AEA0
X-Xss-Protection: 1; mode=block
mc: <DEBUG> Response Time:  253.0017ms

–debug allows you to see not just the final message, but more detailed information.

-l means –with-lock, which means immutable

If we now return to the web interface, our new bucket will appear there.

That’s all for now. We have created a secure storage and are ready to move on to integration with Veeam.

You can also make sure that everything works perfectly:

c:minio>mc admin info veeamS3

●  172.17.32.52:9000
   Uptime: 32 minutes
   Version: 2020-08-16T18:39:38Z
   Network: 1/1 OK
   Drives: 6/6 OK
0 B Used, 1 Bucket, 0 Objects
6 drives online, 0 drives offline

MinIO and Veeam

Attention! If for some incredible reason you want to work via HTTP, then at HKEY_LOCAL_MACHINE SOFTWARE Veeam Veeam Backup and Replication create a DWORD key SOBRArchiveS3DisableTLS. Set its value to 1 and remember that we strongly disapprove of such behavior and do not recommend it to anyone.

Attention again! If, due to some misunderstanding, you continue to use Windows 2008 R2, then when you try to connect MinIO to Veeam, you will most likely receive an error similar to the following: Failed to establish connection to Amazon S3 endpoint. It is treated with an official patch from Microsoft

Well, the preparations are over, let’s open the VBR interface and go to the Backup Infrastructure tab, where we will call the wizard for adding a new repository.

Of course, we are interested in Object storage, namely S3 Compatible. In the opened wizard, set the name, go through the steps with the address and account. If required, do not forget to specify the gate through which requests to the storage will be proxied.

Then select the bucket, folder and tick the Make recent backups immutable checkbox. Or we don’t. But since we have made a repository with support for this function, it will be a sin not to use it.

Next> Finish and enjoy the result.

Now you need to add it to the SOBR repository as a Capacity Tier. To do this, either create a new one, or edit the existing one. We are interested in the Capacity Tier step.

Here we need to choose which scenario we will work with. All options are described quite well in another article, so I will not repeat myself.

And upon completion of the wizard, tasks for copying or transferring backups will be automatically launched. But if it is not in your plans to immediately issue the load on all systems, then be sure to set the permissible intervals for working by clicking the Windows button.

And of course, you can make separate Backup Copy jobs. Some people think that this is even more convenient, since they are somewhat more transparent and predictable for the user who does not want to delve into the details of the work of the capital shooting gallery. And there are enough details, so once again I recommend the corresponding article from the link above.

And finally, the answer to the insidious question: what will happen if you still take and try to remove the backup from the Immutable store?

Here’s the answer:

That’s all for today. By true tradition, catch a list of useful topics on the topic:
– Manual Using MinIO with Veeam

– Example for use MinIO with Veeam Backup for Office 365
– General setup manual S3 Storages in Veeam
– Branch on our forum about S3 storage.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *