Making MinIO and Postgresql Work Together

Why and how to store objects using MinIO as an example”

As it was before

I will show code examples in the most simplified form possible so that no difficulties arise.
The project has a functionality for uploading images for restaurants/dishes/category of dishes via REST API. We upload images in the most primitive way.

func createRestaurant(ctx *gin.Context) {
    // чтение запроса и валидация его для сохранения ресторана в приложении 
    // ...
  
    // Забираем изображения ресторана из тела запроса	
    form, err := ctx.MultipartForm()
	files := form.File["image"]
	if len(files) == 0 {
		ctx.JSON(http.StatusBadRequest, gin.H{
			"error": "no photos",
		})
		return
	}

    // ....
}

After all the preparations, we went to the DB and within the transaction we saved a record about the restaurant in the first table, and then made a record or records about the images associated with our restaurant. We have a connection one-to-many, One restaurant can have multiple profile images.

Everything seems to be fine, the scheme works like a Swiss watch, but we programmers always want to redo something and that is why, when I found about such functionality of MinIO as integration, I immediately went to implement it on my project.

As it is now

Now after uploading images to S3 storage, you no longer need to touch the database yourself, everything will be done for us
To achieve this, you need to set up our MinIO server, a guide to deploying and working with the MinIO server is all in the same article that I indicated above. For the test environment, I will attach docker-compose.yaml, where we have Postgresql and MinIO

version: '3.8'
services:
  minio:
    container_name: minio
    image: minio/minio
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    command: server /data --console-address ":9001"
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio_data:/data

  postgres:
    container_name: postgres
    image: postgres:alpine
    environment:
      POSTGRES_DB: postgres
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: 5432
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  minio_data:
  postgres_data

Postgesql

In our database, we will create a table that will store a record of all images.

CREATE TABLE IF NOT EXISTS images (
    key TEXT,
    value JSONB
);

Minio

To send any events, you need a bucket in which images will be stored. For this, we will make these commands.

# создания алиаса для нашего сервера, чтобы потом его переиспользовать
mc alias set myminio http://minio:9000 minio minio123
mc mb myminio/menus # создания бакета menus 

Setting up a connection to the database

mc admin config set myminio notify_postgres:minio-postgres connection_string="user=postgres password=5432 host=postgres dbname=postgres port=5432 sslmode=disable" table="images" format="namespace"

After this we will be asked to restart the service

mc admin service restart myminio

And finally, set MinIO to make a record in the database after each image is loaded into the menus bucket.

mc event add myminio/menus arn:minio:sqs::minio-postgres:postgresql --event put

Testing

Of course, you need to test the application, I made a test request to create a restaurant using Swagger

Now in the DB you can see such a large json with data about the object

Now for our application, to track the image belonging to a certain restaurant or dish, we will put the metadata when loading the image

func makeMetadata(opts *SaveObjectOptions) map[string]string {
	return map[string]string{
		"restaurant_id": opts.RestaurantID,
		"category_id":   opts.CategoryID,
		"dish_id":       opts.DishID,
	}
}

func (c *Client) SaveImage(ctx context.Context, opts *SaveObjectOptions) error {
	info, err := c.minioClient.FPutObject(ctx, menusBucket, opts.FileName, opts.FilePath, minio.PutObjectOptions{
		ContentType:  "image/png",
		UserMetadata: makeMetadata(opts),
	})
	if err != nil {
		c.log.Error("failed to upload image", zap.Error(err))
		return err
	}

	c.log.Info("successfully uploaded of size", zap.String("filename", opts.FileName), zap.Int64("size", info.Size))

	return nil
}

Conclusion

I hope this article will help someone to achieve their goals.