REST API based on Snake (Python, Mamba, Hydra and Fast API)

Today I want to try something new and start exploring the world of Python. This article provides a step-by-step tutorial on how to implement a simple REST API using Python, Fast API, Hydra and Mamba. Moreover, I will briefly describe how to pack all these snakes into one Docker image and make them work together. All code is posted on my

GitHub

Let’s start with a multiple explanation of why I chose this topic.

▍ Why is it interesting?

First, I wanted to share my knowledge as I recently had the opportunity to implement a REST API based on Python. Choosing a framework turned out to be simple –

FAST API

. We also needed a dependency management tool, so we chose Mamba. For

configuration management

and loading we decided to choose Hydra. We felt that all of these tools worked well and provided all the necessary features, so this combination seemed quite acceptable. Unfortunately, when we moved on to integrating the tools and trying to get them to work together, it wasn’t that easy. Moreover, it turned out that there are quite a few information resources and examples on this topic. That is why I decided to write this article.

▍ What is Mamba?

Mamba is a deadly genus of snakes. But seriously, Mamba is a tool for managing dependencies in a project and for

creating virtual Python environments

. It was created based on

Anaconda

, but it was supposed to be much faster, and from my brief experience with Mamba, I can say that it is really fast. Due to the fact that it is based on Conda, we have access to all ready-made packages from the Conda repositories. Moreover, the Mamba API is generally very similar to Conda, making it easy for Conda users to migrate to Mamba.

▍ What is Fast API?

It’s a tool that is probably known to almost everyone in the Python community: an asynchronous, fast, and lightweight tool for building REST APIs. Probably today it is a tool that can be recommended to anyone who wants to start learning Python and REST. It contains all the functions to create a working API, along with

web sockets

and streaming support. Moreover, the FAST API uses Python type annotations, so code completion in the IDE works quite well (at least in PyCharm). In my opinion, another rather useful feature is the built-in swagger support. In fact, I was surprised by her presence.

▍ What is Hydra?

Hydra is a multi-headed monster from ancient Greek mythology. Hydra is an open source tool for managing and configuring Python-based applications. It is based on the Omega-Conf library. Quote from the main page of the tool: “A key feature is the ability to dynamically create a hierarchical configuration by composing and redefining it using configuration files and the command line.” For me, the hierarchical configuration described in the quote turned out to be very useful. In my case, it worked quite well and provided a cleaner separation of configuration files.

▍ Realization

1. Let’s start the project by creating

environment.yaml

with our environment configuration.

name: greeter-service
channels:
  - conda-forge
dependencies:
  - python=3.8.13
  - pip
  - fastapi
  - uvicorn[standard]
  - hydra-core
  - pytest

The file contains the name of our new virtual environment (greeter-service), as well as the source from which to download dependencies (conda-forge), and a complete list of dependencies required for the application to work properly. Thanks to Mamba, I can set up my entire environment in minutes with one simple command:

mamba env create -n greeter-service --file environment.yaml

When installing Mamba, I recommend using

tutorial

written by the Mamba authors themselves.

2. In this step I will define the file config.yaml with all the configuration required by the application.

app:
  port: ${oc.env:GREETER_API_PORT,8070}
  version: ${oc.env:GREETER_API_VERSION,v1}
  greet_message: "Hello, "

Nothing special here, pretty simple file

.yaml

with a little magic hooked up by reading environment variables. This is the entire configuration that I will use in my tutorial. The structure is pretty standard:

port where our API will run

API version to be used in endpoints

The only non-standard element is the parameter

greet_message

A that contains the body of the message that will be returned to the user.

3. I add a file config.pyThe one responsible for reading the Hydra configuration.

import os

import hydra
from hydra import compose

hydra.initialize_config_dir(config_dir=os.getenv('GREETER_CONFIG_DIR'))

api_config = compose(config_name="config")

First, I initialize the Hydra context based on the folder

config

along the way

./

. Hydra will use environment variables or take the root folder of the project. I then use Hydra’s compositing method to read in the configuration defined in the previous step.

4. Next, I implement the first API endpoint. I will set it in a file health_check.pysince it will be responsible for handling status check requests.

from fastapi import APIRouter

health_router = APIRouter(prefix='/health', tags=['health_checks'])


@health_router.get('', status_code=200)
def is_ok():
    return 'Ok'

The code is simple and clear. It’s just a FAST API router with one method that returns when called

Ok

and HTTP code 200.

5. At this stage, I create a file greeter.pyThe responsible for processing incoming requests.

from fastapi import APIRouter

from api.config import api_config

greeting_router = APIRouter(tags=['greet'])


@greeting_router.get('/greet/{name}', status_code=200)
def say_hello(name: str):
    return api_config.app.greet_message + name

Another simple basic FAST API endpoint that takes a username as input. It then concatenates the passed name with the message format read from the configuration and returns the completed welcome message to the user.

6. Now I implement the file main.pywhich communicates with the routers from the previous steps.

import uvicorn
from fastapi import FastAPI, APIRouter

from api.config import api_config
from api.greeter_api import greeting_router
from api.health_check import health_router

main_api = FastAPI()

main_router = APIRouter(prefix=f'/{api_config.app.version}')
main_router.include_router(health_router)
main_router.include_router(greeting_router)

main_api.include_router(main_router)


def start():
    uvicorn.run(main_api, host="0.0.0.0", port=api_config.app.port)

It’s just plain FAST API code. What’s notable here is that I’m adding the version as a base prefix to all endpoints. The most important method here is the method

start

in which I manually start the server

uvicorn

(this is not a typo, but the real name of the server) on the port read from the configuration.

My simple service is ready for testing, but fear not, this is not the end of our tutorial. Now I will talk about how to make it work as a Docker image.

7. I will start this part with the file definition setup.py.

from setuptools import setup

setup(
    name="greeter-service",
    version='1.0',
    packages=['api'],
    entry_points={
        'console_scripts': [
            'greeter-service = api.main:start',
        ]
    }
)

The most important parameter in this script is

entry_points

; essentially, it defines which Python method is responsible for the application. In this case, this is the method

start

from

main.py

. It also specifies the name of a Python service that can be used to execute the application from the command line.

8. Now is the time to prepare Dockerfile.

FROM condaforge/mambaforge

WORKDIR /greeter-service

COPY environment.yaml environment.yaml

RUN mamba env create -n greeter-service --file environment.yaml

COPY api api
COPY config config
COPY setup.py setup.py

ENV PATH /opt/conda/envs/greeter-service/bin:$PATH

RUN /bin/bash -c "source activate greeter-service" && python setup.py install

What’s going on here?

For starters, I’m using the official Mamba image because I don’t want to waste time installing Mamba in Docker from scratch.

Then I set greeter-service as my working folder and add CONFID_DIR as a new environment variable.

This variable will be used by Hydra as the path to the application’s configuration files. In the next step, I copied the environment file and used it to create the Mamba virtual environment in Docker.

The next few lines are regular copies of the application code and configuration folders.

The last two lines are a kind of hack for Conda, which doesn’t work very well with Docker and to some extent with the shell itself.

Without this hack, we would see exception messages like this:

Your shell has not been properly configured to use 'conda activate and you will not be able to execute conda activate greeter-service

. Unfortunately, no other fix seems to work here, at least in my case. Some related links can be found

here

and

here

9. The icing on the cake is the Docker compose file, which greatly simplifies customizing the Docker image.

version: "3.9"

services:
  greeter-api:
    build: .
    command: greeter-service
    ports:
      - "8070:8070"
    environment:
      - GREETER_CONFIG_DIR=/greeter-service/config
      - GREETER_API_PORT=8070

So far, this is just one Docker service with two environment variables used – the path to the configuration folder and the port. Compose will build the Docker image based on

Dockerfile

in a local folder. Next, he uses

greeter-service

as a container start command. It will also open and bind local port 8070 to port 8070 of the container.

Voila, the implementation is ready. It’s time to do some testing.

▍ Testing

As in my other articles, to run tests on a working API, I use

Postman

. Let’s start testing by setting up a Docker image. To set up the entire testing environment, a simple command is enough

docker compose build && docker compose up

at least if everything works as intended.

Docker is up and running so I can test the API. One or two simple queries and I’ll make sure everything works as it should. Let’s start from the end point

greet

And now the end point

health

Some logs from the Docker container to prove that we didn’t just draw these screenshots:

We have completed coding and testing, it’s time to briefly sum up.

▍ Conclusion

The integration was implemented, tested and described using not the most well-known tools that I decided to use. The example is quite simple, but contains everything you might need to run anywhere. Moreover, it can be easily extended and used as a foundation for more complex projects. I hope the article was useful for you.