Launching ComfyUI on AMD Radeon GPU in Docker

In this article, I want to share my containerization experience using AMD's Radeon graphics card. As an example, we'll take ComfyUI and put it in a container. And at the end, to demonstrate successful operation, we'll create an anime-style video.

Everything below was tested on an AMD Radeon RX 7900 XTX.

ROCm by AMD

In October 2023, AMD took up support PyTorch for their own video cards. AMD's ROCm platform is an analogue of Nvidia's CUDA, which provides acceleration of massive parallel computing. The advantage of AMD is that the ROCm code is open and their video cards are usually cheaper than Nvidia's. But one of AMD's significant disadvantages is that the community of programmers using Radeon video cards is smaller than GeForce, which leads to difficulties in finding solutions to emerging problems.

To be able to connect a video card to a container, ROCm is required. Therefore, the set of programs for Ubuntu will consist of git, make, docker and ROCm. The OS and the first three programs are well known to everyone, we will not analyze their installation here, and ROCm can be installed by instructions.

Brief installation instructions for ROCm

sudo apt update
sudo apt install "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"
sudo usermod -a -G render,video $LOGNAME
wget https://repo.radeon.com/amdgpu-install/6.1.2/ubuntu/jammy/amdgpu-install_6.1.60102-1_all.deb
sudo apt install ./amdgpu-install_6.1.60102-1_all.deb

# При ошибке необходимо выполнить:
sudo chown -Rv _apt:root /var/cache/apt/archives/partial/
sudo chmod -Rv 700 /var/cache/apt/archives/partial/

sudo apt update
sudo apt install amdgpu-dkms rocm

Containerization

For containerization AMD Offers Custom Image Creation based on the executable file build.sh. This file collects images from Dockerfile according to the parameters passed. The assembled image serves as a basis for placing any project into a container.

By selecting only the most necessary packages and limiting the combination to loading only the latest versions, you can get a compact and easy-to-read base image.

Base image code

# https://github.com/ROCm/ROCm-docker/blob/master/dev/Dockerfile-ubuntu-22.04-complete
# https://github.com/microsoft/onnxruntime/blob/main/tools/ci_build/github/pai/rocm-ci-pipeline-env.Dockerfile
FROM ubuntu:22.04

ARG ROCM_VERSION=6.1.3
ARG AMDGPU_VERSION=${ROCM_VERSION}
ARG APT_PREF='Package: *\nPin: release o=repo.radeon.com\nPin-Priority: 600'
RUN echo "$APT_PREF" > /etc/apt/preferences.d/rocm-pin-600

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get install -y \
    libopenblas-dev \
    ninja-build \
    build-essential \
    pkg-config \
    curl \
    wget \
    git \
    make

RUN curl -sL https://repo.radeon.com/rocm/rocm.gpg.key | apt-key add - && \
    printf "deb [arch=amd64] https://repo.radeon.com/rocm/apt/$ROCM_VERSION/ jammy main" | tee /etc/apt/sources.list.d/rocm.list && \
    printf "deb [arch=amd64] https://repo.radeon.com/amdgpu/$AMDGPU_VERSION/ubuntu jammy main" | tee /etc/apt/sources.list.d/amdgpu.list
RUN apt-get update && apt-get install -y \
    rocm-dev \
    rocm-libs

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Install Conda
ENV PATH /opt/miniconda/bin:${PATH}
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh --no-check-certificate && /bin/bash ~/miniconda.sh -b -p /opt/miniconda && \
    conda init bash && \
    conda config --set auto_activate_base false && \
    conda update --all && \
    rm ~/miniconda.sh && conda clean -ya
ENV PYTHON_VERSION=3.11
RUN conda install python=${PYTHON_VERSION} pip

# https://github.com/comfyanonymous/ComfyUI/issues/3698
ENV TORCH_BLAS_PREFER_HIPBLASLT=0
RUN pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.1

RUN pip install transformers \
    peft \
    sentencepiece \
    scipy \
    protobuf --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.1

# https://github.com/agrocylo/bitsandbytes-rocm
# or
# https://github.com/arlo-phoenix/bitsandbytes-rocm-5.6
ENV PYTORCH_ROCM_ARCH=gfx900,gfx906,gfx908,gfx90a,gfx1030,gfx1100,gfx1101,gfx940,gfx941,gfx942
ENV BITSANDBYTES_VERSION=62353b0200b8557026c176e74ac48b84b953a854
RUN git clone https://github.com/arlo-phoenix/bitsandbytes-rocm-5.6 /bitsandbytes && \
    cd /bitsandbytes && \
    git checkout ${BITSANDBYTES_VERSION} && \
    make hip ROCM_TARGET=${PYTORCH_ROCM_ARCH} ROCM_HOME=/opt/rocm/ && \
    pip install . --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.1

Based on the base image, ComfyUI can be placed into a container as follows

FROM hardandheavy/transformers-rocm:2.1.0

EXPOSE 80

# To upgrade to the higher version, you need to wait for the problem to be resolved
# https://github.com/abetlen/llama-cpp-python/issues/1481
ENV LLAMA_CPP_PYTHON_VERSION=0.2.56
ENV DAMDGPU_TARGETS=gfx900;gfx906;gfx908;gfx90a;gfx1030;gfx1100;gfx1101;gfx940;gfx941;gfx942
RUN CMAKE_ARGS="-DLLAMA_HIPBLAS=ON -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++ -DAMDGPU_TARGETS=${DAMDGPU_TARGETS}" pip install llama-cpp-python==${LLAMA_CPP_PYTHON_VERSION}

WORKDIR /app
COPY ./docker/Makefile ./Makefile

CMD make run

The first line loads the base image hardandheavy/transformers-rocmwhich already contains the necessary packages for ComfyUI;
Lines 5 through 9 install llama-cpp-python (lines 5 through 6). This package is required for various ComfyUI extensions to work;
Line 12 copies a set of commands for loading and running ComfyUI.

The Makefile code should be read from line 14. First, the environment is prepared: the repository is copied and ComfyUI packages are installed. Then, the program directory is moved and it is launched.

COMFYUI_CHECK_SEED_FILE = /check/comfyui-check-seed-file
MANAGER_CHECK_SEED_FILE = /check/manager-check-seed-file

init:
	if [ ! -f $(COMFYUI_CHECK_SEED_FILE) ]; then \
		git clone https://github.com/comfyanonymous/ComfyUI /comfyui && \
		cd /comfyui && \
		pip install -r requirements.txt && \
		touch $(COMFYUI_CHECK_SEED_FILE); fi
	if [ ! -f $(MANAGER_CHECK_SEED_FILE) ]; then \
		git clone https://github.com/ltdrdata/ComfyUI-Manager /comfyui/custom_nodes/ComfyUI-Manager && \
		touch $(MANAGER_CHECK_SEED_FILE); fi

run: init
	cd /comfyui && \
	python main.py --listen 0.0.0.0 --port 80

Launch ComfyUI

After building ComfyUI, you can launch the project with three lines:

git clone https://github.com/HardAndHeavy/comfyui-rocm-docker
cd comfyui-rocm-docker
make run

The first time you launch it, there will be a lengthy initialization process. Once this process is complete, ComfyUI will be available at http://localhost.

Making anime

To create an anime based on the video, we will use the article Unsampling for AnimateDiff/Hotshot – An Inner-Reflections Guide and a comment from Catzwhich listed the necessary resources for the work. And for the convenience of reading the article, the text will be duplicated as a short translation with a small adaptation to our created image.

The settings files, original video and generation results can be downloaded from link.

Once you have entered ComfyUI, you need to click the “Load” button and load the anime.json configuration file. After loading, the configured connection of multiple nodes will be displayed on the screen. Some nodes will be marked in red, as they do not have the necessary extensions. To fix this, you need to click the “Manager” button, then “Install Missing Custom Nodes” and, having marked all the extensions, click “Install”. After installation, the program will prompt you to reboot, for which you need to click “Restart”. And then refresh the page by pressing “F5”.

It is important to note that ComfyUI for this image may be outdated, and updated extensions for it will be incompatible. Therefore, you need to update ComfyUI by clicking the “Manager” button, and then “Update ComfyUI”.

Before copying resources to nodes, you need to provide access to the data directory, since the container was launched as the root user. To do this, run the command sudo chmod -R 777 ./data.

Required resources:

For the “LoadVideo” node, select the video file original.mp4. Set “framerate” to a quarter of the original value to get a quick result. And for the initial check of the result, set “imagers_limit” to 5 frames. To generate all frames, “imagers_limit” must be set to 0.

For the “Video Combine” node, we set “frame_rate” equal to the frame rate from the “LoadVideo” node. In our case, it is 15, i.e. 60 frames from the original video divided by “framerate”.

For the positive Promt node (the “CLIPTextEncodeSDXL” in green), edit the text at the top and bottom equally. In our case, the text will be: “masterpiece, smooth color, 1 boy, dark hair, simple background, white shirt, blue trousers, barefoot, anime style, stairs, green grass”.

After all the settings, we are ready to create anime. Click “Queue Prompt” and get anime from video (file anime.mp4).

For each specific video (node ”LoadVideo”) and model (node ”Load Checkpoint”), it is necessary to adjust the positive and negative prompt. The negative prompt requires adjustment less often. It is also useful to try changing:

“strength” and “end_percent” for the “Apply Advanced ControlNet” node;
“resolution” for “Realistic Lineart”;
“cfg” for “SamplerCustom”.

In conclusion, let's try changing the parameters suggested above, and the result will be a couple more examples:

Origami style animation

Load the angle.json configuration file.

Required resources:

For the positive prompt node, enter the text: “((masterpiece, best quality)), Origami young man, folding sculpture, wearing white origami shirt, blue origami jeans, dark origami hair, origami barefoot, depth of field, detailed, sharp, 8k resolution, very detailed, cinematic lighting, trending on artstation, hyperdetailed, stairs, green grass”.

The output is an origami-style video (angle.mp4 file).

The origami idea is taken from the video AnimateDiff Tutorial.

Anime by Studio Ghibli

Load the cartoon.json configuration file.

For the positive prompt node, enter the text: “anime key visual, 70s Art, stylized by Studio Ghibli, Shinji Aramaki, 1 boy, dark hair, simple background, white shirt, blue trousers, barefoot, anime style, stairs, green grass”

The output is a video in the anime style from Studio Ghibli (file cartoon.mp4).

That's all for now. Thank you all for your attention! I wish you to unleash all your creative ideas with ComfyUI and, of course, use more containerization!