Stable Diffusion and AMD

Foreword

Stable Diffusion is a neural network that can turn your text into an image, in a very simple way. But the problem stems from the implementation of PyTorch, which is based on CUDA technology. And how to be?

CPU usage

InvokeAI made it possible to perform calculations on the processor, instead of the video card. On its 16 threads, a picture in 512×512 was generated for 5 minutes, or even more. This is not an option for us…

AMD ROCM

ROCm is AMD’s CUDA Code Repeater. Yes, it is inferior in speed, but compared to the CPU, it works instantly. ROCm is perhaps the best way out of the situation, with the exception of one point, there is no talk of any support for Windows. So this time Linux users are happy.

ONNX

Perhaps the most universal solution in this situation. Library diffusers offers a ready-made solution, where you just have to send your requests with a simple Python script. It has several options for working:

  • CPU

  • DirectML (DirectX 12)

  • CUDA

However, not all so simple. This implementation cannot in VAE, LoRA, Upscale (at the time of writing it was open PR). Plus, apart from that you must convert your .ckpt models to ONNX format.

ONNX and laptops

One very interesting moment. If your laptop has a GPU in the processor:

Then there is a chance that you will not be able to run out of the box. DirectX has one problem when it takes GPU Device 0 which has ~512mb VRAM and ONNXRuntime will just catch an exception. To do this, you have to go through file searches in the hope of finding a function call InferenceSession and write to it in the call the ID of your discrete video card with pens, adding provider_options=[{'device_id': 1}].

It might look something like this:

ort.InferenceSession(path, providers=[provider], provider_options=[{'device_id': 1}], sess_options=sess_options)

Upscale (Vulkan)

As I already said, ONNX does not have native support for Upscale models. However, there are open Vulkan implementations of the same Real-ESRGAN, Real LSR… Which work rather quickly. You just need to call them from the command line with the correct parameters.

Conclusion

Now all that’s left is to find a suitable UI implementation for Stable Diffusion and download upscaler if you’re on Windows.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *