4x image upscaling using ESRGAN

In this article, I want to analyze the use of the ESRGAN pre-trained neural network to increase the image resolution by four times using the tensorflow hub.

Tools

I will use the python 3.10 programming language, the development environment is jupyter notebook.

Let’s import the necessary libraries. To work with neural networks, we will use tensorflow. To work with the already trained ESRGAN model, we will use the tensorflow hub. We also need a library for working with images, take PILLOW.

from PIL import Image
import tensorflow as tf
import tensorflow_hub as hub

Image preparation

To work with a pretrained neural network, the input image must meet the following criteria:

The picture must be floating point, converted using tf.cast(image, tf.float32).
4-dimensional input parameters, [batch_size, height, width, 3]. To apply ultra-high resolution to a single image, use tf.expand_dims(image, 0) to add a batch dimension.
To display an image, you need to convert it back to uint8using tf.cast(tf.clip_by_value(image[index_of_image_to_display], 0, 255), tf.uint8)

The input image. Size 640×480 pixels. (cardiac ultrasound depicted)

# определим путь к входному изображению
image_path="./result_images/frame6.jpg"

# преобразуем входное изображения согласно условиям
hr_image = tf.image.decode_image(tf.io.read_file(image_path))
image = tf.expand_dims(hr_image, 0)
image = tf.cast(image, tf.float32)

The image must be a floating point image converted using tf.cast(image, tf.float32).
The image must be a floating point image converted using tf.cast(image, tf.float32).
The image must be a floating point image converted using tf.cast(image, tf.float32).

Working with ESRGAN

The model is in storage tensorflow hub.

# проведем загрузку модели
model = hub.load("https://tfhub.dev/captain-pool/esrgan-tf2/1")

# выполним 4x увеличения изображения
super_resolution = model(image)

Image post-processing

Let’s convert the 4x image back to uint8 and save it to a folder.

image = np.asarray(super_resolution)
image = tf.clip_by_value(image, 0, 255)
image = Image.fromarray(tf.cast(image[0], tf.uint8).numpy())
image.save('./SR.jpg'% count)

output image. Resolution 2560 * 1920 pixels.

Conclusion

This article explores how, using the ESRGAN pre-trained neural network, you can quadruple an image using a minimum amount of code.

The image must be a floating point image converted using tf.cast(image, tf.float32).
The image must be a floating point image converted using tf.cast(image, tf.float32).
The image must be a floating point image converted using tf.cast(image, tf.float32).
The image must be 4-dimensional, [batch_size, height, width, 3]. To apply ultra high resolution to a single image, use tf.expand_dims(image, 0)add a batch dimension.
To display an image, don’t forget to convert it back to uint8using Image must be a floating point image converted using tf.cast(image, tf.float32).
The image must be 4-dimensional, [batch_size, height, width, 3]. To apply ultra high resolution to a single image, use tf.expand_dims(image, 0)add a batch dimension.
To display an image, don’t forget to convert it back to uint8using
The image must be a floating point image converted using tf.cast(image, tf.float32).
The image must be 4-dimensional, [batch_size, height, width, 3]. To apply ultra high resolution to a single image, use tf.expand_dims(image, 0)add a batch dimension.
To display an image, don’t forget to convert it back to uint8using