Making the right anaglyph with your own hands

Hello everyone. Today I will share a method for creating high-quality anaglyph images with minimal effort. We will not make dinosaurs: we will take any photo from the family archive and get a full-fledged 3D image. I will say right away that the concept is not new, but we will use modern developments and even write a simple plugin for GIMP, which, however, will have to be abandoned …

About anaglyph

I anticipate the questions “Why anaglyph?” A technology from the last century with a bunch of shortcomings, and with the advent of 3D screens/glasses and VR helmets, it is considered long dead. The reason is simple: modern 3D requires special equipment and it, compared to the simplest red-blue glasses, costs several orders of magnitude more. And you can also make glasses yourself from improvised means.

I will reveal the technology right away. It is known that for binocular vision we need two images: separately for the left and right eyes. Technically, these are two angles of an object with a parallax of about 6.5 cm (the distance between the eyes). And, at first glance, they are practically indistinguishable from each other, especially when the object is located one and a half meters from the eyes. Based on the tiny difference in the images, our brain easily and very accurately determines the distances to all objects viewed at a given moment in time.

That is, according to the theory, it is impossible to obtain a stereo image (SIRDS, stereo pairs, anaglyph) from one picture, because both eyes will receive identical images. However, the problem is easily solved if there is a height map (or depth map) of the given picture. We simply build a voxel surface and color each voxel with the color of the corresponding pixel of the original photograph.

The finished 3D surface can be rotated in any direction and generate an image in the required angles. Given the tiny parallax required for stereo images, we can get a new angle of our photo with minimal distortion. These photos will be the source for our anaglyphs, or stereo pairs.

Schematic representation of the pipeline for obtaining anaglyph from a single image

Before we start implementing, let's decide on the tools. First of all, it's GIMP – a cross-platform graphic editor.

I also found this for demonstration purposes downloaded on the Internet the photo below. The picture of Madame with her children, as I believe, was taken at the beginning of the last century. By the way, I also have such a photo of my great-grandfather with his wife and children.

The presented test image is monochrome, but there are no restrictions on color – you can use any photos: family heirlooms, parties, cats and even the tricks of neural networks. In general, everything that is gathering dust in terabytes in the clouds, on phones, archived hard drives. But you should take into account that anaglyph is famous for poor color rendering.

And, of course, we will need anaglyph glasses to contemplate our homemade masterpieces. You can make them yourself, but it is easier and cheaper to buy them online.

Generating a height map

This operation can be done manually. Some slow resources even have videos showing step by step how easy and simple it is. But I

I strongly recommend not to do this

: firstly, you will get tired and lose a lot of useful time, secondly, the result will be an order of magnitude worse than the height map generated by modern AI algorithms. I will tell you about the latter.

A quick search on the Internet shows that everything is developing rapidly here: automatic depth map construction is almost a separate discipline of computer vision sports.
Everything is available online and even free, so all we need is a computer with a browser and internet access.

As I already wrote, there are many implementations, all of them generate depth maps of varying levels of detail and at different speeds. Personally, I use only two resources:

• Tiling Zoe Depth — in my opinion, a more detailed map, but a lot of blur. An additional plus of the solution is that you can upload images in batches. It works on Google Colab, so you will need a Google account to access it.
• Depth Anything V2 — sharper edges and foreground details. Works in Hugging Face. No authorization required, I did not encounter resource limitations.

You can also test other algorithms.

on Hugging Face

by typing the keyword “depth” in the search bar.

Generating a Depth Map with Tiling Zoe Depth

Launch Chrome and go to GitHub using the link

BillFSmith/TilingZoeDepth

and from there to Google Colab via the “v3” link in README.md.

In Google Colab, we must connect the capacities of the remote execution environment: in the upper right corner, click “Connect… GPU” and wait for the green check mark. Run the code by pressing the key combination Ctrl + F9 or by clicking on the picture with a triangle in a circle.
As a result, we get a button to load the image for which we want to generate a height map. You can select several images at a time. This is convenient, since the formation of the computing environment takes quite a lot of time.

During the installation of dependencies, there will be a warning – just ignore it. During the generation, the height maps will be displayed in low and high resolutions. There is no point in downloading them, since after the calculations are complete, the browser will ask for permission to accept them “in bulk”. We agree.

If you need to process additional images, then run it again and select a new batch of images. The process will be faster, since all the libraries have already been downloaded and the runtime environment is configured.

When you're done processing, be sure to disable the resources in the upper right corner. I vaguely remember that Google can ban you for leaving them idle. I haven't checked, but there was a moment when it assured me for quite a long time that it had no free accelerators to run. Perhaps this is a kind of “soft” ban.

Generating a Depth Map with Depth Anything V2

Follow the link

Depth-Anything-V2

If the space is running, everything is simple: we immediately start working – upload an image and click “Compute Depth”.

Otherwise, you will have to start it and wait until it rises.

Finally, we download the depth map from the link “Grayscale depth map” under our images. The lack of a batch mode is compensated by the constant readiness of the runtime environment and the speed of generation.

If you don't like the depth map a little

You may find that automatic algorithms do not determine the distance quite correctly. This is easy to fix.

Load the depth map into GIMP and open the dialog «Color → Curves» (Colors → Curves — from here on I will indicate the names of elements of the English interface of GIMP in brackets). This is necessary to correct the brightness curve (the image is monochrome). On the abscissa axis we have the pixel color, on the ordinate axis – the distance to the viewer, the histogram – the number of pixels of a given color. Zero implies the concept of “far” (infinity, horizon), and the upper right corner is the screen. At the very beginning, the dependence is linear, and it can be corrected.

If you take vintage photos, like the ones in this article, the salon shots were taken on a painted backdrop. ZoeDepth's algorithm can add volume to them that clearly shouldn't be there. Here it recognized the tree, bushes, and some oddities on the left, but Depth Anything V2 ignored them.

By adjusting the curve, you can collapse the back layers and get rid of the incorrect background. Similarly, you can add more volume to the foreground details. Try it, it's not difficult, and in practice the essence is captured much faster.

But you shouldn’t abuse the tool, since careless manipulations can result in unwanted artifacts, such as “cardboard figures.”

The depth map can also be adjusted using standard GIMP tools.

Making anaglyph

So, we have an image and a depth map, now we start to form two angles.

Load the image and the height map into GIMP as layers. I suggest adding them to the menu right away. «Image → Accuracy» install “16 bit floating point” (Image → Precision: 16 bit floating point). This will allow us to slightly reduce our negative impact on the image quality. We move the layer with the height map to the very bottom.

Go to the layer with the main image and clone it, as we need to get two identical layers. Activate the top layer.

Of course, GIMP does not implement a full-fledged voxel engine, but the filter “Bias” (Displace) produces the result we need. Based on the height map, the pixels of the original image are shifted in the required direction, and the resulting voids are filled based on the interpolation of the color of neighboring pixels.

Make the top layer active via the menu “Filters → Projection → Offset” (Filters → Map → Displace) we call up a dialog with filter settings. By the top button “Auxiliary entrance” (Aux input) load the depth map – everything is clear there, just poke the mouse. Do not touch the second button, since there is no need for this: it allows you to have different depth maps for vertical and horizontal shifts. In the parameters Sampler please specify Linear. Next, we break the connection between the horizontal and vertical shift (we only need the horizontal shift) and set it. For a good perception of volume, 6-10 pixels are enough. Click “OK”.

Artifacts when shifting

Sometimes a situation arises when the “contour” is torn off when shifting the image – a kind of “halo” effect appears. This happens when the depth difference is too large or when the depth map smoothing is too small. The more the image is shifted, the more distinct this effect appears.

If you make an anaglyph as described in various sources on the Internet – a pair of the original image and an image with a shift, the result will be ugly. Therefore, we do it more simply: you need to move not one image by 10 pixels, but both images in different directions by 5 pixels. I always do this for my anaglyphs.

At this stage, we already have two different angles in the upper layers. We can immediately make a stereo pair for Google Cardboard or use it for other methods of forming three-dimensional images.

Well, we will continue to make our anaglyph. Let's immediately decide that it will be for red-blue glasses. The top layer is for the left eye, that is, red. Activate it, and in the menu

«Color` → Components → Channel Mixer»

(Colors → Components → Channel Mixer) reset the blue and green channels. For the future, save this configuration using the “+” button next to the list

“Profiles”

(Presets) under the name “Red”. So, next time it will be enough to select it from the drop-down list, and not to enter values ​​manually in several fields.

The second layer contains the image for the right eye. Just like the left (red) image, through the menu “Color → Components → Channel Mixer” adjust the color channels: this time, completely reset everything related to red. Similarly, save it to a profile named “Blue”.

Let's go back to the layers: the top layer is red, the middle layer is blue, the bottom layer is the depth map. Activate the top (red) layer and select a little higher “Mode” (Mode) of layer blending “Difference” (Difference). The picture changes immediately: the color is normalized, noticeable blue and red contours appear where the contents of the layers do not match.

Improving perception

There is another parameter that is worth adjusting – the shift of the right and left images relative to each other. Google images by the line “anaglyph parallax”, everything will be clearly drawn on them. In fact, this moves or brings the three-dimensional image relative to the monitor plane.

I do this because I don't like red/blue halos around faces or key details in the image. I try to keep minor things in the foreground or background as far apart as possible, and the center of the composition, which we focus on when viewing, is exactly on the plane of the screen.

The shift is done in the layer properties. Right-click on the layer, “change layer attributes → offset X” (Edit Layer Attributes → Offset X).

The image is ready for viewing, all that remains is to export it in the required format via the menu “File → Export As…”so that in the future it can simply be displayed on a screen and viewed through glasses with red-blue lenses.

I described the manual method, you can now start gradually converting your photo bank into a three-dimensional form. However, I promised that we would write a plugin for GIMP to reduce the mileage and number of mouse clicks.

Plugin

We will write in Python using the standard GIMP Python-Fu module. It would be tempting to implement a full conversion cycle – from forming a height map to generating a finished image. However, the version of Python in GIMP is not fresh at all and it will not be possible to hook up modern modules. In addition, we will be limited by the computing resources of the local machine, and somewhere Google's Teslas are idle 🙂

Therefore, I propose to significantly reduce the requirements for the plugin. It will become very simple and stupid: in fact, a macro that receives two images (a photo and a depth map) as input and performs all the manipulations that had to be done manually.

I provide the text of the plugin, explanations will be below:

#!/usr/bin/env python

from gimpfu import *

def generate_anaglyph(img, drawable, gtype, displacement, shift, depthmap) :

    if pdb.gimp_image_get_precision(img) != PRECISION_DOUBLE_LINEAR:
        pdb.gimp_image_convert_precision(img, PRECISION_DOUBLE_LINEAR)

    layer = pdb.gimp_image_get_active_layer(img)

    layerL = pdb.gimp_layer_copy(layer, False)
    pdb.gimp_layer_set_name(layerL, "Left")

    layerR = pdb.gimp_layer_copy(layer, False)
    pdb.gimp_layer_set_name(layerR, "Right")

    img.add_layer(layerR, 0)
    img.add_layer(layerL, 0)

    displacement /= 2;
    pdb.plug_in_displace(img, layerL, -displacement, 0, True, False, depthmap, depthmap, 2)
    pdb.plug_in_displace(img, layerR, +displacement, 0, True, False, depthmap, depthmap, 2)

    # Color anaglyph
    if gtype == 0: 

        pdb.plug_in_colors_channel_mixer(img, layerL, 0,  1.0, 0.0, 0.0,
                                                          0.0, 0.0, 0.0, 
                                                          0.0, 0.0, 0.0 )

        pdb.plug_in_colors_channel_mixer(img, layerR, 0,  0.0, 0.0, 0.0,
                                                          0.0, 1.0, 0.0,
                                                          0.0, 0.0, 1.0 )

    # Optimized anaglyph
    elif gtype == 1:

        pdb.plug_in_colors_channel_mixer(img, layerL, 0,  0.0, 0.7, 0.3,
                                                          0.0, 0.0, 0.0, 
                                                          0.0, 0.0, 0.0 )

        pdb.plug_in_colors_channel_mixer(img, layerR, 0,  0.0, 0.0, 0.0,
                                                          0.0, 1.0, 0.0,
                                                          0.0, 0.0, 1.0 )

    pdb.gimp_layer_set_mode(layerL, LAYER_MODE_DIFFERENCE)

    shiftL = int(shift/2)
    pdb.gimp_layer_set_offsets(layerL, -shiftL, 0)
    pdb.gimp_layer_set_offsets(layerR, shift-shiftL, 0)

    return

register(
    "generate_anaglyph",
    "Generate anaglyph image",
    "Longer description of doing stuff",
    "Alef13",
    "Alef13",
    "2024",
    "<Image>/Filters/Test/Anaglyph image",
    "RGB, RGB*, GRAY*",
    [
        (PF_OPTION, "gtype", "Glasses type", 1, ("Color anaglyph", "Optimized anaglyph")),
        (PF_SLIDER, "displacement",  "Horizontal displacement", 10, (0, 50, 0.1)),
        (PF_SLIDER, "shift",  "Horizontal shift", 0, (-25, 25, 0.1)),
        (PF_LAYER, "depthmap", "Depth map layer", None),
    ],
    [],
    generate_anaglyph)

main()

First of all we register our function

generate_anaglyph

as a GIMP filter. The parameter specifies the path where it will appear in the menu

“Filters”

. Also, when registering, we specified the function parameters, value ranges, default values, lists, etc. GIMP will automatically generate a form for calling the plugin, in which we can adjust these parameters. What the form looks like, we will see below.

Let's look at the filter function itself. First, we set the calculation precision to 16 bits with floating point – just in case we forgot to do it earlier.

Clone the original layer twice. I decided not to modify the original image and make both angles in separate layers.

We move (rotate) each layer: left – to the right, right – to the left. The total distance is equal to the displacement parameter.

Based on the GType parameter, which is the index of the drop-down list, we determine what type of anaglyph we need. The fact is that decomposition into red and blue-green channels does not give the most correct color picture of the anaglyph. In addition, there are glasses in color pairs other than red-blue (details in the English “Wikipedia”). And here I provide the possibility of making an image for specific glasses. With regard to red-blue glasses, in order to somehow preserve the color rendering, you can play with the matrices in accordance with this resource. In the plugin, for example, the so-called optimized anaglyph and color anaglyph are implemented, the rest is easy to do yourself.

We set the blending mode of the top layer and shift the layers relative to each other.

The plugin is ready, all that remains is to learn how to use it.

Let's take a look at the GIMP settings, section “Edit → Options” (Edit → Preferences), then «Catalogs → Plugins» (Folders → Plug-ins). Place the plugin in one of these folders. After restarting GIMP, our plugin will appear in the menu «Filters → test → Anaglyph image».

Load the image and depth map into GIMP as separate layers. Make the layer containing the image active and call the plugin through the menu.

In the window with parameters, be sure to specify the layer with the map as the depth map and adjust the other parameters if necessary. Click “OK”.

Our anaglyph is ready, we can export it to a file. If you don't like the result, just delete the two upper color layers and run the filter again, but with different parameters. This is exactly why I created two separate layers.

You probably noticed that both the manual method and the plugin use pixels in the parameters, so for a specific image you have to determine them by selection. I won't give you any universal numbers – they depend on the focus of the image, resolution, image size, etc. Perhaps, if you understand the mathematics of constructing this 3D a little, you can make yourself a table with comfortable numbers (I immediately remembered the markings on the optical sight of a sniper rifle :). But for now, we just rely on intuition and experience.

The final

So, we have abandoned the plugin, but I suggest going further — abandoning GIMP itself. There are several reasons: a scattering of different types of glasses on my desk, Google Cardboard, other viewing devices. Generating content for each device becomes tedious, and storing several versions of 3D images is irrational. Therefore, I plan to keep them in the form of the original photo and depth map, and generate an image for a specific device at the viewing stage.

There are also options for placing the depth map in the metadata of the image itself. I have already conducted a small study of these methods, and perhaps this is enough for a separate article. In the meantime, I suggest you refresh your impressions of the summer holiday, but already in 3D format 🙂