Computer vision lessons in Python + OpenCV from the very beginning. Part 4

In the last lesson, we got acquainted with median filtering, custom filters, and edge detection. Let me remind you that the selected contour can be used to search for an area of ​​interest in the image and to find various features. In particular, here’s what you can do next with the path:

· Identify various geometric primitives (straight lines, circles).

· Convert points into chains and analyze them separately.

· Describe as a graph and apply graph algorithms to it.

Let’s continue to study preprocessing methods. For example, an image can be made contrasting:

import cv2

img = cv2.imread('MyPhoto.jpg', 1)
cv2.imshow("Original image",img)

# CLAHE (Contrast Limited Adaptive Histogram Equalization)
clahe = cv2.createCLAHE(clipLimit=3., tileGridSize=(8,8))

lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)  # convert from BGR to LAB color space
l, a, b = cv2.split(lab)  # split on 3 different channels

l2 = clahe.apply(l)  # apply CLAHE to the L-channel

lab = cv2.merge((l2,a,b))  # merge channels
img2 = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)  # convert from LAB to BGR
cv2.imshow('Increased contrast', img2)

cv2.waitKey(0)
cv2.destroyAllWindows()

Let’s test this program on the same painfully familiar picture with palm trees from previous lessons:

Or, here, for example, a photo taken at night:

I note that this program used an adaptive contrast enhancement algorithm. Here, I think, some explanations are required. First, the program creates an adaptive histogram. In this case, the image is divided into 8 by 8 blocks and a histogram is built separately for each:

# CLAHE (Contrast Limited Adaptive Histogram Equalization)
clahe = cv2.createCLAHE(clipLimit=3., tileGridSize=(8,8))

Next, to superimpose the aligned histograms on the image, it is converted to the LAB format, which is one lightness channel L and two color channels A and B:

lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)  # convert from BGR to LAB color space
l, a, b = cv2.split(lab)  # split on 3 different channels

Finally, we can apply the equalized histograms (only to the L – lightness channel):

l2 = clahe.apply(l)  # apply CLAHE to the L-channel

Then we combine these channels and perform the inverse transformation:

lab = cv2.merge((l2,a,b))  # merge channels
img2 = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)  # convert from LAB to BGR
cv2.imshow('Increased contrast', img2)

Separately, it is worth mentioning the clipLimit parameter. It will determine how much more contrast the photo will become. For example, let’s set it to 10 and get this:

As you can see, even some details in the distance appeared here. True, we cannot increase this parameter indefinitely, there is a certain limit at which noise will spoil the photo. Here’s what we’ll see if we set it to 100:

And now let’s try to select a contour for the same image (without first increasing the contrast):

And if you increase the contrast first:

As you can see, the silhouettes of people are more clearly visible here.

Okay, since we’re talking about contours, let’s look at other ways to identify features. For example, special points, namely, corners, bends, as well as some conspicuous features of certain objects. For example, on a human face, the special points are the eyes, nose, corners of the mouth. The search for singular points, in fact, is also somewhere on the border of the classification of computer vision stages given in the first lesson.

Let’s start with the Harris detector. The Harris detector is an algorithm for finding corners in an image. OpenCV has such a function called cornerHarris. Let’s test it out:

# Python программа для иллюстрации
# обнаружение угла с
# Метод определения угла Харриса

import cv2
import numpy as np

# путь к указанному входному изображению и
# изображение загружается с помощью команды imread
image = cv2.imread('DSCN1311.jpg')

# конвертировать входное изображение в Цветовое пространство в оттенках серого
operatedImage = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# изменить тип данных
# установка 32-битной плавающей запятой
operatedImage = np.float32(operatedImage)

# применить метод cv2.cornerHarris
# для определения углов с соответствующими
# значения в качестве входных параметров
dest = cv2.cornerHarris(operatedImage, 2, 5, 0.07)

# Результаты отмечены через расширенные углы
dest = cv2.dilate(dest, None)

# Возвращаясь к исходному изображению,
# с оптимальным пороговым значением
image[dest > 0.01 * dest.max()] = [0, 0, 255]

# окно с выводимым изображением с углами
cv2.imshow('Image with Borders', image)

if cv2.waitKey(0) & 0xff == 27:
    cv2.destroyAllWindows()

And here’s what happens if you apply it to the image:

As you can see, the algorithm determined the corners of windows, doors and walls quite well. And yet, however, he took for corners where there are no corners. Well, it happens that such algorithms do not always work perfectly.

Okay, we have identified special points. What’s next? And then, about the same as with the contours:

Make different geometric shapes from points, for example, triangles.

· Convert points into chains and analyze them separately.

· Describe as a graph and apply graph algorithms to it.

In the comments to the last lesson, they wrote: “I hope to continue, because here it is: “Describe as a graph and apply graph algorithms to it.” very interested.” Well, let’s try to do this. The easiest way, of course, is to turn a set of singular points into a graph (in the case of a contour, there are more points and therefore it is more difficult to work with it). First, let’s reduce the number of points by raising the threshold:

# Возвращаясь к исходному изображению,
# с оптимальным пороговым значением
image[dest > 0.05 * dest.max()] = [0, 0, 255]

Now, for clarity, we will remove the picture, leaving only one point:

# Возвращаясь к исходному изображению,
# с оптимальным пороговым значением
img_blank = np.zeros(image.shape)
img_blank[dest > 0.05 * dest.max()] = [0, 0, 255]

# окно с выводимым изображением с углами
cv2.imshow('Image with Borders', img_blank)

How can we turn these points into a graph? First, let’s remember how to work with structures such as a graph. You can read about it in the article: Graph traversal algorithms in Python and C# – Developer Library Programming Store (programstore.ru). In short, a graph can be represented as a list of vertices (points) and their connections (edges). You can also represent the graph as a matrix of connections. How we will represent the graph, we will decide later, for starters, let’s try to sort through all the points and count their number:

heigh=img_blank.shape[0]
width=img_blank.shape[1]
count=0
for x in range(0,width):
    for y in range(0,heigh):
        if img_blank[y,x,2]==255:
            print(x,y)
            count+=1
print("Всего",count,"точек")

I got 2829, moreover, many points are nearby, with a difference of only one pixel (yes, this can be seen from the previous picture):

How can these points be thinned out? One way is to combine close points into one, the center of which will be some arithmetic mean of these close points. But there is one problem. If we calculate the distance of each point from each, then this will be the number of points in the square. The number 2829 to the power of 2 is a little more than 8 million. And if there are even more points? Clearly the approach is irrational. As an option, you can immediately collect close points into a cluster during the bypass, and only then calculate their average coordinates.

For starters, let’s just try to group the points. To do this, we need a few helper functions:

#Получить последнюю точку в списке списочков и одиночных точек
def get_last_point(ls):
    lls = len(ls)
    if lls>0:
        item=ls[lls - 1]
        if type(item)==list:
            if len(item)>0:
                x,y=item[len(item)-1]
            else:
                return 0, 0, False
        else:
            x,y=item
        return x,y,True
    return 0,0,False

#Вычислить расстояние между точками
def get_distance(p1,p2):
    x1,y1=p1
    x2,y2=p2
    l = math.sqrt((x1 - x2) * (x1 - x2) + (y1 - y2) * (y1 - y2))  # Евклидово расстояние между точками
    return l

#Добавить точку в кластер 
def add_point_to_claster(x,y,ls):
    lls = len(ls)
    item = ls[lls - 1]
    if type(item) == list:
        item.append((x,y))
    else:
        x1,y1=item
        item=[(x1,y1)]
        item.append((x,y))
        ls[lls-1]=item

The idea is that we have a point represented as a tuple of two numbers (its coordinates). We go through the raster, determine if there is a point and if there is, then add it to the list. But if this point is close to the last point of the list, then we turn this last point into a list consisting of the last and current point. Further, if the next point is close, we will also add it to this list. If not, we will start building a new list. As a result, we get a list of lists and single points (if any). Of course, it could have been made simpler, a list of lists, and if this is a single point, then the sublist has a length of 1. But, on the other hand, from this example we will also learn how to determine the type of a variable in Python.

How to determine that the points are close to each other? It is necessary to calculate the Euclidean distance, and if it is less than the threshold, then the points are close. I took the threshold here “from the ceiling”, set it equal to 3.

Here is how we use these functions:

heigh=img_blank.shape[0]
width=img_blank.shape[1]
points=[]
for x in range(0,width):
    for y in range(0,heigh):
        if img_blank[y,x,2]==255:
            x1, y1, point_is = get_last_point(points)
            if point_is:
                l=get_distance((x1,y1),(x,y))
                if l<3:
                    add_point_to_claster(x,y,points)
                    continue
            points.append((x,y))
for point in points:
    print(point)

Here’s what happened in the end:

As a result, the number of elements in the list became 591. Now it remains to turn them into points:

 def calk_center(ls):
    ix=0
    iy=0
    l=float(len(ls))
    for point in ls:
        x,y=point
        ix+=x
        iy+=y
    return round(ix/l),round(iy/l)

Let’s display these points on the screen:

centers=[]
for point in points:
    if type(point)==list:
        centers.append(calk_center(point))
    else:
        centers.append(point)

img_blank1 = np.zeros(image.shape)
for point in centers:
    print(point)
    x,y=point
    img_blank1[y,x,2]=255

Here’s what happened in the end:

It turned out, however, not very well. Tellingly, even increasing the threshold does not improve the quality of point merging. In fact, bypassing the points through the sweep was not a very good idea, and this is exactly the mistake: neighboring points that are nearby could fall into different lists. In the next lesson, I will tell you how to fix this flaw.

Well, in the conclusion of the lesson, the full text of the resulting program for finding and thinning singular points:

# Python программа для иллюстрации
# обнаружение угла с
# Метод определения угла Харриса

import cv2
import numpy as np
import math

#Получить последнюю точку в списке списоков и одиночных точек
def get_last_point(ls):
    lls = len(ls)
    if lls>0:
        item=ls[lls - 1]
        if type(item)==list:
            if len(item)>0:
                x,y=item[len(item)-1]
            else:
                return 0, 0, False
        else:
            x,y=item
        return x,y,True
    return 0,0,False

#Вычслить расстояние между точками
def get_distance(p1,p2):
    x1,y1=p1
    x2,y2=p2
    l = math.sqrt((x1 - x2) * (x1 - x2) + (y1 - y2) * (y1 - y2))  # Евклидово расстояние между последней и текущей точкой
    return l

#Добавить точку в кластер
def add_point_to_claster(x,y,ls):
    lls = len(ls)
    item = ls[lls - 1]
    if type(item) == list:
        item.append((x,y))
    else:
        x1,y1=item
        item=[(x1,y1)]
        item.append((x,y))
        ls[lls-1]=item

def calk_center(ls):
    ix=0
    iy=0
    l=float(len(ls))
    for point in ls:
        x,y=point
        ix+=x
        iy+=y
    return round(ix/l),round(iy/l)

# путь к указанному входному изображению и
# изображение загружается с помощью команды imread
image = cv2.imread('DSCN1311.jpg')

# конвертировать входное изображение в Цветовое пространство в оттенкахсерого
operatedImage = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# изменить тип данных
# установка 32-битной плавающей запятой
operatedImage = np.float32(operatedImage)

# применить метод cv2.cornerHarris
# для определения углов с соответствующими
# значения в качестве входных параметров
dest = cv2.cornerHarris(operatedImage, 2, 5, 0.07)

# Результаты отмечены через расширенные углы
dest = cv2.dilate(dest, None)

# Возвращаясь к исходному изображению,
# с оптимальным пороговым значением
img_blank = np.zeros(image.shape)
img_blank[dest > 0.05 * dest.max()] = [0, 0, 255]

heigh=img_blank.shape[0]
width=img_blank.shape[1]
points=[]
for x in range(0,width):
    for y in range(0,heigh):
        if img_blank[y,x,2]==255:
            x1, y1, point_is = get_last_point(points)
            if point_is:
                l=get_distance((x1,y1),(x,y))
                if l<3:
                    add_point_to_claster(x,y,points)
                    continue
            points.append((x,y))
centers=[]
for point in points:
    if type(point)==list:
        centers.append(calk_center(point))
    else:
        centers.append(point)

img_blank1 = np.zeros(image.shape)
for point in centers:
    print(point)
    x,y=point
    img_blank1[y,x,2]=255

# окно с выводимым изображением с углами
cv2.imshow('Image with Borders', img_blank1)

if cv2.waitKey(0) & 0xff == 27:
    cv2.destroyAllWindows()