How to Build a Facial Recognition System with Elasticsearch and Python

The translation of the material was prepared as part of a practical intensive “Centralized logging systems Elastic stack”


Have you ever tried to look for objects in images? Elasticsearch can help you store, analyze, and search for objects in images or videos.

In this quick tutorial, we’ll show you how to build a face recognition system using Python. Learn more about how to detect and encode information about appearance – and find matches in searches.

Brief description of the basics

Need a refresh? Let’s take a quick look at a few basic concepts.

Face recognition

Face recognition is the process of identifying a person by their face, for example, to implement an authentication mechanism (like unlocking a smartphone). It captures, analyzes and compares patterns based on the details of a person’s face. This process can be divided into three stages:

  • Face detection: Identification of human faces in digital images.

  • Facial data encoding: Digitizing facial features

  • Face matching: Find and compare facial features

We will look at each stage using our example.

128-dimensional vector

Facial features can be converted into a set of digital information for storage and analysis.

Vector data type

Elasticsearch offers a datatype dense_vector for storing dense vectors of floating values. The maximum number of elements in a vector should not exceed 2048, which is enough to store representations of facial features.

Now let’s implement all of these concepts.

Anything you need

To detect faces and encode information, you need the following:

Please note that we have tested the following instructions on Ubuntu 20.04 LTS and Ubuntu 18.04 LTS. Some changes may be required depending on your operating system.

Install Python and Python Libraries

Ubuntu 20 and other Debian Linux versions come with Python 3 installed. If yours is not the same package, download and install Python by this link

To make sure your version is the most recent, you can run the following command:

sudo apt update 
sudo apt upgrade

Make sure your Python version is 3.x:

python3 -V

Install pip3 to manage Python libraries:

sudo apt install -y python3-pip

Install cmake required for the face_recognition library to work:

pip3 install CMake

Add the cmake bin folder to your $ PATH directory:

export PATH=$CMake_bin_folder:$PATH

Finally, install the following libraries before we start scripting our main program:

pip3 install dlib 
pip3 install numpy 
pip3 install face_recognition  
pip3 install elasticsearch

Detecting and encoding face information from an image

Using the face_recognition library, we can detect faces in the image and convert the facial features to a 128-dimensional vector.

Create a file getVectorFromPicture.py:

touch getVectorFromPicture.py

Supplement the file with the following script:

import face_recognition 
import numpy as np 
import sys 
image = face_recognition.load_image_file("$PATH_TO_IMAGE") 
# detect the faces from the images  
face_locations = face_recognition.face_locations(image) 
# encode the 128-dimension face encoding for each face in the image 
face_encodings = face_recognition.face_encodings(image, face_locations) 
# Display the 128-dimension for each face detected 
for face_encoding in face_encodings: 
      print("Face found ==>  ", face_encoding.tolist())

Let’s do getVectorFromPicture.pyto get a representation of facial features for the images of Elastic’s founders. In the script, you must modify the $ PATH_TO_IMAGE variable to set the name of the image file.

We can now save the representation of facial features in Elasticsearch.

First, let’s create an index with a display containing a field of type dense_vector:

# Store the face 128-dimension in Elasticsearch 
## Create the mapping 
curl -XPUT "http://localhost:9200/faces" -H 'Content-Type: application/json' -d' 
{ 
  "mappings" : { 
      "properties" : { 
        "face_name" : { 
          "type" : "keyword" 
        }, 
        "face_encoding" : { 
          "type" : "dense_vector", 
          "dims" : 128 
        } 
      } 
    } 
}'

We need to create one document for each face representation, this can be done with Index API:

## Index the face feature representation
curl -XPOST "http://localhost:9200/faces/_doc" -H 'Content-Type: application/json' -d'
{
  "face_name": "name",
  "face_encoding": [
     -0.14664565,
     0.07806452,
     0.03944433,
     ...
     ...
     ...
     -0.03167224,
     -0.13942884
  ]
}'

Face matching

Let’s say we indexed four documents in Elasticsearch that contain individual images of the Elastic founders’ faces. We can now use a different image from our founders to match individual images.

Create a file recognizeFaces.py:

touch recognizeFaces.py

In this scenario, we will extract vectors for each face found in the input image and use those vectors when creating a request to submit to Elasticsearch:

Import the libraries:

import face_recognition 
import numpy as np 
from elasticsearch import Elasticsearch 
import sys

Add the following section to connect to Elasticsearch:

# Connect to Elasticsearch cluster 
from elasticsearch import Elasticsearch 
es = Elasticsearch( 
cloud_id="cluster-1:dXMa5Fx...", 
     http_auth=("elastic", "<password>"), 
)

We will use the function cosineSimilarity to calculate the degree of cosine similarity between given query vectors and vectors of documents stored in Elasticsearch.

i=0 
for face_encoding in face_encodings: 
        i += 1 
        print("Face",i) 
        response = es.search( 
        index="faces", 
        body={ 
         "size": 1, 
         "_source": "face_name", 
        "query": { 
        "script_score": { 
                 "query" : { 
                     "match_all": {} 
                 }, 
         "script": { 
                "source": "cosineSimilarity(params.query_vector, 'face_encoding')", 
                 "params": { 
                 "query_vector":face_encoding.tolist() 
                } 
           } 
          } 
         } 
        } 
        )

Suppose a value less than 0.93 is considered an unknown person:

for hit in response['hits']['hits']: 
                #double score=float(hit['_score']) 
                if (float(hit['_score']) > 0.93): 
                    print("==> This face  match with ", hit['_source']['face_name'], ",the score is" ,hit['_score']) 
                else: 
                        print("==> Unknown face")

Let’s run our script:

The script was able to detect all faces with a match of more than 0.93.

Take it one step further with advanced search

Face recognition and search can be combined for advanced use cases. You can use Elasticsearch to create more complex queries like geo-queries, query-dsl-bool-query and search-aggregations

For example, the following query applies a cosineSimilarity search to a specific location within a 200 km radius:

GET /_search 
{ 
  "query": { 
    "script_score": { 
      "query": { 
    "bool": { 
      "must": { 
        "match_all": {} 
      }, 
      "filter": { 
        "geo_distance": { 
          "distance": "200km", 
          "pin.location": { 
            "lat": 40, 
            "lon": -70 
          } 
        } 
      } 
    } 
  }, 
       "script": { 
                "source": "cosineSimilarity(params.query_vector, 'face_encoding')", 
                 "params": { 
                 "query_vector":[ 
                        -0.14664565,
                       0.07806452,
                       0.03944433,
                       ...
                       ...
                       ...
                       -0.03167224,
                       -0.13942884
                    ] 
                } 
           } 
    } 
  } 
}

Combination cosineSimilarity with other requests, Elasticsearch gives you unlimited options for more complex use cases.

Conclusion

Face recognition can be useful in many cases, and you may already be using it in your daily life. The concepts described above can be generalized to any object detection in images or videos, so you can extend your use case to a very broad application.

Elasticsearch can help solve complex problems. Try this with Free 14-Day Elastic Cloud Trial, our official managed Elasticsearch offering, and let us know your opinion on our forums for discussion


Learn more about the express course “Centralized logging systems Elastic stack”

Similar Posts

Leave a Reply