How to Build a Facial Recognition System with Elasticsearch and Python
The translation of the material was prepared as part of a practical intensive “Centralized logging systems Elastic stack”…
Have you ever tried to look for objects in images? Elasticsearch can help you store, analyze, and search for objects in images or videos.
In this quick tutorial, we’ll show you how to build a face recognition system using Python. Learn more about how to detect and encode information about appearance – and find matches in searches.
Brief description of the basics
Need a refresh? Let’s take a quick look at a few basic concepts.
Face recognition
Face recognition is the process of identifying a person by their face, for example, to implement an authentication mechanism (like unlocking a smartphone). It captures, analyzes and compares patterns based on the details of a person’s face. This process can be divided into three stages:
Face detection: Identification of human faces in digital images.
Facial data encoding: Digitizing facial features
Face matching: Find and compare facial features
We will look at each stage using our example.
128-dimensional vector
Facial features can be converted into a set of digital information for storage and analysis.
Vector data type
Elasticsearch offers a datatype dense_vector for storing dense vectors of floating values. The maximum number of elements in a vector should not exceed 2048, which is enough to store representations of facial features.
Now let’s implement all of these concepts.
Anything you need
To detect faces and encode information, you need the following:
Python: In this example, we will use Python 3, which you can be found here… The installation and boot order depends on your operating system.
Elasticsearch Cluster: You can create a cluster in Free Trial Elastic Cloud…
Face recognition library: A simple Python face recognition library.
Python Elasticsearch client: Official Python client for Elasticsearch.
Please note that we have tested the following instructions on Ubuntu 20.04 LTS and Ubuntu 18.04 LTS. Some changes may be required depending on your operating system.
Install Python and Python Libraries
Ubuntu 20 and other Debian Linux versions come with Python 3 installed. If yours is not the same package, download and install Python by this link…
To make sure your version is the most recent, you can run the following command:
sudo apt update
sudo apt upgradeMake sure your Python version is 3.x:
python3 -VInstall pip3 to manage Python libraries:
sudo apt install -y python3-pipInstall cmake required for the face_recognition library to work:
pip3 install CMakeAdd the cmake bin folder to your $ PATH directory:
export PATH=$CMake_bin_folder:$PATHFinally, install the following libraries before we start scripting our main program:
pip3 install dlib
pip3 install numpy
pip3 install face_recognition
pip3 install elasticsearchDetecting and encoding face information from an image
Using the face_recognition library, we can detect faces in the image and convert the facial features to a 128-dimensional vector.
Create a file getVectorFromPicture.py:
touch getVectorFromPicture.pySupplement the file with the following script:
import face_recognition
import numpy as np
import sys
image = face_recognition.load_image_file("$PATH_TO_IMAGE")
# detect the faces from the images
face_locations = face_recognition.face_locations(image)
# encode the 128-dimension face encoding for each face in the image
face_encodings = face_recognition.face_encodings(image, face_locations)
# Display the 128-dimension for each face detected
for face_encoding in face_encodings:
print("Face found ==> ", face_encoding.tolist())Let’s do getVectorFromPicture.pyto get a representation of facial features for the images of Elastic’s founders. In the script, you must modify the $ PATH_TO_IMAGE variable to set the name of the image file.
We can now save the representation of facial features in Elasticsearch.
First, let’s create an index with a display containing a field of type dense_vector:
# Store the face 128-dimension in Elasticsearch
## Create the mapping
curl -XPUT "http://localhost:9200/faces" -H 'Content-Type: application/json' -d'
{
"mappings" : {
"properties" : {
"face_name" : {
"type" : "keyword"
},
"face_encoding" : {
"type" : "dense_vector",
"dims" : 128
}
}
}
}'We need to create one document for each face representation, this can be done with Index API:
## Index the face feature representation
curl -XPOST "http://localhost:9200/faces/_doc" -H 'Content-Type: application/json' -d'
{
"face_name": "name",
"face_encoding": [
-0.14664565,
0.07806452,
0.03944433,
...
...
...
-0.03167224,
-0.13942884
]
}'Face matching
Let’s say we indexed four documents in Elasticsearch that contain individual images of the Elastic founders’ faces. We can now use a different image from our founders to match individual images.
Create a file recognizeFaces.py:
touch recognizeFaces.pyIn this scenario, we will extract vectors for each face found in the input image and use those vectors when creating a request to submit to Elasticsearch:
Import the libraries:
import face_recognition
import numpy as np
from elasticsearch import Elasticsearch
import sysAdd the following section to connect to Elasticsearch:
# Connect to Elasticsearch cluster
from elasticsearch import Elasticsearch
es = Elasticsearch(
cloud_id="cluster-1:dXMa5Fx...",
http_auth=("elastic", "<password>"),
)We will use the function cosineSimilarity to calculate the degree of cosine similarity between given query vectors and vectors of documents stored in Elasticsearch.
i=0
for face_encoding in face_encodings:
i += 1
print("Face",i)
response = es.search(
index="faces",
body={
"size": 1,
"_source": "face_name",
"query": {
"script_score": {
"query" : {
"match_all": {}
},
"script": {
"source": "cosineSimilarity(params.query_vector, 'face_encoding')",
"params": {
"query_vector":face_encoding.tolist()
}
}
}
}
}
)Suppose a value less than 0.93 is considered an unknown person:
for hit in response['hits']['hits']:
#double score=float(hit['_score'])
if (float(hit['_score']) > 0.93):
print("==> This face match with ", hit['_source']['face_name'], ",the score is" ,hit['_score'])
else:
print("==> Unknown face")Let’s run our script:
The script was able to detect all faces with a match of more than 0.93.
Take it one step further with advanced search
Face recognition and search can be combined for advanced use cases. You can use Elasticsearch to create more complex queries like geo-queries, query-dsl-bool-query and search-aggregations…
For example, the following query applies a cosineSimilarity search to a specific location within a 200 km radius:
GET /_search
{
"query": {
"script_score": {
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": "200km",
"pin.location": {
"lat": 40,
"lon": -70
}
}
}
}
},
"script": {
"source": "cosineSimilarity(params.query_vector, 'face_encoding')",
"params": {
"query_vector":[
-0.14664565,
0.07806452,
0.03944433,
...
...
...
-0.03167224,
-0.13942884
]
}
}
}
}
}Combination cosineSimilarity with other requests, Elasticsearch gives you unlimited options for more complex use cases.
Conclusion
Face recognition can be useful in many cases, and you may already be using it in your daily life. The concepts described above can be generalized to any object detection in images or videos, so you can extend your use case to a very broad application.
Elasticsearch can help solve complex problems. Try this with Free 14-Day Elastic Cloud Trial, our official managed Elasticsearch offering, and let us know your opinion on our forums for discussion…
Learn more about the express course “Centralized logging systems Elastic stack”…
