SCARA robot sorts colored caps

In the company's laboratory Insystem We often use our framework ODRS(more details here) to solve various problems in the field of computer vision. One of such problems was the setup SCARA robot for sorting colored plastic bottle caps. This project was driven by the need to automate the recycling process, increasing its efficiency and accuracy. Using ODRS, we were able to train the robot to recognize caps of different colors and then sort them into categories. The following tasks were completed during the work:

• Collection of a dataset consisting of annotated photographs of colored bottle caps on a conveyor belt.

• Training a neural network detector using the ODRS framework.

• Integrating the detector into the robot program

Collecting the dataset

One of the key stages in solving computer vision problems is collecting a dataset, and our task was no exception. To capture images, we used a camera suspended above a conveyor belt. In the future, such a camera and similar scenes will be used to feed data into a neural network detector.

Main characteristics:

• Categories: The dataset includes 10 different colors of bottle caps.

• Color Labels:

  • Blue

  • Brown

  • Green

  • Purple

  • Orange

  • Pink

  • Purple

  • Red

  • White

  • Yellow

Stages of dataset collection and preparation:

1. Photographing lids: Caps of different colors are placed on a conveyor belt and photographed from different angles and lighting conditions to produce a variety of images.

2. Annotating images: Each image goes through an annotation process where the positions and colors of the caps are specified. This is done so that the model can accurately determine the location and color of each object in the image.

   

3. Data verification and validation: Annotations are checked for accuracy and completeness to ensure that all features are correctly labeled and the data is ready to be used to train models.

4. Formation of the final dataset: The finished images with annotations are collected into a single dataset, which is then used to train and test machine learning models.

   In the end, 1000 photographs were collected, containing more than 15,000 objects. To expand the set, arguments were used, which eventually amounted to 2400 images:

   • 50% chance of horizontal reflection

   • Random exposure adjustment in the range from -24% to +24%

   • Randomly applies a Gaussian blur with a radius between 0 and 5.25 pixels

   • Applying salt and pepper noise to 1% of pixels

Training the detector

A well-collected dataset is already 90% of the success, but choosing the right model is also important. To quickly select and train the optimal detector, we used our recommendation framework ODRS. Initially, after analyzing the images, a list of recommendations was formed:

Based on the list of recommendations, a model was selected YOLOv5l. Using tools ODRS we trained this detector, the results are described below:

We managed to achieve a high mAP0.5 ~ 0.94 value for almost all classes. Unfortunately, due to the insufficient number of pink and yellow caps when collecting the dataset, the metrics for these objects were significantly lower. However, this is quite enough to implement the first version of our project and conduct tests. The notebook with the code can be found at kaggle.

Integrating the detector into the robot program

A simple but effective script was developed for integration into the robot program. This script processed the inference results, determining the class of the object and calculating the coordinates of the center of the bounding box. Then this data was transmitted to the robot via the Modbus protocol. Thus, the robot received the necessary information for precise positioning and performing its tasks.

Summary

In a short period of time, we managed to collect a dataset, train the detector and integrate it into the robot program. We will be glad to likes and stars, thank you! Useful links:

1. Dataset with caps

2. Laptop with code

3. Github ODRS