DIY AR monitor in industry

Well, I value your opinion, so this article will focus on implementing a simple and inexpensive AR solution for displaying the parameters of a data collection system. If you are interested, then welcome to the cut!

❯ Home

Some elements of this article will overlap with the context of the previous one, so I urge you to read it in order to have a full understanding of what is happening. Thank you!

So, as I said earlier, the device discussed in the article is part of the “ecosystem” of a software package that collects enterprise technological data. Below I will try to describe the design of the device and the software implementation of data exchange between the device and the mobile application.

❯ Device housing

The device body does not have any complex design and looks like this:

To save time and not have to worry about adjusting the optical system, I used a ready-made solution as a base, to which I made my own modifications. Fortunately, after the presentation of the Google Glass headset in 2015, Indian comrades “produced” DIY versions of such cases. Below is a more detailed description of the device elements:

As you can see, the optical system consists of the following elements:

Projection glass;
Focusing lens;
Mirror.

Projection glass — one of the important elements of the device on which the quality of the projection depends. Below is a photo of the projection glass used:

As projection glass, it is necessary to use specialized glass with a metal coating to ensure effective reflection of the light of the projected image. To make such glass at home, you just need to delaminate a DVD, since CDs use a special coating to effectively reflect the laser beam from the readable surface. The presence of a coating on the glass can be determined by the metallic reflection when the glass is rotated.

Focusing lens — everything is simple here, this lens is necessary to form the focal length of the projected image in order to correctly combine the image of reality and projection. This lens was taken from a cheap VR box and was cut to fit the dimensions of the output “window”.

Mirror — here, not everything is so simple, as practice has shown, you cannot use an ordinary glass mirror as a mirror (as the Indians used). When using a conventional mirror, there is a large projection stratification due to double reflection. The best solution is to use a metal mirror, which is used in laser systems. Often these mirrors are quite expensive, but for a DIY solution a metal mirror made from an aluminum HDD “pancake” is quite suitable, which is what I put into practice.

As you can see in the image above, there are small obscuring elements on the body that look like an accordion. I introduced these elements into the design to combat the halo effect in the projection, since the light emanating from the display was reflected from the walls of the case.

❯ Electronics

The circuit diagram of the device is not complicated, I chose the ESP32 module as the “brains”, mostly due to the presence of a Bluetooth interface, and an inexpensive 0.66-inch OLED module with a resolution of 64×48 was chosen as the display. Why this particular module? — it is more compact than LCD and has higher pixel brightness. Below is a schematic diagram of the device:

To provide power in this prototype, a Li-on battery with a capacity of 250 mAh was used, and a popular board based on TP4056 was used as a charging module. Below you can see the layout of the electronics elements in the device case:

Front view:

❯ Software

The software functionality provides data exchange in JSON format between the installer’s smartphone and the AR monitor using the mobile application of the data collection system.

The AR monitor microsoftware was developed in the Arduino IDE and is not particularly complex. Below is the device code:

Main

#include <BLEDevice.h>
#include <BLEServer.h>
#include <BLEUtils.h>
#include <BLE2902.h>
#include <ArduinoJson.h>
#include <Wire.h> 
#include "SSD1306Wire.h"              // Использую русифицированную библиотеку дисплея

SSD1306Wire  display(0x3c, 4, 5);
#define batPin 34
#define DURATION 10000

int battery = 0;
float bat   = 0;
int count   = 0;
float data  = 0;
char *unit  = ""; 
char *leg  = ""; 
long timeSinceLastModeSwitch = 0;



BLEServer *pServer = NULL;
BLECharacteristic *pCharacteristic = NULL;
bool deviceConnected = false;
uint8_t txValue = 0;

#define SERVICE_UUID        "4fafc201-1fb5-459e-8fcc-c5c9c331914b"
#define CHARACTERISTIC_UUID "beb5483e-36e1-4688-b7f5-ea07361b26a8"

void send_json(String json){
      dsjson(json);
}

class MyServerCallbacks : public BLEServerCallbacks {
    void onConnect(BLEServer* pServer) {
      deviceConnected = true;
    }

    void onDisconnect(BLEServer* pServer) {
      deviceConnected = false;
    }
};



class MyCharacteristicCallbacks : public BLECharacteristicCallbacks {
    void onWrite(BLECharacteristic *pCharacteristic) {
        std::string rxValue = pCharacteristic->getValue();
        if (!rxValue.empty()) {
            send_json(rxValue.c_str());              // Обработка полученных данных
        }
    }
};

void setup() {
  display.init();
  display.flipScreenVertically();
  display.setFont(ArialMT_Plain_10);
  BLEDevice::init("AR Monitor");
  pServer = BLEDevice::createServer();
  pServer->setCallbacks(new MyServerCallbacks());
  
  BLEService *pService = pServer->createService(BLEUUID(SERVICE_UUID));
  pCharacteristic = pService->createCharacteristic(
                      BLEUUID(CHARACTERISTIC_UUID),
                      BLECharacteristic::PROPERTY_NOTIFY | BLECharacteristic::PROPERTY_WRITE
                    );

  pCharacteristic->addDescriptor(new BLE2902());
  pCharacteristic->setCallbacks(new MyCharacteristicCallbacks());
  
  pService->start();
  
  BLEAdvertising *pAdvertising = pServer->getAdvertising();
  pAdvertising->start();
}

void loop() { 
   process();
}

DataProcess

void process(){
     if (millis() - timeSinceLastModeSwitch > DURATION) {
     int anbat = map(analogRead(batPin), 0, 4095, 0, 420);
           bat = anbat*0.01;
     int bata  = bat*100;
       battery = map(bata, 257, 419, 0, 100);
    timeSinceLastModeSwitch = millis();
  }
   if (!deviceConnected) {
       display_text(18,"УСТРОЙСТВО ГОТОВО К ПОДКЛЮЧЕНИЮ","CYBRX","tech", battery);   
    }else{
       display_text(18, leg, String(data), unit, battery);
    } 
   
delay(10);
}

void dsjson(String json){
  StaticJsonDocument<200> doc;
  deserializeJson(doc, json);
 
  leg       = doc["legend"];  // Имя отображаемого параметра
  data      = doc["data"];    // Значение параметра
  unit      = doc["unit"];    // Единица измерения 
}

DisplayProcess

void display_text(int posY, String texts, String data_bt, String unit_bt, int bat_2){
     int co = texts.length()*6;
     int point;
     int positionLine;
     if(co > 120){
        count++;
        point = co - count;
        positionLine = point;
     if(count > co+60){
        count = 0;
        }
     }else { 
       positionLine = 64;
     }
    display.clear();
    display.setTextAlignment(TEXT_ALIGN_CENTER);
    display.setFont(Font5x7);
    display.drawString(positionLine, posY, texts);
    display.setFont(ArialMT_Plain_16);
    display.drawString(64, 32, data_bt);
    display.setFont(Font5x7);
    display.setTextAlignment(TEXT_ALIGN_RIGHT);
    display.drawString(96, 56, unit_bt);
    if(bat != 0){
    display.drawProgressBar(32, 56, 20, 6, bat_2);
       }
    display.display(); 
}

➤ Exchange in the mobile application

The application implements the following logic: The user does not need to manually add an AR device to display data; the application implements search and automatic connection of an AR monitor. This function is implemented in the following class:

BLE device search class

public class BleScanner {
    private BluetoothAdapter bluetoothAdapter;
    private BluetoothLeScanner bluetoothLeScanner;
    private boolean scanning;
    private ScanCallback scanCallback;
    private Handler handler;
    private ScanResultListener scanResultListener;

    public BleScanner() {
        bluetoothAdapter = BluetoothAdapter.getDefaultAdapter();
        bluetoothLeScanner = bluetoothAdapter.getBluetoothLeScanner();
        scanning = false;
        handler = new Handler(Looper.getMainLooper());
        setupScanCallback();
    }

    public void setScanResultListener(ScanResultListener listener) {
        this.scanResultListener = listener;
    }

    private void setupScanCallback() {
        scanCallback = new ScanCallback() {
            @SuppressLint("MissingPermission")
            @Override
            public void onScanResult(int callbackType, ScanResult result) {
                super.onScanResult(callbackType, result);
                BluetoothDevice device = result.getDevice();
                if (device.getName() != null) {                                                     // Пропускаем отправку в слушетель устройства не с именем AR Monitor
                    String dev;
                    try {
                        dev = convert_to_utf_8(device.getName());                                   // Проверяем на кириллицу
                    } catch (UnsupportedEncodingException e) {
                        throw new RuntimeException(e);
                    }
                    if (dev.equals("AR Monitor")) {                                             // Если нашли наше устройство, то отправляем его в слушатель
                         scanResultListener.onDeviceFound(device);
                         stopScan();
                    }
                }
            }

            @Override
            public void onBatchScanResults(List<ScanResult> results) {
                super.onBatchScanResults(results);
                // Handle batch scan results if needed
            }

            @Override
            public void onScanFailed(int errorCode) {
                super.onScanFailed(errorCode);
                // Handle scan failure
                scanResultListener.onScanFailed(errorCode);
            }
        };
    }

    @SuppressLint("MissingPermission")
    public void startScan() {
        if (!scanning && bluetoothLeScanner != null) {
            scanning = true;
            bluetoothLeScanner.startScan(scanCallback);
            handler.postDelayed(this::stopScan, 10000);                                    // Останавливаем сканирование после 10 сек
        }
    }

    @SuppressLint("MissingPermission")
    public void stopScan() {
        if (scanning && bluetoothLeScanner != null) {
            scanning = false;
            bluetoothLeScanner.stopScan(scanCallback);
        }
    }

    public interface ScanResultListener {
        void onDeviceFound(BluetoothDevice device);

        void onScanFailed(int errorCode);
    }

    private  String convert_to_utf_8(String data) throws UnsupportedEncodingException {
        String return_data = "";
        if(data !=null) {
            byte[] ptext = data.getBytes(getEncoding(data));
            return_data =  new String(ptext, StandardCharsets.UTF_8);;
        }
        return return_data;
    }
    public static String getEncoding(String str) {
        String encode = "GB2312";
        try {
            if (str.equals(new String(str.getBytes(encode), encode))) {
                return encode;
            }
        } catch (Exception ignored) {}
        encode = "ISO-8859-1";
        try {
            if (str.equals(new String(str.getBytes(encode), encode))) {
                return encode;
            }
        } catch (Exception ignored) {}
        encode = "UTF-8";
        try {
            if (str.equals(new String(str.getBytes(encode), encode))) {
                return encode;
            }
        } catch (Exception ignored) {}
        encode = "GBK";
        try {
            if (str.equals(new String(str.getBytes(encode), encode))) {
                return encode;
            }
        } catch (Exception ignored) {}
        return "";
    }
}

The device search is launched using the startScan() method and, if our AR monitor is nearby, it returns the MAC address of our device to initialize the connection. Next, the received MAC address is stored in the application memory. To work with BLE connection, the following class is implemented:

BLE connection management class

public class BLEManager {

    private static final UUID SERVICE_UUID = UUID.fromString("4fafc201-1fb5-459e-8fcc-c5c9c331914b");
    private static final UUID CHARACTERISTIC_UUID = UUID.fromString("beb5483e-36e1-4688-b7f5-ea07361b26a8");

    private final Context context;
    private BluetoothAdapter bluetoothAdapter;
    private BluetoothGatt bluetoothGatt;
    private BluetoothGattCharacteristic characteristic;
    private boolean connected;
    public BLEManager(Context context) {
        this.context = context;
        BluetoothManager bluetoothManager = (BluetoothManager) context.getSystemService(Context.BLUETOOTH_SERVICE);
        if (bluetoothManager != null) {
            bluetoothAdapter = bluetoothManager.getAdapter();
        }
    }

    @SuppressLint("MissingPermission")
    public void connectToDevice() {
        if (bluetoothAdapter == null || !bluetoothAdapter.isEnabled()) {
            return;
        }
        // Адрес ESP32
        String DEVICE_ADDRESS = new MySharedPreferences(context).getString("VrMAC", "00:00:00:00:00");
        if(!Objects.equals(DEVICE_ADDRESS, "00:00:00:00:00")) {
            BluetoothDevice device = bluetoothAdapter.getRemoteDevice(DEVICE_ADDRESS);
            bluetoothGatt = device.connectGatt(context, false, gattCallback);
        }
    }

    private final BluetoothGattCallback gattCallback = new BluetoothGattCallback() {
        @SuppressLint("MissingPermission")
        @Override
        public void onConnectionStateChange(BluetoothGatt gatt, int status, int newState) {
            super.onConnectionStateChange(gatt, status, newState);
            if (newState == BluetoothGatt.STATE_CONNECTED) {
                connected = true;
                gatt.discoverServices();
            } else if (newState == BluetoothGatt.STATE_DISCONNECTED) {
                connected = false;
            }
        }

        @Override
        public void onServicesDiscovered(BluetoothGatt gatt, int status) {
            super.onServicesDiscovered(gatt, status);
            if (status == BluetoothGatt.GATT_SUCCESS) {
                BluetoothGattService service = gatt.getService(SERVICE_UUID);
                characteristic = service.getCharacteristic(CHARACTERISTIC_UUID);
            }
        }
        @Override
        public void onCharacteristicChanged(BluetoothGatt gatt, BluetoothGattCharacteristic characteristic) {
            super.onCharacteristicChanged(gatt, characteristic);
            if (CHARACTERISTIC_UUID.equals(characteristic.getUuid())) {
                byte[] data = characteristic.getValue();
                String dataStr = new String(data);                                                  // Здесь можно обработать полученные данные
            }
        }
    };

    @SuppressLint("MissingPermission")
    public void sendData(String data) {
        if (bluetoothGatt != null && characteristic != null) {
            characteristic.setValue(data.getBytes());
            bluetoothGatt.writeCharacteristic(characteristic);

        }
    }

    @SuppressLint("MissingPermission")
    public void disconnect() {
        if (bluetoothGatt != null) {
            bluetoothGatt.disconnect();
            bluetoothGatt.close();
        }
    }
    public boolean isConnected() {
        return connected;
    }
}

To connect to a device, the connectToDevice() method is used, and to transfer data to the device, the sendData() method is used, where a string in JSON format is passed as an argument. Below is the transfer function for data to the device:

Data transfer function to AR monitor

private void sendToAr(String legend, float data, String unit){
    if(bleManager.isConnected) {
      JSONObject json = new JSONObject();
                 json.put("legend", legend);
                 json.put("data",     data);
                 json.put("unit",     unit);
      
      bleManager.sendData(json.toString()); // Отправка JSON по BLE
    }
  }

This function is implemented in Foreground Service in which a cyclic request for the required parameter is performed from the data acquisition system, and the received data is transmitted to the device using the function described above. Activation Foreground service in the application is performed using the “switch” element “Broadcasting data to AR device”

❯ Results

In this article, I tried to explain in a simplified way how this device and the software for its operation are implemented. As you can see, the device does not have any complex solutions and is quite accessible for implementation. Below is a list of hardware implementation costs:

Microcontroller ESP-32S – ＄2.26;
Display module SSD1306 – ＄2.06;
Li-po battery 250mAh – ＄2.04;
Charge module TP4056 – ＄1.12 (for 5 pcs);
Other components – ＄1;
Total cost of components: ~＄7.6.

Thank you to everyone who took the time to read this article and if you have any questions, welcome to the comments! All the best, success and interesting projects!

Testing of the first prototype in 2020

_{PS: This decision did not bypass my hobby: I have been riding a unicycle for a long time and decided to use this AR monitor to display telemetry, after adding a couple of classes to the WheelLog application for working with the device, I really liked the result.}