DIY Project Controls with Bluetooth Gamepad. Part 2 (ESP32)

The first part described how to connect a Bluetooth gamepad to Arduino. Then, slightly outdated, although still available, components were used. Now it's time to figure out how to do the same on the ESP32 platform.

This article is divided into two parts, and although the task is the same in both, the methods for implementing it differ depending on the chosen platform.

Part 1 (Arduino) – the first part is recommended for reading, it tells what it is USB Host Shield and how to use it, this library will be needed on ESP32 too

Part 2 (ESP32) – you are here

Next there will be a lot of text and not a single picture – almost the entire second part is devoted to the actual process of porting the library to the ESP32. Therefore, if you just need a ready-made code for embedding in the project – scroll down, there will be links to the ready-made library and an example of use.

What Bluetooth is in ESP32

In short, in Bluetooth, in addition to different versions, there are also two different standards:

You can read more about the differences between these Bluetooth standards, for example, in the article Bluetooth Low Energy: A Detailed Guide for Beginners. Some Bluetooth devices can support only one of the standards, while others can support both at the same time (Dual mode Bluetooth).

ESP32 variants also come in different versions:

The first ones are the most common, and if you already have an ESP32, then it is very likely that it will support Dual mode and therefore it will be suitable for connecting a gamepad, the vast majority of which support only Bluetooth Classic.

Bluetooth API in ESP32

It's worth making a note here that development for ESP32 can be done using two different frameworks (espruino I am deliberately not including in this list):

• ESP-IDF (Espressif IoT Development Framework) – a native ESP32 framework that can be used from the command line (CLI) or via convenient IDE plugins (VSCode and Eclipse)

• ESP32 Arduino Core – An Arduino-compatible shell over ESP-IDF, allowing you to write code in the familiar Arduino style and using familiar functions from standard libraries. ESP32 Arduino Core can be used with both the regular Arduino IDE and VSCode or PlatformIO

The second option, in principle, allows you to implement most of the most common scenarios for Arduino projects, however, it should be taken into account that in terms of functionality it still loses to the first option – not everything that is in the ESP32 is, in principle, present in the form of a standard Arduino API, so sometimes you will still have to use the API from the ESP-IDF, which will lead to code in a mixed style. For example, I do not really like this neighborhood of xTaskCreate and loop in one project.

Bluetooth support in the ESP32 Arduino Core mainly comes down to classes for working with BLE services, and for Bluetooth Classic there is only an implementation of BluetoothSerial.

Therefore, to fully work with Bluetooth, you need to start developing using ESP-IDF. After the usual Arduino IDE, of course, it takes some time to set up the environment, study the documentation minimally and get used to the features of the multitasking FreeRTOS, but all this is not at all difficult – ESP-IDF still remains a fairly high-level framework, when using which you will not have to understand the intricacies of the microprocessor registers, and your code will not half consist of assembler inserts. The documentation for ESP-IDF is present in sufficient volume and high-quality form directly from the vendor.

So, what opportunities for working with Bluetooth does it provide? ESP-IDF ?

• Bluedroid stack, supports both BLE and Bluetooth Classic

• Nimble stack, supports only BLE

• VHCI (Virtual Host Controller Interface) – low-level access to the Bluetooth controller using HCI commands, it's something like a standardized API at the Bluetooth chip level

Although the first two stacks are considered high-level Bluetooth APIs, to use them you will have to understand in detail many of the subtleties of Bluetooth, including various GATT/GAP/SDP/L2CAP and so on. Of course, there are examples in ESP-IDF for all of the listed APIs, but these examples are so abstract that they do not make things any easier – there is no ready-made code to quickly connect a Bluetooth device to ESP32.

Google suggests several more possible options for solving the problem:

• BlueKitchen BTstack – another implementation of the BT stack, opensource, there are ports for many processor architectures (including ESP32), free use for non-commercial purposes is possible. This version of the Bluetooth API looks simpler than Bluedroid, there is even a ready-made example with connecting a keyboard, from which, probably, a more complex version for a gamepad can be made. This option still does not look fast

• There is a project on GitHub (https://github.com/aed3/PS4-esp32) – 300+ stars, which is not surprising, since this is the only working and ready-to-use example of connecting DualShock 4 to ESP32. In this example, there is a hardcoded BT address of the device for connection and there is no full pairing support.

• Second project (https://github.com/StryderUK/BluetoothHID) is more advanced, but the repository is clearly abandoned, requires replacing files inside the SDK, and is tied to a specific, already old, version of the SDK, which I would not like to develop on. I have not checked how this example will behave with the current version of the SDK. Dealing with potential issues of backward compatibility of ESP-IDF versions is also not a quick option.

All the above-described implementation options are completely inferior to that old library in terms of speed and convenience. USB Host Shield 2.0which was used in the first part of the article for the classic Arduino board. So I wondered, is there a way to simply transfer all those Bluetooth developments that are there to the ESP32?

Porting Bluetooth stack from USB Host Shield 2.0 to ESP32

First, I looked into the library source code. The main part of the library source code, as expected, implements support for various USB devices – wired keyboards, mice, flash drives, and so on. Working with Bluetooth comes down to connecting a special type of USB device – Bluetooth Dongle (B.T.D.). The processing of data packets for BTD is wrapped in a kind of state machine, which is essentially a simplified implementation of the Bluetooth stack, but still sufficient for connecting simple BT devices, including gamepads. The main thing I found was that all the code for exchanging data packets contains the abbreviation HCI throughout. For those who have been working with Bluetooth at a low level for a long time, this is probably an obvious conclusion, but for me it was a godsend, meaning that it was possible to take the code from USB Host Shield 2.0, along with all the BT device support it had, and port it to the ESP32, simply replacing the HCI command exchange via the USB controller with API calls. VHCI ESP32.

Upon closer examination of the source code, a couple of peculiarities were discovered:

1. The Bluetooth state machine itself reads packets from the USB input buffer, i.e. does Polling, and immediately processes these packets

2. Sending outgoing packets towards the BT device in the library occurs directly from the state machine, right inside the methods that are busy processing incoming packets.

As it turned out, such a scheme of work without changes cannot be transferred to ESP32:

1. There is no Polling mechanism in ESP32, the VHCI interface delivers incoming data packets to the callback function.

2. VHCI methods for sending packets do not work if called from within data receiving callbacks

To avoid completely rewriting the Bluetooth state machine, when porting the library to ESP32 I did the following:

1. The callback function for reading incoming packets from BT does not process them – instead, all packets are saved in a separate ringbuffer (in fact, in two different buffers – one for HCI packets, the second for ACL packets).

2. A separate thread reads packets from ring buffers and delivers them to state machine methods. In this case, when handlers start calling data sending functions, it turns out that this happens outside the reading callbacks.

As a result, the new scheme turned out to be quite efficient and fast. At least, the ESP32 manages to process all 1000 packets per second that the DualShock 4 sends with data on the state of the gamepad buttons and joysticks.

Then there were cosmetic edits such as replacing debug logging with the esp_log library adopted in ESP32, adding the missing #include and #define, after which the new component (this is the name of the “library” adopted in ESP-IDF) was ready.

BTD_VHCI component for ESP-IDF

You can download the finished component from here – btd_vhci

If you do not have ESP-IDF installed yet, you need to do so.

In the project, you need to make the necessary sdkconfig settings (done via F1: ESP-IDF: SDK Configuration Editor)

Hidden text

Required options in sdkconfig:

#
# Bluetooth
#
CONFIG_BT_ENABLED=y
CONFIG_BT_CONTROLLER_ONLY=y
CONFIG_BT_CONTROLLER_ENABLED=y
#
# Controller Options
#
CONFIG_BTDM_CTRL_MODE_BR_EDR_ONLY=y
CONFIG_BTDM_CTRL_BR_EDR_MAX_ACL_CONN=2
CONFIG_BTDM_CTRL_BR_EDR_MAX_SYNC_CONN=0
CONFIG_BTDM_CTRL_BR_EDR_SCO_DATA_PATH_HCI=y
CONFIG_BTDM_CTRL_BR_EDR_SCO_DATA_PATH_EFF=0
CONFIG_BTDM_CTRL_PCM_ROLE_EFF=0
CONFIG_BTDM_CTRL_PCM_POLAR_EFF=0
CONFIG_BTDM_CTRL_LEGACY_AUTH_VENDOR_EVT=y
CONFIG_BTDM_CTRL_LEGACY_AUTH_VENDOR_EVT_EFF=y
CONFIG_BTDM_CTRL_BLE_MAX_CONN_EFF=0
CONFIG_BTDM_CTRL_BR_EDR_MAX_ACL_CONN_EFF=2
CONFIG_BTDM_CTRL_BR_EDR_MAX_SYNC_CONN_EFF=0
CONFIG_BTDM_CTRL_PINNED_TO_CORE_0=y
CONFIG_BTDM_CTRL_PINNED_TO_CORE=0
CONFIG_BTDM_CTRL_HCI_MODE_VHCI=y

The same in Menuconfig:

Next, you need to add a dependency to the project btd_vhci:

• Method 1: via IDF Component Manager, run the command idf.py add-dependency -pink0d/btd_vhci (the command is run in the terminal, which can be opened via F1: ESP-IDF: Open ESP-IDF Terminal)

• Method 2: Copy the contents of the Github repository to a directory components\btd_vhci inside the project and add manually REQUIRES btd_vhci nvs_flash in CMakeLists.txt for the main component

To use the C++ classes from the library, you should rename main.c to main.cpp and update the file name in CMakeLists.txt component main, and add a modifier to the entry point of the application extern "C"

What else should be in the project and inside app_main:

• Global instance of the class for receiving data from the BT device. In the example below it is PS4BT PS4;

• nvs_flash_init() – initialization of flash memory for internal needs of the ESP32 Bluetooth controller

• btd_vhci_init() library initialization

• xTaskCreatePinnedToCore(...) start main task

• btd_vhci_autoconnect(...) launching an automatic connection with a BT device. When launched, this task tries to find a previously saved BT device address, and if such an address was previously saved in the ESP32 flash memory, the Bluetooth controller will begin to wait for a connection with it. If after 30 seconds the connection is not established, or if there was no saved address at all, the Bluetooth controller will go into pairing mode. If pairing is successful, the address of the connected device will be written to the flash memory

Main task code:

• Must contain btd_vhci_mutex_lock(); And btd_vhci_mutex_unlock(); when accessing library classes because they are not thread safe

• The example below performs a simple poll of the PlayStation 4 controller status and outputs the data to the console.

An example in the form of a finished project is on GitHub – btd_vhci_examples_ESP-IDF

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "nvs_flash.h"

#include "PS4BT.h"
#include "btd_vhci.h"

PS4BT PS4;

bool printAngle, printTouch;
uint8_t oldL2Value, oldR2Value;

static const char *LOG_TAG = "main";

// print controller status
void ps4_print() {
    if (PS4.connected()) {
        if (PS4.getAnalogHat(LeftHatX) > 137 || PS4.getAnalogHat(LeftHatX) < 117 || PS4.getAnalogHat(LeftHatY) > 137 || PS4.getAnalogHat(LeftHatY) < 117 || PS4.getAnalogHat(RightHatX) > 137 || PS4.getAnalogHat(RightHatX) < 117 || PS4.getAnalogHat(RightHatY) > 137 || PS4.getAnalogHat(RightHatY) < 117) {
        ESP_LOGI(LOG_TAG, "L_x = %d, L_y = %d, R_x = %d, R_y = %d",
                    PS4.getAnalogHat(LeftHatX),PS4.getAnalogHat(LeftHatY),
                    PS4.getAnalogHat(RightHatX),PS4.getAnalogHat(RightHatY));
        }

        if (PS4.getAnalogButton(L2) || PS4.getAnalogButton(R2)) { // These are the only analog buttons on the PS4 controller
        ESP_LOGI(LOG_TAG, "L2 = %d, R2 = %d",PS4.getAnalogButton(L2),PS4.getAnalogButton(R2));
        }
        if (PS4.getAnalogButton(L2) != oldL2Value || PS4.getAnalogButton(R2) != oldR2Value) // Only write value if it's different
        PS4.setRumbleOn(PS4.getAnalogButton(L2), PS4.getAnalogButton(R2));
        oldL2Value = PS4.getAnalogButton(L2);
        oldR2Value = PS4.getAnalogButton(R2);

        if (PS4.getButtonClick(PS))
        ESP_LOGI(LOG_TAG, "PS");
        if (PS4.getButtonClick(TRIANGLE)) {
        ESP_LOGI(LOG_TAG, "Triangle");
        PS4.setRumbleOn(RumbleLow);
        }
        if (PS4.getButtonClick(CIRCLE)) {
        ESP_LOGI(LOG_TAG, "Circle");
        PS4.setRumbleOn(RumbleHigh);
        }
        if (PS4.getButtonClick(CROSS)) {
        ESP_LOGI(LOG_TAG, "Cross");
        PS4.setLedFlash(10, 10); // Set it to blink rapidly
        }
        if (PS4.getButtonClick(SQUARE)) {
        ESP_LOGI(LOG_TAG, "Square");
        PS4.setLedFlash(0, 0); // Turn off blinking
        }

        if (PS4.getButtonClick(UP)) {
        ESP_LOGI(LOG_TAG, "UP");
        PS4.setLed(Red);
        } if (PS4.getButtonClick(RIGHT)) {
        ESP_LOGI(LOG_TAG, "RIGHT");
        PS4.setLed(Blue);
        } if (PS4.getButtonClick(DOWN)) {
        ESP_LOGI(LOG_TAG, "DOWN");
        PS4.setLed(Yellow);
        } if (PS4.getButtonClick(LEFT)) {
        ESP_LOGI(LOG_TAG, "LEFT");
        PS4.setLed(Green);
        }

        if (PS4.getButtonClick(L1))
        ESP_LOGI(LOG_TAG, "L1");
        if (PS4.getButtonClick(L3))
        ESP_LOGI(LOG_TAG, "L3");
        if (PS4.getButtonClick(R1))
        ESP_LOGI(LOG_TAG, "R1");
        if (PS4.getButtonClick(R3))
        ESP_LOGI(LOG_TAG, "R3");

        if (PS4.getButtonClick(SHARE))
        ESP_LOGI(LOG_TAG, "SHARE");
        if (PS4.getButtonClick(OPTIONS)) {
        ESP_LOGI(LOG_TAG, "OPTIONS");
        printAngle = !printAngle;
        }
        if (PS4.getButtonClick(TOUCHPAD)) {
        ESP_LOGI(LOG_TAG, "TOUCHPAD");
        printTouch = !printTouch;
        }

        if (printAngle) { // Print angle calculated using the accelerometer only
        ESP_LOGI(LOG_TAG,"Pitch: %lf Roll: %lf", PS4.getAngle(Pitch), PS4.getAngle(Roll));        
        }

        if (printTouch) { // Print the x, y coordinates of the touchpad
            if (PS4.isTouching(0) || PS4.isTouching(1)) // Print newline and carriage return if any of the fingers are touching the touchpad
                ESP_LOGI(LOG_TAG, "");
            for (uint8_t i = 0; i < 2; i++) { // The touchpad track two fingers
                if (PS4.isTouching(i)) { // Print the position of the finger if it is touching the touchpad
                ESP_LOGI(LOG_TAG, "X = %d, Y = %d",PS4.getX(i),PS4.getY(i));          
                }
            }
        }
    }
}

void ps4_loop_task(void *task_params) {
    while (1) { 
        btd_vhci_mutex_lock();      // lock mutex so controller's data is not updated meanwhile
        ps4_print();                // print PS4 status
        btd_vhci_mutex_unlock();    // unlock mutex
        vTaskDelay(1);
    }
}

extern "C" void app_main(void)
{
    esp_err_t ret;

    // initialize flash
    ret = nvs_flash_init();
    if (ret == ESP_ERR_NVS_NO_FREE_PAGES || ret == ESP_ERR_NVS_NEW_VERSION_FOUND) {
        ESP_ERROR_CHECK(nvs_flash_erase());
        ret = nvs_flash_init();
    }
    ESP_ERROR_CHECK( ret );

    // initilize the library
    ret = btd_vhci_init();
    if (ret != ESP_OK) {
        ESP_LOGE(LOG_TAG, "BTD init error!");
    }
    ESP_ERROR_CHECK( ret );

    // run example code
    xTaskCreatePinnedToCore(ps4_loop_task,"ps4_loop_task",10*1024,NULL,2,NULL,1);

    // run auto connect task
    btd_vhci_autoconnect(&PS4);

    while (1) {       
        vTaskDelay(pdMS_TO_TICKS(100));
    }
    // main task should not return
}

This is where the library's functionality ends, leaving room for DIY projects.

So far I've only managed to transfer keyboard-mouse classes (BTHID) and classes for the most common controllers (PS4, PS5, Xbox) to the library, and I haven't even managed to test all of this with real devices. Of course, I'd like to transfer the Serial Port Profile (SPP), but it's already noticeably more complicated, and will also require a general rethink of multithreading and thread safety within the library.

Using BTD_VHCI from ESP32 Arduino Core

Using the same code with the ESP32 Arduino Core framework is theoretically possible too. I must warn you that it will not be any easier or faster than switching to ESP-IDF straight away.

The reason is that the ESP32 Arduino Core consists of a ready-made set of libraries, pre-compiled with the default sdkconfig, which includes the Bludroid stack, and there seems to be no access directly to the VHCI functions. Espressif provides the ability to customize the config through a tool called Library Builderthat is, the ability to install specific options for ESP32 still exists. To do this, in addition to installing the ESP32 Arduino Core in the Arduino IDE, you need to perform a whole sequence of steps:

1. Install ESP-IDF and Library Builder

2. Change the necessary options in sdkconfig, which is used when building libraries

3. Build your own custom ESP32 ArduinoCore build (as described in the documentation, it takes several hours)

4. Replace the sdk downloaded via Board Manager in Arduino IDE with your custom build

I haven't gone this route yet, so I won't give a detailed description in the current article, which has already turned out to be too voluminous. Perhaps, if there is interest in this topic, I will still check the compatibility of the library with the rebuilt ESP32 Arduino Core and make an addition in the form of the third part of the article.