How I created a device that can remotely emulate a mouse and keyboard on stm32
1. How did the idea come about?
Hello everyone, I just recently started studying the protocol USB on STM32F103C8namely HID devices. I'm the kind of person who doesn't really like theory, but loves to learn everything in practice, so I immediately started thinking about a future project. And I remembered that I recently ordered a wifi module for myself – ESP8266.
And then an idea came to me: why not combine STM32f103c8 and ESP8266 into one device – USBDeviceHacker (that's what I called him). My idea was to control the mouse and keyboard remotely using an api that would run on a web server running on the ESP. Accordingly, ESP will act as master devices, and STM32 in the role slave.
Many may ask: why not make everything much simpler and instead wifi use bluetoothhowever use wifi will be much more interesting, because our web server will be deployed on a local network, and, accordingly, each device in it will be able to control the device simply by sending HTTP request for the required ip.
2. Project model
Before starting implementation, you need to describe the operating model of the device, I’ll start with ESP8266. The Wifi module connects to an external access point and creates a web server on its local network. In its api 8 endpoint (POST), each of which is responsible for its own functionality.
Points /mouse And /keyboard allow you to directly access the mouse and keyboard, i.e. use according to their official documentation. Dot /animation already uses ready-made functionality that makes working with the device easier.
After sending a correct request ESP8266 forms a package and sends it via UART directly to STM32F103C8. Blue pill accepts data and checks it for correctness, if everything is good, then it sends a report to USB.
The complete architecture of the project can be viewed in miro.
3. ESP8266 implementation
I wrote the firmware for ESP8266 using PlathormIO, in Visual Studio Code. I will not dwell too much on explaining the code, but will simply tell everything in general terms, otherwise the article may become very lengthy. In any case, you can view everything yourself, the repository is public, here link.
In general, the project structure looks like this:
I designed the working folder as follows: it consists of 4 folders: configs, controllers, modules, routes, and files main.h And main.cpp. Function InitRouter() creates endpoints using the library ESP8266WebServer.hand redirects them to controllers. For controllers, instead of classes, I decided to use namespace, simply because it is clearer and more convenient.
void InitRouter(){
server.on("/mouse/set", Mouse::set);
server.on("/mouse/remove", Mouse::remove);
server.on("/mouse/click", Mouse::click);
server.on("/keyboard/set", Keyboard::set);
server.on("/keyboard/remove", Keyboard::remove);
server.on("/keyboard/click", Keyboard::click);
server.on("/animation/set", Animation::set);
server.on("/animation/remove", Animation::remove);
}And then all the main actions take place in the controllers. Here is an example controller Animation.h:
For JSON parsing I use the library ArduinoJson.h. It is very convenient and greatly facilitates the work; with it you can easily parse, filter data and create JSON objects yourself.
The package for STM32 is formed in an array data and sent using the function SendDataWithWait(uin8_t *data, uint8_t len) class serial. To check the correctness of the data, a checksum is added to the last byte of the package. crc8.
string SendDataWithWait(uint8_t *data, uint16_t len){
uint8_t data_out[len+2] = {HEADER,};
for(int i = 0; i<len; i++){
data_out[i+1] = data[i];
}
data_out[len+1] = crc8(data_out, len+1);
Serial.write(data_out, len+2);
if(AwaitResponse()){
if (status){
status = false;
return "ok";
}
return "error";
}
return "timeout";
}ESP8266 waits for response from STM32 in function AwaitResponse()and then sends a response to the client. All work with the client part is implemented in the file ClientProcessingModule.h.
One of the problems when creating a web server is its dynamic ip. When connected to a router or modem, the device is given ipwhich changes over time. Because of this, you have to constantly connect the ESP8266 to the serial port and watch it manually. There can be two options to solve the problem: 1st, if the device connects to the router, you can configure static ip; 2nd send dynamic ip to the web server from static ip.
I don’t currently use any of the solutions, simply because I don’t need it yet.
4. Connection
To get started with STM32, you need to understand how to connect the ESP8266 to it. Since the blue pill will be connected via USB, the ESP will be powered from it. The VCC and CH_EN pins are connected to the 3.3V pin on the STM32, GND to GND, RX to B10, TX to B11 and that's it.
On STM32 pins B10 and B11 are TX and RX of USART3 respectively.
5. Implementation of STM32f103C8
I wrote the firmware for STM in CubeIDE. The code for it is very simple, it is written entirely in C. The protocols connected are UASRT3(HAL), which receives requests from the ESP, and, in fact, USB, which is configured as an HID device. Also added LED on PC13.
The project structure consists of 3 folders: inits, modules, controllers. Data from USART3 is received in a file UsartController.cV HAL_UART_RxCpltCallbackand they are received one at a time and collected in an array bufferafter which the function is called ParsingDatawhich checks for correctness using crc8and fills the structure Action verified data.
After receiving the parcel, the desired case V main.cin which the data processing function is launched, where the main work happens. Next, the data, depending on the device and command, forms a report, which is sent via USB.
In order to send reports from both mice and keyboards at the same time, you need to use the function HID_MOUSE_ReportDescwhich is in the file usbd_hid.cwrite down the following instructions:
0x05, 0x01, /* Usage Page (Generic Desktop) */
0x09, 0x02, /* Usage (Mouse) */
0xA1, 0x01, /* Collection (Application) */
0x09, 0x01, /* Usage (Pointer) */
0xA1, 0x00, /* Collection (Physical) */
0x85, 0x01, /* Report ID */
0x05, 0x09, /* Usage Page (Buttons) */
0x19, 0x01, /* Usage Minimum (01) */
0x29, 0x03, /* Usage Maximum (03) */
0x15, 0x00, /* Logical Minimum (0) */
0x25, 0x01, /* Logical Maximum (0) */
0x95, 0x03, /* Report Count (3) */
0x75, 0x01, /* Report Size (1) */
0x81, 0x02, /* Input (Data, Variable, Absolute) */
0x95, 0x01, /* Report Count (1) */
0x75, 0x05, /* Report Size (5) */
0x81, 0x01, /* Input (Constant) ;5 bit padding */
0x05, 0x01, /* Usage Page (Generic Desktop) */
0x09, 0x30, /* Usage (X) */
0x09, 0x31, /* Usage (Y) */
0x15, 0x81, /* Logical Minimum (-127) */
0x25, 0x7F, /* Logical Maximum (127) */
0x75, 0x08, /* Report Size (8) */
0x95, 0x02, /* Report Count (2) */
0x81, 0x06, /* Input (Data, Variable, Relative) */
0xC0, 0xC0,/* End Collection,End Collection */
//
0x09, 0x06, /* Usage (Keyboard) */
0xA1, 0x01, /* Collection (Application) */
0x85, 0x02, /* Report ID */
0x05, 0x07, /* Usage (Key codes) */
0x19, 0xE0, /* Usage Minimum (224) */
0x29, 0xE7, /* Usage Maximum (231) */
0x15, 0x00, /* Logical Minimum (0) */
0x25, 0x01, /* Logical Maximum (1) */
0x75, 0x01, /* Report Size (1) */
0x95, 0x08, /* Report Count (8) */
0x81, 0x02, /* Input (Data, Variable, Absolute) */
0x95, 0x01, /* Report Count (1) */
0x75, 0x08, /* Report Size (8) */
0x81, 0x01, /* Input (Constant) ;5 bit padding */
0x95, 0x05, /* Report Count (5) */
0x75, 0x01, /* Report Size (1) */
0x05, 0x08, /* Usage Page (Page# for LEDs) */
0x19, 0x01, /* Usage Minimum (01) */
0x29, 0x05, /* Usage Maximum (05) */
0x91, 0x02, /* Output (Data, Variable, Absolute) */
0x95, 0x01, /* Report Count (1) */
0x75, 0x03, /* Report Size (3) */
0x91, 0x01, /* Output (Constant) */
0x95, 0x06, /* Report Count (1) */
0x75, 0x08, /* Report Size (3) */
0x15, 0x00, /* Logical Minimum (0) */
0x25, 0x65, /* Logical Maximum (101) */
0x05, 0x07, /* Usage (Key codes) */
0x19, 0x00, /* Usage Minimum (00) */
0x29, 0x65, /* Usage Maximum (101) */
0x81, 0x00, /* Input (Data, Array) */
0xC0 /* End Collection,End Collection */Where the report id of the mouse will be 0x01, the report id of the keyboard will be 0x02, i.e. For example, data for a mouse will need to be sent in this format:
uint8_t data_out[5] = {0x01, 0, 0, 0, 0}; // первый байт - report id
USBD_HID_SendReport(&hUsbDeviceFS, data_out, 5);Special attention needs to be paid animationsbecause they add ready-made functionality to the device. There are currently 3 animations available:
1) smooth mouse movement – the mouse moves smoothly along the x or y axis, you can set the direction and speed of movement from 1 to 10 inclusive;
2)automatic typing (only Latin characters, numbers and space work) – alternate text typing, you can set the text and repetitions of its writing, for example, type “Hello World” every 10 seconds.
3)move the mouse in a circle – the mouse simply moves in a circle, you can set the speed from 1 to 10 inclusive and the radius of movement.
Also, animations 1 and 3 can be set to the status of pressing mouse buttons, for example, let the mouse move in a circle with RMB or LMB pressed, or all at once. There are three statuses available: LMB, RMB and mouse wheel button.
API documentation here.
6. Tests
Now we come to the most interesting part, namely the tests. Connect the device to the PC (laptop) and wait for the ESP to connect to the modem. I distributed the access point from my phone, and also connected to it from my laptop to send requests from it. As a result, I will start from the laptop, and stop from the phone, sending a request to the endpoint /remove.
Here is a link to a video with tests:
That's all, I think, it turned out to be a very interesting project that definitely deserved to spend time and effort on it. I leave a link to public git repositorywhere you can get acquainted with all the details. Subscribe to my tg channelin which I publish information about my projects and life in general. I wish you all good luck, I hope to see you again.
