# A simple example of AI to control a robot. TensorFlow + Node Js

A few words about me: my hobby is robotics. At the moment I am experimenting with a walking robot based on SunFounder PiCrawler.

Recently, the topic of artificial intelligence (AI) has become increasingly popular. The reason for this is, among other things, the improvement of mobile devices and computers – they are becoming more powerful and compact.

In this article, I will try to explain in simple words how you can use AI to control a robot using the ready-made TensorFlow library.

## What is AI?

In short, it is a data processing tool built on the basis of a neural network. To create our own AI, we decide which input parameters we process and which we expect as output. Then we collect a huge amount of this data, be it images or something else, and “train” the neural network based on it. As a result, we get a model that will process our values.

## A little about the PiCrawler robot

This is a device purchased on AliExpress. To control it, you need a Raspberry Pi 4 mini computer. The robot has four legs, an expansion board connected via an i2c interface, a camera, an ultrasonic sensor and various outputs.

I connected a WT901 mini-gyro (any other one), also purchased on the Chinese marketplace and connected via the i2c interface, to this robot. There is software for controlling the legs.

## Formulation of the problem

As an example, I will take the following robot control task – to maintain a horizontal position depending on the angle of inclination of the supporting surface.

Since there is a gyroscope inside the robot, we will read the angles of inclination from it `x, y` and, based on this data, determine the position of the legs `y1, y2, y3, y4`,

Where `x, y, y1, y2, y3, y4` – numbers.

The model API uses Tensors – arrays with numeric values ​​- as input variables. The result is a similar situation.

In our case, the input data format will be `[x, y]`and weekends `[y1, y2, y3, y4]`

In picture form, this model can be represented as follows:

## Writing code for the TensorFlow model

So what we have:

• Input data in format `[x, y]`

• Output data in format `[y1, y2, y3, y4]`

Based on this, you can already build a simple layers model in Tensorflow – as input parameters, an array of two elements is `inputShape: [2]`output – `units: 4` which corresponds to an array of four elements.

``````const tf = require("@tensorflow/tfjs-node");
const model = tf.sequential();

model.add(tf.layers.dense({ inputShape: [2], units: 32, activation: "relu" }));
model.add(tf.layers.dense({ units: 32, activation: "relu6" }));
model.add(tf.layers.dense({ units: 32, activation: "relu6" }));
model.add(tf.layers.dense({ units: 4, activation: "relu" }));

model.summary();
model.compile({
optimizer: tf.train.sgd(0.0001),
loss: "meanSquaredError",
});``````

And so, our model is ready! Isn’t it simple?

## Collection of training data. Model training.

What is training? This is the process of processing various variables and their further analysis to identify any patterns. It sounds a little confusing, but let’s understand it using our example.

What data will we collect and process? These are all the same model parameters – tilt angles and Y coordinates for the robot’s legs. But when collecting data, we do the opposite – we set the corresponding coordinates of the robot’s legs and record the angles of inclination. In other words, we tilt the robot by changing its leg positions and remembering the angles of inclination. Here it is important to analyze how and in which direction to tilt it.

Let’s agree that we have objects:

• `gyroScope` – to receive data from the gyroscope. Let there be a method `getData()` which returns an object `{ X, Y }` with tilt angles.

• `leftArm, rightArm, leftArmBack, rightArmBack` – objects with a method `setCoords` to set the coordinates of the robot’s legs. We are interested in the coordinate along the Y axis

Let’s record our first data!

Everything is simple here – this is a neutral position.

Y – leg coordinates [-80, -80, -80, -80] – Gyroscope data [0, 0].

Next, changing the coordinates of the front legs, we tilt the robot, for example in increments of 10-20 mm.

Leg coordinates – [-50, -50, -80, -80]gyroscope data [8, 0].

Leg coordinates – [-30, -30, -80, -80]gyroscope data [14, 0].

Now we tilt it in the other direction.

To do this, change the coordinates of the opposite legs.

Leg coordinates – [-80, -80, -50, -50]gyroscope data [-7, 0].

Leg coordinates – [-80, -80, -30, -30]gyroscope data [-12, 0].

And so on, changing the coordinates of the legs, we write down the resulting angles of inclination.

We save this data in a separate file as an array

``````const x_train = [
[0, 0],
[8, 0],
[14, 0],
[-7, 0],
[-12, 0]
//...... и множество остальных данных
];

const y_train = [
[-80, -80, -80, -80],
[-50, -50, -80, -80],
[-30, -30, -80, -80],
[-80, -80, -50, -50],
[-80, -80, -30, -30],
//...... и множество остальных данных
];``````

So we have a set of training data.

Let’s start training the model

To train the model, you need to write the following code:

``````model
.fit(tf.tensor2d(x_train), tf.tensor2d(y_train), {
epochs: 300,
batchSize: 4,
callbacks: {
onEpochEnd(e, l) {
console.log(e, l);
},
},
});``````

``````onEpochEnd(e, l) {
console.log(e, l);
},``````

so that you can see the current loss of accuracy in the logs. This will come in handy later.

And so, let’s run our script `node model.js`

You can see the following logs in the console:

``````Epoch 298 / 300
eta=0.0 ===========================================================>
22ms 1380us/step - loss=245.84
297 { loss: 245.83786010742188 }
Epoch 299 / 300
eta=0.0 ===========================================================>
23ms 1466us/step - loss=245.56
298 { loss: 245.56163024902344 }
Epoch 300 / 300
eta=0.0 ===========================================================>
23ms 1457us/step - loss=245.76
299 { loss: 245.76048278808594 }``````

As seen, `loss` parameter is too large. Why is this happening? The answer is simple – our data is too small. In this case, I’ll just expand them by adding the same values ​​several times:

``````const x_train = [
[0, 0],
//....те же самые строчки х10
[0, 0],
[8, 0],
//....те же самые строчки х10
[8, 0],
[14, 0],
//....те же самые строчки х10
[14, 0],
[-7, 0],
//....те же самые строчки х10
[-7, 0],
[-12, 0]
//....те же самые строчки х10
[-12, 0]
//...... и множество остальных данных
];

const y_train = [
[-80, -80, -80, -80],
//....те же самые строчки х10
[-80, -80, -80, -80],
[-50, -50, -80, -80],
//....те же самые строчки х10
[-50, -50, -80, -80],
[-30, -30, -80, -80],
//....те же самые строчки х10
[-30, -30, -80, -80],
[-80, -80, -50, -50],
//....те же самые строчки х10
[-80, -80, -50, -50],
[-80, -80, -30, -30],
//....те же самые строчки х10
[-80, -80, -30, -30],
//...... и множество остальных данных
];``````

Let’s run the script again. The latest logs in the console look like this:

``````296 { loss: 0.3628617525100708 }
Epoch 298 / 300
eta=0.0 ===========================================================>
61ms 358us/step - loss=0.363
297 { loss: 0.3628309965133667 }
Epoch 299 / 300
eta=0.0 ===========================================================>
57ms 338us/step - loss=0.364
298 { loss: 0.36354658007621765 }
Epoch 300 / 300
eta=0.0 ===========================================================>
58ms 341us/step - loss=0.363
299 { loss: 0.3632044792175293 }``````

Great! Now let’s add the code to save the model:

``````model
.fit(tf.tensor2d(x_train), tf.tensor2d(y_train), {
epochs: 300,
batchSize: 4,
callbacks: {
onEpochEnd(e, l) {
console.log(e, l);
},
},
}).then(() => model.save('file://model-js'));``````

After running the script, our model will be saved in this folder:

## We use the created model to control the robot

Now comes the practical part. We can load our model and use it to determine the position of the legs based on the angles. To load the model we will write the following code:

``````const tf = require("@tensorflow/tfjs-node");

async function initModel() {
return model;
}

initModel().then(model => {
/////// После загрузки модели можно передать в нее данные с гироскопа X и Y
const result = model.predict(tf.tensor2d([[X, Y]])).dataSync();
});``````

Method `model.predict()` used to obtain leg coordinates.
How the algorithm works – the gyroscope reads the angles several times per second and transmits them to `model.predict()`. The result obtained, in the form of an array of four elements, is used to set the coordinates of the robot’s legs.

Abstract code might look like this –

``````const tf = require("@tensorflow/tfjs-node");

async function initModel() {
return model;
}

initModel().then(model => {
setInterval(async () => {
const { X, Y } = await gyroScope.getData();
const result = await model.predict(tf.tensor2d([X, Y])).dataSync();

leftArm.setCoords(defaultX, result[0], defaultZ);
rightArm.setCoords(defaultX, result[1], defaultZ);
leftArmBack.setCoords(defaultX, result[2], defaultZ);
rightArmBack.setCoords(defaultX, result[3], defaultZ);
}, 30);
});``````

(Here I do not provide the implementation of the gyroScope and leftArm, rightArm, leftArmBack, rightArmBack objects)

That’s all! Now, by tilting the robot in different directions, it will automatically move its legs.

## Conclusion

As you can see from the example, using AI does not look particularly difficult. It is more important to understand what problem it should solve.

My plans are to continue experimenting with gyroscope data processing and various robot behavior models.