AI in the heart of Africa. How we went to Rwanda for the largest machine learning conference ICLR 2023

Hello! My name is Alexander Korotin. I am an AIRI Fellow and Head of the Skoltech Research Group. My area of ​​interest is generative models for transferring data between domains. My colleagues and I have made great strides in improving the efficiency of such models and presented our results at ICLR 2023, a prestigious conference on artificial intelligence that took place this spring in Rwanda (you can read the articles accompanying our reports Here And Here). About how this event went in the very center of Africa, I tell in the text below.

International Conference on Learning Representations

The International Conference on Learning Representations, or ICLR for short, is a fairly prominent annual event in the world of machine learning. The conference is regularly attended by major experts in artificial intelligence, and its works have the highest rank (A*) according to CORE. It is important that the review process is organized openly, and the referee’s decisions are always accompanied by sound arguments.

The steepness of this event can be judged, if only because there is fake conferenceconducted by a “predatory” publishing house WASET. This is warned by a big announcement on main page of the site this conference.

The first ICLR conference took place in 2013 in Scottsdale, Arizona, USA, and until 2020 all subsequent conferences were held on the North American continent or in Europe. But in 2020, the city of Addis Ababa, the capital of Ethiopia, located in East Africa, was chosen as the venue for this event.

This choice organizers explained visa problems faced by many conference participants, especially Africans. In general, the organizers are quite reverent about the issues of equality and inclusiveness, giving this great attention. Therefore, this time the event was held in the city of Kigali, the capital and largest city of Rwanda.

Road to Africa

When we learned that our papers had been accepted for a conference to be held in Central Africa, we did not hesitate for long. Given the current foreign policy situation, our chances of performing live in the same States were close to zero. In addition to us, other Russian research groups from AIRI, Skoltech, HSE and Yandex made the same decision.

Even logistical difficulties did not frighten us: it took 16 hours to fly with a change in Istanbul. In this sense, the idea of ​​the organizers worked, although the flight across the Atlantic Ocean seems to have scared away many of our American colleagues.

A couple of weeks before we left, we were vaccinated against yellow fever and stocked up on malaria pills. It was not so easy to do this, given that the medicines we need are not produced in Russia. In addition, I went to the doctor separately to talk about how else you can prepare for visiting Africa.

The conference lasted five days – from 1 to 5 May. We arrived the day before the start to check in without haste, take the badges from the organizing committee and take a short walk around the neighborhood. For us, as for many who read this text, the first association with Rwanda is the infamous genocide that happened here almost 30 years ago. This fact added a little anxiety.

But these fears turned out to be unfounded. Kigali is the most ordinary city, somewhat reminiscent of the seaside south of Russia, only there is no sea there, only mountains. The only warning sign was the machine gunners surrounding the building of the Kigali Convention Center, or, in Russian, the Kigali Convention Center, where our event was held.

How was the conference

The first three days of the conference were held in approximately the same format. In the morning, an invited talk was held in the large conference hall. After the coffee break, people dispersed to their oral and poster sections. According to the rules of the conference, each oral presentation must also be presented as a poster. After lunch, there was another invited talk, and then again thematic sections.

The last two days were devoted to workshops. True, they were no longer so interesting – I visited only a part of them. It is difficult to estimate the total number of participants, but some conclusions can be drawn from the number of accepted papers: all of them over 1200.

In general, I liked the organization, although some grouping of poster sections on one topic was missing. They often took place at different times and it was not always possible for scientists working within the same subject to get together. There was also lunch during the afternoon poster sections: people came up to the posters with plates in their hands.

Our research

Two papers were accepted from our group at ICLR 2023. It so happened that the second was written later than the first and was its direct continuation, so we combined them into one presentation and one poster, which I spoke on the first and third days. My colleagues also participated in the poster presentations: Daniil Selikhanovich and Evgeny Burnaev, our leader.

Our group deals with the transfer of data from one domain to another using neural networks. This problem arises, for example, when there is a lack of quality data to create a sufficient training set. It can be solved by generative models that allow you to synthesize some data from others – in other words, transfer one domain to another. For example, a person drew a sketch, and the neural network created a digital picture based on it, or a satellite took a picture, and the neural network improved its detail.

Usually, to solve such problems, paired training samples are needed, sets of input-output images, the knowledge of which the neural network learns to generalize and distribute to new incoming input images. As a rule, paired data is very difficult or expensive to collect, and often unpaired datasets have to be dispensed with, making it difficult to achieve really good results. Now, the approaches to transferring one domain to another are largely heuristic – the training parameters have to be configured “by hand”. Our goal was to correct this situation by formulating a rigorous mathematical approach for this operation.

We turned to the work of the Soviet mathematician and economist Leonid Kantorovich. Based on his ideas about optimal transport of goods (optimal transport theory), we came up with a new algorithm for calculating plans for optimal transport of data between domains. The algorithm is called Neural Optimal Transport (NOT). It was created on the basis of deep neural networks and the use of independent data sets.

In our articles and reports (first And second), we have shown that the new algorithm is superior to existing methods in a number of experiments, including the problem of image styling. At the same time, it has much fewer hyperparameters that are difficult to tune than other existing methods, and the result of its work is well interpretable.

We hope that the results of our work will increase the speed and quality of data transfer from one domain to another and, as a result, will help to create large and high-quality training sets. This hope is fueled by the great interest shown to our posters by other participants of the conference. If you are also interested in our study, I attach a link to our repository with the implementation of our algorithm on the PyTorch framework. You can also find a poster in the repository.

What else was interesting?

In the main part of the conference, I liked several reports. Among them is the report of Xing Chao Liu and his colleagues, which proposes domain transfer approach based on the theory of optimal transportation and models with simple differential equations; report by Tuomas Kyunkänniemi and colleagues, in which they talked about roles of ImageNet dataset classes in a popular metric for generative FID models; report by Zahra Kadhodaye and colleagues who developed probabilistic model based on Markov wavelets for image synthesis. Also, I can’t help noticing work our colleagues from AIRI, who, like us, are engaged in optimal transport.

What did we do in our free time?

It would be wrong to spend almost a week in an exotic African country and devote it exclusively to the conference. First of all, we got acquainted with the local cuisine. Almost all of it seemed to me quite spicy, especially tea: it seems to be made entirely of ginger there.

On the fourth day, when the workshops began, we went to the Genocide Memorial Center. This quiet place left a rather sad impression.

In the late afternoon we visited the local market, which we didn’t really like. First, it was very crowded. Secondly, we were immediately surrounded by local merchants, who constantly beckoned us to buy something from them. And thirdly, the souvenirs that we took there turned out to be just as expensive as in the airport shop, but much worse in quality.

On the last day of our stay, we agreed to participate in a safari. This pleasure is not cheap – $ 235 per person. But it took a whole day and was quite exciting.

The driver picked us up early in the morning from our hotels and took us to the Akagera National Park, located on the border with Tanzania. The car was a spacious six-seat jeep with an opening roof. It so happened that there were just five of us, scientists from AIRI and Skoltech, so everyone was on the road.

The safari consisted of the following: the driver drove around the park for 7-8 hours and looked for animals for us to look at and photograph. Once we stopped for lunch with a local analogue of shawarma, and another time – to take pictures in a large meadow surrounded by animals. For the whole day we managed to look at zebras, giraffes, antelopes, buffaloes and several other exotic animals, but we did not manage to catch elephants and lions. But we were lucky with the weather: on the day of the safari it was about 20 degrees.

Conclusion

Summarizing my impressions of this trip, I was satisfied. All the difficulties from preparing for the road and the hardships of the flight (we got back a day) more than compensate for the experience that we received and shared with others.

I am especially pleased that the most important event in the world of machine learning does not close the doors for Russians who have something to share with their foreign colleagues. I hope that in the future I will also be able to attend ICLR and other conferences of this level.