What do scientific data visualization and game dev have in common – we discuss in the new podcast “ITMO Research_”

This is the second part of the interview with Andrey Karsakov (kapc3d), candidate of technical sciences, senior researcher at the National Center for Cognitive Development, associate professor of the Department of Digital Transformations.

Since 2012, Andrey has been working in the scientific group Visualization and Computer Graphics. He is engaged in large-scale applied projects at the state and international level.


Photo Hello lightbulb (Unsplash.com)

In the first part of the conversation, we discussed the experience of AR support of mass events:

  • Bullets and audio version on youtube
  • Text transcript on Habré

Scientific data visualization and process modeling

Timecode (at Apple Podcasts) – 40:00

dmitrykabanov: You were not only involved in AR, you have a lot of things related to data visualization, starting with the training of masters and ending with projects for large customers?

kapc3d: Let’s even say this: these projects, although they are large and quite loud, but this is 20% of the total volume of tasks that we do. We are a visualization team. Engaged in precisely scientific visualization and developed (and still are developing) various tools for visualizing data related to research. On this branch, we continue to work with large customers and small private ones, let’s call it our internal customers – guys from other scientific groups who need tools. For example, to visualize data so that it is convenient to work with it. We came to augmented reality due to the fact that we use the technology of the gaming industry to work with “serious visualization”.

Dmitry: It is worth explaining what you can mean by data visualization for a wide variety of projects. For example, the calculation of the movement of people at a particular station.

Andrew: Yes, you mentioned a project that is not only related to visualization. Visualization is a representation of either numerical or abstract information. We love working with dynamic forms when something in time still happens. Visualization can be different, ranging from infographics. But visualization and infographics are two different things. Data visualization is more of an engineering process. His task is to show the information content of the data that you have. If they are wrapped in a beautiful shell, this is an additional plus.

This can be visualization of air flow, tomography, MRI images. In such data, the spatial dimension is determined by the physical and chemical properties of natural objects, and we cannot change them. From the point of view of data visualization, in particular, abstract data, we can use the visual channel of space as we please.

When there are external restrictions, such as the fact that you cannot show the temperature of a particular city elsewhere. Or if you show the calculated data of pedestrian behavior, it is clear that you need to demonstrate how people actually behave.

Another of the additional branches of scientific activity in our laboratory is the development of artificial intelligence systems just for pedestrian mobility. We are engaged in the visualization of such physical processes. We had a project on instrumentation and visualization of hydrometeorological data. They were engaged in forecasting the level, temperature, current of our sea, ocean, precipitation, wind, and geomagnetic situation. We work with urbanism, development. Due to the fact that we are doing the visual part, we play the role of some integrators, collectors in just one heap, because we provide the final user interface. Therefore, a specific subject area is not very important for us; we have worked with a large number of industries: medicine, urban studies, and a large-scale project is underway with fintech.

Gamedev and Master’s program

Timecode (at Apple Podcasts) – 48:52

dmitrykabanov: It turns out that in computer games this all comes together in an interdisciplinary thing, when there is data that is not tied to specific objects that need to be visualized, and directly game objects that somehow behave, including being guided and building their behavior thanks to artificial intelligence systems.

kapc3d: Yes.

Dmitry: Did you come to this on the basis of the general trend for interdisciplinary programs at the university or did you start to deal with this topic based on your interests?

Andrew: Rather, the second. This is evolutionary development, rather than some kind of adjustment to trends. Previously, the branch with scientific visualization, with simulators, simulators, and the branch of the consumer industry of computer games lived quite far from each other. Different technologies, different levels of equipment. For serious games certain computer capacities are needed, so that everything is honestly considered, these are some real simulators built on real models (for example, combat simulators). On the other hand, computer games work on simplification for the sake of optimization. It is not always necessary to reliably reproduce something in order not to lose the player’s interest. But the last three years, these areas have come close to each other, began to mix.

The tools that are used in game development have actively gone into the scientific field. And a lot of high-tech things from the “serious development” began to fall into the igrostroy. These are artificial intelligence systems, generative methods. We naturally got there. We were engaged in scientific visualization, but I always wanted to take part in the development of computer games. The thought arose, why not start mixing. Why not start teaching children who want to go into the gaming industry, and it is from the master’s level that they give them the competencies that we have from the point of view of the academic part. That is, that often highly qualified personnel are not enough: more freedom in decision making, to constantly study, study, and seek new solutions. Companies just do not have enough. They are looking for joons to teach them for themselves, or professional leaders of the top management level.

Dmitry: A logical question arises. The layman imagines work on games as a kind of creative process and for him the scientific component takes up 5-10% percent maximum. How are you doing with this? How do you explain to people that this piece has weight?

Andrew: Games are about business. Now in most cases, if you want to go into the industry, be prepared that this is a business where people just make money in most cases. When we talk about some kind of creative component, these are either some solitary or small projects that people together do for the soul. There is more creativity. There they take ready-made tools and use them for the sake of their creative potential.

On the other hand, there is a business that brings a lot of money. Last year, the gaming industry doubled the revenue of the film industry in terms of revenue. This is a lot of money. Concrete steps need to be taken to recapture the cost of this development. To do this, we need qualified personnel to build processes correctly, efficiently and make optimal decisions at the right time. There is nowhere without science. Everything that we see in interesting game projects, everything that catches us there, for all this there is a very powerful R&D base.

Disney and Piksarovskaya team has made a huge contribution to the development of computer graphics. They have new chips and technologies in every new cartoon or film that they release on SigGraph just like scientific publications. For example, the Disney team came up with a method of simulating snow in the “Frozen” so that it would slide correctly and leave traces. An incredible amount of man-hours is spent on this, and if you made a system that allows you to reproduce it, then the production cost is immediately reduced, but you immediately spent on R&D.

In each project there are moments that greatly develop the industry. If we talk about game projects, our Sasha – techlide – is trying to drag our entire team into Star Citizen, which has been under development for a long time. I will not give any estimates on it. Everything looks very cool. If you look at the technical component that they realize inside, it’s just the craziness of technology inside.

There is almost a completely seamless universe, and not at the level of No Man’s Sky, but in real terms. When you can take off from the planet, see somewhere there: “Oh, a small luminous point in the sky, let’s go and I’ll reach it.” You board your spaceship, take off from the surface of the planet, fly into orbit, see a space station, fly up to it, see its landing docks, board there, get out of the ship, go for a walk on it – everything is seamless.

And inside there is a large number of technologies, you can go crazy. There are awesome economic models that allow you to reproduce the life of this whole world, various models that allow you to realistically generate the surfaces of the planets so that they are as unique as possible. When you see that your identical objects begin to repeat themselves, then to make it look as natural as possible is another task.

Cinema – there are interesting cases. On Netflix went out “Brandashmyg“Which is the” Black Mirror. ” Remember, there was an interactive series where you could choose the development of events. But this is not quite right. There a limited number of branches can develop. The film industry is now slightly in the other direction is changing in terms of film production. Everything is greatly simplified there, that is, it turns into a computer game. You can see almost all the special effects that come in real time right on the set together with your actors.

Immediately edit everything, dynamically change. This simplifies the process, it becomes more creative than before, when the director had to think through everything in his head. There is more creativity in terms of technical capabilities. You can add a lot of computer graphics.

No need to go to nature to take beautiful shots. It is enough to put a person in front of the green screen, plus or minus correctly set the light. By the way, the toolkit of the conditional game engine will help with this, when you know what kind of environment you will have. So you reduce further post-production labor. In the film industry, this is noticeable. And even on the example of professional events. Previously, conferences like the CG Event were mainly devoted to film production, special effects, and television pieces. In recent years, the block for the production of computer games and the block for gaming technologies in digital media have been actively entering. For example, in Moscow, almost a separate section was under the Unreal Engine.

Dmitry: What can you say about prerequisites for those who want to go to the igrostroy? What you need to know and what can be advised to those who are willing to devote their time to this?

Andrew: A slippery question. I can fly a little, but I have a certain position. If you want to get into game dev, you don’t have to go get specialized education for this. It all depends on who you want to be in this industry. You can easily go after college with a basic knowledge of computer literacy to QE – just test the games.

Everything, you got into the industry. Then you understand how you want to develop, study, move, anywhere. On the other hand, we do not have an education system for the game industry in Russia, although in Europe and the USA it has long been quite an academic branch in undergraduate and graduate programs. We have not taken this seriously yet. When the gaming industry began to actively develop, we had the dashing 90s, where nothing was developed at the academy. Many want to learn, but do not understand why.

We launched a magistracyfirstly, technological. We do not teach managers how much they want. We are teaching high-class technical specialists. To get to our magistracy, you need to have good basic knowledge of programming, especially object-oriented, higher mathematics, linear algebra, probability theory.

The main thing is not to be lazy. We have the opportunity to enter the portfolio if there are projects, participation in conferences, showcases or work experience in the field of games, interactive technologies. We select it rather hard, we look at motivation. This year, 2-3 people went through the portfolio, and we do not regret it. The guys are motivated.

You can listen and support us here:

Meanwhile, on the English version of Habr: memory, cyberpunk and startups.