How AI helps to overcome the monopoly in sports advertising and what do GPUs and dedicated servers have to do with it
Sports competitions today are filled with advertising, from commercials on screens to static company logos on stadium billboards. The airwaves cover many countries, with different brands and different advertising laws. Thanks to a breakthrough in the development of AI and AR technologies, it becomes possible to replace information on the field for each audience, directly during the broadcast of the match.
Advertising in sports arenas was originally static and targeted to people attending a specific event in a specific city. Later, LED screens appeared that showed successive commercials. The next step is that they began to insert a “green background” into the videos on the screens for a split second, with the help of which the AI determined the space and inserted unique text.
Now you can replace anything with anything. The challenge is to ensure that the replacement is done discreetly and realistically. The largest players in this market use special cameras. From the photographs one can conclude that there are specialized sensors that read positioning and adjustment information. That is, popular solutions still have hardware “crutches”.
All this requires enormous computing power, since introductory information and the original broadcast are processed by specialized software online. That is, in addition to cameras and sensors, servers are added for computing. The equipment is expensive, the market is “closed”, and so are the technologies. And this is where AI and cloud GPU servers come to the rescue.
Rent dedicated and virtual GPU servers with professional graphics cards NVIDIA RTX A5000 / A4000 and Tesla H100/A100, as well as RTX4090 gaming cards in reliable TIER III class data centers in Moscow, the Netherlands and Iceland We accept payment for HOSTKEY services in the Netherlands in rubles to the account of the Russian company. Payment using bank cards, including the MIR card, bank transfer and electronic money. Hourly rental of GPU servers.
Expensive, static and not for everyone
HOSTKEY has a startup client PTF Lab, who developed his own technology for introducing virtual advertising and integrating digital content (such as augmented reality) in a multiregional mode. Moreover, their solution promises seamless (and cost-free) integration of advertising directly into the video stream.
Once upon a time, they saw information about HOSTKEY in the Open Data Science (ODS) community and turned to us when they needed cloud power, evaluating our service, prices and flexible line of available options.
The startup’s goals are noble, understandable and quite achievable:
Get away from expensive “closed” equipment and complex settings, shifting the task of advertising placement and framing, taking into account the people and objects overlapping it, to artificial intelligence.
Due to this, cover relatively small events (for example, arena fights) and bring technology to the masses.
Make sports advertising accessible and relatively inexpensive.
Moreover, the startup has set an ambitious goal to surpass the solutions of monopolistic giants in flexibility. For example, displaying “virtual advertising” during replays, using angles from any camera, and not just from certain angles.
How it works?
The video signal from the site of a sporting event can be processed using computing power not only at the site itself, but also in the cloud. This allows you to flexibly distribute the load and choose when to apply advertising: before broadcasting, or already during broadcasting, taking into account different regions. Also, the use of cloud services allows you to use advertising in locations where it is impossible to bring a server (and in principle it is more convenient).
Object segmentation is built on a neural network architecture from U-Net; neural networks are responsible for the location of objects to detect and compare key points. But the task is not trivial, so all the solutions and neural networks had to be reworked and trained for ourselves. It is especially difficult on broadcasts of martial arts, where literally everything is unpredictable from the point of view of the picture: light sources, shadows, camera angles, the mesh covering sponsor logos and the bodies of fighters and referees.
Neural networks are not used everywhere. Sometimes, to solve a problem, it is enough to be smart and use simple algorithms. Thus, tracking algorithms combine neural network methods and systems of linear and nonlinear equations.
A significant portion of the GPU is consumed by segmentation. The better the identification of people and objects in the frame and their separation by plan and type, the more natural and beautiful the frame will look after superimposing advertising.
A separate task is related to lighting and shadows, which must be taken into account in augmented reality when rendering a scene. The realism of shadows is a key element in assessing the “believability” of a picture.
“Sports” neural networks are traditionally trained both on real broadcasts using human markings and on synthetic models. And here Blender comes to the rescue, in which the company builds 3D models of the ring, fighters, judges, and receives both rendered “real” frames from the required angles, and a segmentation mask or the position of objects and cameras needed for training. Tagging real data is time consuming, expensive, but high quality for specific venues or types of competitions. Synthetic data with less realism provides more data for training.
The main difficulty is that the location of the event may change. In one case it will be a boxing ring with ropes, and in the other it will be an arena with mesh walls, which creates difficulties for segmentation.
Camera tracking and advertising position are determined by comparing a cloud of points from a 3D model of the venue with their actual position in the frame. This allows position detection even for hand-held cameras with erratic motion. After reconstructing a 3D frame from 2D (determining the desired angle), the advertisement is directly rendered in a 3D engine and combined with the video broadcast frame.
A 3D scene has to be built before starting work, and in fact we have a virtual copy of the venue in the frame, into which we fit real people and objects through render masks. It sounds difficult, but with the proper power and optimized neural networks, these somersaults can be performed instantly and seamlessly.
Why does the project need GPU computing and server rental?
PTF Lab has its own servers (and the possibility of using them directly at the competition site has already been mentioned), but it is more convenient to use remote resources, since the service provider’s engineers are responsible for the operation of the equipment, and the company spends fewer resources on this. And the client votes with rubles, and economical options are always preferable.
Also, the amount of capacity required by the company is constantly growing. If necessary, it is easy to scale when renting (up to changing the server configuration to suit the company’s needs).
In the future, a startup may need a lot of cloud capacity, and renting it is easier than buying and selling servers as demand for services rises and falls.
Currently, rented servers and GPU power are used in the following areas:
Training neural networks on GPU (segmentation of people and other objects in sports broadcasts; tracking of a virtual 3D camera).
Data backup (videos from events, datasets, etc.).
The direct process of working with video is testing cloud production, when the company’s software is deployed on remote servers and video signals pass through it: input without graphics, output one or more signals with graphics.
In addition to our own computing power at HOSTKEY, the startup is currently using the following GPU configurations:
AMD Ryzen 9 5950X 3.4GHz (16 cores)/128Gb/1Tb NVMe SSD+12Tb HDD/2xRTX 3090+PSU]
Xeon E3-1230 3.2GHz (4 cores)/16Gb/2x12Tb HDD/PSU]
AMD Ryzen 9 5950X 3.4GHz (16 cores)/128Gb/1Tb NVMe SSD+12Tb HDD/2xRTX 3090+PSU+HDMI emulator]
AMD Ryzen 9 5950X 3.4GHz (16 cores)/128Gb/1Tb NVMe SSD+12Tb HDD/2xRTX 3090+PSU]
AMD EPYC 7402P 2.8GHz (24 cores)/384Gb/2×1.92Tb U3 NVMe SSD/4xRTX 4090+2xPSU]
As you can see, the GPU solutions are mainly rented based on the RTX4090/3090, but as the requirements for computing power increase, the startup wants to consider servers with more powerful professional cards that provide better stability during continuous 24/7 operation. In the case of 3090, renting the same A5000 may be even cheaper.
PTF Lab is at the beginning of its journey, but the result is already excellent, and we at HOSTKEY wish them success and development, especially in overcoming the monopoly of sports augmented reality. The witty David always defeats the clumsy, proprietary Goliath.
If you have questions for your PTF Lab colleagues, you can ask them in the comments.
Rent dedicated and virtual GPU servers with professional graphics cards NVIDIA RTX A5000 / A4000 and Tesla H100/A100, as well as RTX4090 gaming cards in reliable TIER III class data centers in Moscow, the Netherlands and Iceland We accept payment for HOSTKEY services in the Netherlands in rubles to the account of the Russian company. Payment using bank cards, including the MIR card, bank transfer and electronic money. Hourly rental of GPU servers.
