AI trainer, neuroeducator, assessor, crowd and marker – who are all these people and what is the difference?

Many companies have recently introduced the position of “AI trainer” (AI trainer), but simply markers/assessors have not gone away. What is this – just a beautiful naming wrapper over the same ones or something conceptually new?

Let's try to figure this out and clearly answer the question about the differences.

As an easy introduction, it’s worth telling what all these people do in general.

Technologies require data, but not just data, but well and correctly prepared data. If you already have a complete set of data and you just need to put things in order, then, conditionally, a few good data engineers/analysts will be enough for you.

But if you don’t have data (you need to extract/collect/generate/validate it), or you need to assign certain characteristics to this data according to the technical specifications, then you can’t do without people. Almost. For such data you will need marking — the very assignment of characteristics to data.

Good markings are very important, the closest analogy is the engine (technology and algorithms) and fuel (markings). If you pour bad fuel into even the best engine in the world, then the performance will be worse and the entire technology will very soon need to be overhauled. Therefore, good markup is really important, although many companies do not really like to do it well.

The markup can be very different and on any type of data: highlighting objects in video, transcribing audio into text, field tasks – we haven’t seen anything on our platform. _{(how do you like “Humming the anthems of other countries”?)}

Actually, all these people are doing the same thing globally – markings. But there’s a lot more nuances.

What is the difference between all these words (and the phenomena behind them)?

Their differences lie in depth and expertise.

AI trainer (AI trainer)

The very appearance of the term “AI trainer” practically “coincided” with the boom of large language models (LLM) – and this is not an accident. The emergence of very complex technologies such as multimodal LLMs is greatly raising the bar in the required markup. If we are preparing data for such fundamental technologies or wrappers over them, then we can no longer entrust it to an unprepared person – we need the person to at least basicly understand the insides of the technology, as well as to understand well the task assigned to him.

To get a good result, a person needs to be adapted and prepared for the task – sometimes the marking instructions can take 40 A4 sheets, where the basics are described, and what to do with corner-cases and everything else necessary. The ability to understand and follow such instructions in itself imposes a certain level of requirements for a person’s education and qualifications.

And the tasks are sometimes quite complex, for example, working with “sensitive” topics such as politics, religion, suicide, ethical behavior, and so on. Mistakes in such topics can affect millions of people and pose huge reputational risks, so this markup is truly complex and intelligent, and that is why the term “AI coach” was invented for it.

If you come across the term “neuroeducator”, it’s right there, just a joke or slang, simply because the process of “educating” technology’s reactions to something is similar to raising children/animals in the initial stages of their lives.

Marker, assessor and labeler

In fact, all these words are synonyms and mean the same thing, but simply appeared from a different subject area.

Marker – simply because he is engaged in marking. “Labeler” is the same thing, just an anglicism – in English the process of marking is often called “label/labeling”, literally – putting labels on something. That is, simply assigning characteristics to your data.

With an assessor, everything is trickier, let's look at the official definition: “An assessor is an official vested with judicial power in Ancient Rome and medieval Europe.” In fact, a person who makes some kind of decision.

I believe the term was coined and implemented by a Russian company whose assessors first made a decision about the results of something (for example, search results), and then the term simply expanded to any data markup and markup in general.

For such markers, the scope for tasks (and terminology, accordingly) can be enormous – from very low-skilled work to expert work, from narrow specialization to the widest possible range of issues.

Low-skilled work, for example, is mechanically tracing clear objects with a brush – something like this was how data was prepared for a model that greatly changed the layout of computer vision, I wrote about this in my article:

SAM's adventure in Japan or how computer vision sees a geisha

The model turned out cool, so, alas, there is no way without such low-skilled markings, but after the release of such models there will be less of it.

There is a narrow specialization – this is when a person deals with a wide class of tasks, but only one subject area that does not require abovepreparation, for example, by traffic markings or the results of a recommendation algorithm.

Another related example is people who know other languages ​​(very different, from large international ones to specific dialects of a region or small languages); such problems are often solved.

But often extra preparation is required.

Experts and markup – everything is complicated

It often happens that we need narrow expertise in something if our technology affects it. For example, if we are trying to teach our technology to determine the originality of something (for example, a 17th century painting), then we will need an expert in the field of culture and, ideally, that very 17th century painting.

He does not need to understand the depth of the technology, simple instructions are enough, but he will be required to have expert knowledge in this subject area. Here, without a specialized tower and many years of work in this industry, there is no way.

Another example is medicine. To mark up medical images (for example), we need doctors who have a good understanding of the subject area. It is impossible to give such a task to a person, even with good training, but not in this subject area.

In fact, such experts are rather simply expert markers, although much depends on the setting of the task.

Crowd from markers

Crowd is an Anglicism, literally “crowd”. In relation to markup, this is the crowd of markers, that is, people who want to do markup, but do not specialize in it.

Almost like freelancing, like some kind of part-time job. Crowd is very feared by techies who need good markup, since this is the most unprepared public for creating markup. I have an article in the works about how to properly cook crowd so that you don’t feel ashamed, but it will come later.

In general, the stereotypes that crowding is bad are not entirely true. It’s very bad for crowd if you throw off the technical specifications and don’t make allowance for the fact that there are a lot of people there who know absolutely nothing about the inner workings of technology and AI. But the crowd with which you work correctly – you train, explain, control, b̶Ѷе̶т̶е̶ ̶п̶а̶л̶к̶о̶й̶ ̶п̶о̶ ̶г̶о̶р̶б̶у̶ ̶к̶о̶г̶д̶а̶ ̶н̶а̶д̶о̶ and valid You write down the results (you can use them too) – this is a good way to do markup.

As conclusions

All these people do markup, but their markup is very different. Terms are often abused, often used out of ignorance, but if you try to very roughly answer the question in the title, the answer will look like this:

AI trainer — “educates” technologies to respond correctly to complex and ambiguous situations or teaches very complex things.
Marker/assessor/expert — they do a wide range of markings, from low-skilled to highly expert.
Crowd – this is a general designation for people who are generally ready to do markup as a freelancer.

I hope you found it helpful.

Good weekend!

My other articles about markup:

SAM's adventure in Japan or how computer vision sees a geisha

Will LLMs replace humans in data tagging for AI?