Bayes algorithm for data analytics

Article author: Artem Mikhailov

Bayes algorithm is a statistical method that is used to determine the likelihood of an event based on prior knowledge of that event. This method is based on probability theory, which allows us to estimate the probability of a random event based on its significance and the frequency of its occurrence.

It was named after the English mathematician Thomas Bayes, who lived in the 18th century and made a significant contribution to the development of probability theory. He also researched the theory of problem solving based on empirical data.

The essence of the Bayesian algorithm is to update the posterior probabilities for the model parameters based on the prior probabilities and new observations. That is, when analyzing data, it is necessary to determine the probabilities that the desired parameter takes a certain value. As new data becomes available, the probabilities for the model parameters are updated, taking into account prior knowledge and new data.

One of the most popular applications of the Bayesian algorithm is data analysis. The Bayesian algorithm can be used to determine the probability of an event based on the data already available, making it a very useful tool for forecasting and decision making in various fields such as medicine, business, and finance.

In this article, we will look at the basic principles of this algorithm and how to apply it in practice.

Application of Bayes algorithm in data analysis

Prior probability distribution

is a distribution that is used to describe prior knowledge about the model parameters before the data was received. It is used in Bayesian statistics and allows estimation of unknown model parameters based on data and prior knowledge.

The specification of the prior probability distribution depends on the type of model and the parameters to be estimated.

For example, if we are estimating the mean of some parameter, we can use the normal distribution as a prior distribution, where the mean and variance are chosen based on prior knowledge of the parameter. If you want to estimate the percentage of success in some experiment, you can use the beta distribution.

Specifying a prior distribution requires some prior knowledge of the model and parameters. Often such knowledge is based on expert opinion or previous research. If knowledge is not enough, non-informative distributions can be used, which have little effect on parameter estimates.

In addition, the prior distribution may change during data analysis, for example, after the first observations are obtained. In such cases, so-called updated priors can be used, which take new data into account.

Bayesian approach is a method of identifying and modifying probability estimates based on data, and these estimates can be expressed using a probability distribution. The essence of the approach is to update a priori knowledge (a priori probabilities) based on new data, which are also called observations.

The Bayesian approach is based on the Bayes formula, which allows us to calculate posterior probabilities based on prior probabilities and observations. Bayes formula looks like this:

P(A|B) = P(B|A) * P(A) / P(B)P(A|B) = P(B|A) * P(A) / P(B),

where P(A|B) is the posterior probability of hypothesis A and P(B|A) is the probability of observing data B if hypothesis A is true. P(A) is the prior probability of hypothesis A and P(B) is the probability of observing data B in any case.

Thus, we can use Bayes’s formula to update the prior knowledge of the hypothesis based on new data. If we have a priori knowledge about the hypothesis, we can use this data to estimate the posterior probability of the hypothesis when we get more data.

Likelihood probability is a measure of how likely it is that the data we observe was generated by a particular hypothesis.

To use likelihood in Bayes’ algorithm, we need to determine the probabilities of the hypotheses we want to test based on the available data. Then you need to determine the probability that the data was generated by each of these hypotheses.

To determine the likelihood, the likelihood function is used, which is defined as the probability of obtaining a certain set of data, provided that some hypothesis is true. In other words, it measures how likely it is that the data was generated using a given hypothesis.

Once we’ve determined the likelihoods for each hypothesis, we can use Bayes’ theorem to recalculate their probabilities based on the new data we get. This allows us to more accurately determine the probability that each hypothesis is true.

Let’s say we have data about a patient’s medical condition and we want to determine if it’s likely that they have diabetes. We can use likelihood probability to determine the likelihood that these data are consistent with a patient with or without diabetes. We can then use the Bayesian algorithm to update our probabilistic representations based on additional medical data, such as blood test results, to more accurately determine the likelihood of a given patient having diabetes.

In summary, likelihood is used to determine the likelihood of having a certain hypothesis based on available data, and Bayes’ algorithm allows you to adjust the probabilities based on new data.

Multinomial Bayes Model is a statistical classification algorithm based on probability theory. It is used to determine which class a new object belongs to based on a probabilistic assessment of its characteristics.

The multinomial naive Bayes model is used, for example, in the tasks of filtering spam emails in e-mail, determining the tone of texts, or classifying goods into categories.

To use this model, you need:

1. Prepare a training sample with a description of objects and their features. Each object must be assigned to one of the classes into which we will classify them.

2. Estimate the probabilities of occurrence of each feature for each class based on the training sample.

3. Using the obtained probabilities, classify the new object.

A naive Bayesian classifier is considered naive because of the assumption that the features of an object are independent. Although this assumption may not always be justified in practice, the method can still give good results and is widely used in data analysis.

Consider in practice

Imagine that we have an online store, and we want to decide which product categories of our store are most used by our customers:

male

or

female

.

To solve this problem, we can use the Bayesian algorithm. Let’s define a few options related to this task:

F – the number of purchases for a certain period made by women

M – the number of purchases for a certain period made by men

Then we can write our probability equation as follows:

P(F|M) = P(F) * P(M|F) / P(M)

P(F|M) – the probability that the majority of purchases were made by women, if the number of purchases of men and women differs

P(F) is the prior probability that the purchases were made by women

P(M|F) – the probability that a certain category of goods is more popular with men if women make more purchases

P(M) is the prior probability that the purchases were made by men

We can set the prior probability arbitrarily, for example, as 0.5 – that is, we assume that the number of purchases of men and women is approximately equal.
Suppose we have collected data on which product categories men and women buy and estimated the probabilities of these purchases.

For example, we found that:

– 60% of women buy cosmetics
– 40% of men buy cosmetics
– 65% of women buy food
– 35% of men buy food

Then we can calculate P(M) as the sum of two options:

P(M) = P(M|F) * P(F) + P(M|¬F) * P(¬F)

Where P(¬F) is the probability that the purchases were made by men, given that women make fewer purchases. This can be calculated as follows:

P(¬F) = 1 - P(F)

Then we can calculate the probability that the purchases were made by women. Let’s use python:

# задаем априорную вероятность
P_F = 0.5

# задаем вероятности покупок
P_cosmetics_given_F = 0.6
P_cosmetics_given_not_F = 0.4
P_food_given_F = 0.65
P_food_given_not_F = 0.35

# задаем количество покупок
purchases_F = 200
purchases_M = 150

# рассчитываем апостериорную вероятность
P_M_given_F = (P_F * P_cosmetics_given_not_F**purchases_M * P_food_given_not_F**purchases_M) / ((P_cosmetics_given_not_F**purchases_M * P_F * P_food_given_not_F**purchases_M) + (P_cosmetics_given_F**purchases_M * (1 - P_F) * P_food_given_F**purchases_F))
print("Вероятность того, что большинство покупок было совершено женщинами:", 1-P_M_given_F)

In this example, we got the probability that the majority of purchases were made by women. Its value depends on the probabilities given at the beginning. The presented example can be modified to solve other problems, where it is necessary to determine the probability of an event, knowing other events.

In the second example Let’s imagine that we are conducting a study on what percentage of people living in different regions own a car. We have data on the population, the number of cars, and the total number of people living in different regions.

Let’s say we have the following data:

| Регион | Население | Автомобили | Владельцы автомобилей |

|--------|-----------|------------|-----------------------|

| Регион 1 | 100000 | 30000 | 15000 |

| Регион 2 | 200000 | 40000 | 30000 |

| Регион 3 | 150000 | 20000 | 12000 |

Now we want to calculate the probabilities that a person living in each of the regions owns a car.

First, we can calculate the probability of owning a car in general:

P(car) = (15000 + 30000 + 12000) / (100000 + 200000 + 150000) = 0.25

Then we can calculate the probability of owning a car in each of the regions:

P(car|Region 1) = 15000 / 100000 = 0.15
P(car|Region 2) = 30000 / 200000 = 0.15
P(car|Region 3) = 12000 / 150000 = 0.08

Next, we can apply the Bayes formula to calculate the probability of owning a car in each of the regions:

P(Region 1|car) = P(car|Region 1) P(Region 1) / P(car) = 0.15 0.333 / 0.25 = 0.2
P(Region 2|car) = P(car|Region 2) P(Region 2) / P(car) = 0.15 0.667 / 0.25 = 0.4
P(Region 3|car) = P(car|Region 3) P(Region 3) / P(car) = 0.08 0.444 / 0.25 = 0.142

Thus, we can conclude that the percentage of car owners in Region 2 is higher than in other regions – 40% versus 20% and 14.2%.

Advantages and disadvantages

Advantages:

1. Flexibility. The Bayes algorithm is suitable for various classification tasks such as text classification, sentiment analysis, or image classification.

2. High precision. Bayes’ algorithm can achieve high classification accuracy, especially in problems with a large number of parameters.

3. Good noise handling. Bayes’ algorithm can remove noise from the data, thereby improving performance.

Flaws:

1. Necessity of the assumption of independence. Bayes’ algorithm works best when all features are independent of each other.

2. The cost of calculations. Bayes’ algorithm can be computationally expensive, especially when there are a large number of parameters.

3. Limitations in application. Bayes’ algorithm may not work properly if the data is too small or the features are highly correlated.

Finally, The Bayesian algorithm is a powerful tool for solving various classification and prediction problems.

This tool is often used in the work of a systems analyst. And how to build a career in this area from zero to middle, will tell the leading systems analyst at open lesson in OTUS. The teacher will talk about who is suitable for this profession, what initial knowledge and skills are needed, using examples of successful cases, he will show the ways for a beginner to develop in this area and give recommendations. You can sign up link.