What is Big Data?
With the development of technology, the amount of data began to increase exponentially. Traditional tools no longer cover the need for processing and storing information. To process data whose volume exceeds hundreds of terabytes and is constantly increasing, special algorithms have been created. They are called “big data”.
Today, information is collected in huge volumes from various sources: the Internet, contact centers, mobile devices, etc. Most often, such data does not have a clear structure and ordering, so a person cannot use them for any activity. To automate the analysis, “big data” technologies are used.
When did the first big data appear?
Big data appeared in the 60-70s of the last century along with the first data centers (data centers). In 2005, companies began to understand the extent of content created by users of Internet services (Facebook, YouTube, etc.). At the same time, the first platform designed to interact with large data sets, Hadoop, began work. Today it is a large stack of technologies for processing information. A little later, NoSQL began to gain popularity – a set of methods for creating big data management systems.
The amount of information generated began to increase with the advent of large Internet services. Users upload photos, browse content, like, etc. All this information is collected in large volumes for further analysis, after which you can make improvements to the services. For example, social networks use big data to show users relevant ads (that is, one that matches their needs and interests) in a target. This allows social networks to sell businesses the opportunity to conduct accurate advertising campaigns.
Key Features of Big Data
At the very beginning of the article, we identified three main properties of big data from a generally accepted definition. Let’s open them in more detail:
- Volume. From the name “big data” it becomes clear that they contain a lot of information. And this is true: companies can receive dozens of terabytes of various data daily, some hundreds of petabytes. That is, “big data” would not have been without volume.
- Speed. Big data arrives and is processed from different sources with high speed. In the absence of this property, information can no longer be called “big data”. And they are also generated without stopping.
- Diversity. Big data contains information related to different types. This is one of the main differences from simple data – they are always structured and can be immediately stored in a database.
Over the past few years, the popularity of big data has increased, as a result of which they have received two additional properties (characteristics): value and reliability. The value is determined by each company in its own way. Experts evaluate whether the information received will benefit the business. And reliability shows whether the data used can be trusted (how true they are), because inaccurate information can harm the company and its activities.
How do they work with them?
Big data carries a lot of useful information, on the basis of which companies create new opportunities and form business models. Work with big data is divided into 3 stages: integration, management and analysis.
Stage 1. Integration
At this stage, the company integrates technologies and systems into its work that allow it to collect large amounts of information from various sources. Data processing and formatting mechanisms are being introduced to simplify the work of analysts with “big data”.
2 stage. Control
The received data needs to be stored somewhere, this issue is resolved before working with them. The decision is made on the basis of many criteria, the main of which are preferences in format and processing technology. As a rule, companies use local storage, public or private cloud services for storage.
3 stage. Analysis
Big data begins to benefit after analysis. This is the final stage of interaction with them. For this, machine learning, an association of learning rules, genetic algorithms and other technologies are used. After analyzing the data, only the most valuable for the business remains.
Big Data Examples
In general terms, with “big data” sorted out. But the important question remains – where can they be applied in practice? Answer: in any field of activity that operates with the data necessary for analysis. Let’s look at some real world examples. This will help you better understand why big data is needed and how you can benefit from it.
Big Data in banks
In the Russian banking sector, Sberbank was the first to use big data. On the basis of “big data” and a biometric system, in 2014 they developed a system for identifying a client’s identity using photographs. The principle of operation is very simple: comparing the current image with a photo from the database that employees take when issuing a bank card. The new system has reduced cases of fraud by 10 times.
Today, Sberbank continues to use big data in its work: the collection and analysis of information allows you to manage risks, fight fraud, assess the creditworthiness of customers, manage queues in branches and much more.
Another example from the Russian banking sector is VTB24. The company began implementing “big data” a little later than Sberbank. Today, they use big data to segment and manage customer outflows, generate financial statements, analyze reviews on the Internet, and much more.
Alfa-Bank Big Data helps to monitor the brand image on the Internet, assess the creditworthiness of new customers, personalize content, manage risks, etc.
Big data in business
Many people mistakenly believe that working with big data is relevant only for the banking sector and IT companies. This refutes the example of the Magnitogorsk Iron and Steel Works, which developed the Sniper service to reduce raw material costs in production. The technology collects large amounts of information, analyzes them and gives recommendations for optimizing the costs of materials.
Surgutneftegas uses a special system to track the main business processes in real time. This helps in automating product accounting, pricing, providing staff with the right data, etc.
Big Data in Marketing
Marketers use big data to predict the results of advertising campaigns. The analysis also helps in identifying the most interested audience. A striking example of “big data” in marketing is Google Trends. The system receives a huge amount of data, and after analysis, the user can assess the seasonality of a product (work, service).
Difficulty using
Where there are great opportunities, great difficulties await there. This rule has not bypassed big data.
The first difficulty companies face is that big data takes up a lot of space. Yes, storage technologies are constantly improving, but at the same time, the amount of data is growing steadily (on average, twice every two years).
The acquisition of a huge storage does not solve all the problems. There will be no sense from simple data storage, you need to work with them to get benefits. From here another complication arises – arranging the processing of the received big data.
Now analysts spend 50-80% of working time to bring information into a form acceptable to the client. Companies have to hire more professionals, which increases costs.
And another problem is the rapid development of big data. New tools and services for work regularly appear (for example, Hbase). Businesses have to spend a lot of time and money in order to “be in trend” and not lag behind development.
Thus, big data is a combination of technologies for processing large amounts of information (hundreds of terabytes and more) and today few people deny their importance in the future. Their popularity will grow and distribution in the business will increase. Subsequently, they will develop technologies for analysis automation and not only large companies, but also medium-sized and small ones will work with big data.
