Big Data. How IT employees improve their qualifications

There is a myth that large IT companies hire specialists who can do everything and are ready to work on any projects. We do not think so, and therefore decided to tell how our employees from the Data Management Center learn.

Big Data Workshop

First, a little about the Center. All the magic of creating data warehouses, implementing BI systems, as well as organizing work with data and developing the Data Governance direction takes place here. By the way, recently the first proprietary software product of the Jet BI Center was included in the register of Russian software. Our Data Management Center employs a strong team of architects and designers of intelligent systems, and other specialists who gain experience while working on projects and go through various training programs, which two of our employees, Sergey and Anton, will talk about in this post.

There is a plan

The specifics of working with data is such that we constantly have to learn something new: approaches change, new products appear, the company often enters into new partnership agreements with vendors. Therefore, we meet at least once every two months to discuss and adjust the development plan. At these meetings, managers ask us in which direction we want to develop, and based on our wishes and needs for projects, a single solution is developed, what should be done in the context of learning and self-development.

Of course, we do not set new tasks for the study of specific technologies every two months, but having determined the vector of development, we set milestones in the form of intermediate results that bring us closer to the goal. We have extensive topics, for example, Big Data, Data Lake, Hadoop, etc., and it is unrealistic to study everything at once in 2 months. Therefore, the leader formulates the stages.

If a new technology appears on a project, our individual development plans stipulate that in two months we will need to know certain aspects of the topic. This knowledge is tested by experts – more experienced employees. Accordingly, time is allocated to work on such an individual development plan (IDP).

Before qualifying meetings, you can talk with more experienced colleagues, for example, architects, who, based on the results of the conversation, will tell your immediate supervisor whether you can assume that you have figured out a specific stage and move on to the next.

Anton:

“I was simultaneously solving work problems and developing within the framework of my individual plan. It happens when the tasks for the IPR do not quite coincide with the tasks for the project: for example, the project has been transferred, there is a task, but there is no project. Here, somehow independently, not to the detriment of the time allocated for the project, you have to master something. “

Usually the task is divided into four large stages: 1) basic study of possibilities, theoretical understanding; 2) application on a project, mastering practical knowledge under the guidance of a mentor; 3) independent application of technology on a project without the help of more qualified specialists. 4) obtaining a certificate in this area (if possible), or passing a referral exam.

Vendors

Oracle. Initially, our department, and now a large Center, was created on the basis of Oracle products: we dealt with the Oracle database, the Oracle BI reporting system, Apex and Data Integrator. We have been actively studying these products since 2008.

Sergey:

“Since 2008, I have been responsible for the Oracle Data Integrator business. I have been trained several times, for example, using Data Integrator to build data warehouses. Before the pandemic, all training was in-person, I went to Oracle training classes, performed practical and theoretical tasks. It was serious and dense training, in my opinion, more thorough than Zoom or Skype. Now we are actively consolidating all this in practice. “

Anton:

“I have been trained in Oracle relational databases in various courses. These courses were always held in the training center, the lessons lasted the entire working day, and the practical tasks were worked out in the classroom on working machines with tuned sandboxes. The training took from 3 to 5 days from morning to evening. It makes sense to take courses to structure knowledge and fill in the gaps. This is not necessary to obtain a certificate, but for further work. For example, I took the Oracle SQL exam, but I did not undergo any training: I prepared myself and passed it myself. “

Arenadata. The course we took from Arenadata was called “Arenadata DB Operations”. It included a theoretical block and practical tasks that we performed in a specially organized “sandbox”. For this, a platform was prepared, each of us was given access, and allocated our own instance of the cluster, on which you can experiment with installing and running the DB. You can check the software that comes with the delivery, the execution of the simplest commands, for example, creating tables, assigning roles, etc.

The program was very rich: in 4 days we covered the whole course. It was difficult to perceive quite complex technical information for 8 hours, the course was not conducted by appointment. But from the handout, we still have presentations for each day and scripts for laboratory work. As a result, we passed the certification exam, and only a few passed the first time. The tasks in it were not so much theoretical as to find a specific solution, for example, “Which team is responsible for installing the server?” etc. As a result, we were able to successfully pass the exam and get certified.

“Unidata”. In 2020, the Center began contacting Unidata, and they offered to join their webinars on MDM systems. The training took place during quarantine, when many left to work remotely, and the course was called “Unidata strengthens the fight against coronavirus” J.

All lessons took 5 days in the format of online webinars with a teacher and a chat, where you could ask questions, to which he later gave answers. In terms of efficiency, this is slightly inferior to face-to-face classes, where you can clarify any incomprehensible points at once, and the teacher will also give examples. Still, an online teacher cannot immediately answer all the questions that are asked, because 30-40 people can participate in the webinar.

All our employees who had the opportunity to combine the course with their project uploads attended the webinars. Now we are actively consolidating all this in practice. The lessons provide a general understanding of a cluster of MDM systems and what the platform consists of.

Informatica. We have recently been trained on the American platform Informatica. It was organized by DIS Group, Informatica’s representative in Russia and the CIS countries. And the course was called “Data Integration for Developers”. We covered the main points on the use of computer science in the power center. The training took 5 days, each lesson had an introductory part and then practice.

Why all this?

The need for training arises primarily due to the fact that certificates and knowledge are required to participate in any tenders and competitions. When there is no practical experience in the application of a specific technology, you need to at least master the theoretical knowledge and obtain certificates. In the future, this can help to apply knowledge in practice in specific projects.

If a company has not been trained and does not have a certificate, then work on vendor platforms is, in principle, possible, but with certified specialists, partnership agreements are concluded, and all processes become much easier.

Through trial and error, you can learn anything, but another big plus of the courses passed is that you can use the stored materials: it is difficult to master the entire amount of information and remember all aspects, and when knowledge is really needed in practice, you can refer to the materials again. They are often more well-designed than those found on the Internet, and this is also a big plus. Professionals who teach classes correctly structure information and convey it more effectively.

We would be grateful if you would share in the comments how the training is going in your company.