They Manage Big Data in Fintech. How We Find Top DWH Specialists
A sign of the times – businesses are quickly acquiring huge amounts of data. If there is big data, then you need a team that can deal with it. I am Olga, a recruiter at Clevertec. Our specialists develop corporate data warehouses. I will tell you from my own experience how we find them.
Who is needed to work with DWH?
Ideal team scheme:
Architect (methodologist)
Business analyst
Systems Analyst
Developer
Tester
The number of teams needed depends on the repository. The architect remains the lead link, the keeper of the concept. The rest of the set of specialists is duplicated to work in different areas.
We strengthen DWH teams with systems analysts, developers and testers.
People with experience are always needed to work with big data. The resume usually shows how the person started working with regular databases. From project to project, these databases became larger and, accordingly, the list of tools and technologies that the specialist mastered expanded.
Here is an example of the transition from working with databases to DWH
I have 10 years of experience working with databases in the IT-fintech sector, five of which were with Oracle. When you start working with a DB, you constantly encounter problems that force you to dig up a lot of information in search of a solution. You look for answers, go deeper, and so you gain experience.
Over time, most tasks on a project become routine. I had a choice: expand my stack and area of responsibility within the current project or try my hand at a new one. As a result, I switched from OLTP systems to DWH.
What names do DWH specialists hide under?
It's interesting to look for people to work with DWH because of how differently they call themselves. When I first started looking and did some preliminary research on the topic, I was surprised by the abundance of options.
Database systems analyst, DWH analyst, SQL developer, ETL developer, data engineer, DB developer/analyst and the list goes on.
Why does this happen? It's about the organization of work on projects and the division of responsibilities. For example, not all projects have a clear boundary between system analysts and DWH developers. Therefore, a person whose competencies are needed by the business calls himself a DB analyst in his CV, but in fact he can successfully perform the tasks of a developer.
To avoid tripping over these conventions, I always search for signal beacons. For example, a bank corporate storage needs an analyst. Knowing the project stack, I first look for people with a connection SQL+Oracle and I draw attention to the general background.
And then, in the process of communication, we find out together whether the candidate and the vacancy are suitable for each other. My task is to find a successful combination of hard and soft skills. It is impossible to say unequivocally which is more important. You can safely turn a blind eye to the most pumped-up hard skills if the person does not find a common language with the team and does not share its values.
Hard skills for working with DWH
Each data warehouse is unique. The set of technologies is determined at the stage of creating the warehouse. Many factors influence the final result: what kind of data, how often it needs to be accessed, what is its volume, what resources the company has, what time frames for forming processes and how much the company is ready to spend on technologies.
All of this forms the necessary stack of a candidate. It is impossible to make an exhaustive list of must have skills. We have summarized the tools and technologies that are most often needed for our team.
Developer DWH owns technologies that support data warehouses.
Works with relational databases (RDBMS) and NoSQL, ETL tools – Informatica, data programming languages: SQL, PL/SQL, Java, Python and C++. Knows DBMS: Oracle, SQL Server and new technologies – Hadoop. Business analytics tools may be needed in the work: Tableau or Power BI.
Systems Analyst DWH understands the architecture of storages, the concepts of building DWH and ETL, but the key for him is an advanced level of SQL. A big plus is an understanding of multi-tier architectures, client-server and distributed systems, microservice architecture, WSDL, basic knowledge of the HTTP protocol, SOAP, experience working with a bundle of Oracle + ETL Informatica, Git.
Where to look for candidates with the required experience?
A common requirement in job descriptions is experience in a similar field. For us, this is fintech. There are many banks and other financial organizations, they work closely with big data to improve business indicators. This means that the market for specialists in this field has already been formed. At the same time, DWH specialists cannot be called “popular” and offers for them do not pour out of the horn of plenty.
But experience in fintech will not be a decisive factor in choosing.
It is immediately obvious: the right person has a completely unique way of thinking, a habit of working hard and almost never stopping. Usually their eyes are shining, they are immersed in their work with great interest, and learning something new in their field is not a problem for them.
To fill a vacancy, you usually don't have to use highly specialized channels for searching. These specialists live on large platforms – hh.ru, LinkedIn. Then only the recruiter's skill and a quality offer decide.
Of course, there are periods of calm: the weather on the market is unfavorable, everyone is in the waiting stage. But here you just need to be patient and wait for that very “goldfish”, of which, by the way, there are many in our team.
This was my experience of finding the right people for the team. In the comments, let's discuss your views or unusual cases from practice.
