How a business analyst can organize work in DWH (Data Warehouse) projects. Work structure
Hello analysts!
Last time we figured out how a business analyst can establish interaction with stakeholders and turn their conflicting requirements into a single system.
Today I want to talk about the further work of an analyst in data warehouse (DWH) projects. How to go all the way from collecting requirements to implementing a ready-made solution, while maintaining a balance between the interests of business and IT.
We have already identified user needs and reached consensus on the goals of the system. Now it's time to translate these requirements into real DWH. And here we are faced with no fewer pitfalls than at the approval stage.
How to avoid misunderstandings between business and IT during design? How to make sure that the developed solution really solves the customer’s problems? And how can we help users transition painlessly to the new system?
I will share with you my experience of participating in DWH projects from the planning stage to supporting the finished solution.
Stage of collecting and analyzing business requirements
The beginning of any DWH project is a deep understanding of business needs.
Determining business needs is one of the key and most time-consuming stages in DWH projects.
It is important to agree on the research plan with the project manager from the business side – determine a list of respondents from different departments, develop an interview questionnaire, and distribute roles in the project team.
Next, interviews are conducted with company employees – usually 2-3 representatives from each key department. The goal is to collect and document their needs and wishes for DWH as completely as possible.
After a series of interviews, the collected requirements are analyzed and systematized. It is important to identify and eliminate all discrepancies and contradictions between the wishes of different departments.
Next, a working group is formed with the participation of business and IT representatives to jointly discuss the research results. The goal is to arrive at an agreed vision of the goals and priorities of the DWH project.
The result of the stage is a formalized BRD (Business Requirement Document) requirements document, as well as recommendations for organizing subsequent stages of the project – composition of teams, selection of performers, step-by-step work plan.
Thus, it is at the stage of gathering requirements that a solid foundation is laid for the DWH project, which allows minimizing risks in subsequent stages.
Formation of detailed business requirements
After defining the general goals and objectives of the DWH, the stage of detailed elaboration and documentation of business requirements begins.
At this stage it is necessary:
Conduct an in-depth analysis of existing business processes, highlight the key processes that the system should cover.
Collect and analyze analytical techniques and reports used in the company. Identify gaps that DWH needs to close.
Conduct an inventory of customer information systems and data. Understand which systems will become data sources for DWH.
Elaborate and document the functional and non-functional requirements for DWH in the BRD.
Formulate KPIs and metrics that the system should calculate.
Conduct a series of workshops and agree on the BRD with all key departments.
The result of this stage is a detailed requirements document that is understandable to both business users and developers. It will form the basis for the design and development of a DWH solution
Information Research
At the stage of information research there is a transition from business requirements to technical ones.
Main tasks at this stage:
Analyze methods for calculating KPIs and indicators that DWH should support. Determine data requirements.
Identify all necessary business concepts (products, customers, transactions, etc.) and agree on their description with the customer.
Develop a semantic data model that describes entities and relationships between them in terms of the subject area.
Design an information model of data sources – what data is taken from each system.
Determine requirements and limitations for source data based on KPI and reporting requirements.
Develop DWH architecture options, including software selection, ETL workflow, data sources and data warehouses.
Make a list of risks and difficult areas of project implementation.
The result is an information and architectural model of the system, which is then refined and detailed during the design and development stages.
Solution concept based on DWH technologies
At the concept stage, business requirements and technical capabilities are combined into a holistic vision of the solution.
Main tasks at this stage:
Develop a solution concept document based on the research conducted.
Present the concept for discussion to a working group with the participation of business and IT representatives.
Agree on the priority tasks that DWH will solve within the first phase.
Approve criteria for assessing project results.
Distribute roles and responsibilities within the project team.
Receive final confirmation that the concept meets the customer's expectations.
Based on the results of the stage, a final concept document is formed, which records the main parameters of the project for subsequent development.
DWH design stage
At the design stage, a detailed technical design of the DWH is created based on the approved concept.
Key tasks of the stage:
Develop a conceptual data model – entities, attributes, relationships. This is the logical representation of data in DWH.
Design a physical data storage scheme – tables, fields, keys, indexes. This is already a physical model in the selected DBMS.
Select ETL tools and design processes for extracting, transforming and loading data.
Develop the architecture of the entire system – what components are located where, how they interact.
Design interfaces and reports in a business intelligence system.
The output is a set of documentation for the implementation of software, databases, ETL processes, reporting and interfaces.
Implementation and maintenance stage
After developing and testing the DWH, the important stage of introducing the system into commercial operation begins.
The main goal of the stage is to ensure the effective use of the system to solve business problems and continuous development in accordance with new needs.
Main goals:
Develop analytical reports, dashboards and KPIs that solve the customer’s priority tasks.
Organize training for key users and administrators on how to use the system.
Provide support to users after launch, help if questions arise.
Regularly collect feedback from users and record wishes for system development.
Gradually increase DWH functionality according to business priorities.
Proper implementation of DWH is critical to fully utilize the full power of the system for business purposes.
Useful document
Friends, in addition to the described stages of work on the DWH project, I would like to share a useful tool – a detailed structure of work in such projects.
In the attached file a detailed hierarchical structure of tasks that must be performed at each stage is presented – from identifying business needs to implementing a ready-made solution.
It includes work on research and analysis of business requirements, design of architecture and data model, development of ETL processes, interfaces and reporting, testing and system implementation.
For each type of work, a detailed description of the steps and tasks that must be performed in practice in real DWH projects is provided.
This is a very useful tool for both business analysts and project managers. It helps to understand the completeness and complexity of tasks, optimize resource allocation and manage risks at each stage.
I recommend studying the work structure before starting a new project to create a data warehouse. And then use it as a checklist and navigator when planning and monitoring the progress of the project. I'm sure this will increase your chances of success!
So, we have looked at the main stages of a business analyst’s work in data warehouse projects – from collecting requirements to implementing a ready-made solution.
As you can see, this is a rather long and labor-intensive path that requires careful planning, clear distribution of roles in the project team and constant interaction between business and IT.
The analyst is required to have the following skills:
deep understanding of business processes and customer needs;
ability to identify and structure requirements;
ability to find compromises between business and IT;
knowledge of design methodologies and best practices in the field of BI and DWH.
In this article, I tried to give a general idea of the analyst’s tasks at each stage, talk about typical pitfalls and ways to overcome them.
In the following publications, I will analyze specific cases in detail and give examples from my experience in implementing DWH solutions in various companies.
In the meantime, good luck to you in leveling up your analytical superpowers and building advanced data warehouses! Share your valuable experience in the comments – this will help us all become even better in our difficult, but so necessary task!
