Business Goals
Our Client is a UK-based international retailer with operations in multiple European countries. They had independent business units operating from various regions. For business reporting and decision making, data generated across all business units were collated at a group level.
They had structured data, semi-structured data like XMLs and unstructured data like images and audio files. Each business unit had its tools and data management methodologies. So, gathering varied data from different sources, in different formats, and different regions is challenging. The data extraction, transformation and analysis were carried out predominantly through manual processes.
Quality and availability of the data in time for business decisions was an issue. They wanted to establish a robust data governance strategy because it is crucial for any organisation that uses data to drive business growth. They wanted automated processes to effectively manage risks, reduce costs, and execute business objectives.
How NeosAlpha helped
Our Azure integration engineers organised multiple workshops with the retailer’s IT team to understand their current data structure and lineage. They architected a program to break down the data silos through a collaborative process with stakeholders from separate business units. As part of this data governance program, data was organised and accurately entered into systems.
We delivered a fully automated Azure data lake initiative for reference and transactional data from different business units. The solution included:
- Identify: Documenting data from different data sources and watermark every section for periodic transformation intervals.
- Ingest: Using data pipelines in azure data factory to ingest and validate data for quality on a periodic storage in azure data lake.
- Egress: Retrieve data from Data Lake and convert into staging tables for Azure Sql Server.
- Transform/Stage: Modeling data to load in data warehouse tables for agile reports.
- Load: Ingest data in clients warehouse.
- Analytics and Reporting: Creating minimal reports in Power BI for better solutions.
- Error Handling: Working with Power BI reports for quick solutions.
Results
NeosAlpha delivered a centralised data management solution using Azure’s capabilities and services. Following are the benefits brought in by this solution.
- Single source of truth. Data sets from different business units shared a common terminology and view.
- Improved data quality. The data was complete, consistent and compliant.
- Automation of data management helped reduce costs.
- Real-time data available to analyse and generate business intelligence reports
