Fielmann AG, Controlling Analytics – Definition and implementation of an AWS Analytics Lakehouse architecture.
About the Client:
As a publicly listed family business, the Fielmann Group has more than 900 locations in Germany and Europe, and supplies more than 27 million customers across Europe with glasses, contact lenses and hearing aid. In addition to its exceptional customer focus and the fair price of its products, Fielmann has been providing customers for decades with high quality and excellent service. Fielmann is the price leader, and also regularly sets standards for new technological developments within the industry.
The Challenge:
Via an omnichannel business model with digital sales channels, the Central European market leader supplies both brick-and-mortar and online retail. In different departments across the entire customer organization, different product teams work on the collection, evaluation and provision of data from the sales and eCommerce systems.
The Fielmann Group has established a central data analytics team that provides various evaluations and dashboards within the customer organization in an easily accessible and usable way. The aim is to enable the business to make its decisions data- and insight-driven. Here, the “one-stop store” idea is pursued of making all relevant data available at a central location. Depending on the skills of the analysts, there are opportunities here to prepare data themselves, to carry out efficient ad-hoc evaluations, to create management reports or to gain new insights with machine learning.
The omni-channel business model and the associated digital transformation require an agile, scalable and high-performance analytics architecture. In addition to new data sources (data mesh architectures, streaming data, APIs), legacy applications (classic databases) must also be integrated efficiently as part of mapping the entire customer journey on- and offline. Furthermore, the requirements for the availability of information from a wide range of stakeholders (controlling & finance, sales, marketing, logistics, etc.) must be mapped.
The Implementation:
Implementation of the Architecture: “Analytics Lakehouse”
PROTOS Technologie GmbH supports the Analytics division in planning and automated provisioning of the AWS infrastructure for various ETL pipelines, provisioning of ready-to-consume data sources as well as with explorative data analyses in order to optimally provide stakeholders with enriched data. PROTOS supports the migration of existing infrastructure components, data services, pipelines, and data artifacts for the new architecture.
In order to implement the processing with Big Data frameworks and modern database technologies sustainably and for the most diverse data producers or consumers, previous analytical data pipelines are migrated and converted from a data warehouse concept to an “analytics lakehouse architecture” based on AWS services (AWS S3, AWS Redshift, AWS Glue, AWS Lambda). In addition, PROTOS supports with components for an easy and efficient handling of data across departments, such as dbt for data provisioning and state-of-the-art reporting tools for visualization and adhoc analysis.
Provision of the Infrastructure: “Automation”
The modernized infrastructure is deployed using Infrastructre as Code (IaC), making it consistent and under version control. AWS CDK and hashicorp Terraform are used for this purpose. By using AWS managed services such as AWS Codebuild, the CICD infrastructure does not need to be manually maintained.
The automated deployments for the individual components as well as the customization of infrastructure and ETL jobs are built using DevOps best practices (AWS Codebuild + Codepipeline, GitHub Actions). The high level of automation and potential of serverless/elastic cloud architectures is reinforced by the use of AWS Elastic Container Service (ECS), AWS Lambda in conjunction with AWS Step Functions.
Implementation of Data Pipelines
To enable different data consumers to work efficiently with the data, the focus is on high quality data by enriching it and putting it into an analytics-ready structure. PROTOS supports the analytics team at Fielmann, particularly in the development of data pipelines and ETL jobs. Spark is used on AWS Glue, AWS Redshift and in combination with AWS Lambda. This provides the product teams with ready-to-consume data sources.
The Benefits:
Data availability maps to the greatest added value. Relevant data is available centrally and harmonized across teams. By using AWS managed services, scaling and flexibility has been achieved and thus the reduction of time-to-market, for example, in the development for management dashboards. Maintainability and further development are future-proof.
Further Information:
For more information on cloud, infrastructure-as-code, terraform, serverless and DevOps, feel free to check out the PROTOS Technologie blog.
Your PROTOS Team
