Architecture
The Data Quality Framework implementation contains 3 physical modules:
The input Datasets
The input datasets form the foundational layer of the Data Quality Framework. These datasets are sourced from various origins, such as databases, data warehouses, or external data providers. They contain raw data that needs to be validated, cleaned, and transformed before it can be used for decision-making or further processing.
The Data Quality Engine
The Data Quality Engine is the core component of the Data Quality Framework. It is responsible for executing data profiling, applying data quality controls, and managing the results of these checks. The engine operates on the input datasets, ensuring that they meet the required quality standards before being processed further.
The Datamarts
The Datamarts are the final destination for the cleansed and validated data within the Data Quality Framework. After the data has passed through the Data Quality Engine, it is organized into datamarts for efficient storage, retrieval, and analysis.