Objectives and Benefits
Objectives
Summary
While Data Quality rules help formalize business requirements for Data to be considered fit for purpose, the objectives of data quality controls are to put in place mechanisms to enforce the rules and ensure compliance.
Data Quality controls are implemented to check that the data is consistent with the rules at various points of the data pipeline.
For a given Data Quality rule, there might be a necessity to implement several controls in various systems as Data is replicated, transformed and dispatched throughout the information system.
Beyond rule-based approach, it is possible to adopt statistical / ML based approach to implement data quality controls to identify anomalies based on known historical issues or trends.
At an airport, security checks, baggage checks, and aircraft technical inspections are standard procedures to ensure the safety and efficiency of air travel. Similarly, data quality controls at the enterprise are designed to ensure that information processed and used across the company is accurate, complete, and reliable.
Just as airport security inspections ensure that all items carried by passengers are safe and compliant with regulations, data quality checks verify data integrity. This includes validating the accuracy and completeness of data, ensuring that the data has not been inappropriately altered and that it accurately reflects reality.
Similar to preventative checks on aircraft to avoid in-flight problems, data quality checks aim to prevent problems before they become major challenges. By detecting and resolving inconsistencies or errors at the earliest stages, the enterprise minimizes the risks associated with poor data quality that could otherwise impact business decisions or customer satisfaction.
As customs procedures and passport checks ensure compliance with international laws, data quality checks help support regulatory compliance (as GDPR). They ensure that data meets all relevant legal and regulatory requirements, reducing the risk of breaches that could result in sanctions or reputational damage.
Defining data quality rules begins with a thorough understanding of the business requirements and objectives associated with each type of data. Here are the key steps to define these rules:
Identification of business requirements: For the enterprise, it involves understanding the specific needs of different players in the travel industry, such as airlines, travel agencies, and destination managers. This step requires close interactions with these parties to capture their data expectations, including:
Accuracy: Flight data, such as departure and arrival times, must be extremely accurate to avoid major disruptions to passenger itineraries.
Completeness: It is crucial that reservation information contains all necessary details, such as passenger information, seat preferences and special requests.
Validity: Pricing information must comply with complex and varied structures, depending on the pricing rules specific to each airline and each type of ticket.
Freshness: Ensure that flight schedules in the reservation system are regularly updated to reflect real-time changes. Data freshness is crucial to maintaining service reliability, especially in situations of last minute changes.
Consistency: Ensure that service descriptions, terms of sale and cancellation policies are presented consistently across all platforms and points of sale. Consistency helps build user confidence in the enterprise system.
Uniqueness: Guarantee that a reservation number is unique and is not repeated throughout the system. This avoids confusion and errors in processing reservations.
Formalization of criteria: This means setting up specific and measurable criteria requirements for the enterprise. This includes:
Format: Rules may stipulate that airport codes in reservation data must follow a standard format.
Time constraints: Booking date data must precede the travel date, and flight times must match the operational time slots of the airports concerned.
Documentation: Each data quality rule must be clearly documented to facilitate its understanding and application by all users of the enterprise system. This documentation should explain the purpose of the rule, how it is to be applied, and the procedures to follow in the event of non-compliance. For example, documenting a rule regarding the validity of fare information would explain how to verify that fares meet contractual agreements with airlines.
Review and validation: These rules must be regularly reviewed and validated by experts in the field to ensure their relevance and effectiveness. This involves periodic testing and compliance audits to ensure that the rules remain in step with changing industry standards and regulatory requirements.
These steps ensure that the enterprise has a data system of high accuracy and integrity, but also that they respond agilely to the changing needs of the travel industry.
Defining standard dimensions to structure Data Quality controls and analyze results is a key objective of the Data Quality framework.
Similarly, to dimensions in the measurement of physical objects (e.g., length, width, height), data quality dimensions provide a vocabulary for measuring data quality against a set of standardized properties.
Having standardized measurements will help the enterprise evaluate the level of Data Quality in a unified manner across teams and organizational units. This capability is an important prerequisite to make informed decisions on improvements that need to be implemented across the organization.
Benefits
Some of the benefits of Controls are the following:
Optimization
Data controls enable proactive and continuous monitoring of data quality. By quickly detecting and resolving data quality issues, these controls minimize interruptions and optimize business process performance. This ensures that the enterprise services remain efficient and reliable, essential to maintaining customer satisfaction and company reputation.
Accuracy
By ensuring that data used across the enterprise is accurate and meets established standards, data quality checks improve the accuracy of reporting and analysis. This improvement helps the enterprise make strategic decisions based on reliable and accurate information, crucial for the development and deployment of new products and services.
Compliance
Data Quality checks ensure that all data processed and stored by the enterprise meets current regulatory standards, such as GDPR for data protection. This helps prevent data breaches that could result in costly penalties and damage to the company's reputation.