Data Quality Audits and Assessments

Whatsapp

Every organisation that works with meaningful volumes of data should conduct periodic data quality audit and assessments, to ensure that data quality is adequate and that they are not exposed to unnecessary risks.

Data Quality Audits provide an accurate measure of your data quality

Have you failed a financial or regulatory audit?

Are you implementing a new system?

Is your data appropriate to deliver the required reports?

How will you address known (and unknown) data quality issues?

How bad is your data quality really?

Table of Contents

There should be no debate that poor-quality data costs money.

Indeed, according to Gartner Research, “the average financial impact of poor data quality on organizations is $9.7 million per year.” But which end of the scale do you fall on? Is the quality of your data better, or worse, than average?

A data quality audit or assessment will give you an accurate answer and should be performed before engaging in any significant data-related project.

Why do you need a Data Quality Audit?

Does your data comply with data policies and standards?

How will the Protection of Personal Information (PoPIA) impact your business?

How does your data support your requirements to comply with regulations and legislation such as RDARR, PoPIA, the Solvency and Assessment Management regime, and many more?

Where is your data held, and what changes do you need to make in order to be compliant?

Can you identify where data does not comply with required business standards, and track improvements over time?

If you have concerns in these areas then contact Master Data Management for a data quality audit. Understand how we use market-leading data quality tools supported by experienced consultants, to rapidly assess the fitness of your environment to meet your specific business needs. 

When to perform a data quality audit

Every organisation that works with meaningful volumes of data should conduct periodic data quality assessments, to ensure that data quality is adequate and that they are not exposed to unnecessary risks.

However, there are some specific circumstances that should trigger a data quality audit, including:

Before making an important, data-dependent decision.

Understand where existing data may need to be enhanced to be updated to be suitable for its new use.

For example, one client invested tens of millions in a new client segmentation model, only to find that the data points proposed by the external consultant were poorly captured and useless for analytics.

A data quality audit would have allowed them to plan the project better, focus on the data they had, and deliver the required outcomes.

Before implementing a new system:

Any data migration carries risk. Whether you are rolling out a new ERP, implementing master data management, or moving your EDW to the cloud, a data audit identifies data risks before they become issues.

For example, one client rolled out an integrated planning application not realising that the data in their ERP was not sufficiently accurate, complete, or consistent to be used in the new planning system. 

The result - the project failed to deliver the expected returns.

A data quality audit would have allowed them to plan the project better, adjust for the data issues, and deliver the required outcomes.

To comply with regulations:

Data issues can, in turn. lead to non-compliance with data-dependent regulations such as the PFMA, PoPIA or the Basel Accords. For example, at one client we helped them to identify and resolve data issues that had caused them to fail a financial audit

Before beginning a data clean-up:

It may sound obvious, but many organisations engage in data cleansing with no idea of how bad the problem is. A data quality assessment gives you a baseline against which you can prioritise, identify opportunities for process or automated improvement, and measure the progress of manual cleansing.

Stop relying on assumptions about your data quality

Each data quality assessment is tailored to your specific business drivers and can be delivered much more quickly, and more accurately than manual/SQL approaches.

A data quality assessment typically comprises 5 steps:

1. Information gathering: A high-level understanding of business goals, processes and known issues
2. Identification of data: Confirm which data is in scope, where it is stored, and how to access it.
3. Data profiling: Measure data quality including completeness, uniqueness, consistency and referential integrity
4. Confirmation of findings: Discussions with stakeholders to confirm findings and impact
5. Report: Finalise reports and other deliverables with business context

Actual measures of data quality issues (known and previously unknown) will be exposed to your business and technical stakeholders to allow you to accurately identify the scope and risk of any remediation effort.

Deliverables of a data quality assessment typically include

• Review of key business objectives and relevance of approach in this regard. 
• Various assessment reports and recommendations (sample exec summary below)
• Roadmap/plan for delivery

Reuse your investment

The last thing you need is to invest millions in new systems only to find that the data quality issues that plagued you previously have resurfaced within months of your go-live date.

Our approach allows new systems to reuse data quality rules and automate data cleansing and management processes both to improve data quality and to maintain desired levels.

FAQ

What is the difference between a data quality audit and an assessment?

A data quality audit is the process of evaluating the quality of your business data to ensure its accuracy and reliability, typically in response to a specific event. A data quality assessment is repeated periodically, to ensure that data quality standards are being met and maintained or improved.

Why are data quality assessments important?

Data quality assessments are important because they ensure that your business decisions and operations are based on accurate and reliable data. Poor data quality can lead to incorrect insights and decisions, which can have serious consequences for your business, and is proven to cost businesses millions each year due to operational errors and issues.

How often should I perform a data quality assessment?"

The frequency of data quality assessments depends on the nature of your business and the volume of data you handle. However, it's recommended to perform a data quality assessment at least once a year, and more frequently for critical data sets.

What are the common data quality issues?

Common data quality issues include missing and incomplete data, incorrect data, duplicate data, inconsistent data, and outdated data.

How can I improve data quality?

To improve data quality, you can use data profiling tools to identify issues, implement data cleansing processes, and establish data quality metrics to measure and monitor data quality regularly.

Why do I need to use a data profiling tool?

Data quality issues are frequently poorly understood and hidden. Data profiling tools, like Trillium Discovery, run predefined rules to automatically discover data quality issues inherent in the data. They make unknown data issues visible to business data stewards and other stakeholders, quickly and at scale. This keeps business owners in the loop and helps to build integrity

By contrast. SQL approaches depend on (typically) junior technical resources asking the right questions. Questions that they do not have the subject matter expertise to ask.

Data profiling is proven to provide results that are up to 90% more accurate in a fraction of the time taken to deliver similar insights using SQL or BI tools

Whatsapp