Should you care about data transparency?

Yes, data transparency delivers value no matter how you define it!

What is data transparency?

Data transparency refers to the openness and accessibility of data collection, use, and sharing practices. It is the act of making data easily accessible and understandable to individuals, which helps promote accountability and prevent misuse of data.

It is essential to easily access and work with the right data to:

• make informed, confident decisions
• complete projects in less time than before
• improve customer service and performance
• track financial obligations, team performance and allocation of resources, 
• and to allow comparison to other similar reports that may present conflicting results.

The cost of not finding information

Why are most institutions still struggling with hidden and lost data?

The status quo of data chaos and opacity is understandable. Companies are buried under an avalanche of data, and the problem is getting worse.

More businesses are shifting to digital processes and platforms which in turn are generating more and more information. Everything spits out data, and institutions need to make sense of that data to understand their customers, and to understand the trends that are affecting their market and powering their business.

Between 80% and 90% of this data is so-called dark data - operational data that is left unanalysed. The more data an organisation collects directly correlates to how difficult it is to manage, analyze and achieve these values. It’s not for lack of interest, value, or investment. 

Dark data creates risk

On the one hand, dark data presents significant security and compliance risks.

Enterprises now store enormous volumes of data across different cloud platforms, often with little or no visibility into which data resides where. It’s critical to adopt a more sophisticated approach that provides comprehensive visibility into the organization’s asset footprint and enables specific steps to reduce risk.

Dark data has unquantified value

On the other hand, dark data may have value, but it is hard to quantify if organisations don't know what is being stored.

Storing this data typically incurs more expense than its value. Destroying it might be too risky, but analyzing it can be costly. A lack of understanding of the data on behalf of your data analysts and data scientists can bloat your big data costs by creating unnecessary delays and data quality issues whenever they touch your data.

"Everyone talks about going digital, but if you’re not capitalising on data streams that are generated through your digital channels, then you’re going digital without listening (to your customers)"

Aidan Millar, Chief Data Officer, DNB Bank

Moving from dark data to deep data

Deep data refers to big data that is of high quality, relevant, and actionable.

It provides answers and solves problems by extracting information from data sets that are hosted on a complex and distributed architecture, with the implementation of data analysis algorithms and techniques. AI-driven consumer and product insights can be used to make companies stand out from their competitors.

We create deep data by improving the integrity of the data we hold. This means adding both business context and quality to the data we plan to store and analyse.

Enter the data supply chain

To make better business decisions, leading organisations are repurposing a well-understood discipline, supply chain management, to better manage data as it enters the organisation, is stored, used, and distributed for analysis. The data supply chain is built on four pillars of data governance:

1. Data discovery, classification and cataloguing of data assets to make them easy to find, access and secure
2. Building business context through metadata
3. Providing robust and current data lineage
4. Measuring data's worth

Data Transparency and PoPI Act

South Africa's Protection of Personal Information Act (PoPIA) takes transparency seriously.

Condition 2 regulates how personal data must be collected (to protect the rights of the data subject), limits its use to a specific, legitimate purpose, and requires the consent of the data subject, except under specifically defined conditions.

The Act requires that personal information is protected from unauthorised access, which in turn requires that PII be identified and that any breach is reported to both the regulator and the data subject.

In addition, PoPIA gives the data subject the right to request a responsible party to confirm, free of charge, whether or not the responsible party holds personal information about the data subject, and to request further information in this regard. A Single Customer View (SCV) is important for PoPIA compliance as it helps organisations with a single point of reference for this requirement. 

Data transparency is a material requirement for compliance with the PoPI Act.

Transparency is essential for building trust with customers, as it helps to re-establish trust between consumers and businesses that handle their data.

Additional points to explain data transparency

Here are some key points to further explain data transparency:

• Data transparency provides customers with an inside look into how their data is collected and used. Customers should know why their data is needed, how it is gathered, where it is stored, and how it is protected

• Data transparency is an ongoing process that should be maintained throughout the entire relationship between a business and its customers. If new processes expose data to different parts of an organisation or a customer's data will be used for purposes other than what they initially agreed to, they should be made aware of it.

• Data transparency is an essential component of compliance with data protection regulations, such as the GDPR and the PoPIA. Data controllers must be able to demonstrate that personal data are processed transparently in relation to the data subject.

• Organizations can use user-friendly procedures to inform people about the purpose of holding personal data, how it is processed, and intentions for transferring data to other parties. Clear language should be used to state the organization’s efforts to be transparent and its compliance with applicable regulations.

• Transparency is also essential for all the organizations handling the data throughout the data’s life cycle.

In summary, data transparency is about giving people the ability to see how data is used, collected, and shared. It is an essential aspect of building trust with customers and ensuring compliance with data protection regulations. Whatever your goal, you should care about data transparency.

FAQ

What is data transparency?

Data transparency refers to the practice of making data easily accessible and understandable to those who need it, such as stakeholders or the public. This can involve providing clear and concise information about the source of the data, how it was collected, and any limitations or biases that may be present. The goal of data transparency is to increase trust in the data and the organisations using it, as well as to encourage accountability and informed decision-making.

Why is data transparency important?

Data transparency is important because it promotes trust, accountability and informed decision-making. By providing clear and accessible information about the data being used and how it is being used, individuals and organisations can make more informed decisions through data integrity, and trust is built between data providers and users. This can lead to better accountability, increased public trust, and ultimately better outcomes for businesses and society as a whole. Additionally, data transparency can help prevent unethical or illegal practices related to data usage, such as data breaches, misuse of personal information, and discriminatory practices helping to support data privacy initiatives.

Is data transparency important for AI

Yes, data transparency is crucial for AI.  For AI systems to be trustworthy and ethical, it is important to have transparency around the data that is being used to train and operate the system. This means understanding where the data came from, how it was collected, how it was labelled, and any biases or limitations that may be present in the data. Data transparency can also help to identify errors or biases in the AI system's decision-making, and can facilitate accountability and oversight.

Some examples of data transparency include:

1. Making public records and government data available online.
2. Disclosing data collection and usage policies to users and stakeholders.
3. Providing clear and concise data visualisations that are easy to understand.
4. Allowing individuals to access, review and correct their personal data held by companies and organisations.
5. Conducting regular audits of data collection and usage practices to ensure compliance with data privacy laws and regulations.
6. Publishing reports on how data is collected, analysed and used by companies and organisations.
7. Providing explanations for AI-powered decisions and actions based on data.Providing transparency into the data sources and methodologies used in scientific research.

Data management plays a critical role in enabling data transparency. 

Effective data management practices ensure that data is collected, stored, processed and shared in a way that promotes transparency and accountability. Here are some ways in which data management enables data transparency:

1. Clear data policies: Data governance establishes clear policies around data collection, usage, and sharing that promote transparency and ensure compliance with relevant laws and regulations.

2. Data quality assurance: Data management processes help to maintain data quality, ensuring that data is accurate, complete and consistent. This is crucial for building trust in data and ensuring that decisions based on the data are informed and accurate.

3. Data traceability: Effective data management ensures that data can be traced back to its source, enabling transparency in data collection and usage.

4. Data accessibility: Data management practices ensure that data is accessible to stakeholders, allowing them to review and verify data and supporting transparency.

5. Data security: Data management practices help to ensure that data is protected against unauthorised access, maintaining confidentiality and privacy and promoting transparency.