Data quality vital for sound BI decisions

Business intelligence (BI) is touted as the enabler of sound business decisions through the use of accurate reports. However, the high failure rate of BI implementations over the years shows that simply incorporating BI into the organisation and producing reports is not a silver bullet, writes Gary Allemann, senior consultant at Master Data Management. 
Fundamental to the success of a BI project is the quality of the data, and if businesses do not address data quality (DQ), it is akin to putting the cart before the horse. DQ tools are therefore an imperative for organisations that are embarking on a BI project, as well as those companies that have already implemented a BI solution but are not yielding the expected results.
When an organisation has a BI solution in place, CEOs and decision makers are led to believe they are making decisions based on fact, when often the reports show only a version of the truth that may be inaccurate or have been manipulated.
The reality is that quality data is vital in making quality business decisions. It is very well having a highly sophisticated BI solution that generates reams of reports for businesses to base decisions on, but if the data these reports are generated from is poor, the reports will be correspondingly poor and bad decisions are likely to be made.
While the failure of BI may be attributed to a number of factors, research shows that poor data quality is the major contributor towards the high incidence of failed BI implementations. This has led to a growing trend for organisations to begin looking at DQ solutions in their own right, rather than simply as part of the overarching BI system or as an add-on solution.
Historically, DQ budgets have been spent on manual, unrepeatable processes, which fail to yield on-going improvements and leads to a lack of understanding from business as to why money needs to be spent in this area.
The decisions makers also do not realise that business problems can be a result of data problems, since data is technical and belongs to information technology (IT). The relationship between data and business processes has not been clearly understood. However, all business processes rely on data.
Poor data quality is often hidden from the business, since IT will spend vast amounts of time and money producing results that business can make sense of in the form of reports, but the information contained in reports may not bear a complete resemblance to the underlying data. Business assumes that reporting is based on correct, quality data, but this may not always be the case.
The recent partnership between Qlikview, a BI vendor, and Trillium Software, a specialist best-of-breed DQ solutions provider, highlights a growing trend for BI vendors to incorporate comprehensive data quality software as part of a complete BI offering, due to growing customer demand.
Decision makers are now realising the connection, and are beginning to understand the need for better quality data in order to drive better decision making.
The implications of poor data quality are highly dependent on the particular business, but since every critical business process relies on data from the supply chain to credit risk to invoicing and so on, the effectiveness of these processes relies on accurate underlying data. For example, if the data received for invoicing is incorrect then incorrect invoices will be issued.
This results in poor collections and has a negative impact on the company’s reputation. Even legitimate invoices may be queried or have delayed payment, resulting in cashflow problems and the massive, unnecessary expense of resolving each query. Therefore, poor data quality can introduce a host of problems to any organisation including risk, financial and damage to the organisation’s reputation.
When it comes to DQ, there are two paths a business can take. The company can either accept poor quality data, or implement a sustainable, repeatable and auditable solution that ensures data is captured “right” the first time, managing DQ proactively and preventing quality issues before they become problems.
DQ solutions should be selected much like any other business critical tool, based on its ability to solve a specific business problem.
This means that users should evaluate data quality solutions in their own right, just as they would evaluate any other critical infrastructure. They should consider both technical aspects – such as the ability of the solution to integrate into data sources, and to support both batch and real time data cleansing – and business aspects.
Other questions that should be asked include: does the platform provide a quick start in the form of pre-packaged knowledge of the data? Does the vendor bring data management experience to the table or are they simply dropping a product that is not their core focus?
The importance of DQ quality is a growing trend within the BI space, as highlighted by the fact that pure play DQ vendors such as Trillium have seen impressive growth in Europe and the USA, even in the grips of a massive recession. While South Africa tends to lag behind the technology curve, the 2011 ITWeb BI Survey shows that a quarter of all respondents have cited data quality as a reason for the failure of BI.
While only 10% of the respondents stated that a DQ solution was on the cards, users are beginning to see investment into best-of-breed DQ solutions in the country, particularly within financial services, and following international trends, consumers can expect growth in this space in the next few years.