Recently I had a discussion on the question if the goal of data management would be to achieve good data quality or if data quality is an enabler for the most different business purposes. Data quality (DQ) is, in my personal opinion, an extremely interesting
You probably use checklists to record and efficiently execute a wide range of daily work tasks. But if you don’t use data warehouse testing checklists for developing and monitoring your data warehouse quality assurance (QA), you’re missing an enormous boost in productivity and proficiency. Procedural
Today’s Data Warehoues (DW) source data often originates from a large variety of data formats as illustrated in Figure 1. Original sources may consist of external data, reference data, business transactions, production raw data and more. The multitude of data sources each has its own
Effective Data Warehoues (DW) data source profiling is often an overlooked step in data warehouse data preparation. DW project teams need to understand all quality aspects of source data before preparation for downstream consumption. Beyond simple visual examination, you need to profile, visualize, detect outliers,
This quick guide provides an overview of the basic concepts in fault tree analysis technique, as it applies to data quality. For some more well-known and useful root cause analysis techniques, please check out the: 5 whys analysis Fishbone diagram Barrier analysis Pareto analysis Definition
Data warehousing for business intelligence and “big data initiatives” continues to gain significance as organizations become more aware of the benefits of decision oriented data warehouses. However, a key issue, with the rapid development and implementation of data warehouses, is that data quality defects are
Fixing data quality issues is only part of the solution. In order for these not to reoccur, one needs to identify the root cause and prevent from further creating poor data quality. As we know, for every effect there is a cause, and that’s the
Data warehousing and business intelligence users assume, and need, trustworthy data.In the Gartner Group’s Online IT Glossary, data integrity and data integrity testing are defined as follows:Data Integrity: the quality of the data residing in data repositories and database objects. The measurement which users consider
There are different takes on what veracity refers to. The overall consensus is that data veracity reflects the truthfulness of a data set and your level of confidence or trust in it. I’ll take this a step further and say that data veracity is your
In the 6 weeks of my course, Data Quality Improvement at BCIT, I try to impart to them an understanding of how to ‘discover’ the instances of poor data quality in the data sets they are responsible for so that they can begin to help