Aptelisense Team Blog

Data governance - monitoring your data quality

10 Feb 2012

Many organizations encounter difficulties with the quality of their data especially the larger the organization gets. The problem compounds the larger the organization gets because the requirements to 'slice and dice' the data for business intelligence, reporting and specific application requirements continually grow. Once the data has been 'sliced and diced' the data quality begins to get out of control and quite often the problem simply gets assigned to the 'too hard basket'. The reasons for poor data quality are varied, but here are some typical scenarios:

No formal data governance role in the architecture team

This is typical when organizations are immature because it can be seen as a costly overhead that the organization doesn't need for a problem that doesn't exist. This decision or lack of decision will nearly always comes back to bite the organization the larger it grows.

Lack of tools to automate data quality monitoring

Although there are various methods for monitoring data quality, here are some common reasons why they may not be used:

• Products costs are too high

• Organization does not have a business sponsor to justify the cost

• Products are tied to specific technology and thus more than one solution is required

• Organization requires a monitoring solution to be developed in house which is too costly

• No budget for Operations team to manage the product or tool

Poor data validation in applications

There are many reasons why applications do not enforce data standards:

• Project pressures to complete application roll-outs

• Lack of organizational data standards

• Programmers not trained in organizational data standards

• Organizational data standards not enforced

• Data quality is omitted from application testing

The hidden side-effects and costs to your organization

The lack of control over data quality can have a compounded effect on your organization. The evidence of this will start to be visible when reports from different systems or tools highlight inconsistencies. Along side this there will be random quality issues with application behavior and business intelligence may not be as accurate as expected. Customers may start to see these issues when it becomes apparent that their details are not the same across products and they may experience difficulties trying to change some of their personal details.

One of the most serious side-effects of not monitoring data quality is fraud. By monitoring data quality across different systems you can be alerted to suspicious events before they escalate into opportunity for fraud.

Where to next?

If your organization is currently experiencing any of these issues or side-effects, there are some simple strategies you can take to stop the problem escalating further:

• Assign an architectural role to provide data governance. Define data standards and ascertain the scale of the problem starting with the most critical / valuable data in your organization. There is a lot more to data governance than simple data quality control but this is the best place to start.

• Implement simple controls that force standards across new applications and data repositories. This could be part of the standard project risk assessment methodology.

• Start monitoring your critical data for inconsistencies. Use a product that is simple to install, technology independent and can spot issues as they happen. This product should not require any changes to your systems or applications and monitor your live data sources while not adding any extra risk to your organization.

What product should you use?

Thought you'd never ask :o) We recommend the use of our Compliance Automation Server (CAS).

CAS meets all the requirements defined above and more. Using CAS you could immediately start to enforce data quality across your customer data and check that all your customer details are the same across all your systems. Taking this one step further, why not validate your customer details with other organizations to ensure the possibilities for fraud are minimal and address any Anti-Money Laundering requirements you may have. After this you could use CAS to perform data quality validation across your other data repositories.


Background on Aptelisense Compliance Automation Server (CAS)

27 Jun 2011

What does it all mean?

The general definition of the word compliance means: conforming to a rule. This can be applied across lots of areas in business but in its simplest form it has the purest meaning for our product goal. With this goal in mind, we continually strive to make CAS simple to operate and construct rules that allow you to test and validate data from diverse types of data source. From this goal of simple compliance we aim to make CAS usable by less technical business users in order to remove the dependencies on IT departments. Another goal of CAS is to allow you to drive down your operational compliance costs and your internal and external auditor costs.

What's the future?

Because compliance applies across most areas of business especially across business systems, we intend to drive CAS into the core areas of your business. This will enable you to achieve a greater return on investment with the same solution. Another intention is that CAS will interface with key enterprise software solutions to allow seamless data validation and reporting with these solutions (you may like to guess which these solutions might be :o). The third area we will be focusing on, will be reporting and specifically the auditor reporting.

Although we have a very focused product strategy which is built using customer validation, we take very seriously customer feedback. We will always consider and respond to your feedback and ideas.