Aptelisense Team Blog

Data governance - monitoring your data quality

10 Feb 2012

Many organizations encounter difficulties with the quality of their data especially the larger the organization gets. The problem compounds the larger the organization gets because the requirements to 'slice and dice' the data for business intelligence, reporting and specific application requirements continually grow. Once the data has been 'sliced and diced' the data quality begins to get out of control and quite often the problem simply gets assigned to the 'too hard basket'. The reasons for poor data quality are varied, but here are some typical scenarios:

No formal data governance role in the architecture team

This is typical when organizations are immature because it can be seen as a costly overhead that the organization doesn't need for a problem that doesn't exist. This decision or lack of decision will nearly always comes back to bite the organization the larger it grows.

Lack of tools to automate data quality monitoring

Although there are various methods for monitoring data quality, here are some common reasons why they may not be used:

• Products costs are too high

• Organization does not have a business sponsor to justify the cost

• Products are tied to specific technology and thus more than one solution is required

• Organization requires a monitoring solution to be developed in house which is too costly

• No budget for Operations team to manage the product or tool

Poor data validation in applications

There are many reasons why applications do not enforce data standards:

• Project pressures to complete application roll-outs

• Lack of organizational data standards

• Programmers not trained in organizational data standards

• Organizational data standards not enforced

• Data quality is omitted from application testing

The hidden side-effects and costs to your organization

The lack of control over data quality can have a compounded effect on your organization. The evidence of this will start to be visible when reports from different systems or tools highlight inconsistencies. Along side this there will be random quality issues with application behavior and business intelligence may not be as accurate as expected. Customers may start to see these issues when it becomes apparent that their details are not the same across products and they may experience difficulties trying to change some of their personal details.

One of the most serious side-effects of not monitoring data quality is fraud. By monitoring data quality across different systems you can be alerted to suspicious events before they escalate into opportunity for fraud.

Where to next?

If your organization is currently experiencing any of these issues or side-effects, there are some simple strategies you can take to stop the problem escalating further:

• Assign an architectural role to provide data governance. Define data standards and ascertain the scale of the problem starting with the most critical / valuable data in your organization. There is a lot more to data governance than simple data quality control but this is the best place to start.

• Implement simple controls that force standards across new applications and data repositories. This could be part of the standard project risk assessment methodology.

• Start monitoring your critical data for inconsistencies. Use a product that is simple to install, technology independent and can spot issues as they happen. This product should not require any changes to your systems or applications and monitor your live data sources while not adding any extra risk to your organization.

What product should you use?

Thought you'd never ask :o) We recommend the use of our Compliance Automation Server (CAS).

CAS meets all the requirements defined above and more. Using CAS you could immediately start to enforce data quality across your customer data and check that all your customer details are the same across all your systems. Taking this one step further, why not validate your customer details with other organizations to ensure the possibilities for fraud are minimal and address any Anti-Money Laundering requirements you may have. After this you could use CAS to perform data quality validation across your other data repositories.


Compliance Automation Server 2.2 supports remote data validation

17 Jan 2012

We are excited to announce that Compliance Automation Server (CAS) version 2.2 now supports CAS to CAS remote data validation! This allows organizations to securely validate data in real-time with other organizations.

This new function now enables organization to organization real-time data validation in the following scenarios:

Banking and Finance: Anti-Money Laundering and Countering Financing of Terrorism. The prime principle being Know Your Customer. CAS now enables different organizations to validate customer details in real-time with each other.

Insurance: Policy validation. Insurance companies can now validate in real-time any customer details with other organizations to ensure that the customer information is not fraudulent. This can now take place before a policy is issued. Organizations using this functionality can become aware of any suspicious customer changes as they take place rather than when a suspicious insurance claim is lodged.

Procurement: Organizations can now validate data with suppliers in real-time such as product stock levels and expected delivery dates.

External Regulatory: External regulators can constantly monitor and validate organizations performance in real-time. For example Energy regulators can monitor energy supplier compliance commitments and immediately be aware if any of these commitments are not met.

Finance: Financial regulators can monitor the current exposure level of Financial organizations and immediately be aware if the exposure is out of bounds.

• Government Agencies: Agencies such as Internal/Inland Revenue can automate data checks with other agencies or external organizations.

These are just a few of the typical scenarios that CAS can enable.

Under the hood

Remote data validation can now take place from any Reactor Rule (RR). The RR simply passes the data to a defined Remote Reactor Query (RRQ). The RRQ definition points to a Collector on a remote CAS. When the RR passes the data to the RRQ the data is passed via an encrypted SSL session to the remote CAS Collector. The data is now available to any Reactor on the remote CAS that is configured to validate the data. This can include running local database queries or passing the data to another remote CAS for further validation. A Reactor can pass data to any number of RRQs defined on any number of remote CAS and check the values returned from each.

By default the Reactor on the remote CAS will delete any data received from the source CAS after it has finished processing the data. By default the Reactor will also not store event, node or alert information in the remote CAS database and will not generate alerts. This is designed to limit the information shared between organizations and to allow the remote RRQ to run silently. These defaults can be overridden if required.


Compliance Automation Server 2.2 Released

17 Jan 2012

Version 2.2 of Aptelisense Compliance Automation Server (CAS) is now available.

The major focus of this version is to enable CAS to CAS Remote Reactor Query.

If you already have a license you can get the latest from your account product releases page.

Here are the major changes:

• Reactor Rules can now validate data on remote Compliance Servers. This allows organizations to securely validate data with other organizations who also run CAS. For more information see the latest Aptelisense Blog and Compliance product pages.

• Various bug fixes


Accurately track, bill and rate-shape Internet usage with Proxy To Radius Accounting

1 Dec 2011

Aptelisense has released Version 2.0 of Aptelisense Proxy To Radius Accounting (P2R).

P2R receives log data via syslog from remote systems and monitors Internet activity against network IP addresses, updating Radius servers with users' Active Directory authentication details (account and group) and bandwidth allocation plan. This enables organizations to accurately track, bill and rate-shape users' Internet usage.

P2R has been running successfully at a large University since 2008 where it integrates several Bluecoat SG proxy appliances with Allot Subscriber Management Platform (SMP) appliances. P2R enables the University to monitor student and staff Internet traffic usage against network IP addresses and continuously update the SMP appliance with user IP address, Active Directory authentication details (account and group) and bandwidth allocation plan. This solution allows the University to accurately track, bill and rate-shape their users' Internet usage. P2R still performs this vital solution for the University today and has done so for 3 years!


P2R is also suitable for use in Telcos and Internet Service Providers as it has the flexibility to integrate different and disparate types of proxy servers and Radius servers.


P2R Version 2.0 is proxy and server agnostic and easily configurable. It allows one or more devices to send log data via syslog to P2R and supports one or more destination Radius devices. This allows each source device to be mapped to a specific Radius device for ultimate flexibility.

For more information and to see the live running demo go to: Proxy To Radius Accounting.


New opportunity to join our Sales team

3 Nov 2011

We are looking for an experienced sales person to join our Wellington based team. If you are interested in an exciting future within our organization, go to our careers page.


Compliance Automation Server 2.1 Released

1 Nov 2011

Version 2.1 of Aptelisense Compliance Automation Server (CAS) is now available.

The major focus of this version is improvements to the Auditor interface, SQL Datasource and Reactor Query.

If you already have a license you can get the latest from your account product releases page.

Here are the major changes:

• The Auditor interface now allows the user to choose the interface type at login time. The Auditor can now login in read-only mode as an Administrator, Authorizer or Analyst.

• The SQL Datasource can now be configured with a user supplied SQL query (expert option). This allows any tables to be queried in a database and the resultant records processed by an individual Collector. 

• A Reactor Query can now specify the result as a sum() of column values.

• A Reactor Query can now be configured so the Rules are Or'd or And'd together.

• The Authorizer and Analyst alert management interfaces allow alerts to be viewed by Reactor. This enables the Analyst social network map to display associations from just one Reactor.

• Various minor changes to help and error messages

• Various bug fixes


Compliance Automation Server 2.0 Released

12 Aug 2011

Version 2.0 of Aptelisense Compliance Automation Server (CAS) is now available.

The major focus of this version is advanced data correlation plus the new Analyst Work Bench interface.

If you already have a license you can get the latest from your account product releases page.

Here's what's new:

• CAS will now automatically analyze and correlate data across data source by type and association. This allows events and alerts to be viewed using a social network map for easy visual assessment.

• New Analyst role. The Analyst role can manage alerts and events using the new Analyst Work Bench interface.

• Changes to the way CAS is licensed allows for a more flexible approach in pricing

• Controls can now be edited

• Additional standard types added to Controls

• Initial positioning option added to SQL Datasource configuration

• Improvements to Reactor Queries to allow percentage value and time period comparisons

• Performance improvements

• Various bug fixes


NZ Serious Fraud Office Annual 2011 Report

2 Aug 2011

The recent New Zealand Serious Fraud Office 2011 annual report (http://www.sfo.govt.nz/f55,13828/SFO_Annual_Report_to_30_Jun_2011-Final.pdf) indicates that a common reason why so many financial institutions suffer fraud is because they are not monitoring their key financial indicators against their external compliance commitments such as their Deed of Trust, SEC or FSA. The common financial indicators that indicate a financial organizations health are:

• Liquidity position and minimum capital ratio

• Debt exposure to third parties

• Liquid assets and undrawn committed facilities against future commitments

• Suspicious and fraudulent behavior

It appears that financial institutions need to place more focus on real-time monitoring of their external compliance commitments. Aptelisense Compliance Automation Server (CAS) can provide this capability. See: financial monitoring using CAS.


New opportunity to join our Sales team

1 Jul 2011

We are looking for an experienced sales person to join our Wellington based team. If you are interested in an exciting future within our organization, go to our careers page.


Background on Aptelisense Compliance Automation Server (CAS)

27 Jun 2011

What does it all mean?

The general definition of the word compliance means: conforming to a rule. This can be applied across lots of areas in business but in its simplest form it has the purest meaning for our product goal. With this goal in mind, we continually strive to make CAS simple to operate and construct rules that allow you to test and validate data from diverse types of data source. From this goal of simple compliance we aim to make CAS usable by less technical business users in order to remove the dependencies on IT departments. Another goal of CAS is to allow you to drive down your operational compliance costs and your internal and external auditor costs.

What's the future?

Because compliance applies across most areas of business especially across business systems, we intend to drive CAS into the core areas of your business. This will enable you to achieve a greater return on investment with the same solution. Another intention is that CAS will interface with key enterprise software solutions to allow seamless data validation and reporting with these solutions (you may like to guess which these solutions might be :o). The third area we will be focusing on, will be reporting and specifically the auditor reporting.

Although we have a very focused product strategy which is built using customer validation, we take very seriously customer feedback. We will always consider and respond to your feedback and ideas.


New Aptelisense website goes live

21 Jun 2011

We are excited to publish our new look website today. The new look and feel is in anticipation of version 2.0 of Compliance Automation Server and our new DocumentMonitor tool. The feedback has been great so far!


Compliance 1.5 Now Available To Download

14 Jan 2011

The latest version of Aptelisense Compliance Automation Server (1.5) is now available.

The changes in this version have focused on increased security, performance and simplicity of installation and operation.

If you already have a license you can get the latest from your account product releases page.

Here's what's new:

• The Compliance server database is now fully encrypted, embedded and requires zero administration

• Automatic and configurable Compliance server database backups 

• The Compliance server installer is less than half the size of previous releases and takes up much less disk space (not that it was a problem before :)

• The performance of SSL connections has been dramatically improved

• Improved control over Windows Event Datasource

• Various bug fixes


Focus on Aptelisense and Fraud on Stuff.co.nz

1 Nov 2010

Great article on www.stuff.co.nz/technology today. Good to see a focus on fraud on the front page.


NZ Serious Fraud Office Suspects Fraud Rife

17 Oct 2010

Here's an interesting article that highlights that the number of cases of fraud reported is the tip of the iceberg. A statement from the New Zealand Serious Fraud Office indicates that the cases investigated by the SFO has only scratched the surface mainly due to limited resources.

http://www.stuff.co.nz/the-press/news/4225893/SFO-suspects-fraud-rife


Compliance 1.4 Now Available to Download

17 Oct 2010

The latest version of Aptelisense Compliance Automation Server (1.4) is now available.

This is our most advanced version yet as it adds the new Reactor Query functionality which allows any Reactor Rule to dynamically validate data values against any defined SQL database and then test the results before continuing processing of any following rules.

If you already have a license you can get the latest from your account product releases page.

Here's what's new:

• New Reactor Query

• Support for the main databases: Oracle, SQL Server, Sybase, Ingres, Informix, DB2, PostgreSQL, MySQL

• Improved system alerts

• Support for monthly licensing

• Various bug fixes


Compliance 1.3 Now Available to Download

17 Sep 2010

The latest version of Aptelisense Compliance Automation Server (1.3) is now available. If you already have a license you can get the latest from your account product releases page. If you want to try Compliance for the first time, just create an account for access to the download.

Here's what's new:

• New Dashboard

• Improved admin reporting capability

• Improved Reactor Rules

• Added support for more Databases

• Various bug fixes


Compliance 1.2 Now Available to Download

8 Jun 2010

The latest version of Aptelisense Compliance Automation Server (1.2) is now available. If you already have a license you can get the latest from your account product releases page. If you want to try Compliance for the first time, just create an account for access to the download.

Here's what's new:

• Support for Windows 2008 Server

• Improved syslog Collector performance

• Improved policy save time

• Pro port can now be modified via the dashboard

• Improved connection time for dashboard logins

• Various bug fixes


Compliance 1.1 Now Available to Download

16 Mar 2010

The latest version of Aptelisense Compliance Automation Server (1.1) is now available. If you already have a license you can get the latest from your account product releases page. If you want to try Compliance for the first time, just create an account for access to the download.

Here's what's new:

• Added reverse of configuration change if rejected by authorizer

• Added override to memory performance optimizer

• Added override to system performance monitor

• Added alerts and events view to administrator Reactor status panel

• Improved the logging

• Added display of Aptelisense Pro log entries to Server Logs panel


Aptelisense website goes live

29 Oct 2009

We have released our new look website today. The website offers only limited functionality while the closed beta testing of the Compliance Automation Server is being conducted. We have received very positive feedback on the look and feel and we are looking forward to pushing the rest of it live after completion of the beta test program.


Aptelisense Compliance version 1 enters closed beta testing

11 Oct 2009

Compliance Automation Server version 1 is now available in a closed beta testing program. The beta program will allow a select group of organisations to test the frozen feature set before the Compliance product is made generally available. We are still accepting requests from organisations to join the program although we anticipate the program will be complete with by December. If you are interested contact us.