Is Open Data At Risk From Poor Data Quality?

Open data is a concept that’s new to many people in the UK. The idea is fairly straightforward. By opening up records used by the public sector, the British public can see how their data is being used. The idea was introduced by the coalition government in May 2010, but has yet to gain traction, and the British public have not yet had the opportunity to scrutinise government spending. In theory, open data should make the public sector more efficient and effective, and it should give British citizens more confidence in the way their tax money is being spent. Local governments are supposed to publish records for all spending over £500, and we should all be able to see figures on crime, civil servant salaries and government contracts. But in order to inspire confidence, people need to see two things. They must know that data is being published in the manner it should be published, and they need to know that data is accurate. According to the press, data quality is becoming a serious risk that could derail the open data […]
By |September 16th, 2014|Blog, Data Quality|Comments Off

Is Customer Data Integration vital to business success?

In today’s economy, it is imperative to put the customer at the heart of the business; treating each customer as an individual to maximise lifetime value, coupled with the ability to personalise mailings or to send lifestyle information to the right person at the right time is vital to business success. When you consider that the average business never hears from more than 90% of its unhappy customers, the importance of putting the customer at the heart of the business becomes even more apparent. Customer Data Integration (CDI), or the process of consolidating and managing customer information from all available sources, if properly carried out, ensures that all relevant departments in the organisation have access to the most current and complete view of customer information. Most companies have customer data, possibly in disparate database containers which makes it difficult to manage effectively, resulting in out of date and conflicting customer data.  Although most companies believe having an integrated view of customer data is critical, only a very small percentage of companies actually manage to achieve this. The inability to consolidate data from disparate databases results […]
By |September 10th, 2014|Blog, Data Management, Data Quality|Comments Off

Is There Such a Thing as Too Much Data?

We live in an age where all businesses are generating data, and harvesting data at the same time. The flow of information, and the speed that it is created, have led to the age of Big Data – where massive amounts of ones and zeros are created, transmitted and stored. Every day, another huge chunk of data is broadcast for the consumption of anyone who wants to access it. The Internet of Things (IoT) is one big contributor to data generation. Machines are increasingly being equipped with sensors and can report back their state continuously, across the factory or across the globe. Thousands of these sensors can be remotely monitored, making traditional machinery more connected, and more visible to the managers who have invested in it. And data is being harvested at incredible speed. Most websites are hooked up to Google Analytics, Google’s website statistics package; this retains data for at least 25 months (although some users report seeing older data) on every website it monitors. Google also collects […]
By |September 2nd, 2014|Blog, Data|Comments Off

I was a duplicate Record!

As a point of contact for Recruitment agencies at our company, I regularly receive telephone calls from them promoting their services. This week I received a telephone call from a large online recruitment agency. As the caller started “his script” it felt very familiar and I very quickly realised that I had received the very same telephone call from this company a week ago. I listened politely to what he had to say and then I explained to him that we did not require his services at the moment and pointed out to him that I had been called last week by someone from his company. He assured me that from my records I was last called in June. I was 100% certain that I received a call from this agency last week. He finally admitted (quietly) that there was probably a duplicate record of me on their system! This situation was embarrassing for the caller and an interruption and annoyance to me. Last week, as a company we also received a letter from a large supplier informing us that they were changing their system and […]
By |August 26th, 2014|Blog, Data, Data Quality|Comments Off

7 Stops on the Route to Data Quality

The road to quality data can seem like a long, difficult journey. But it’s one that every business eventually needs to make. As data ages, the business becomes less efficient; it’s not possible to maintain operational efficiency if you can’t make robust decisions. Like many tasks, data quality is best viewed as a long-term concern, and something that will be tackled in controlled phases. We’ve split the journey into seven distinct parts. Each one is a stop you’ll make on the way.   1. Get Out Of the Pit When tackling a data quality problem, taking the first step is often the most important part of the process. With a long, difficult challenge ahead, it can be surprisingly difficult to get the wheels in motion. To make progress with any problem, we have to recognise it and stop denying its existence. Own up. The organisation has a data quality problem; it’s not impossible to fix. Most – perhaps all – businesses will eventually find themselves in the same position; […]

Improve Data Quality by Playing as a Team

Data warehouses are key to the way modern businesses make decisions. The data warehouse is the hub of business intelligence. A data warehouse is essentially a repository where data is stored, organised, categorised and integrated into various systems. Data warehouses receive data from many sources. According to industry research, 95 per cent of Fortune 500 companies use data warehousing, or are planning to develop a data warehouse. However, potential failure rates are high – between 50 per cent and 90 per cent. Understanding failure is key in preventing negative outcomes. Why Data Quality Matters According to the International Journal of Latest Trends in Computing, there are 20 factors affecting the success off data warehousing projects. Reasons include poor data quality, the failure of management to recognise the benefits of data warehousing, and a knowledge gap between researchers and practitioners that prevents meaningful progress. Ted Friedman, one of Gartner’s key analysts, said “Consistency and accuracy of data is critical to success with business intelligence, […]

Picture Perfect Customer Data

In today’s data-driven world, with so many points of interaction with customers and so many data points about customers, pointing to a consistent and comprehensive representation of customer data is a complex challenge. This is why maintaining a customer relationship management (CRM) system with a current and complete picture of every customer is more important than ever. Your business needs every customer interaction to have the benefit of accurate, up-to-date data, such as name, postal address, telephone number, and email address. Unfortunately, even when you achieve accurate, up-to-date data it doesn’t stay that way because data decays. Customers get married, move, change phone numbers, and switch their primary email address. Snapshots of customer data are taken from different sources, which do not change at the same time, and not all customer data attributes within a source stay up-to-date. For example, a call center might be the best source for telephone number, while a billing system could be the best source for postal address, and confirmations sent by a self-service web portal may verify the best email address. External reference databases can also provide alternative […]

Data Cleansing and Enhancement

If you follow the DQ Global blog, you’ll be aware that we often write articles on data decay: the rate at which data becomes unusable. This is one aspect of the challenge businesses face as they build contact databases. While data quality is partly affected by the method of capture, data is also subject to natural decay that will cause a database to become less useful over time – even if data is captured correctly on day one. Data decay is a costly problem that no business can escape; the cost of fixing broken databases increases as the data becomes more decayed. Additionally, the cost increases if the record is left unchecked. According to DQ Global estimates, 20 to 30 % of business data decays each year, which means most business databases will unavoidably suffer without robust data quality initiatives. Our numbers relate specifically to B2B data, including details of companies and employees. When tackling data decay, there are two key processes that will heal the dataset and […]

Data Quality – who’s responsible?

The answer is – everyone in the business.  Unfortunately, all too often no one seems to take responsibility or realise its value. So just why is data quality so undervalued?  The underlying reason is that there is a denial mindset regarding ownership of the data content.  In many organisations, IT builds and owns the database container and the data users take ownership of the content, or perhaps not.  It’s the classic ‘Everybody, Somebody, Anybody, Nobody’ no ownership story: Data quality management is an important job and Everybody is sure that Somebody will do it.  Anybody can do it, but Nobody does.  Somebody got angry about that because it was Everybody’s job. Everybody thinks that Anybody can do it, but Nobody realises that Everybody won’t do it. In the end, Everybody blames Somebody when Nobody does what Anybody could do. Accurate data is undoubtedly the cornerstone of industry, but a lack of standardised data prevents efficient information exchange between departments and subsidiaries and impedes decision-making and understanding of business problems.  Paradoxically, many businesses have masses […]

Why Data Should Be a Business Asset – The 1-10-100 Rule

If you run a business, or even just run a household, you’ll understand that chaos creates waste. How many times have you rushed to find an important document, such as a birth certificate, only to have to pay out for an emergency replacement? How many hours have you wasted searching for lost keys, lost passports and lost letters? How many times have you wasted money on an emergency callout for a problem that could have been located and fixed? Just as chaos at home is disruptive, disorganisation and chaos can have huge effects on business profitability. If employees don’t work to conventions, they risk creating problems for themselves, and for other employees down the line. Use of IT systems is strictly controlled for precisely this reason, and businesses work hard to ensure that data is stored in a controlled way. Otherwise, the chaos spreads like a virus. But what happens when data is captured correctly, then naturally decays? Even the best laid plans cannot protect against this inevitable, and costly, data […]