Data warehouses are key to the way modern businesses make decisions. The data warehouse is the hub of business intelligence. A data warehouse is essentially a repository where data is stored, organised, categorised and integrated into various systems.
Data warehouses receive data from many sources. According to industry research, 95 per cent of Fortune 500 companies use data warehousing, or are planning to develop a data warehouse. However, potential failure rates are high – between 50 per cent and 90 per cent. Understanding failure is key in preventing negative outcomes.
Why Data Quality Matters
According to the International Journal of Latest Trends in Computing, there are 20 factors affecting the success off data warehousing projects. Reasons include poor data quality, the failure of management to recognise the benefits of data warehousing, and a knowledge gap between researchers and practitioners that prevents meaningful progress.
Ted Friedman, one of Gartner’s key analysts, said “Consistency and accuracy of data is critical to success with business intelligence, and data quality must be viewed as a business issue and responsibility, not just an IT problem.” This is a key principle in data quality projects: the ability for the whole organisation to aim for a common goal.
We often talk about getting buy-in from the CEO when commencing data quality assignments. In fact, staff at every level must understand that they have a specific part to play. When dealing with data, it’s all too easy to dismiss the subject as being too technical, too inaccessible, or too complicated for the rest of the business. Yet the IT department only stores and maintains access to data: it doesn’t generate it. That is why the business needs to open up data quality to all.
Forming a Team
Team building is big business. From office-based logic challenges to activity weekends, businesses invest a great deal of time and money in “play”.
But team building isn’t just about having a good time, solving a problem or trying new sports. There’s a reason why companies set up team building weekends. These away-days or activities are designed to awaken our desire to communicate and cooperate with people we may not have much in common with. Organisational psychology is all about inspiring better working practice, and one of the best ways to open the lines of communication is through recreation.
Friedman hints at an important principle in data quality: the need to work as a team across the business, with everyone playing a part and working towards the success of the team. This analogy can be applied nicely across most businesses, large and small.
When the business works as a team, and each player understands their objectives, they have the chance to achieve more and reach higher as a unit. This is a critical factor if the data quality project is to be a success, and the end result is a more effective data warehouse that is less likely to fail.
How Data Quality Becomes Team Work
In data quality projects, the employees are the players: the people tasked with achieving a goal. To achieve success, all players need to be pulling their weight. They must understand their own needs and each others’; they must be able to play to their strengths and understand when they need help from others.
The players also need to play in accordance with the rules. The governing body imposes the rules of the game. In this case, they are the governance team in charge of the data quality initiative; the people who are driving it forward.
Part of the governance team’s job is to buy the equipment that that the players need – the data quality software. The equipment needs to be fit for purpose, suitable for long term use, and of a good enough quality to support the players as their needs change.
Managers referee the players as they carry out their tasks. Day to day, it’s the managers that ensure results are achieved consistently. Managers also guide the players in acquiring the necessary skills that they need to complete their role.
Winning the League
The key to a successful game is motivation. In every data quality project, each player needs to feel inspired by the end result, and must feel a duty to their team mates to meet the right standards of performance. Each player has a role to play: a position to fulfil.
The players all have responsibilities to the managers and coaches, too. Together, everyone will make complicated decisions for the good of the game. Each player must be focused on achieving the required results through consistent effort, rather than being focused on their own needs to the detriment of others’.
In data quality, there are requirements to be met, rules to be followed and consequences, both good and bad. And while the failure figures for data warehousing projects sound discouraging, they need not be. In sport, and in IT projects, a well-organised team of average players will normally beat a disorganised team of good players.
Meeting Your Data Quality Goal
In order to foster success, management and employees need to understand that data quality is a team game – not a solo sport. If the business invests tens of thousands of pounds in a data warehouse, or a single customer view, it needs to know that its team members are united in making that investment a success.
Without cooperation, data quality becomes a side issue. People do not understand how they can influence data quality. They don’t appreciate how data can be recorded incorrectly, or how it can decay. They find it more difficult to understand the consequences of poor data quality, and feel powerless to do anything about it. They will almost certainly experience the down side of poor data quality, through wasted resource, wasted money or wasted time.
But with the right preparation, and with everyone doing their part, companies can improve their data quality and move up the league tables to become champions.
