As we all know, data is all pervasive in today’s business world. Now generated in huge quantities, it resides in the Dataverse. But what is the Dataverse, and how do we effectively collect and manage all the data in it?
What is the Dataverse and what does it do?
The Dataverse is an open-source Microsoft application that allows users to securely store and manage data that’s used by business applications. It also has the capability to:
- Unify data from different sources and formats into a common form.
- Access and process data from a universal structure of tables.
- Use Power Apps to build rich applications that use the data stored within.
What Can DQ Global do for the Dataverse?
While excellent for storage and analysis, bad data entering the Dataverse can render this process fruitless, as the information that is produced can end up providing insults, rather than insights.
At DQ Global, we believe that storage and analysis are important parts of the process, but only half of the battle. The other half is the content. Bad data coming in means bad insights going out. You do not put diesel in a petrol car and expect it to run well!
By making sure that the data being collected is of a high quality, we can ensure that the insights generated from that data are of the same high quality.
How Does That Work in Practice?
On its journey data will go through various stages before it enters the Dataverse and is stored and analysed.
It begins with collection, which happens in two ways, through human entry when using applications and through other sources like machine data, which is collected in a more automated fashion. The next step is ingestion, or data in flight, and finally storage, or data at rest.
Once the data is at rest, this is where the analysis and visualisation take place, which leaves us with the insights that drive our businesses forward.
We believe in the old saying of “prevention is better than cure”. It is during the collection part of the journey that it is best to fix any broken data, before it gets into the Dataverse, and it will help reduce cost and mitigate risk. For example, at the collection stage it may cost you £1 to fix the broken data, but further down the line at the data in flight or data at rest stages, it may cost you £10 or £100, respectively.
Stopping bad data getting in
Why do we want to clean and fix the data on collection? There are a number of reasons:
- To prevent bad data entering the Dataverse.
- To clean up fields and records.
- To enrich existing data.
- To acquire new data and records.
To do this, we take advantage of Microsoft’s Power Component Framework, that has allowed us to create some PCF controls that help with data entry. For example, using our PCF, you can create an address search field that will allow you to search for over 240 countries and principalities around the world, drop them into your application through the PCFs and populate your data. This immediately cuts down key stokes, increases the accuracy of the data collection, and helps with the deduplication and formatting of your data.
Our PCFs can also help with other important collection points like phones, emails, and websites. There are many tasks they can help with, from simple ones like confirming the data is a telephone number, email address or website address, to more the more complex like formatting the data to making sure a telephone number is in the correct international or local format. We can also go as far as verifying that an email address or telephone number is active and working, without ringing the number, or delivering the email.
What is this good for?
This process can be applied to several different entities, such as companies, contacts, leads and addresses. As for practical uses, it can be utilised in several ways:
- Anti-Money Laundering checks.
- Suppression (Deceased, Moved or Preference).
- Know Your Customer.
- Politically exposed person searches.
- Identity resolution.
- Duplicate detection.
- After mastering your data, adding firmographics, demographics geographics.
Ultimately, ensuring your data’s quality on collection will not only mean high quality data for you and your business, but it will also ensure high quality data for all once it enters the Dataverse.
At DQ Global, we have spent 25 years championing the data quality cause and empowering organisations to achieve more with their customer data.
We have built a range of products that put data quality and management front and centre, helping businesses improve revenue, save money, and reduce risk. You can find out more about our data quality solutions here.
Martin Doyle, DQ Global CEO, recently spoke at this years’ Scottish Summit on Dataverse Data Management, which you can find, in its entirety, here.
You can also check out our free “DQ Guide to Data Quality in 2021” here, which looks at data quality as a business problem and the steps you can take to fix it.
