Imagine a meeting to decide whether to hire another person. You open a report. A colleague brings up a spreadsheet with a different number. Someone asks about canceled orders, someone else about VAT. Eventually, you discover that each person is comparing a different period. The decision has to wait while you work out which number to use.
A better-looking chart won’t help here. You need to know what the numbers mean and whether they’re reliable enough for this particular decision. A new tool may make some of the work easier, but it won’t resolve a disagreement about what you’re counting.
I see the gradual loss of trust as a bigger problem than one delayed meeting. People often catch an obvious discrepancy before making a decision. But once they stop trusting the numbers, they stop using the reports. They pull data from their own sources and create their own summaries. Then the argument becomes whose numbers are right.
That bypasses shared reporting and weakens the data team’s credibility. Fixing one total may not be enough. People need to understand where the number comes from, what it means, and why they can rely on it.
Assess the data against the question it needs to answer. An order summary may be enough to plan next week’s workload. To calculate customer profitability, it may be missing costs, discounts, or returns. The same data can support one decision well and be insufficient for another.
Imperfect data can be useful when you understand its limits
With incomplete or inaccurate data, I want to know why it is that way, whether we can improve it, and whether the gap could change our decision. Not every use requires perfectly clean data. But an unknown level of error can materially change what a result means.
Google Analytics is one example I use. It helps me follow interest in articles, activity on a website, and trends. It does not have a complete picture of every visitor: Google explains the gaps caused by declined consent and how modeling may address them. For an exact count of units sold for accounting, I would check the order or accounting system.
Ratios and trends need checks, too. A change in measurement settings or in the share of visitors we can observe can change the chart without a real change in behavior. Comparing two button colors also requires a properly designed test; high traffic alone does not make the comparison fair.
Choose one decision you need to make
You don’t have to examine every database in the company. Start with something specific: do we need more people, should we change the price of a project, or should we order more stock? Pick a number that matters to that decision and walk it through the following six checks.
Suppose you want to know which projects are profitable. Revenue gives you part of the answer. You also need to know which costs belong in the calculation, whether they’ve all been recorded, and how they’re allocated across projects. Those questions deserve an answer before you start comparing margins.
1. Does everyone mean the same thing by this number?
Ask two people to explain an important metric. When does an order count: when it’s accepted, delivered, or paid for? And who counts as an active customer: someone with a valid contract, or someone who actually bought something in the past three months?
Different definitions can make sense. But a single comparison needs a consistent definition. Write down what is included, what is excluded, the period, and the source.
Also name someone who can explain the definition and help agree on changes. You do not need a long document. A few sentences and one disputed example often help reveal where people disagree.
2. Can you trace the number back to its source?
Take one value from the report and try to reach the original records. Which system did it come from? What was added up, recalculated, or excluded along the way? Will a later correction to an order also appear in the report?
The CEO doesn’t need to read SQL. But someone in the company should be able to explain the calculation and demonstrate it with a few specific records. When you find an error, you’ll have somewhere to start and a better chance of understanding which other results it affects.
3. Is the data current enough for this decision?
Yesterday’s figures are fine for some decisions and too late for others. For a monthly profitability review, you want a closed accounting period. To promise a delivery date, you need to know what’s in stock now. How current the report needs to be depends on what you use it for.
Agree on an acceptable delay and show the last successful data update in the report. Opening the page does not refresh the data. If the latest load failed, make that visible before someone promises a customer a delivery date based on old information.
4. What’s missing from the total?
A total can look reasonable even when some inputs are missing. A margin report might include all revenue while leaving out costs that couldn’t be assigned to a project. The result then looks better than reality.
Alongside the total, look at excluded or unmatched records. Check missing currencies, duplicates, and unusual values. A negative quantity might be an error, or it might be a correctly recorded return. You need to know which.
Also distinguish zero from a missing value or a failed data load. Zero means something different from “we don’t know.” If the report displays them the same way, it hides uncertainty that may matter.
5. Does someone know how to fix an error?
Finding a discrepancy is a start. Who decides which value is correct? Is the error in the order, the calculation, or just the display? And who will finish the correction and check it?
Fixing a number in a slide for leadership does not remove the error from the next report. Trace the cause and correct it in the source system, calculation, or display. Record what changed and why.
If you need a temporary manual adjustment now, label it. Also agree on who will address the cause and check the next report.
6. What will you do differently because of the report?
Now return to the original question. Does the number help you decide whether to hire someone or change a price? Do you know when a difference calls for action and when you need to investigate first?
You do not need an automatic rule for every situation. You do need to know who will look at the result and what they will do next. If the chart only gets added to a presentation each month, ask who uses it and for what.
The answer may not be a new system
These checks may show that you only need to agree on a revenue definition, add a missing cost, or put someone in charge of monitoring updates. You may also find that the current setup is sufficient for the decision at hand. That’s a useful result too.
An initial audit should show which numbers you can rely on, where you need caution, and what’s worth fixing first. Buying a platform is a possible next step. On the data audit for owners and leadership teams page, I explain how I approach this kind of independent review.
What to bring to your next meeting
Choose one report you regularly use to make decisions. Instead of going through every chart, ask these six questions about its most important number:
- Do we all understand what this number means?
- Can we trace it to the original records and explain the calculation?
- Is the information current enough for our decision?
- Do we know what’s missing from the result and why?
- Is it clear who will fix and verify an error?
- Do we know what we’ll do based on the result?
If an answer gives you pause, you have a specific place to start. You don’t have to stop making decisions or rebuild your reporting. First, establish whether the uncertainty could change the decision you’re working on.
For the broader context, see my article on moving from ETL to AI agents. The same question comes up with AI: do we understand the inputs well enough to rely on the result?
Further reading: Open Data & Metadata Quality covers accuracy, completeness, consistency, timeliness, and fitness for a particular use. The six checks above are a practical approach to business decisions, not a certification test.