An AI agent can receive a question, find data, write a query, and return an answer. For leadership, getting a result without waiting for the data team is appealing. To make that result useful for a decision, the agent also needs the rules the company uses to calculate and interpret its numbers.
An agent does not replace ETL: pipelines still bring in and prepare the data. What changes is the work on top of it. Here, an agent means a model that chooses steps and uses tools within a goal, instructions, and permissions. For data work, it might retrieve a definition, run SQL, and prepare an answer for review.
In my work with agents, I consider a well-framed question essential. If I give a vague task and the agent lacks the context, I cannot expect a reliable answer. It also needs to recognize what still needs clarification and ask. Testing therefore starts with whether it understood the request.
Start with a question: how many orders do we have?
A hypothetical example shows why even a simple question may not have an obvious answer. A model can write syntactically correct SQL:
SELECT COUNT(order_id)
FROM orders This counts non-null order_id values. It does not establish whether the company needs rows or unique orders. It does not address cancellations or sales channels.
For our example, assume the table holds confirmed purchases and the rule is to count unique online orders that have not been canceled. For that scope, the query would be:
SELECT COUNT(DISTINCT order_id)
FROM orders
WHERE order_status <> 'canceled'
AND order_stream = 'online' 
This is a simplified illustration, not a ready-to-use query for any company. A time-based report would also need a period and a rule for assigning orders to it. The team must know how to handle missing statuses and later corrections.
If only a senior analyst knows the rule, the agent has nowhere to retrieve it. It may ask, or start guessing. A missing definition is an environment problem; we also need to evaluate the model, which can make mistakes despite good inputs.
Store the definition alongside the calculation
For an important metric, record its meaning, source, calculation, filters, and exceptions. Name an owner who can resolve ambiguity and explain where the metric is used. The definition should be accessible to people and the agent.
This hypothetical excerpt matches the SQL above. It deliberately includes both filters. It is not a universal format or an example of a deployed system:
{
"entity": "orders",
"business_name": "Orders",
"description": "An order represents a confirmed customer purchase.",
"primary_key": "order_id",
"default_metric": {
"name": "Order count",
"calculation": "count(distinct order_id)",
"filters": {
"order_status": { "not_equal": "canceled" },
"order_stream": { "equal": "online" }
}
}
} 
The formula is only part of the agreement. For margin, you also need to define included costs, shipping, discounts, returns, and marketplace fees. A model’s general knowledge cannot replace the company’s decision on how to count these items.
A number also needs the purpose of the question
Extend the example: “How did Slovakia perform in the first quarter compared with last year?” First, establish whether we mean orders, revenue, or margin. And whether Slovakia means the market, customer country, delivery address, or accounting entity. Those select different data.
This is a hypothetical situation, not an anonymized client story. Sales, marketing, logistics, and finance may need different views for their work. The agent needs to identify the one that fits the decision.
Three kinds of context come together here:
- Business context: what the company needs to learn and why the decision matters now.
- Domain context: what the team is working on and how it uses its tools and reports.
- Data context: which metrics, tables, filters, currencies, and time rules answer the question.

I find the analogy of a new colleague useful. Showing an analyst the database is not enough. They need current definitions, an explanation of historical tables, and someone they can ask. We need to make the same material available to the agent. If everyone is waiting for the only person who knows a rule, start by writing it down.
What the data team needs to prepare
You do not need one long document about the whole company. For the chosen question, you need a traceable path from definition to sources and output. I would start with these parts:
Show which data to use and what it means
Show which tables are original sources, intermediate outputs, and reporting tables. Describe relationships and join keys. Names and descriptions should explain columns; identically named items should not silently mean different things.
A postal code, for example, is not an integer just because it contains digits. An amount needs a currency, and a timestamp needs a time zone. A refund needs a rule assigning it to either the original order’s period or the processing period. The company must make that choice and record it with the definition.

Define what the agent may read and change
The agent needs to know where to run a query, find tests, and read logs. Define what it may only read and which changes require approval. APIs, the command line, and MCP can expose repeatable work. For file changes, versioning and diff review help, for example through GitHub and CI.
When choosing a tool, check its actual available operations. Browser interaction can work, but stable procedures and action records can be harder to maintain for recurring tasks. An API alone does not guarantee reliability either.


Check the answer and the cost of getting it
Alongside data tests, prepare questions with expected answers, including exceptions. These evaluations should check the use of context as well as the calculation: did the agent recognize an unclear question, use the current rule, and answer what the company needed to know? Check model, definition, or procedure changes against the same examples. Agreements between a source, transformation, and report should specify what data they exchange and what checks must catch.
Good context may reduce wasted attempts, but fewer queries or tokens are not guaranteed. Retrieving context also has a cost. Measure attempts, total costs, and human corrections per usable answer. More queries for more questions are different from dozens of failed attempts on one task.
A review should improve the next answer
At the start, someone from the data team should check the question, query, and answer. Correct syntax is insufficient: the output must match the definition and the decision it supports. Set ongoing oversight according to task reliability and the consequences of an error.
When a problem appears, do more than rewrite the commentary. A missing filter belongs in the query, an unclear meaning in the definition, and a new edge case in the tests. If a business rule changes, record the decision and its effective date. The next task should use that correction.
That is how I approach building context: as ongoing work that feeds findings from outputs back into the source material and evaluations. A document written once cannot cover every future question or change in the company.
This also protects trust in the data. A confidently written wrong answer can damage more than one meeting. People need to see the source, limitations, and a way to fix mistakes.
First step: one question from request to verification
I would choose the first rules based on the process we want to improve. I want to know where the problem hurts most, what costs the company the most money, or where its main revenue comes from. I look for either savings in work and cost or an opportunity to create more value. I do not have one universal rule for every company.
From there, choose a question a good analyst can answer and whose result someone will use. You do not have to document the entire warehouse first. Keep the scope small enough to establish whether the agent completed the task.
- Choose the decision. Record the question, who needs the answer, and what they will do with it.
- Store the rules. Make definitions, filters, exceptions, an owner, and source links easy to find.
- Prepare data and access. Describe the required tables, joins, types, currencies, and timestamps. Set permissions.
- Agree on verification. Define the expected result, difficult cases, and the person reviewing the answer. The team keeps a version history and reviews changes.
- Complete the task. Track the output, costs, and human corrections. Feed findings back into definitions, procedures, and tests.
Once that process works, expand the set to, for example, ten common questions and their exceptions. Treat the number as a starting suggestion, not evidence that the whole company is ready. Use verified results to decide on further scope.
If you are deciding how to connect data, metrics, and AI, we can review your data and AI systems architecture. For a shared brief with the team, see the From ETL to AI agents workshop. If you do not yet know why the numbers are unreliable, start with a data audit.