If there’s one thing this series keeps coming back to, it’s that agentic workflows don’t usually break on the model. They break on the data underneath the evaluation. This article takes that idea somewhere we haven’t been yet: what happens when your agent has to reason across several linked tables that live inside a scoped app. I was involved in a top Indian bank engagement and a few others alongside Kowsalya, so I asked her to walk through this one herself. — Ragu Ramakrishnan, PM, AI Data Kit

Kowsalya is an AI Architect on the Product Excellence AI team at ServiceNow.

 

The Setup: An Agent That Was Fine, Data That Wasn’t

We were testing an agent in the Software Asset Management scope. Its job is to answer the questions a SAM admin actually asks — is this publisher compliant, and how is its license being used across subscriptions and installs. To get there it has to read across three linked tables: the product catalog, the subscription records, and the install telemetry. We pointed it at a publisher with hundreds of products, and the allocation came back completely empty. Zero subscriptions, zero installs. Your first thought is that the agent is broken somewhere.

It wasn’t. We went and looked at the tables ourselves. The subscription table had almost nothing in it, and not a single row for this publisher. The install telemetry had a couple thousand rows, but they were nearly all for one or two other publishers. So the catalog was seeded and the rest of it never was. The agent had done everything right, against data that simply wasn’t there. That’s not a bug in the agent. It’s a hole in the data, and the only reason we caught it is that we were grading the agent against something specific instead of skimming a few answers and calling it a day.

 

Easy to Miss — and Easy to Get Wrong When You Fix It

The obvious fix is to seed some subscriptions for that publisher. The mistake is seeding them as a flat table of rows with no product behind them, or generating product, subscription, and install as three separate runs that don’t know about each other. Compliance reasoning only holds up if those tables link the way they do in production. A subscription has to point at a real product. An install has to point at a real subscription. The numbers have to add up, because that’s the math the agent is doing. Throw a heap of unconnected rows at it and every tool call will succeed while never once testing whether the reasoning across the relationship is right.

There’s a second catch too. These tables live inside a scoped application, so generating the rows is only half the work. Actually getting them into the real tables the agent queries means dealing with the same permission boundary any scoped app puts up.

 

Where the Multi-Table Generator Earns Its Place

This is where AI Data Kit’s multi-table generator did the heavy lifting. Four things in particular made the difference:

Selecting all the tables in one pass meant product, subscription, and install were generated together, not as three runs that just happen to share a few IDs. It surfaced the dependent tables automatically , so we found out what the selected tables relied on up front instead of halfway through a generation run. The relationship view let us confirm how the three tables actually connect before we generated anything, which beat guessing at the join. And it suggested a related table we hadn’t thought to include on the first pass.

Put together, that gave us a dataset shaped like an actual SAM deployment, with product, subscription, and install properly wired to each other, rather than three piles of rows sitting next to one another.

 

Designing the Scenarios, Not Just the Records

We built the dataset around the situations the agent has to get right, rather than a stack of clean, compliant products:

Scenario

|

Share

|

What it tests

|
|

Compliant: subscriptions cover installs

|

45%

|

Reports “compliant” correctly, doesn’t over-flag

|
|

Over-allocated: installs exceed subscriptions

|

20%

|

Correctly flags a compliance breach

|
|

Catalog entries but zero telemetry

|

10%

|

Reports “no data” plainly — the exact gap that started this

|
|

Ambiguous multi-match (shared name fragment)

|

15%

|

Product resolution handles a broad search

|
|

Duplicate subscription from a rerun

|

10%

|

Doesn’t double-count allocation on a rerun

|

 

Choosing that spread on purpose is what separates “it worked on the products I happened to try” from a test that actually reflects what a real SAM estate looks like — the empty-telemetry publisher included.

 

The Payoff: Data You Can Grade an Agent Against

Once the three tables were linked properly, with that deliberately empty publisher sitting in the mix, the evaluation stopped being guesswork. The same dataset backed every run, so the scores were comparable to each other. When a number moved, we knew the agent had changed, not the test.

Run

|

Score

|

What it caught

|
|

Run 1

|

78%

|

Missed one over-allocation flag

|
|

Run 2

|

82%

|

Correctly reported “no data” for the zero-telemetry publisher

|
|

Run 3

|

86%

|

Ambiguous multi-match resolved correctly

|
|

Run 4

|

90%

|

Rerun duplicate correctly not double-counted

|

 

Having a fixed yardstick is really the whole point. Generate the data once, push it into the real SAM tables, and reuse it unchanged on every run. Then each improvement you see is genuinely the agent getting better, not the test quietly shifting under you. And it shows the agent handles that data-coverage gap deliberately, which is very different from getting lucky on it.

 

The Pattern Worth Remembering

Forget the SAM details for a second, because this applies to any agent that reasons across linked tables — a case and its escalation, an order and its line items, a product and its subscriptions. If the answer depends on how those tables relate, your test data has to keep that relationship intact. That includes the case where the relationship is missing on purpose, because that happens in production too. And when the tables sit in a scoped app, assume the permission boundary is going to show up at some point. Leave yourself time to work through it instead of reading a stall as bad data.

Multi-table generation gives you data that’s shaped like the real thing. Getting it into scoped tables safely and repeatably is the part that still takes real time, so plan for it.

 

_ Could This Be Your Story? _

If you’re building an agent that has to reason across linked ServiceNow tables and you’re running into the same scoped-app wall, get in touch with Ragu Ramakrishnan on the ServiceNow Community. We’d be glad to tell your story next.

 

Kowsalya Srinivasan is an AI Architect on the Product Excellence AI team at ServiceNow.

Ragu Ramakrishnan is Product Manager, AI Data Kit & Evals at ServiceNow.