[Docs index](/docs.md) / [Tool Policies](/docs/tool-policies/overview.md) / Playbook: Build a Query Intent Classifier

---

# Playbook: Build a Query Intent Classifier

You wrote a rule that blocks any warehouse query containing the word `delete`. It works, until someone asks a perfectly reasonable question about the `deleted_orders` report and gets blocked, and until someone else gets a destructive statement through by wrapping it in a way your pattern did not expect. Patterns look at text. What you care about is what the query would do. This playbook walks you through building a small tool that reads a query and says what it intends, then using that tool as a classifier so your rules act on intent instead of spelling.

## What you will build

By the end of this playbook, you will have:

- A custom tool that reads a query and returns a label, a confidence score, and the tables it touches
- A rule that uses the tool as a classifier to block queries that change data
- A second rule that uses the same classifier to keep restricted tables off limits
- A cheap first check in front of the classifier, so most queries never wait for it
- A tested setup where you can see exactly what the classifier decided and why
- An agent that reviews blocked queries each week and tells you where the classifier is wrong

## What you need before you start

- An Assist workspace where you are a workspace admin.
- An AI client connected to your workspace through an MCP server, or the chat in Assist.
- A tool your team uses to query data. This playbook calls it `warehouse_query`, with a `sql` parameter. If you have not built one, see [Playbook: Put guardrails on warehouse queries](playbook-warehouse-query-guardrails.md).
- Ten to twenty real queries from your team, including a few you would want blocked. You will use them to check the classifier.

## Step 1: Decide what the classifier should answer

A classifier is only as useful as the question you ask it. Keep the question narrow and the answer structured. For queries, three things are enough:

| Field | What it holds |
|-------|---------------|
| `label` | One of `read`, `write`, or `admin` |
| `confidence` | A number from 0 to 1 |
| `tables` | The tables the query touches |

`read` means the query only returns data. `write` means it adds, changes, or removes rows. `admin` means it changes the structure of the warehouse or who can access it.

Avoid asking for a verdict such as "safe" or "unsafe". A label describes the query. Your rules decide what to do about it. That separation lets you change what is allowed without rebuilding the classifier.

## Step 2: Build the classifier tool

In your AI client, describe the tool:

> "Create a tool called 'classify_query_intent'. It takes a 'sql' parameter with a query and an optional 'dialect' parameter that is one of snowflake, athena, or postgres. It returns an object with three fields: 'label', which is one of read, write, or admin; 'confidence', a number from 0 to 1; and 'tables', a list of the fully qualified tables the query touches. The tool must not run the query or connect to the warehouse. It only reads the text."

The last two sentences matter most. A classifier runs outside your rule list, and its input comes from whatever the AI wrote in the call being checked. A tool that only reads and labels is safe to use this way. A tool that connects to a system or changes something is not.

Ask the AI how it plans to decide. A good answer combines parsing the statement with a model's judgment for the unclear cases. Push back if the plan is a list of keywords, because that is the pattern you are trying to get away from.

> "Handle these cases: a select that contains the word delete inside a string or a column name should be read. A statement with several commands separated by semicolons should take the most serious label among them. A select into a new table is write. A query inside a comment should be ignored."

## Step 3: Check the classifier against real queries

Before any rule depends on it, find out how good it is. Give it your real queries:

> "Run classify_query_intent on each of these and show me the results in a table with the query, the label, the confidence, and the tables."

Then paste your list. Look for:

- **Wrong labels.** A write labeled as read is the serious kind. A read labeled as write is an inconvenience.
- **Low confidence.** Note the queries where confidence is below 0.8. These are the ones your rules need a plan for.
- **Missing tables.** If the list of tables is incomplete, a rule about restricted tables will have gaps.

Refine the tool until it gets your list right:

> "It labeled 'create temporary table t as select ...' as read. That should be write. Also, it missed the table inside the subquery in number 7."

Note how long each run takes. The call being checked waits for the classifier, so a classifier that takes four seconds adds four seconds to every query your team runs.

## Step 4: Add the classifier to a rule

In Assist, open **Permissions** and select the **Tool Policies** tab. Click **New Rule**.

1. Under **Action**, select **Deny**.
2. Under **Applies to**, leave **Workspace**.
3. In **Tool**, type `warehouse` and press **Tab** to complete `warehouse_query`.
4. Click **Add classifier**.
5. In **Classifier tool**, type `classify` and press **Tab**. When the name completes, the line under the field lists the parameters the classifier takes.
6. Click **Draft input**. The **Input** box fills in. Because both tools have a parameter called `sql`, the classifier's `sql` is set to read from the call:

   ```
   {
     "sql": "$.sql",
     "dialect": "snowflake"
   }
   ```

   A value that starts with `$` is read from the call being checked. Any other value is sent as written. Set `dialect` to the one your warehouse uses.

7. Under **Expect**, set the first clause to `label` **in** `write, admin`.
8. Under **On error**, leave **Fail closed**.
9. Set **Timeout (ms)** to a little more than the slowest run you saw in Step 3.
10. **Reason**: "Queries that change data or structure are not allowed."
11. **Message returned to the model on deny**: "This query would change data. Only read queries are allowed. Rewrite it as a select, or tell the user to ask data engineering."

Click **Create Rule**.

## Step 5: Decide what to do when the classifier is unsure

Your rule blocks when the label is `write` or `admin`. It says nothing about confidence. A query labeled `read` with a confidence of 0.4 goes through.

Add a second rule for the uncertain cases. Click **New Rule** and set it up the same way, with a different expectation:

1. **Action**: **Deny**, **Tool**: `warehouse_query`.
2. **Add classifier**, choose `classify_query_intent`, and click **Draft input**.
3. Under **Expect**, set the clause to `confidence` **lt** `0.7`.
4. **Reason**: "The query could not be classified with confidence."
5. **Message returned to the model on deny**: "This query is unclear. Simplify it, split it into separate statements, and try again."

Both rules use the same classifier with the same input. The classifier runs once per call, and both rules read its answer.

## Step 6: Put a cheap check in front

Most queries are plain selects. There is no reason to make every one of them wait for a classifier.

Conditions on a rule are checked before classifiers. If a condition fails, the classifier does not run. Use that to skip the classifier when the query obviously only reads.

Edit your first rule and click **Add condition**:

1. Choose the path `sql`.
2. Set **Operator** to **matches**.
3. Enter a pattern that describes a query worth a closer look, and tick **ignore case**:

   ```
   \b(insert|update|delete|merge|truncate|drop|alter|create|grant|into)\b
   ```

Now the classifier only runs for queries that mention one of those words. The query about the `deleted_orders` report still reaches the classifier, because `deleted_orders` does not contain `delete` as a whole word. If it did, the classifier would label it `read` and let it through. The pattern decides which queries get judged. The classifier does the judging.

Be careful not to make the pattern the gatekeeper. If a dangerous statement could avoid every word in your pattern, leave the condition off and accept the wait.

## Step 7: Keep restricted tables off limits

The classifier returns the tables a query touches, including ones inside subqueries that a pattern would miss. Click **New Rule**:

1. **Action**: **Deny**, **Tool**: `warehouse_query`.
2. **Add classifier**, choose `classify_query_intent`, and click **Draft input**.
3. Under **Expect**, set the clause path to `tables[*]`, the operator to **matches**, and the pattern to `^(payroll|hr_private)\.`. Tick **ignore case**.
4. **Reason**: "Payroll and HR tables are not available through the warehouse tool."
5. Under **Placement**, choose **Top of chain**, so it applies to everyone before any exception.

With a list path such as `tables[*]`, the clause matches if any table in the list matches.

## Step 8: Test in the simulator

The test box in the rule editor skips classifiers. Select the **Simulator** tab and switch to **Tool call**.

Choose a person, enter `warehouse_query` in **Tool**, click **Fill from tool**, and set `sql` to one of your real queries. Click **Evaluate**.

In the **Evaluation trace**, the classifier line shows what the classifier returned and which clause decided. For example:

```
classify_query_intent → label in ["write","admin"] → "write"
```

Run your whole list from Step 3 through the simulator. For each query, confirm the decision is the one you expect. If the classifier fails or times out, the trace says so and shows that the rule failed closed.

## Step 9: Add a weekly review agent

A classifier drifts. Your team starts writing new kinds of queries, and its accuracy changes without anyone noticing. Set up an agent to keep watch.

In your AI client:

> "Create an agent called 'Policy Review' that runs every Monday at 8am. It should look at the tool calls that were blocked by policy in the last seven days for warehouse_query, group them by the rule that blocked them, and for each group show the three most common queries. For any query that looks like it only reads data, flag it as a possible false block. Post the summary to the data team channel."

Each week you get a short list of queries to look at. For the ones the classifier got wrong, go back to your AI client, show it the query, and ask it to fix the tool. Your rules do not change. They pick up the improved classifier on the next call.

## What you built

Your rules now act on what a query does. A question about the `deleted_orders` report goes through. A statement that changes data is blocked however it is written. Restricted tables are caught even inside a subquery. Queries the classifier cannot read with confidence are sent back to be simplified. Plain selects skip the classifier entirely, so most of your team never waits.

When the classifier fails, the rule fails closed and the call is blocked, so an outage in the classifier never becomes a gap in your guardrails. And every week, an agent tells you where the classifier needs work.

## Where to go next

- **Classify other things.** The same approach works for outbound messages ("does this contain customer account numbers?"), purchase orders ("is this within the requester's spending limit?"), and support replies ("does this promise a refund?").
- **Reuse one classifier across tools.** If you have query tools for several systems, one rule with a tool pattern such as `*_query` covers all of them.
- **Build a review app.** Ask your AI client for a sandcastle app that lists the week's blocked queries with the classifier's label, and lets a reviewer mark each one as correct or incorrect. Feed the incorrect ones back into the classifier.
- **Give trusted groups a lighter check.** Add an allow rule for data engineering above the classifier rules, so their calls skip it.

## Related guides

- [Using a tool as a classifier](using-a-classifier.md)
- [Writing conditions](writing-conditions.md)
- [Testing rules with the simulator](testing-rules-with-the-simulator.md)
- [Playbook: Put guardrails on warehouse queries](playbook-warehouse-query-guardrails.md)
- [Troubleshooting](troubleshooting.md)

---

## Navigation

### In this section: Tool Policies

- [Tool Policies](/docs/tool-policies/overview.md)
- [Use Cases and Playbooks](/docs/tool-policies/use-cases.md)
- [Creating a Rule](/docs/tool-policies/creating-a-rule.md)
- [Writing Conditions](/docs/tool-policies/writing-conditions.md)
- [Using a Tool as a Classifier](/docs/tool-policies/using-a-classifier.md)
- [Testing Rules with the Simulator](/docs/tool-policies/testing-rules-with-the-simulator.md)
- [Creating a Rule from a Tool Call](/docs/tool-policies/creating-a-rule-from-a-tool-call.md)
- [Managing the Rule List](/docs/tool-policies/managing-the-rule-list.md)
- [Troubleshooting](/docs/tool-policies/troubleshooting.md)

#### Playbooks

- **Playbook: Build a Query Intent Classifier** (current)
- [Playbook: Control Where Your Tools Can Send Data](/docs/tool-policies/playbook-outbound-request-allowlist.md)
- [Playbook: Give a Scheduled Agent Only the Access It Needs](/docs/tool-policies/playbook-scheduled-agent-guardrails.md)
- [Playbook: Put Guardrails on Warehouse Queries](/docs/tool-policies/playbook-warehouse-query-guardrails.md)

### Other sections

- [Tool Creation](/docs/tool-creation/overview.md)
- [Subagents](/docs/subagents/overview.md)
- [Agent Skills](/docs/agent-skills/overview.md)
- [Sandcastles](/docs/sandcastles/overview.md)
- [MCP Servers](/docs/mcp-servers/overview.md)
- [Scheduled Triggers](/docs/scheduled-triggers/overview.md)
- [Agent Filesystem](/docs/agent-filesystem/overview.md)
- [Workspace Permissions](/docs/workspace-permissions/overview.md)
- [Workspace Billing](/docs/workspace-billing/overview.md)
- [Chat Sharing](/docs/chat-sharing/overview.md)

[Back to docs index](/docs.md)