> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sengol.io/llms.txt
> Use this file to discover all available pages before exploring further.

# AI developer

> Evaluate your agent in code, record signed evidence, guard live calls, and gate releases.

You build the agent. Your job with Sengol is to wire evaluation into the
agent's code and its release pipeline, without slowing either down.

**1. Get credentials**

An API URL and, on a `local`-auth install, an `sk_` token from your platform operator.

**2. Evaluate locally**

Score responses in code. No appliance needed.

**3. Add an LLM judge**

Catch what rules cannot, such as an answer that contradicts its source.

**4. Record signed evidence**

Send each verdict to the appliance as a signed record.

**5. Guard live calls**

Withhold a failing response before the user sees it.

**6. Gate releases**

Fail the CI build when the agent misses its thresholds.

**7. Register the agent**

Propose it as an intake. An approver registers it.

## 1. Get credentials

Ask your platform operator for:

* the API URL, for example `https://sengol.internal.example`
* the install's tenant id
* a personal `sk_` token with the **developer** role ([how operators mint one](https://docs.sengol.io/journeys/platform-operator.md#5-issue-api-tokens)). `sk_` tokens exist only in `local` auth mode. On an install that signs people in through an IdP they cannot be minted, `sengol gate` and the evidence sinks ignore a `sengol login` session, and this journey does not cover that setup. See [Credentials by journey](https://docs.sengol.io/guides/credentials.md)

```bash theme={null}
export SENGOL_API_URL=https://sengol.internal.example
export SENGOL_TENANT_ID=acme-bank
export SENGOL_API_TOKEN=sk_acme-bank_...
```

Install the SDK from the Sengol source you were given. Add the
`openai-judge` or `anthropic-judge` extra if you will use a hosted LLM judge:

```bash theme={null}
pip install "./sengol[openai-judge]"
```

## 2. Evaluate locally

An `EvalSuite` runs a list of evaluators over one input. Deterministic
evaluators run first. If a CRITICAL one fails, the LLM evaluators are
skipped. Otherwise, all LLM evaluators run concurrently.

`NullAuditStore` signs the record and stores nothing, which is what you want
while iterating:

```python evaluate.py theme={null}
import asyncio

from sengol.build.suite import EvalSuite
from sengol.core.types import LLMEvalInput
from sengol.evaluators import PIIEvaluator, RequiredDisclosureEvaluator, ResponseLengthEvaluator
from sengol.governance.audit_store import NullAuditStore

suite = EvalSuite(
    [
        ResponseLengthEvaluator(min_length=20),
        PIIEvaluator(),
        RequiredDisclosureEvaluator(["past performance"]),
    ],
    audit_store=NullAuditStore(),
    tenant_id="local",
)

result, record = asyncio.run(
    suite.run(
        LLMEvalInput(
            agent_id="fund-advisor",
            agent_version="1.0.0",
            prompt="What is the management fee on Fund X?",
            response="The fee is 0.42%. Email me at jane.doe@example.com.",
        )
    )
)
print("overall_passed:", result.overall_passed)
for score in result.scores:
    print(f"  {score.evaluator}: {score.passed} {score.failure_mode or ''}")
```

```bash theme={null}
python evaluate.py
```

```text theme={null}
overall_passed: False
  ResponseLengthEvaluator: True
  PIIEvaluator: False PII_DETECTED
  RequiredDisclosureEvaluator: False REQUIRED_DISCLOSURE_MISSING
```

Every score is a pass or a fail with a named `failure_mode`. There are no
confidence thresholds to tune. See the [evaluator
library](https://docs.sengol.io/reference/evaluator-library.md) for every evaluator and its failure
modes.

## 3. Add an LLM judge

Some checks need a model. Give an LLM evaluator a judge client. This one
catches a response that contradicts the source document:

```python judge.py theme={null}
import asyncio

from sengol.build.suite import EvalSuite
from sengol.core.types import LLMEvalInput
from sengol.evaluators import FaithfulnessEvaluator, PIIEvaluator
from sengol.governance.audit_store import NullAuditStore
from sengol.judges import OpenAICompatibleJudgeClient

judge = OpenAICompatibleJudgeClient(model="gpt-4o-mini")  # reads OPENAI_API_KEY
suite = EvalSuite(
    [PIIEvaluator(), FaithfulnessEvaluator(judge=judge)],
    audit_store=NullAuditStore(),
    tenant_id="local",
)

result, _ = asyncio.run(
    suite.run(
        LLMEvalInput(
            agent_id="fund-advisor",
            agent_version="1.0.0",
            prompt="What is the management fee on Fund X?",
            response="The management fee on Fund X is 2.5%.",
            context="Fund X fact sheet: management fee 0.42% per year.",
        )
    )
)
for score in result.scores:
    print(f"{score.evaluator}: passed={score.passed} {score.failure_mode or ''}")
```

```bash theme={null}
python judge.py
```

```text theme={null}
PIIEvaluator: passed=True
FaithfulnessEvaluator: passed=False FAITHFULNESS_FAIL
```

* `OpenAICompatibleJudgeClient(base_url=...)` reaches any OpenAI-compatible
  endpoint, including one your bank hosts. `JUDGE_LLM_BASE_URL` sets it too.
* `AnthropicJudgeClient` reads `ANTHROPIC_API_KEY`.
* Any object with an `async judge_async(request) -> JudgeResponse` method is a
  judge. Use this to plug in an in-house model.

> [!NOTE]
>
> Sengol never holds your model-provider keys. The judge client reads them
> from your process environment.

## 4. Record signed evidence

Swap `NullAuditStore` for `ApiAuditSink`. Each verdict becomes a record that
the appliance signs and chains. The sink spools to disk first
(`~/.sengol/spool/audit.wal`), so an unreachable API does not lose records:

```python record.py theme={null}
import asyncio
import os

from sengol.api.sink import ApiAuditSink
from sengol.build.suite import EvalSuite
from sengol.core.types import LLMEvalInput
from sengol.evaluators import PIIEvaluator, RequiredDisclosureEvaluator


async def main():
    sink = ApiAuditSink(os.environ["SENGOL_API_URL"], os.environ["SENGOL_API_TOKEN"])
    suite = EvalSuite(
        [PIIEvaluator(), RequiredDisclosureEvaluator(["past performance"])],
        audit_store=sink,
        tenant_id=os.environ["SENGOL_TENANT_ID"],
    )
    result, record = await suite.run(
        LLMEvalInput(
            agent_id="fund-advisor",
            agent_version="1.0.0",
            prompt="What is the management fee on Fund X?",
            response="The fee is 0.42%. Past performance is not indicative of future results.",
            policies=["OSFI_E23"],
        )
    )
    print("passed:", result.overall_passed, "record:", record.record_id)
    await sink.close()  # flush the spool before exiting


asyncio.run(main())
```

```bash theme={null}
python record.py
sengol audit list --agent-id fund-advisor --limit 3
```

```text theme={null}
01a10caf-6076-...  2026-10-05T15:29:42Z  fund-advisor  overall_passed=True  severity=INFO
```

`policies` labels the record with the regulations it serves. It does not
choose evaluators; you do that in code or in the
[ControlBook](https://docs.sengol.io/concepts/controlbook.md).

## 5. Guard live calls

`sengol.instrument()` patches the OpenAI, Anthropic and MCP clients in your
process. It checks each prompt before the call and each response after it,
writes a signed record for both, and raises `BlockedError` instead of
returning a failing response:

```python agent.py theme={null}
import sengol
from openai import OpenAI
from sengol import BlockedError
from sengol.evaluators import RequiredDisclosureEvaluator

sengol.configure(
    agent_id="fund-advisor",
    agent_version="1.0.0",
    policies=["OSFI_E23"],
    post_evaluators=[RequiredDisclosureEvaluator(["not financial advice"])],
)
sengol.instrument()  # reads SENGOL_API_URL and SENGOL_API_TOKEN

client = OpenAI()


def ask(question):
    try:
        reply = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": "End every answer with: This is not financial advice."},
                {"role": "user", "content": question},
            ],
        )
        return reply.choices[0].message.content
    except BlockedError as exc:
        return f"[withheld: {exc.failure_mode}]"


print(ask("In one sentence, what is a management fee?"))
print(ask("My SIN is 046 454 286. Is it valid?"))
```

```bash theme={null}
python agent.py
```

```text theme={null}
A management fee is a charge ... This is not financial advice.
[withheld: PII_DETECTED]
```

The second prompt never reaches the model: the default pre-call check is
`PIIEvaluator`. For more control over what happens on a failure, see
[Guardrails](https://docs.sengol.io/guides/guardrails.md).

## 6. Gate releases

Put a dataset of known questions and answers in your repository and run the
gate on every pull request. See [Gate releases in CI](https://docs.sengol.io/guides/ci-gate.md).

## 7. Register the agent

An agent appears on the Console's **Agents** page only after an approver
approves an intake for it. Until then Sengol still ingests its evidence.

In the Console, open **Govern → Intakes → New** and fill in the form.

| Field | Meaning |
| - | - |
| Agent id | The name your agent reports in its records. Up to 256 characters. It cannot start with `tenantlifecycle:`, `retention:`, `legalhold:`, `retention-policy:` or `retention-auto-execute:`. It stays reserved while the intake is submitted and after it is approved or rejected. Only a withdrawn or expired intake releases it. |
| Use case | What the agent does and for whom. |
| Owner | Who is accountable for it, for example an email address. Free text, up to 256 characters. |
| Risk tier | Defaults to `MEDIUM`. `LOW`, `MEDIUM` and `HIGH` are the built-in tiers, and the form accepts any name, so use the tiers your organization has set up. It sets how many approvers and which reviews apply. |
| Data classes | Optional labels for the data it handles, such as `pii`. A review policy can route on them. |
| Policies | Optional policy ids from your [ControlBook](https://docs.sengol.io/concepts/controlbook.md), such as `OSFI_E23`. |
| Labels | Optional `key=value` pairs, one per line. They decide who can see the agent. |
| Capabilities | Optional JSON listing the tools and agents it may call: `{"tools": ["search"], "callees": []}`. Use the names your traces show. A call to anything not listed fails the delegation check. If an authorization policy exists for the agent, it replaces this list. |
| Repository | Optional. Recorded with the intake only. |

You can also submit with your token. A token is not an interactive sign-in,
so name yourself as the requester. Your principal id is the `sub` of your
session token.

```bash theme={null}
JWT=$(curl -s -X POST "$SENGOL_API_URL/v1/auth/token" \
  -H "Content-Type: application/json" \
  -d "{\"grant_type\":\"local_user_token\",\"secret\":\"$SENGOL_API_TOKEN\"}" \
  | python -c 'import sys, json; print(json.load(sys.stdin)["access_token"])')
ME=$(python -c 'import sys, json, base64; p = sys.argv[1].split(".")[1]; print(json.loads(base64.urlsafe_b64decode(p + "=" * (-len(p) % 4)))["sub"])' "$JWT")
AGENT="docs-advisor-$(date +%s)"
curl -sf -X POST "$SENGOL_API_URL/v1/intakes" \
  -H "Authorization: Bearer $JWT" -H "Content-Type: application/json" \
  -d "{\"agent_id\":\"$AGENT\",\"use_case\":\"Answers customer questions\",\"owner\":\"owner@example.com\",\"inherent_risk_tier\":\"HIGH\",\"policies\":[\"OSFI_E23\"],\"labels\":{\"team\":\"retail\"},\"requester_claimed\":\"$ME\"}" \
  | python -c 'import sys, json; d = json.load(sys.stdin); print(d["agent_id"], d["state"])'
```

An intake submitted with a token waits for you. Sign in to the Console, open
the intake and select **Confirm as requester**. Then an approver can approve
it. See [Approve an agent intake](https://docs.sengol.io/journeys/compliance-officer.md#approve-an-agent-intake).

While it is open you can **Withdraw** it. An approver can revise it, and a reviewer
on a routed review step can request changes.
An intake that nobody decides expires after 30 days.

![Intakes page listing two submitted intakes with risk tier, owner and expiry](https://mintcdn.com/sengol/WAzD6pTFOjHwDiMz/images/console-intakes.png?fit=max\&auto=format\&n=WAzD6pTFOjHwDiMz\&q=85\&s=44179c20a8d5a288d16e6ed70d571c06)

Govern → Intakes, filtered to submitted. My submissions lists only your own.

## What to hand to the next person

* **Approvers** need your intake. Tell them its agent id.
* **Compliance** needs your agent id and the evaluators you run, to map them
  in the [ControlBook](https://docs.sengol.io/journeys/compliance-officer.md).
* **Reviewers** see failing records in the Console's **Human review** page.
* **Auditors** export your agent's records by its `agent_id`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.