AI developer

You build the agent. Your job with Sengol is to wire evaluation into the agent’s code and its release pipeline, without slowing either down.

Get credentials

An API URL and, on a local-auth install, an sk_ token from your platform operator.

Evaluate locally

Score responses in code. No appliance needed.

Add an LLM judge

Catch what rules cannot, such as an answer that contradicts its source.

Record signed evidence

Send each verdict to the appliance as a signed record.

Guard live calls

Withhold a failing response before the user sees it.

Gate releases

Fail the CI build when the agent misses its thresholds.

Propose it as an intake. An approver registers it.

1. Get credentials

Ask your platform operator for:

the API URL, for example https://sengol.internal.example
the install’s tenant id
a personal sk_ token with the developer role (how operators mint one). sk_ tokens exist only in local auth mode. On an install that signs people in through an IdP they cannot be minted, sengol gate and the evidence sinks ignore a sengol login session, and this journey does not cover that setup. See Credentials by journey

export SENGOL_API_URL=https://sengol.internal.example
export SENGOL_TENANT_ID=acme-bank
export SENGOL_API_TOKEN=sk_acme-bank_...

Install the SDK from the Sengol source you were given. Add the openai-judge or anthropic-judge extra if you will use a hosted LLM judge:

pip install "./sengol[openai-judge]"

2. Evaluate locally

An EvalSuite runs a list of evaluators over one input. Deterministic evaluators run first. If a CRITICAL one fails, the LLM evaluators are skipped. Otherwise, all LLM evaluators run concurrently. NullAuditStore signs the record and stores nothing, which is what you want while iterating:

evaluate.py

import asyncio

from sengol.build.suite import EvalSuite
from sengol.core.types import LLMEvalInput
from sengol.evaluators import PIIEvaluator, RequiredDisclosureEvaluator, ResponseLengthEvaluator
from sengol.governance.audit_store import NullAuditStore

suite = EvalSuite(
    [
        ResponseLengthEvaluator(min_length=20),
        PIIEvaluator(),
        RequiredDisclosureEvaluator(["past performance"]),
    ],
    audit_store=NullAuditStore(),
    tenant_id="local",
)

result, record = asyncio.run(
    suite.run(
        LLMEvalInput(
            agent_id="fund-advisor",
            agent_version="1.0.0",
            prompt="What is the management fee on Fund X?",
            response="The fee is 0.42%. Email me at jane.doe@example.com.",
        )
    )
)
print("overall_passed:", result.overall_passed)
for score in result.scores:
    print(f"  {score.evaluator}: {score.passed} {score.failure_mode or ''}")

python evaluate.py

overall_passed: False
  ResponseLengthEvaluator: True
  PIIEvaluator: False PII_DETECTED
  RequiredDisclosureEvaluator: False REQUIRED_DISCLOSURE_MISSING

Every score is a pass or a fail with a named failure_mode. There are no confidence thresholds to tune. See the evaluator library for every evaluator and its failure modes.

3. Add an LLM judge

Some checks need a model. Give an LLM evaluator a judge client. This one catches a response that contradicts the source document:

judge.py

import asyncio

from sengol.build.suite import EvalSuite
from sengol.core.types import LLMEvalInput
from sengol.evaluators import FaithfulnessEvaluator, PIIEvaluator
from sengol.governance.audit_store import NullAuditStore
from sengol.judges import OpenAICompatibleJudgeClient

judge = OpenAICompatibleJudgeClient(model="gpt-4o-mini")  # reads OPENAI_API_KEY
suite = EvalSuite(
    [PIIEvaluator(), FaithfulnessEvaluator(judge=judge)],
    audit_store=NullAuditStore(),
    tenant_id="local",
)

result, _ = asyncio.run(
    suite.run(
        LLMEvalInput(
            agent_id="fund-advisor",
            agent_version="1.0.0",
            prompt="What is the management fee on Fund X?",
            response="The management fee on Fund X is 2.5%.",
            context="Fund X fact sheet: management fee 0.42% per year.",
        )
    )
)
for score in result.scores:
    print(f"{score.evaluator}: passed={score.passed} {score.failure_mode or ''}")

python judge.py

PIIEvaluator: passed=True
FaithfulnessEvaluator: passed=False FAITHFULNESS_FAIL

OpenAICompatibleJudgeClient(base_url=...) reaches any OpenAI-compatible endpoint, including one your bank hosts. JUDGE_LLM_BASE_URL sets it too.
AnthropicJudgeClient reads ANTHROPIC_API_KEY.
Any object with an async judge_async(request) -> JudgeResponse method is a judge. Use this to plug in an in-house model.

Sengol never holds your model-provider keys. The judge client reads them from your process environment.

4. Record signed evidence

Swap NullAuditStore for ApiAuditSink. Each verdict becomes a record that the appliance signs and chains. The sink spools to disk first (~/.sengol/spool/audit.wal), so an unreachable API does not lose records:

record.py

import asyncio
import os

from sengol.api.sink import ApiAuditSink
from sengol.build.suite import EvalSuite
from sengol.core.types import LLMEvalInput
from sengol.evaluators import PIIEvaluator, RequiredDisclosureEvaluator


async def main():
    sink = ApiAuditSink(os.environ["SENGOL_API_URL"], os.environ["SENGOL_API_TOKEN"])
    suite = EvalSuite(
        [PIIEvaluator(), RequiredDisclosureEvaluator(["past performance"])],
        audit_store=sink,
        tenant_id=os.environ["SENGOL_TENANT_ID"],
    )
    result, record = await suite.run(
        LLMEvalInput(
            agent_id="fund-advisor",
            agent_version="1.0.0",
            prompt="What is the management fee on Fund X?",
            response="The fee is 0.42%. Past performance is not indicative of future results.",
            policies=["OSFI_E23"],
        )
    )
    print("passed:", result.overall_passed, "record:", record.record_id)
    await sink.close()  # flush the spool before exiting


asyncio.run(main())

python record.py
sengol audit list --agent-id fund-advisor --limit 3

01a10caf-6076-...  2026-10-05T15:29:42Z  fund-advisor  overall_passed=True  severity=INFO

policies labels the record with the regulations it serves. It does not choose evaluators; you do that in code or in the ControlBook.

5. Guard live calls

sengol.instrument() patches the OpenAI, Anthropic and MCP clients in your process. It checks each prompt before the call and each response after it, writes a signed record for both, and raises BlockedError instead of returning a failing response:

agent.py

import sengol
from openai import OpenAI
from sengol import BlockedError
from sengol.evaluators import RequiredDisclosureEvaluator

sengol.configure(
    agent_id="fund-advisor",
    agent_version="1.0.0",
    policies=["OSFI_E23"],
    post_evaluators=[RequiredDisclosureEvaluator(["not financial advice"])],
)
sengol.instrument()  # reads SENGOL_API_URL and SENGOL_API_TOKEN

client = OpenAI()


def ask(question):
    try:
        reply = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": "End every answer with: This is not financial advice."},
                {"role": "user", "content": question},
            ],
        )
        return reply.choices[0].message.content
    except BlockedError as exc:
        return f"[withheld: {exc.failure_mode}]"


print(ask("In one sentence, what is a management fee?"))
print(ask("My SIN is 046 454 286. Is it valid?"))

python agent.py

A management fee is a charge ... This is not financial advice.
[withheld: PII_DETECTED]

The second prompt never reaches the model: the default pre-call check is PIIEvaluator. For more control over what happens on a failure, see Guardrails.

6. Gate releases

Put a dataset of known questions and answers in your repository and run the gate on every pull request. See Gate releases in CI.

7. Register the agent

An agent appears on the Console’s Agents page only after an approver approves an intake for it. Until then Sengol still ingests its evidence. In the Console, open Govern → Intakes → New and fill in the form.

Field	Meaning
Agent id	The name your agent reports in its records. Up to 256 characters. It cannot start with `tenantlifecycle:`, `retention:`, `legalhold:`, `retention-policy:` or `retention-auto-execute:`. It stays reserved while the intake is submitted and after it is approved or rejected. Only a withdrawn or expired intake releases it.
Use case	What the agent does and for whom.
Owner	Who is accountable for it, for example an email address. Free text, up to 256 characters.
Risk tier	Defaults to `MEDIUM`. `LOW`, `MEDIUM` and `HIGH` are the built-in tiers, and the form accepts any name, so use the tiers your organization has set up. It sets how many approvers and which reviews apply.
Data classes	Optional labels for the data it handles, such as `pii`. A review policy can route on them.
Policies	Optional policy ids from your ControlBook, such as `OSFI_E23`.
Labels	Optional `key=value` pairs, one per line. They decide who can see the agent.
Capabilities	Optional JSON listing the tools and agents it may call: `{"tools": ["search"], "callees": []}`. Use the names your traces show. A call to anything not listed fails the delegation check. If an authorization policy exists for the agent, it replaces this list.
Repository	Optional. Recorded with the intake only.

You can also submit with your token. A token is not an interactive sign-in, so name yourself as the requester. Your principal id is the sub of your session token.

JWT=$(curl -s -X POST "$SENGOL_API_URL/v1/auth/token" \
  -H "Content-Type: application/json" \
  -d "{\"grant_type\":\"local_user_token\",\"secret\":\"$SENGOL_API_TOKEN\"}" \
  | python -c 'import sys, json; print(json.load(sys.stdin)["access_token"])')
ME=$(python -c 'import sys, json, base64; p = sys.argv[1].split(".")[1]; print(json.loads(base64.urlsafe_b64decode(p + "=" * (-len(p) % 4)))["sub"])' "$JWT")
AGENT="docs-advisor-$(date +%s)"
curl -sf -X POST "$SENGOL_API_URL/v1/intakes" \
  -H "Authorization: Bearer $JWT" -H "Content-Type: application/json" \
  -d "{\"agent_id\":\"$AGENT\",\"use_case\":\"Answers customer questions\",\"owner\":\"owner@example.com\",\"inherent_risk_tier\":\"HIGH\",\"policies\":[\"OSFI_E23\"],\"labels\":{\"team\":\"retail\"},\"requester_claimed\":\"$ME\"}" \
  | python -c 'import sys, json; d = json.load(sys.stdin); print(d["agent_id"], d["state"])'

An intake submitted with a token waits for you. Sign in to the Console, open the intake and select Confirm as requester. Then an approver can approve it. See Approve an agent intake. While it is open you can Withdraw it. An approver can revise it, and a reviewer on a routed review step can request changes. An intake that nobody decides expires after 30 days.

Intakes page listing two submitted intakes with risk tier, owner and expiry — Govern → Intakes, filtered to submitted. My submissions lists only your own.

What to hand to the next person

Approvers need your intake. Tell them its agent id.
Compliance needs your agent id and the evaluators you run, to map them in the ControlBook.
Reviewers see failing records in the Console’s Human review page.
Auditors export your agent’s records by its agent_id.

Get started

Journeys

Concepts

Guides

Reference

1. Get credentials

2. Evaluate locally

3. Add an LLM judge

4. Record signed evidence

5. Guard live calls

6. Gate releases

7. Register the agent

What to hand to the next person

​1. Get credentials

​2. Evaluate locally

​3. Add an LLM judge

​4. Record signed evidence

​5. Guard live calls

​6. Gate releases

​7. Register the agent

​What to hand to the next person

1. Get credentials

2. Evaluate locally

3. Add an LLM judge

4. Record signed evidence

5. Guard live calls

6. Gate releases

7. Register the agent

What to hand to the next person