Build a CI Agent with Coresource

Your pipeline enforces everything a machine can check mechanically. Coresource lets you put an agent in CI that enforces the standards you define and trust it as a required check.

CI is the most successful quality mechanism software has ever adopted. Its principle is simple: any property you care about gets checked automatically, on every change, at the moment the change is made, or it decays.

But look at what your pipeline actually enforces. Tests, types, lint, dependency scans: every check is mechanical, because until now a check had to be fully specified in advance to run unattended. Everything that takes judgment (does this change respect our architecture? our data boundaries? our regulatory obligations? our API contracts?) stayed with human reviewers. Which means it gets checked when someone remembers, by whoever happens to review, against whatever subset of the rules they hold in their head that day.

That is the biggest unfilled seat in the pipeline: a reviewer that reads code the way a person does, but runs the way a check does.

A CI agent fills it. Coresource is the runtime for building one you can actually trust in the required-checks list.

What a CI agent is

A CI agent is a step in your pipeline like any other: it triggers on a pull request, runs headless, and reports pass, fail, or needs-a-human. The difference is what happens inside the step. Instead of matching patterns, it reads the change in the context of the codebase and judges it against your standards: compliance frameworks, architecture decisions, security plans, coding practices, API guidelines, and control catalogs.

The concept is general. The same shape of agent can hold a codebase to an architecture, catch breaking API changes a schema diff can't see, enforce data-handling rules across services, or keep docs true to the code they describe.

But a pipeline seat comes with unforgiving job requirements, and this is where most attempts die. A required check that is usually right is worse than no check at all: the team either learns to ignore it or learns to trust it wrongly. To hold the seat, an agent must:

• Apply the whole standard, every run. Not the memorable parts. A reviewer that silently skips rules when the diff is large doesn't reduce risk. It launders it.
• Run unattended, indefinitely. Nobody babysits CI. The agent must survive the failures long-running work always hits and behave the same on run five hundred as on run one.
• Show its work. A red X with no reasoning gets overridden; a verdict that feeds a decision a human is accountable for needs an inspectable record.
• Know its edge. When a change raises a genuine judgment call, the agent must surface it as an explicit outcome, not guess and not block forever.

Wire a model to a webhook and you get none of these. They are runtime properties, not model properties. That is precisely the layer Coresource provides.

How Coresource maps to the job

Coresource is a managed agent runtime: you supply the standard to enforce and the model does the reasoning, while the runtime supplies the properties that make the agent trustworthy. Architecturally, three things are true at once: the trigger and the verdict live in your pipeline, the run lives on the platform, and the standard travels with the run as its objective.

Each job requirement above maps directly onto the runtime:

Your standard becomes a durable objective. The documents that define the check (the framework, the security plan, the org's interpretations) are pinned as the objective governing the entire run. They do not fade into prompt history as the context fills with diff; they remain the standard the work is verified against from first action to last.

Coverage is decomposed and verified. The standard breaks down into rules the agent must answer for individually: each comes back marked applicable, satisfied, or flagged, so nothing can be skipped silently. Anything short of a complete, clean verdict fails closed.

Every run produces an auditable record. What was examined, what was concluded, and why, captured as a structured event trace rather than a chat log. Every run has a durable id; its history can be replayed from the first event, by any process, long after the CI job that started it is gone. The verdict on the pull request links to evidence a reviewer, or an auditor, can walk.

Escalation is a first-class result. Pass and fail are not the only outcomes. When the agent needs a human decision, the run pauses durably, checkpointed rather than burning tokens or holding a connection open, and resumes from exactly that state when the answer arrives.

That is the recipe. Now the hardest example we know: the least forgiving standard software ships under.

The stress test: FedRAMP in the pipeline

If a CI agent can be trusted anywhere, it has to earn that trust where the rules are hardest: federal compliance.

The FedRAMP Authorization Act establishes a standardized, reusable approach to security assessment and authorization for cloud services used by federal agencies. "Be FedRAMP compliant" unpacks into hundreds of controls drawn from NIST SP 800-53: access control, audit and accountability, configuration management, incident response, system integrity, supply-chain risk management, and more, each tailored to the specific system and backed by evidence.

Two properties make it the perfect stress test for a CI agent. It is conjunctive: the system must satisfy every applicable control at once, controls interact, and at the Moderate baseline's 323 controls, 322 satisfied and one violated is not 99.7% compliant. It is a finding. And it is continuous: FedRAMP's monitoring guidance requires ongoing assessment that controls are implemented correctly, operating as intended, and producing the desired outcome, for as long as the system runs. A codebase merging dozens of changes a week cannot meet that standard with a quarterly review. The only place continuous assessment can actually live is the pipeline the changes flow through.

Built on Coresource, the compliance agent takes each pull request together with the codebase and the team's control context, and on every change:

• Scopes which controls the change actually touches: a schema migration, a new log line, and a dependency bump implicate different control families.
• Judges the diff in context the way an assessor would: what does this code do to the data flows, boundaries, and obligations the controls protect?
• Reports like a reviewer. Compliant changes pass without ceremony; violations are flagged on the pull request with the control, the reasoning, and the specific code, while the developer still has the context to fix it in minutes.
• Escalates judgment calls, like a genuine trade-off or a gap in the security plan, as explicit findings instead of guesses.
• Leaves evidence behind, so every merge arrives with its assessment attached and continuous monitoring becomes a property of the project's history rather than a report someone assembles.

No tool grants an authorization, and nothing here replaces assessors or the authorizing official. What it replaces is the months between a violation entering the codebase and a human finding it. That rule-by-rule accounting is what makes the verdict worth gating on, and the trace is what makes it worth showing an auditor.

Your pipeline, your standard

Notice what the FedRAMP agent needed from Coresource: a durable objective, rule-by-rule coverage, an auditable record, unattended durability, structured escalation. Notice what none of those are: compliance features.

Swap the control catalog for your own standard and the same build holds: the files below don't change, only the standard you hand the run. An architecture-conformance agent gating merges against your ADRs. A data-handling agent enforcing boundary rules across services. An API-review agent catching contract breaks that no schema diff can see. If your team can write the standard down, a Coresource agent can hold every change to it: mechanically, continuously, with evidence.

And whatever the standard, the pipeline sees a plain check. Every terminal state of the run maps to an outcome:

The pipeline taught software teams that quality you don't check on every change is quality you lose. For twenty years that lesson stopped at the checks a machine could pattern-match. It doesn't anymore.

Try your own CI agent

The entire integration is two files. Point them at a diff and a written standard, and the result is the check.

The agent itself plugs into any CI: your job step supplies pr.diff and standards.md, and the run returns the verdict and the evidence.

import { readFile } from 'node:fs/promises';
import { createAgent } from '@coresourceai/sdk';

const instructions = new URL('./reviewer.md', import.meta.url);

const reviewer = createAgent({
  type: 'agent',
  systemPrompt: await readFile(instructions, 'utf8'),
  model: 'deepseek/deepseek-v4-flash',
  effort: 'high',
  maxTurns: 80,
  outputs: ['verdict.json', 'evidence.md']
});

const result = await reviewer.run({
  prompt: 'Review pr.diff against standards.md. Write both artifacts.',
  files: ['pr.diff', 'standards.md']
});

await reviewer.close();

The system prompt lives in its own file: how to review is prose, and your compliance team can edit it without touching code.

You are a compliance reviewer sitting in CI. Judge the pull-request diff
(`pr.diff`) against every rule in the supplied standard (`standards.md`).

The standard is conjunctive: the change passes only if every applicable rule
is satisfied. Work rule by rule. Do not conclude while any applicable rule is
unexamined; report inapplicable rules as not applicable rather than skipping
them silently. Flag violations with the rule id, the exact code or text, and
the reasoning. If something genuinely needs a human judgment call, ask;
never guess.

Write two artifacts:

- `verdict.json` (machine-readable):
  `{"verdict": "pass" | "fail",
    "rules": [{"id", "applicable", "satisfied", "note"}],
    "findings": [{"rule", "file", "excerpt", "reasoning"}]}`
- `evidence.md` (a reviewer-readable report): what was checked and what was
  found, rule by rule, quoting the exact lines for every finding.

That is the whole agent. Drop it into your CI workflow on GitHub Actions or any other platform and mark the step as required.

Define your standards. Build the agent on Coresource. Make your standards a required check.

Coresource is the managed agent runtime for durable, auditable, long-horizon AI agents. Start with the quickstart, or learn more at coresource.ai.