Reference architecture for a scalable guardrails engine

A practical design for collecting engineering evidence and enforcing standards across repositories, pipelines, and AI workflows.

A platform team sets a test-coverage standard. One service publishes a coverage report from a shared workflow. Another runs tests through a Makefile. A third has a custom pipeline that produces a different report format. All three use the same CI vendor.

The threshold is easy to write. Establishing whether each service meets it, for the code about to merge, takes more work.

This is the architectural problem behind engineering guardrails. Every standard depends on evidence, and that evidence is scattered across source code, CI execution, tool outputs, and external systems. A rule that only works for the shared workflow leaves the other services outside enforcement.

A scalable guardrails engine collects those observations centrally, normalizes them into a common data model, and evaluates standards wherever developers and AI agents need feedback. This article follows one coverage standard through that architecture, including the less tidy cases: incomplete runs, stale reports, and a new rule that would fail half the organization.

Collect evidence where the work happens

The engine needs two complementary views of the software development lifecycle (SDLC): what is in the repository and what actually happens during a pipeline run.

An organization-level version-control integration provides access to the repositories in scope. Code-change events can trigger analysis of Dockerfiles, dependency manifests, Kubernetes configuration, ownership files, and other source-controlled material. Repository permissions and exclusions define the coverage boundary.

CI instrumentation supplies the execution view. A tracer deployed with supported runners can observe processes and trigger collectors around relevant commands. This makes it possible to recognize a test tool invoked through a script or Makefile, rather than relying on a particular workflow step name.

On Linux, filtered syscall tracing can support this approach by notifying a tracer about selected process-execution events. The exact kernel, runner, and container requirements belong in the implementation’s support matrix. Performance claims need measurements from representative workloads.

The important deployment property is central control. The platform team operates the integrations and collectors across the repositories and runners in scope. Teams retain their build logic, while the enforcement system gains a consistent way to observe it.

CI templates still have a useful role in distributing build and test workflows. Central instrumentation adds coverage for teams whose pipelines differ from the template, including teams using the same CI vendor.

Separate collection, evaluation, and feedback

The architecture has three main responsibilities:

• Collect observations from code, pipeline execution, and relevant external systems.
• Store normalized posture data associated with the service and code revision.
• Evaluate policies against that evidence and deliver actionable results.

The data flow looks like this:

Repositories in scope       Instrumented CI runs
          |                         |
     Code collectors           CI collectors
          |                         |
          +------------+------------+
                       |
         Revision-scoped posture data
                       |
           Deterministic evaluation
                       |
          +------------+------------+
          |            |            |
     Agent feedback  PR checks  Deploy gates

Posture data + decision history -> visibility and evidence

CI observations arrive asynchronously. A code collector may finish before tests start, and separate test jobs may finish at different times. The posture store and evaluator have to account for that sequence.

Keeping collection separate from evaluation also avoids collecting the same data for every use case. A coverage observation can support a PR check, an adoption dashboard, and an investigation into which services have stopped publishing reports.

Give different tools a common evidence contract

For the coverage standard, collectors need to turn different tool outputs into the same facts: whether the relevant run completed, what coverage it measured, and where the result came from.

A compact posture document might look like this:

{
  "component": "payments-api",
  "revision": "example-commit-a",
  "testing": {
    "coverage": {
      "completed": true,
      "percentage": 78.5,
      "source": {
        "tool": "go-cover",
        "run_id": "example-run-42",
        "observed_at": "2026-11-05T14:00:00Z"
      }
    }
  }
}

This is an illustrative schema. An implementation can store revision and provenance in an envelope or metadata store rather than inside the document itself. What matters is that the association is explicit and survives queries, retries, and reevaluation.

The collector records 78.5 percent. The policy owns the organization’s 80 percent threshold. Changing that threshold should not require modifying every report parser.

Normalization also needs a semantic contract. Line coverage and branch coverage measure different things. Combining both under a field called percentage would make the JSON look consistent while making the policy unreliable. Collectors should preserve the measurement type, scope, and relevant configuration when those affect comparability.

That contract is the work a policy language cannot do on its own. A rule engine can evaluate structured input, but someone still has to produce trustworthy input from the organization’s actual workflows.

Follow one observation into a guardrail

Consider a collector that parses a coverage report into the coverage object above. In Lunar’s collection interface, publishing that normalized object can be as small as:

lunar collect -j ".testing.coverage" - < normalized-coverage.json

The file contains the completed, percentage, and source fields for the observation. The collection context associates the write with the relevant component and revision. The parser that produces the file is tool-specific; it must distinguish a valid report from a missing, malformed, or incomplete one.

A corresponding policy can use Lunar’s Python assertion API:

from lunar_policy import Check

with Check("coverage", "Tests must meet the coverage standard") as c:
    c.assert_true(
        c.get_value(".testing.coverage.completed"),
        "The required coverage run did not complete",
    )
    c.assert_greater_or_equal(
        c.get_value(".testing.coverage.percentage"),
        80,
        "Test coverage must be at least 80%",
    )

This policy evaluates a completed coverage observation against a threshold. Its correctness depends on the collector’s measurement contract and the engine’s handling of incomplete collection. The snippet shows the collector-to-policy boundary; deploying it also requires collector configuration, policy scope, and an enforcement level.

The same distinction applies to scanner and test-tool detection. Observing a process start proves that a command was invoked. It does not prove that a scan completed, its report was parsed, or its findings met the standard. A hook that matches every invocation of a tool can even match its version command.

Collectors should match the relevant command and capture the evidence the policy actually requires.

Keep incomplete and stale data out of passing decisions

Suppose revision A starts a slow test run. Revision B is pushed, starts a second run, and finishes first. The report from A arrives afterward.

If the store treats the last report received as the latest coverage result, revision B can inherit the wrong evidence. The engine needs to associate writes and decisions with the revision and run they describe, regardless of arrival order. Reruns also need an explicit rule for which attempt supplies the accepted result.

Missing evidence needs equally clear treatment. While an expected test job is running, the guardrail can remain pending. Once collection finishes, a report that never arrived must resolve through defined missing-data or error behavior. It cannot silently reuse an older passing result.

A failed collector should produce a distinguishable error. Otherwise, a developer sees “coverage below threshold” when the real problem is a parser that broke after a tool upgrade. The platform team needs enough context to identify the collector, affected run, and last successful observation.

These distinctions determine whether developers can act on the feedback and whether the platform team can trust the result.

Reuse policies at each enforcement point

During AI-assisted authoring, an agent hook can request evaluation and return relevant violations to the agent’s dev-test loop. A source-level check can catch a prohibited API or a missing configuration field before a PR exists. The agent can then correct its work and run the check again.

A CI-derived coverage report may not exist at that point. The architecture must preserve the difference between an early check passing and the full set of required evidence being available. Reusing policy logic does not mean pretending every stage has identical inputs.

At PR time, the evaluator can combine source analysis with observations from the relevant CI runs. When configured as a required check, a failing guardrail prevents merge. A passing guardrail satisfies that requirement; the PR still has to meet the repository’s other merge conditions.

At deploy time, the gate needs to evaluate the revision or artifact being released. If a service rebuilds after merge, the system must establish how the released artifact relates to the evidence being checked.

For blocking standards, deterministic evaluation means that the same evidence and policy version produce the same result. LLM-assisted assessments can help with judgment-heavy work, such as reviewing a runbook, but a fixed threshold on a variable model score does not make the entire assessment deterministic. Keep those assessments distinct from deterministic blocking checks.

Operate guardrails like a production system

A policy that works for one service still needs a rollout strategy.

For the coverage standard, draft evaluation reveals which services would fail, which produce no usable evidence, and which report a different coverage measure. That gives the platform team a way to fix the collection gaps before asking developers to remediate violations.

Advisory feedback can then show teams the observed value, the threshold, and the next action. A canary group tests the blocking behavior before broader promotion. Exceptions need an owner, a reason, and an expiry date so temporary accommodations do not become permanent blind spots.

The underlying system needs similar operational discipline. A useful design includes:

• Explicit component-to-repository mapping, including services that share a monorepo.
• Versioned collection contracts and policies, with repeatable tests for changes.
• Idempotent writes and bounded retries so duplicate events do not corrupt posture data.
• Queues and concurrency limits that prevent a burst of PRs from exhausting collector capacity.
• Scoped reevaluation so an update to one observation does not trigger unrelated work.
• Decision history recording the evidence, policy version, scope, and enforcement mode.

These are design requirements for a scalable engine. The exact mechanisms vary by implementation.

Collector failures also need isolation and a delivery policy. Limits on runtime and resource use help protect application builds. The engine must separately define whether an infrastructure error blocks delivery or permits it with an explicit warning. Running a collector alongside CI does not, by itself, settle that choice.

Make evidence useful beyond a single PR

Once posture data is collected consistently, platform leaders can answer fleet-level questions: which services meet the coverage standard, which have never produced a usable report, and which stopped reporting after a pipeline change.

Those are different populations. A dashboard that reports only on services with available data can make adoption look much better than it is. Show the coverage boundary and unknown states alongside pass/fail results.

The same data supports other engineering standards: approved runtimes, dependency migrations, Kubernetes resource requirements, valid ownership, and operational readiness. External APIs can supply evidence about on-call schedules or service ownership where code alone is insufficient.

Access and trust remain part of the architecture. Collectors need scoped credentials, sensitive outputs need appropriate handling, and changes to policies need review. Evidence gathered from a team-controlled pipeline also has a trust boundary. Central storage does not automatically make that evidence complete or tamper-resistant.

How Earthly Lunar fits

Earthly Lunar implements the central instrumentation, collection, and normalization layer described here, together with policy evaluation, organization-wide visibility, and enforcement in developer and AI workflows.

The repo integration and the CI/CD Tracer provide visibility into source code and pipeline execution. Collectors turn those signals into structured posture data. Policies evaluate that data, with enforcement that can progress from draft and advisory modes to PR and deploy blocking.

Lunar ships with 300+ pre-built guardrails across testing, deployment, operational readiness, security, and compliance, plus interfaces for custom collectors and policies. Platform teams can use the same engineering standards across human-written and AI-generated changes without distributing new enforcement logic repo by repo.

The build-versus-buy decision includes the ongoing work: maintaining parsers as tools change, handling incomplete runs, preserving revision identity, and operating guardrails across different teams’ workflows. A team with that infrastructure already in place may choose to extend it. A team building it from scratch needs to account for those responsibilities alongside the policy code.

Bring one stalled engineering standard and a representative pipeline to a Lunar architecture walkthrough. Use them to examine what evidence is available, where feedback belongs, and what would be required to enforce the standard across the rest of the organization.

See how Lunar works.