How Kagex works, end‑to‑end

00Engineering thesis

Why the system is shaped the way it is.

A manual audit certifies a single commit. Every commit after it ships unreviewed, because no team can afford an audit per pull request. Kagex closes that gap: it reviews every change with an AI pipeline whose output is deterministic in structure, stable across re-scans, and compounding in quality — then turns the result into a public, verifiable security score.

01System architecture

Three planes: a Next.js app, an async analysis worker, and Claude.

The web app never runs a scan inline. A scan is a durable job: the app enqueues it and returns immediately; a dedicated worker consumes the queue, drives the AI pipeline, and writes results back — keeping request latency flat and scans crash-safe and retryable.

02Multi-agent scan pipeline

Two specialist agents per file, run concurrently, structurally constrained.

Each Solidity file is analyzed by two agents with distinct system prompts and knowledge bases:

• Vulnerability Scanner — grounded in a curated knowledge base of 420+ vulnerability patterns from 800+ documented exploits. Reports every issue, honestly rated by severity and confidence.
• Solidity Expert — reviews against best-practice coding standards: security patterns, gas efficiency, code quality.

Structured outputs — no parse gamble

Every call constrains the response to a JSON schema (output_config.format = json_schema). A success is guaranteed to match the findings schema — no fallback parsing, no dropped finding. A refusal or max_tokens stop reason throws — a truncated analysis fails loudly rather than reporting a partial "all clear".

Prompt caching

The large, static knowledge base carries a cache_control breakpoint, so every per-file call reuses the cached prefix. Learned lessons sit in a separate trailing cached block, keeping the stable KB prefix cache-hot even as lessons evolve.

Failure semantics

• If every agent call fails, the scan fails outright (and is retried) — never reported as clean.
• If some files failed, a would-be PASS is downgraded to PASS_WITH_WARNINGS so the gap is visible.

03Per-scan-type model routing

Match model depth to the scan's job — pay for depth only where it matters.

A PR scan reviews a focused diff; a full audit reviews an entire codebase. Routing them to different models keeps PR feedback fast and cheap while reserving maximum reasoning for deep audits. The chosen model is persisted on each scan.

Overridable per-tier via environment without code changes.

04Security scoring

A 0–100 score across seven categories, computed only from scan results.

The score is severity-weighted and status-aware. It is computed exclusively from findings — it cannot be purchased, boosted, or influenced. Dismissed and resolved issues stop counting; acknowledged issues count at half weight (the risk is real, the team has chosen to carry it).

Model-produced free-text categories are normalized into the seven buckets deterministically (e.g. anything matching /reentran/ → Reentrancy). Each category yields a sub-score; the overall aggregates them — exactly the breakdown exposed on the public profile and scorecard badge.

05Fingerprints & suppression

Content-addressed identity so an issue is the same issue across scans.

Every finding gets a stable fingerprint. Cosmetic changes in the model's output (an added comment, re-quoted string, shifted line) must not create a "new" issue, and a genuine code change must:

fingerprint = sha256( file | category | swcId | normalize(snippet) )

normalize(s): strip block/line comments → collapse whitespace → lowercase

Suppression & status carry-forward

06Regression detection

Every scan is diffed against the right baseline.

A scan is compared against the previous completed scan of the same lineage; each finding is classified new, persisting, or fixed. The comparison scope is chosen so partial and full analyses never contaminate each other:

A "fixed" finding only counts if it held live risk in the baseline and is gone now — and only a full-repo scan updates the repo's public health score, since a PR scan sees only changed files.

07The learning loop — an “immune system”

Every validated finding makes every future scan — on any protocol — better.

When a team validates a finding — confirming a real vulnerability, or marking a false positive — Kagex distills it into a generalized, anonymized lesson and adds it to a shared knowledge base immediately. Confirmed patterns sharpen detection; false-positive patterns cut noise.

Anonymization (never leaks customer code)

A lesson is built from the finding's class-level fields (title, category, SWC id, description, impact, fix) — never raw code. A redaction pass strips protocol-identifying specifics before storage:

in:  transfer to 0xABCDEF123456 of "1000 tokens" costing 12345678 wei
out: transfer to <addr> of <value> costing <num> wei

Convergence & anti-poisoning

• Deduped by (label, category, swcId, normalized title) with a confirmations counter — a pattern confirmed by many teams outranks a one-off, so the highest-signal lessons are injected first.
• Mock/dev findings are skipped, so test data never pollutes the shared KB.
• Lessons ride a dedicated cached prompt block — measurable, versioned, cheap to carry.

08The public score & registry

The score as shared infrastructure — a credit-rating layer for on-chain code.

Any protocol can opt a repository into a public profile. From there the score is readable by humans and machines — open and unfakeable because everything is served live from Kagex.

All read endpoints are no-auth and CORS-open so wallets, explorers, and dashboards can consume them directly. Badges are served live and escape all user-controlled text — a score can neither be faked with a static image nor used as an injection vector.

09Gamification — earned, never bought

Rewards flow only from verifiable improvement, and never touch the score.

Security Points are awarded only for outcomes that make code safer — a rescan-verified fix, a clean PR — never for spend or scan volume. Awards are idempotent (deduped in the database), so re-pushing commits can't farm points. Points drive leagues (Bronze → Titan) and non-purchasable badges.

10GitHub integration

Sign-in, repo access, and label-triggered PR scans.

• OAuth — "sign in with GitHub" is also the grant to read the user's repos. Sessions are stateless jose HS256 JWTs; the access token is encrypted at rest with AES-256-GCM.
• GitHub App — one-click manifest creation; authenticates with an RS256 app JWT and mints short-lived installation tokens for repo/PR file access.
• Webhooks — signed with HMAC and verified in constant time. A PR labeled with the configured scan label enqueues a PR scan; the worker fetches only the PR's changed .sol files and posts results back as a PR comment.

11Security & multi-tenancy

Every row is org-scoped; every boundary is enforced.

Volumetric (L3/L4) DDoS is absorbed at the CDN/edge in front of the app; the controls above are the application-layer complement.

12Reliability

Scans are durable jobs, not fire-and-forget requests.

• Retries — BullMQ retries failed jobs with backoff; only the final attempt marks a scan failed.
• Stuck-scan sweeper — a periodic sweep fails scans that never finish, timing running scans from when work actually began (not enqueue) and giving pending scans a longer grace.
• Honest verdicts — partial coverage downgrades a clean pass; an all-agents-failed scan fails rather than reporting "clean".
• Idempotent side-effects — points awards and lesson writes dedupe at the database layer, so retries and races can't double-count.

13Roadmap

From “review every change” to a security network.