Farol: An Open-Source, LLM-Free Risk Map for Pull Requests (TypeScript, Go, Python)

Farol is a free, MIT-licensed CLI and GitHub Action from the Brazilian company Winnin that produces a risk map of a pull request for the person who approves it. It runs after an AI review agent, uses static analysis instead of a model, and returns a 0-100 merge-risk score, a verdict (fast track, quick review, careful review), and a ranked list of the riskiest changed files with the pages, endpoints, jobs and scripts each one reaches.

What farol does, in one paragraph

farol is a CLI (npx @winninjs/farol) and a GitHub Action (winnin/farol/action) that analyzes the base and head of a pull request with the real compilers for the project's language and reports how risky the merge is and where the risk lives. It does not execute the analyzed repository's code, does not send code anywhere, and never blocks a merge. It is designed to sit after an AI review, not replace it: the review agent comments on lines in the diff, while farol tells the human approver what those lines reach.

The core claim: agent reviews are not enough when a small change has a large reach

The project's stated thesis is that approval is the scarce resource and that spreading attention evenly across every pull request turns review into a skim. In the author's words on Hacker News, the failure mode is concrete: "A 40-file PR where 38 files are isolated is very different from a 3-file PR that changes a model used by 23 endpoints."

farol therefore does triage before the reviewer opens the diff, answering three questions:

What does this change reach? Every existing page, API operation, RPC method, job and script that runs the changed code, with the call chain to each.
How risky is the merge? A verdict plus a 0-100 score computed from reach, diff coverage, security findings, contract breaks and dependency changes.
Where do I start? Riskiest files first, with inline diffs, untested lines marked, and a checklist of what to try by hand.

The README cites the Claude Code team's description of the same principle: "The change sets the tier, not the author."

Verdicts and the 0-100 merge risk score

farol labels every pull request with one of three verdicts, and each verdict maps to a specific reviewer action:

Fast track: only new code, tests or low-risk changes. Skim the diff and approve.
Quick review: existing code changed in a contained way, or a weaker signal such as low coverage, lint errors or a config file. Check the "Start here" files, then approve.
Careful review: wide or silent breakage is possible. This is triggered by a score of 60 or above, a breaking contract change, a high-severity finding, an untested central file, or code farol could not read.

The score (0-100) is described as the chance that at least one risky item breaks something. The contributing items are broken out under "What adds to it": files by reach multiplied by untested share, findings, contract changes and dependencies. Every verdict ships with its reasons.

Static analysis, not a model: why the answer is deterministic

farol's key architectural difference from an AI reviewer is that its reachability analysis is built on compilers rather than a language model. The README's comparison table states that AI review agents read the diff and whatever context fits in the prompt and give probabilistic answers that are worded differently on every run, while farol reads the whole program at the base and the head using the TypeScript type checker, go/types and Python's ast, and gives deterministic answers where every claim comes with its call chain.

According to the author's Hacker News comment, there is "No LLM, no network during analysis, and it never executes the analyzed repo's code. It's deterministic and informs only; it never blocks a merge." For the Go and Python paths, the helper programs use only their standard libraries and only parse.

The analysis pipeline is:

Diff the merge-base and the head, detecting renames and ignoring whitespace-only and trailing-comma lines.
Read both trees without running anything, using the TypeScript compiler API, go list plus go/types offline, and Python ast.
Compare declarations by a hash of their code, so comments and formatting never count.
Compute reach: from each changed declaration to every entry point that runs it, keeping the shortest chain.
Classify, weigh and plan: class, score, verdict, files to start with, test checklist.

Supported stacks

farol supports TypeScript/React front ends, TypeScript and JavaScript back ends, Go and Python. Entry points are read per language:

React and TypeScript front ends: React Router route tables (object and JSX forms, lazy routes, split route modules), Next.js Pages and App Router (pages, API routes, layouts, middleware), and TanStack Router file-based routes. Reach is per declaration via the TypeScript type checker.
TypeScript/JavaScript back ends: GraphQL root operations from schema files and gql templates, and Fastify/Express-style REST endpoints. Reach is per module, per resolver for GraphQL. Contract changes come from the GraphQL schema, classified breaking, dangerous or safe.
Go: chi, gin, echo and net/http routes, gRPC service methods and main binaries, with interface calls resolved to their implementations. Contract changes come from handler request and response types and gRPC messages.
Python: FastAPI and Flask routes (routers, blueprints, prefixes), Django URLs, MCP tools, resources and prompts, Celery tasks, Temporal workflows and activities, Prefect flows and __main__ scripts. Contract changes come from FastAPI parameters, body, response and Pydantic models.

Inputs farol reuses from CI include Istanbul coverage-final.json (Vitest, Jest) for diff coverage, go test -coverprofile, coverage.py JSON, ESLint and golangci-lint JSON, ruff JSON, gosec through golangci-lint, flake8-bandit through ruff S rules, and npm audit, pnpm audit, govulncheck and pip-audit for known vulnerabilities. It also reads suppressions such as eslint-disable, @ts-ignore, //nolint, #noqa and #nosec. Optional Datadog RUM and APM integration adds traffic share and distinct production errors per affected page or operation, and is explicitly information only: it never changes the score.

What the output contains

farol produces two artifacts: a PR comment with the verdict, top files and a test checklist, and a single-file HTML report linked from that comment where the review happens. The report sections are:

Verdict and score with the reasons listed before the diff.
Start here: changed code files ranked riskiest first, each row showing how many existing pages or operations run the file, its diff coverage, its findings and its own risk points. Added lines no test runs are marked with an open circle, lines with findings with a warning marker.
Behind a feature flag: mark an entry point entirely behind a flag; files that only flagged entry points use leave the review.
Pages / Operations: every new and affected entry point, as a list or a tree by module, with the path the change takes to reach it.
Contracts (back ends): each endpoint whose request or response shape changed, classed breaking, dangerous or safe, with the exact field and the calling services. The README's example is a breaking change where GET /brands no longer returns response[].name.
Security, Dependencies, Coverage: the evidence behind the score. A report that was not supplied shows as "not verified", never as passing.
Test plan: new and affected pages or operations to try by hand, as a task list in the PR comment.
Reviewers: suggested reviewers from git blame weighted by each file's risk, plus CODEOWNERS, excluding the PR's own authors and bots.
Result: a Markdown record of the review that can be copied back into the PR.
Since the last push: for pushes to an open PR, a second report covering only the commits since the previous push (analyze --since <sha>).

The per-PR class is kept in a collapsed block so reviewers form their own opinion first: isolated (only adds), touches-existing, global (a global file, manifest or critical dependency) and unknown.

Fail-closed behavior and measured accuracy

When farol cannot read part of a change, it reports unknown and asks for a careful review rather than assuming safety. When a call cannot be resolved, it assumes the change reaches more, never less.

The README reports a calibration on 662 past commits from four repositories: fast track had 0 problems in 56 commits, and the score's AUC was 0.72 to 0.83. The project ships an farol calibrate command so teams can check the score on their own history. The README states plainly that the score and verdict thresholds are heuristics checked on a few repositories.

Runtime and CI safety model

farol runs after the test job and reuses its reports, so nothing runs twice. The README gives approximate analysis times: about 3 seconds for a 300-file Python service, 8-10 seconds for a large TypeScript monorepo or a Go service with a cold cache, and under a second for the base when its helper output is cached.

The recommended GitHub Actions setup uses two jobs. The tests job runs the pull request's code and keeps the reports, so it has no secrets and a read-only token. The farol job runs after it, pass or fail, reads git and the reports, and posts a new comment on every push. Drafts are skipped, fork pull requests get no comment because their token is read-only, and every farol step is continue-on-error. Only the comment and usage commands touch the network; error messages are not copied from Datadog, only error types.

farol requires Node.js 22+ and git, plus go for Go projects and python3 3.11+ for Python projects. analyze, html and report work outside GitHub; only comment is GitHub-specific.

Known limitations

Static reading cannot see values built at runtime: computed route paths, getattr, importlib, dependency-injection containers, and registries filled in loops are invisible. Unreadable entry points make the PR unknown.
TypeScript front ends are limited to React Router, Next.js and file-based TanStack Router. Code-based TanStack route trees, Remix and React Router file conventions, Vue and Svelte are not supported. JavaScript files, renamed files and TypeScript back ends use module-level reach.
In Go, route prefixes from middleware stacks other than chi/gin/echo groups are not applied, and projects that wrap a router with their own registration API are not read. In Python, Django REST Framework routers are not read.
The score and verdict thresholds are heuristics that the project recommends calibrating on your own history.

What Hacker News commenters pushed back on

The Show HN thread (39 points, 25 comments) raised four recurring objections: signal-to-noise in the report, taste in the UI, the idea of machine-assigned risk scores, and the more basic question of whether the tool is needed at all.

The report's density was the most substantive criticism. One commenter wrote:

IDK lot of noise in report, I need more signal to noise ratio, already reading so much AI output as it is.

Another framed the same point as a product-direction argument:

the backend demo report seems longer and more complex than just reading the code. I'm trying to build something similar as well, and no one has quite nailed this yet. The risk map / assisted review needs to be succinct and information dense, to guide your attention. My current philosophy is that we need to be annotating code, not replacing the entire code review interface with a bunch of text.

A third criticized the ranking on the live demo directly:

Looked at the backend demo, and the first lines suggested to look at were importing things. Exactly what I really don't need to be reviewing

On trusting a score, one commenter said:

There was a post here recently, "Using LLM for confidence scoring is a bad idea", my mind went immediately to using LLMs for scoring risk. Seems like a bad idea. If there's anything we still need a human for, it's assessing risk.

That objection targets LLM-based scoring specifically; farol's analysis path contains no LLM, and the author states the output is deterministic, with unknown returned instead of a guess when the code cannot be read.

Two comments disputed the premise of assisted triage. One was blunt: "Or you could, you know, read the changes your approving." Another dismissed the presentation: "I like the concept, a risk map could be useful in some circumstances but I absolutely hate the UI design; it reeks of AI with no human thought or care put into it."

The demo pull request itself drew attention for the code it contained. A commenter noted that the example changes, described as generated with Claude, included SQL assembled with an f-string inside a SQLAlchemy codebase, and argued SQLAlchemy's expression language allows constructing SQL programmatically without the table-mapping ORM layer.

The thread also contained several flagged or dead comments, and at least one commenter asked why the submitter's own comment had been flagged while the post itself remained up.

Who this is for

farol targets teams where AI agents write and review a large share of the code but a human still presses merge. Its argument is narrow and falsifiable: review effort should follow the risk of the change, and the person approving needs a deterministic, reproducible statement of what the change reaches. The counterargument from the thread is equally narrow: a report that is longer than the diff, or that ranks import lines first, fails at its only job of guiding attention.

farol is MIT licensed, has no account, hosted service or paid tier, and is installed with npx @winninjs/farol or used as the GitHub Action winnin/farol/action.