← Back to blog
·8 min read

Test impact across a TypeScript/Python boundary: one fused graph, one PR comment

A one-language code graph stops at the wire. We fused our TypeScript app and Python worker through a 19-line seam spec so one PR comment covers both sides.

test-impact-analysiscicode-intelligence

A code graph built from one language stops at the wire. When a TypeScript API and a Python worker share an HTTP contract, test selection on either side alone is blind to the other, and the same two-file diff can get two opposite verdicts from two graphs of the same repository. We closed that gap on this site with a 19-line seam spec that joins the two graphs at the contract, and a CI job that comments the cross-language blast radius on every pull request that touches it.

The problem

This site has a small Python side. The Factory feed's drain, a script that runs on a developer machine, composes activity notes and POSTs them to /api/factory/notes, a Next.js route whose ingest schema lives in a TypeScript file. That is the contract: one schema, one endpoint, one caller, in two languages, in one repository. (The gate that decides whether those notes may publish is its own post.)

Every pull request here already gets graph-based test selection: a job builds a code graph of src/, hands the diff to Blastline, and runs the tests the diff can reach. That job is TypeScript-only by construction, because src/ is. The Python files are not in its graph, and nothing in a graph of the TypeScript tells it that a one-line change to the ingest schema breaks a function six hundred lines into a Python script.

That blindness fails in two directions. A change to the Python drain alone reaches the TypeScript job as two files with no graph node, which is a correct fail-open: run everything. A change to the schema is worse, because the job can vouch for it. It selects the TypeScript tests that import the schema and says nothing about the Python caller that just broke. The edge from the schema to the drain's post() function is in neither language's graph. It is in the contract between them.

What we did

We declared the contract once, as a seam spec, and let CGraph resolve it against both graphs. The whole file:

{
  "provider": "webapp",
  "api_version": "v1",
  "error_codes": [400, 401, 429, 500],
  "services": [
    {"name": "webapp", "role": "provider", "owned": true},
    {"name": "factory", "role": "consumer", "owned": true}
  ],
  "schemas": [
    {"name": "FactoryNoteIngest", "canonical": "webapp/src/lib/factory/schema.ts"}
  ],
  "endpoints": [
    {"method": "POST", "path": "/api/factory/notes", "response_schema": "FactoryNoteIngest", "path_params": [], "query_params": []}
  ],
  "consumes": [
    {"service": "factory", "method": "POST", "path": "/api/factory/notes", "call_site": {"graph": "factory", "file": "factory-drain.py", "line": 600}}
  ],
  "mirrors": []
}

Three decisions are in those nineteen lines, and each was made against an alternative we rejected.

Anchor to real code, or emit nothing. The schema is anchored to the file that defines it; the consumer is anchored to a file and a line inside the function that makes the request. cgraph seam gen resolves both against the real per-repo graphs and emits a cross-service edge only when a node is there. The alternative is guessing: match URL strings across repositories and draw an edge wherever /api/factory/notes appears. That produces exactly the plausible-but-wrong edge you cannot build a test selector on, and the open-source seam design rejects it for the same reason.

Build both graphs on every pull request, then fuse. The job builds a graph of src/ and a graph of tools/factory/, runs seam gen against the committed spec, seam fuse to produce one graph, and hands that to the Blastline action. It triggers only on paths that can touch the contract: the factory library and route, the Python tools directory, and the MongoDB client the route writes through. The ignore list is a set of claims that changes there cannot affect which tests should run: Markdown, the documentation and drafts directories, posts, images, JSON, plists, shell scripts and YAML. Getting that list wrong in the other direction is how a test-impact tool starts lying, so it stays short, and it already holds one entry we would tighten: the JSON pattern also matches the seam spec itself, so a pull request that changes only seam.json gets no selection from it.

Advisory here, enforced there. This job posts a comment. It does not skip a single test. The repo-wide job is the one that runs the selected subset, and it runs the full suite whenever it fails open. We wanted to read the cross-language selection against reality for a while before letting it gate anything. You cannot learn whether a test-impact tool is honest by trusting it. This is the same posture we take on the developer platforms we build for clients: a new signal earns its authority by being right in public first.

The result

The artifact is the set of comments the job has actually posted. Four pull requests have triggered it since August.

PR What the diff touched Seam job verdict
#38, the end-to-end exercise a comment added to the ingest schema Subset: 4 TypeScript test files, with the Python caller in the blast radius
#41, pseudonym auto-publish factory library and route, plus .env.example Full suite: .env.example has no graph node
#42, Mongo database name the MongoDB client, plus .env.example Full suite: .env.example has no graph node
#46, dry-run cursor fix the Python drain and its test file Subset: 1 of 12 test files

The first row is the one the seam exists for. A two-line comment added to the TypeScript schema, in a pull request opened only to exercise the job and closed without merging, produced a comment that selected the four TypeScript test files the graph reaches from it, and listed, in the same blast radius, the contract and the Python code on the other side of it:

Blast radius (12 dependents)
- endpoint POST /api/factory/notes
- file factory/factory-drain.py (tools/factory/factory-drain.py:1)
- file factory/feed.test.ts (src/lib/factory/feed.test.ts:1)
- file factory/feed.ts (src/lib/factory/feed.ts:1)
- file factory/notes.test.ts (src/lib/factory/notes.test.ts:1)
- file factory/notes.ts (src/lib/factory/notes.ts:1)
- file factory/schema.test.ts (src/lib/factory/schema.test.ts:1)
- file lib/mongodb.test.ts (src/lib/mongodb.test.ts:1)
- file lib/mongodb.ts (src/lib/mongodb.ts:1)
- function main (tools/factory/factory-drain.py:620)
- function post (tools/factory/factory-drain.py:596)
- service factory

That is the comment as posted, with the runner's path prefix removed. No graph of the TypeScript alone contains the second line, the two function lines, or the last one. The walk went from the schema node through the endpoint to the consumer's call site and on to the function that calls it, across the language boundary, because the seam put that edge in the graph.

The two middle rows are correct and unexciting. Both pull requests also edited .env.example, a file with no node in any code graph, so the selector refused to vouch and ran everything. That is the behavior we want, and why a config change runs your whole suite has the full reasoning.

The last row is the clearest picture of what the fusion buys, because by September the repo-wide job existed too, and both jobs commented on the same pull request two seconds apart. The diff touched two Python files and nothing else.

Diagram: the same two-file Python diff goes to two jobs. The repo-wide job, with a graph of src/ only, finds no node for either file and runs its full TypeScript suite of 11 test files. The seam job, with the src/ graph and the tools/factory/ graph fused at POST /api/factory/notes, selects 1 of 12 test files.

The repo-wide job, with its TypeScript graph, reported that two files had no graph node and fell open to its full run, which is bun test across the 11 TypeScript test files; the Python test is not in its suite at all. The seam job, with the Python graph fused in, mapped the same two files to five symbols, found that nothing in the TypeScript depended on the changed functions, and selected one test file of twelve: the Python one. Same diff, same repository, same tool. The only difference was whether the graph knew the other language existed.

Two limits are worth stating plainly, because they are the kind a vendor page leaves out.

The Python test file drives the drain through a subprocess, which no static graph can see. So a schema change puts post() and main() in the blast radius but does not select the Python test; a second consumer anchor at the test's drive site would. And the consumer anchor is a line number. The spec says 600. When the first comment was posted, def post started at line 596 and line 600 fell inside its body, so the anchor resolved to post(); the September fix shifted the file, and def post now sits at line 600 exactly. Neither state was arranged. Nobody has re-anchored the spec, and the next edit above that line will move it again: the anchor then lands on whatever sits at line 600, or, if nothing does, seam gen fails naming the anchor and the job stops before it comments. Newer CGraph releases discover HTTP routes and their callers, for the frameworks they read, without a hand-written spec; this job still pins the release it was verified with.

Takeaway

If your tests live on both sides of a wire, your test selection has to as well. Declare the contract once, anchor it to real code rather than to a URL string, let the graph refuse when it cannot find the anchor, and run the selection as advice until its comments have been right in public for a while. A per-language graph is not wrong about the language it covers. It is silent about the boundary, and the boundary is where the change that hurts usually crosses.

Building something like this?