The pipeline returned nine sections. Every field read the same: N/A.
Technical positioning, token economics, market structure, ecosystem role, regulatory exposure, team and governance, risk matrix, narrative, supply-chain transmission. Nine dimensions, zero information points. The input schema had ten required fields. Ten came back blank: title missing, source missing, category returned "unclassified," domain tag "unclassified," one-line summary missing, author stance "undetermined," project list empty, time sensitivity unassessed, source quality unassessed.
The second stage did the only thing that mattered. It stopped. It printed a diagnostic โ here is what I need, here is what is missing, here is why I will not guess โ and refused to produce a verdict.
That refusal is the rarest behavior in crypto research tooling right now. The default failure mode of an automated analyst is not silence. It is a confident, well-formatted, entirely fictional report.
I have seen the fictional version. Not in a demo. In production, pasted into a governance forum thread, cited by three people who did not check the citations.
Every one of these pipelines is two functions bolted together. Stage one extracts: it reads an article, a GitHub README, a governance post, and emits structured facts. Stage two reasons: it takes those facts and produces a judgment. The contract between them is a schema, and the schema is where the whole thing lives or dies.
In a market where the information half-life is measured in hours, the extraction layer is the load-bearing wall. A reader in a drawdown does not want a thesis. They want to know whether the protocol holding their collateral has an admin key, an unaudited upgrade path, or a treasury that runs dry in eleven weeks. Those are extractable facts. They are not opinions, and they should not be generated.
I learned the difference in 2017, three months deep in the Waves IDEX contracts. The marketing described a liquidity pool. The code described an integer overflow in the trading engine. I wrote the proof-of-concept, submitted it to the repository, and the team patched it in two weeks. The lesson was not "audit things." The lesson was that a description of code and the code itself are different objects, and only one of them is load-bearing.
Same in 2020, six weeks reverse-engineering Compound's cToken rate model in Hardhat. I could not have stress-tested liquidation cascades from a blog post. I needed the collateral factors, the reserve factors, the actual curve parameters. Strip those out and the simulation is fiction with a nice chart on top. By 2022, mapping Mercurial Finance's leverage mechanism after the collapse, the pattern was fixed. A post-mortem without the parameter table is a story. A post-mortem with it is a causal chain.
Here is the structural defect, and it is not a model problem.
The schema conflated three states that must never be conflated: unknown, not applicable, and known-empty. In a typed system these are distinct. NULL means the value exists but we did not retrieve it. An explicit "not applicable" means the value does not exist for this object โ a pure news item with no token has no unlock schedule, and that is a fact, not a gap. An empty string means we looked and found nothing.
The pipeline collapsed all three into one token and passed it downstream. Downstream, a NULL and a "doesn't exist" are indistinguishable, and every conclusion inherits the confusion.
Then the confidence ceiling. The schema had a source-quality field โ official primary, reputable media, general media, social rumor, unverified. It came back unassessed. That single omission caps the credibility of every other field in the report, because a claim's confidence cannot exceed its weakest provenance. A risk matrix built on an anonymous Telegram screenshot and one built on a published audit are not the same document, even when the JSON is byte-identical.
Then the timestamp. The report could not determine whether an event was priced in. That is correct, and it is also unanswerable without three inputs: the exact event time, the historical price series of the affected asset, and derivatives positioning around the window. "Priced in" is not a vibe. It is a computation, and it requires a clock. The code doesn't negotiate with a missing timestamp.
Then the risk matrix. Six categories โ technical, market, operational, regulatory, competitive, narrative โ all unknown. The framework's own rule is that a rating must rest on at least one identified exposure. So it output "cannot be assessed." Good. Because the alternative, a "medium" or a "low," is a category error dressed as diligence. Unknown risk and low risk are opposites in a portfolio, and every dashboard that renders them in the same color is lying to its users.
The genuinely useful artifact in that report was the stopping rule. If the extraction success rate โ the share of requests where the information-point list comes back non-empty โ falls below 95%, the line stops. That is not a research principle. It is statistical process control, borrowed from manufacturing. You do not ship units and inspect the defect rate afterward; you stop the line when the process drifts out of tolerance.
Applied to crypto research, the tolerance band is brutal. Below 95% extraction, every downstream report is a coin flip wearing a lab coat. Below 90% fill rate on any single required field, that entire dimension is permanently dead โ you will keep generating the section, and it will keep being noise.
The fix is not a better model. It is a stricter type. Make the pipeline total: it returns either a report or a structured error, never a half-filled report. Nine of ten fields populated is not 90% correct, it is a malformed object, and the type system should reject it the same way a compiler rejects a missing return. The extraction stage should be forced to declare, per field, whether it found the value, proved it does not exist, or failed to look. Those are three different exit codes and they deserve three different names.
Then instrument it. Track the non-empty rate on the information-point list. Track fill rate across the eight mandatory fields. Track the missing-source-quality ratio, and when it crosses twenty percent, downgrade every conclusion in the system by one confidence level automatically, without asking a human. These are not analytics. They are the fuse box.
The industry is fixated on the wrong failure. Everyone is building guardrails against the model inventing a tokenomics table. The actual defect is upstream and dumber: the extractor silently returned an unfilled template, and nothing between stage one and stage two was typed strictly enough to notice.
The second, less comfortable point: coverage is the enemy of integrity. Once a product commits to producing a nine-dimension report on every protocol in a watchlist, the pressure to fill blanks becomes structural. An empty field looks like a bug to the person paying for the dashboard. So the blanks get filled โ by inference, by analogy to a competitor, by a plausible-sounding prior. The code doesn't care whether the field is true. It only cares that it is populated.
The third: I would rather ship the N/A report. An analyst that says "I have ten empty fields and here is the minimum set I need to fix that" is worth more than one that says "buy," because the first is auditable and the second is a mood.
The next layer of this problem is already being built. In 2026 I worked on a verifiable inference oracle โ zero-knowledge proofs that an off-chain model ran correctly on a given input, 10,000 inferences at 99.9% accuracy on a private testnet. The interesting part was never the accuracy. It was proving the input existed, unmodified, before the model touched it.
Apply that to research pipelines and the question sharpens. Not "did the model reason well," but "can you prove the model was given anything at all?" Until a report can carry a cryptographic receipt for its own inputs, the empty template and the confident verdict will keep looking identical on the page. Only one of them will cost you money โ and you will not know which until the position is already open.