The most dangerous output in any trading pipeline is not a wrong number. It is a well-formed JSON object full of placeholders. I learned this the hard way at 3 a.m. Geneva time, staring at a structured extraction result that looked, superficially, like a deliverable: six fields, clean schema, valid formatting. Every field read ๆชๆไพ, ๆชๅ็ฑป, ๆช่ฏไผฐ. The core opinion field: empty. The information point list: empty array. The pipeline did not fail loudly. It failed politely, and polite failures are the ones that liquidate accounts.
This is a forensic breakdown of that failure mode, because I have now seen it four times across three different signal-generation stacks in the past eighteen months, and each time the post-mortem blamed the wrong layer. The instinct is to audit the model. The correct move is to audit the plumbing. A schema that validates is not a schema that ran.
Context: Why the Hollow Output Problem Is a Bull-Market Disease
In a bull market, nobody audits the null path. Volume is up, funding is flowing, and the engineering priority is always throughput. When I was running real-time signal infrastructure during the 2021 run, our extraction layer processed roughly 12,000 articles per day with a 2.3% parse failure rate that we tolerated because the winners buried the losers. That tolerance is exactly how hollow outputs metastasize.
The architecture most teams run in 2026 looks like this: a scraper fetches raw HTML, a readability extractor pulls the body text, an LLM call structures that text into a fixed schema (title, source, information points, stance, domain, purpose), and downstream signal logic consumes the structured fields. Six stages, five handoffs, one silent killer. The failure I dissected had every downstream field at its template default, and crucially, the field names themselves were preserved: ๆชๅคๆญ, ๆช่ฏไผฐ, ๆชๆไพ are not null โ they are the placeholder strings baked into the prompt template. That distinction is the entire investigation.
When a system runs and produces empty values, you see nulls, empty strings, or explicit error objects. When a system never reaches the assignment step โ or reaches it and throws, then falls back to the template โ you see the template defaults, untouched. The presence of those placeholders tells you the pipeline did not degrade. It short-circuited.
And there was a third signal, the one that made me close my laptop lid and pour the espresso: the prompt instructions themselves were still sitting in the output, verbatim. Strings like "identify stance from the above information points" and "determine domain from the source field" โ instructions meant to be consumed by the model, still present in what was supposed to be a filled result. That is not a model that misunderstood. That is a template that was read and never written. Friction is where the opportunity hides, and here the friction was a five-stage gap between template invocation and template populate.
Core: The Anatomy of a Silent Short-Circuit
The forensic reconstruction, ordered by signal strength, because in incident response the first thing you triage is what is loudest.
Signal one: the empty information point array. This is the root cause, not a symptom. In every extraction architecture I have audited, information points are the atomic unit of the structured output โ the raw material from which stance, purpose, and domain are derived. Downstream fields depend on them via a single traversal loop. Empty list in, empty fields out. Every symptom downstream is epiphenomenal. When I see an empty array where the core output should be, I stop looking at the LLM and start looking at the text extraction stage immediately.
Signal two: template defaults preserved. I have written about this before in the context of institutional risk auditing โ a system that returns its default values is a system that never assigned. It is the difference between a trader who evaluated a position and chose zero, and a trader who never opened the order book. Those are not the same decision, and conflating them is how risk models develop blind spots. The default values are the fingerprint of a path not taken.
Signal three: instruction leakage. This is the most damning forensic artifact and the one most teams overlook. If the prompt text "identify the stance from the above information points" appears in the final output, the populate step did not execute. It is the equivalent of finding an unfilled form in a filing cabinet โ the form was routed there, the clerk received it, and the clerk never wrote in it. When I did my first smart contract audits back at ETH Zurich, this was the re-entrancy pattern: the function that was supposed to decrement the balance before the external call simply... wasn't. The control flow looked right on the surface, and the state was never mutated. Same failure topology, different stack. Forensic accounting for the decentralized age has taught me that the most expensive bugs are the state transitions that silently do not happen.
Signal four: title and source both missing. This is the metalogical canary. If the metadata โ the cheapest field to extract, the one that comes from the document header, not the body โ is absent, the input side is the prime suspect. Scraper returned a login wall. URL returned empty body. The passed string was empty. When title and source are missing together, I assume the text never arrived and work backward from the network layer.
So the diagnosis tree, and I have run this tree professionally enough times to have it taped above my desk:
Input exists and is non-empty?
โ Body extraction yielded >200 characters?
โ LLM response parses as valid JSON?
โ Schema field names match the parser's keys?
โ Information point array correctly traversed and written?The fourth step is the one that ate four hours of my life last quarter. The probabilistic extractor returned information_points. The parser read info_points. The traversal iterated over an undefined key, treated it as an empty iterable, and wrote an empty array downstream โ no exception, no warning, no trace. The model did its job. The code did not. And the parser was so confident about its silence that it also, in a flourish of misplaced helpfulness, passed through the original prompt text into the output payload. That last detail is the tell that separates a model failure from a plumbing failure: a confused model produces wrong content. A broken binding produces no content and leftover scaffolding.
Here is the quantitative frame that makes this urgent rather than academic. In a real-time trading signal stack, an extraction failure does not just cost you the article โ it costs you the signal that article would have generated. At 12,000 documents per day and a conservative 3% hollow-output rate, you are silently discarding roughly 360 potential signals daily. If the historical hit rate on those signals is even 15% with an average edge of 40 basis points, that is a blind spot worth approximately 2.2 basis points of expected daily edge on your full-book exposure โ and it compounds, because the hollow outputs cluster on the same market narratives that generate the strongest signals. The pipeline is not just failing. It is failing selectively, on the exact articles your signal logic was built to catch.
Mapping the invisible grid where value leaks out, and the leak is always at a handoff. This is why I model every extraction pipeline as a liquidity system: capital (information) flows through stages, and the failure mode is never a total halt โ it is a thin, silent bleed at one joint where the pipe diameter steps down to zero and nobody notices because the meter after the joint reads correctly for whatever did get through.
Contrarian: The Model Is Almost Never the Problem, and You Are Auditing the Wrong Layer
The consensus reflex in 2026 is to blame the LLM. Reprompt it, swap it, add chain-of-thought, upgrade the model. I have watched three teams burn a combined six engineer-weeks reprompting a model that was performing perfectly. The model returned well-formed, parseable JSON on every call. The prompt was firing. The output was being generated. The failure was one level down, in the binding between the response and the schema, and no amount of model quality fixes a parser reading the wrong key.
This is the contrarian position I will defend at any conference panel: the structured-extraction layer is a data-engineering problem masquerading as an AI problem, and the masquerade costs teams their entire incident budget. When I rebuilt my EigenLayer threat-model pipeline in 2024, I spent three days tuning the model and two hours finding the actual bug โ a field-name drift between the prompt schema version and the parser schema version, silently reconciled by a fallback that wrote defaults. The fallback was the villain. A fallback that writes template defaults instead of raising is not resilience. It is a system that lies to you at exactly the moment you need the truth, and it does so with clean formatting.
There is a deeper structural point that the atomic-information-point architecture exposes, and it is the same lesson I keep writing about: systems fail at the dependency boundary, not at the node. Every field downstream of the information point array inherits its emptiness. The array is the single point of failure, and it is also the single point of silence โ because an empty iterable is a valid iterable in every language, so the traversal never errors. The failure is legal at every layer simultaneously. That is what makes it invisible, and visibility, not accuracy, is the scarce resource in a real-time stack.
The corollary that teams hate hearing: your extraction pipeline needs a liveness assertion, not just a schema check. A schema validator on the output says "this object has the right shape." A liveness assertion says "this object contains evidence that the populate step ran." The presence of untouched template strings is positive evidence of a dead path โ and most teams are not even looking for it, because they validate types, not provenance. Validation without provenance is decoration.
Takeaway: The Next Watch Is the Null Path, Not the Model
Two roads, and I have walked both. The fast one: hand the raw text to a human analyst and let them do the deconstruction and the deep pass in a single sitting โ bypass the broken pipeline entirely, because in a bull market the only unforgivable delay is waiting on tooling that is quietly returning ๆชๆไพ.
The slow one, the one that actually pays: fix the stage. Profile it as exactly what it is โ a chain of five handoffs โ and instrument every handoff with a liveness probe that fires when template defaults survive to the output. Decide, in writing, whether a failed parse raises or returns; the current answer is returns, and that answer is costing you signals every day.
The forward question I am leaving on the desk: how many of your live systems are currently returning well-formed, schema-valid, completely hollow results โ and how would you even know, given that your dashboards track error rates and not emptiness rates? The error rate on that pipeline was zero. The emptiness rate was the only number that mattered, and nobody was graphing it. Speed is the only moat when the gate opens โ but a gate that opens onto an empty room is worse than a locked one, because you will walk through it and call it progress.