Can an LLM Analyze Stocks? Only If the Context Has a Contract
In the previous post I listed five real failures from a market-analysis pipeline. The conclusion was that four of five had nothing to do with model choice — the model turned structured records into sentences correctly every time, and was correct about wrong inputs.
So the useful question is not which LLM. It is: what does the context have to guarantee before the model is allowed to write a sentence?
Here is the contract I ended up with.
Rule 1 — every field carries its provenance
A value in the context is never a bare number. It carries where it came from and when.
prev_close: 71_400
source: daily_chart.stck_clpr
as_of: 2026-07-27 (KST close)This looks like overhead until the day two fields that mean different things collide. “Yesterday’s close” and “current price” are both a number of the same magnitude in the same currency. Without provenance, nothing in the pipeline can tell them apart — which is exactly how a gap direction ends up inverted.
The cheap version: prefix the key with the source (daily__prev_close,
live__cur_price). You do not need a schema library to make a collision loud.
Rule 2 — completeness is a gate, not a field
The model never sees a partially-filled record. A record either satisfies its required set or it is dropped and counted.
REQUIRED = ("code", "cur_price", "prev_close", "volume")
ready = [r for r in rows if all(r.get(k) is not None for k in REQUIRED)]
missing = len(rows) - len(ready)Two things matter here.
Never default a missing value. get(key, 0) converts an absence into a claim. A
price of zero is not “unknown,” it is a specific and wrong number, and any downstream
formatter will render it as fact.
Report the drop count. If nine of ten records were dropped, the briefing needs to say so. Silence about what was excluded is how a pipeline degrades for weeks without anyone noticing.
Rule 3 — confidence is bounded by data quality
The failure that stuck with me most was a signal that read 0 KRW BUY (67%). The 67%
came from indicator scores, which were fine. Nothing connected data completeness to
confidence, so a record missing its price could still be reported at 67% conviction.
Confidence has to be the minimum of the model’s score and the data’s own quality:
quality = present_fields / len(REQUIRED) # 0.0 .. 1.0
score = min(indicator_score, quality)If that seems crude — it is. It is also strictly better than the two numbers being unrelated, which was the previous state.
Rule 4 — external references are matched against a spec, not judged by plausibility
Any API path, TR code, field name, or symbol that the model produces gets checked against the provider’s specification file before it is used. Not reviewed for plausibility — matched, verbatim, as a string.
This is the check that catches fabricated infrastructure. The endpoint that cost me weeks
of a dead feature looked exactly like the real ones: right prefix, right digit count,
right URL shape. The only thing that distinguished it was that it did not appear in the
spec workbook. A grep found that in a second; a code review never would have.
Rule 5 — the analysis path cannot write
The step that reads data and produces narrative gets read-only access to state. Writes belong to a separate, explicit step.
The reason is not tidiness. If analysis can write, then every exploratory run — every
--dry-run, every reproduction of yesterday’s output — edits the dataset that tomorrow’s
analysis will summarize. Contamination survives and gets averaged in. A crash would have
been kinder.
The test that this actually holds: diff the state file before and after a dry run. If it differs by one byte, the flag is decoration.
What stays outside the model entirely
Some things should never be delegated to the model, regardless of how good it gets:
- Fetching. The model does not choose endpoints. It receives records.
- Arithmetic that has a right answer. Percent change, moving averages, gap direction — computed in code, verified by tests, passed in as values.
- Date and index selection. Rows are matched by explicit date string, never by array position. Reading a daily-bar array one index off produces output that is fluent, internally consistent, and about the wrong day.
What is left for the model is what it is actually good at: turning a verified record set into readable prose, flagging which items are unusual relative to the rest, and phrasing uncertainty honestly.
That is a narrower job than “analyze stocks.” It is also the version that survives contact with production.
The boundary
This pipeline reads market data, computes indicators, and writes a briefing. It does not decide anything. The trading decisions are mine, made separately, and they stay out of what gets published here — partly because the strategy parameters are not the transferable part, and partly because the moment a system like this emits a recommendation, it needs a completely different set of guarantees than the ones above.
The engineering is the part worth sharing. The rest is not a content problem.
This describes the architecture of a data pipeline. It is not investment advice and not a recommendation to buy or sell any security.