Market Data APIs, Measured: Coverage, Latency, Cost

Published Jul 28, 2026 · market-datareliabilityoperations

Every “best market data API” post compares the same four things: monthly price, number of tickers, historical depth, and whether there is a free tier. All four are copied off the provider’s pricing page. None of them predicted a single problem I actually hit.

The things that did decide it were request budget arithmetic, whether the source returns a number for my symbol set at my call time, whether a fetch fits inside its schedule slot, and what a failure looks like when it happens. This post is those four axes, measured on the sources I run.

What is measured here, and what is not

I run four classes of source in production: a broker’s REST API, that same broker’s WebSocket feed, yfinance, and portal HTML for things no API exposes. The numbers below come from those, from the client code and the operational logs.

I have not run Polygon, EODHD, or FMP in production, so this post contains no latency or uptime numbers for them. Publishing a benchmark table for a provider I have not paid for and operated would be exactly the kind of copied-off-the-pricing-page comparison I am complaining about. There is a measurement harness at the end — it is provider-agnostic, and it is how you get your own numbers instead of trusting mine.

Axis 1 — Cost is a request budget, not a monthly price

On a flat-rate or broker-bundled API the dollar cost is fixed, so “cost” moves entirely into a different currency: requests per unit time. My broker’s API caps request rate, so the client enforces its own floor before every call:

def _throttle(self) -> None:
    elapsed = time.time() - self._last_call
    if elapsed < self._min_interval:      # 0.06s
        time.sleep(self._min_interval - elapsed)
    self._last_call = time.time()

_min_interval = 0.06 is roughly 16 requests per second, deliberately under the documented ceiling. That one constant sets the real budget: a watchlist of N symbols fetched one at a time cannot complete faster than N × 0.06 seconds, no matter how fast the network is. 300 symbols is an 18-second floor, before any response time.

The escape hatch is the batch quote endpoint, and this is where reading the spec beats reading the pricing page:

for start in range(0, len(codes), 30):        # domestic: 30 symbols per call
    chunk = codes[start:start + 30]
    params = {}
    for i, code in enumerate(chunk, start=1): # 1-based suffixes: _1 … _30
        params[f"FID_COND_MRKT_DIV_CODE_{i}"] = "J"
        params[f"FID_INPUT_ISCD_{i}"] = code
    out.extend(self._get(MULTI_PRICE_PATH, tr_id=MULTI_PRICE_TR_ID, params=params).get("output", []))

Same 300 symbols: 10 calls instead of 300, a 0.6-second floor instead of 18. Same provider, same plan, same data — 30× fewer requests because of one endpoint.

Two details that only show up in operation:

• The batch limit is not uniform within one provider. Domestic quotes take 30 symbols per call; the overseas equivalent takes 10, and numbers its parameters _01…_10 instead of _1…_30. Same API, same auth, different arithmetic — so the budget has to be computed per market, not per provider.
• The batch response nests differently from the single-symbol one (output versus output2 on the overseas call). Batching is never a drop-in swap; it is a second parser.

So the honest cost question for a rate-limited source is not “how much per month” but: given my symbol count and my schedule interval, does the arithmetic close? Work that out before choosing, and the answer often flips to the provider with the worse pricing page and the better batch endpoint.

Axis 2 — Coverage means your symbols, at your call time

Providers advertise coverage as a count: 100,000+ tickers, 60+ exchanges. The number is almost useless, because the free-source failure mode is not “symbol not found.” It is a 200 OK with a null field.

I hit this on index quotes: one source returned nothing for a major US index through the path I was using, quietly, while every other symbol in the same call worked. It was not an exception and it was not logged as an error — the field was simply None in the report. The fix was to change source for that field, and the general lesson is in how the fetch is now written:

for symbol, (val_key, chg_key) in symbols.items():
    try:
        info = tickers.tickers[symbol].fast_info
        curr = round(float(info.last_price), 2)
        prev = float(info.previous_close or 0)
        result[val_key] = curr
        if chg_key and prev:                       # no prev close → emit no change, not 0%
            result[chg_key] = f"{(curr - prev) / prev * 100:+.2f}%"
    except Exception as e:
        print(f"[index] {symbol} fetch failed: {e}", file=sys.stderr)

Three things here are the coverage lesson, not Python trivia:

1. The try/except is per symbol, inside the loop. One bad symbol degrades one field instead of nulling the whole report.
2. previous_close or 0 then if prev — a missing previous close produces no change field, never a 0.00%. A fabricated zero is worse than a gap, because downstream code cannot tell it from a flat day.
3. The failure is printed with the symbol. A coverage gap you cannot attribute to a symbol is a coverage gap you will re-discover every month.

The only coverage test that means anything: run your actual watchlist against the source, at the hour you actually call it, and count nulls per field. Pre-open, mid-session and post-close give different answers on the same source — a field that is populated during regular hours can be null for hours after the close, and a daily job that runs at the wrong minute sees a source that “does not cover” symbols it covers fine.

Axis 3 — Latency only matters as slot fit

Per-request p50 latency is the most-published and least-useful number in this space. For the two ways market data is actually consumed, neither cares about it directly.

Scheduled batch. The question is whether the whole fetch fits its slot. If a job runs every 5 minutes and the fetch takes 6, you do not have a slow pipeline, you have overlapping jobs corrupting shared state. The number to measure is wall-clock for the complete fetch including your own throttle sleeps — which, per Axis 1, is often the dominant term. A provider with 40 ms responses and a 16 rps cap is a 300-symbol, 18-second fetch. Its response latency was never the constraint.

Streaming. Latency is the wrong metric entirely; freshness is the right one. A socket can be open, healthy, and delivering nothing — I wrote up why that happens and how to detect it. Derive “live” from the age of the last tick, never from connection state.

An honest admission, since this post is about measuring: my pipeline carries no per-call timing instrumentation. There is no p50 in my logs to show you, which is precisely why the last section of this post is a harness rather than a chart. The logs did record failures, though — and those turned out to say more.

Axis 4 — Failure signature, or: most of your outages are yours

Across the operational logs, connection-level failures against the broker’s API host break down like this:

501  total connection failures against the provider host
483  NameResolutionError  ("Failed to resolve … nodename nor servname provided")
 18  Read timed out
  0  HTTP 429 / rate limit rejections
  0  provider 5xx

96% of what looked like provider unavailability was my own machine failing to resolve DNS — a laptop asleep, a network down, a VPN mid-handshake. A representative line, with the endpoint kept and the rest trimmed:

[error] balance query failed: HTTPSConnectionPool(host='<provider>', port=9443):
Max retries exceeded with url: /oauth2/tokenP
(Caused by NameResolutionError(… Failed to resolve '<provider>' [Errno 8] …))

This changes the response, not just the blame. Retrying harder does nothing for a host that cannot resolve — the fix is on the scheduling and host side (run it somewhere that stays awake, gate the job on connectivity, alert on the gap). Retry logic tuned as if the provider were flaky would have burned attempts on a problem no retry can solve.

It also changes how you compare providers. If you rank sources by “errors in my logs” you will rank your own network. Classify by exception type first: DNS and connect errors are yours, read timeouts are shared, 429 and 5xx are theirs. Only the third bucket is a provider comparison.

Worth noting what is absent from that list: zero rate-limit rejections across the whole corpus. The 0.06 s throttle did its job. A client that respects the documented limit turns the most-feared failure class into a non-event, which is a strong argument for spending your first hour on the throttle rather than on provider selection.

Axis 5 (the one nobody lists) — auth is an operating cost

Pricing pages do not have a row for this, and it is a real difference between sources.

yfinance: no auth, no key, no token. Portal HTML: no auth either, and no contract — the markup changes when someone ships a redesign. A broker API: OAuth token, and token issuance is itself rate-limited, which means the token must be cached with the same care as any other persistent state:

if expire - datetime.now() < timedelta(minutes=5):   # refresh early, don't race expiry
    self._token_cache_path.unlink(missing_ok=True)
    return None
...
with tempfile.NamedTemporaryFile("w", dir=cache_dir, suffix=".tmp", delete=False) as f:
    json.dump({"access_token": token, "expire_at": expire_at.isoformat()}, f)
os.replace(tmp_path, self._token_cache_path)         # atomic; concurrent jobs share it

Three properties, each earned: refresh five minutes before expiry so a long job does not die mid-run; write atomically via os.replace because several scheduled jobs read the same cache file; and delete the cache on any parse failure so a corrupted token file self-heals instead of failing every job until someone notices.

That is maybe forty lines of code and a persistent file, versus import yfinance. It is not a reason to avoid broker APIs — they are the only source for some of this data — but it belongs in the comparison, and it never appears in one.

The scorecard

Qualitative, from operating these, on the axes above:

The row that decides most architectures is failure signature. A source that fails loudly can be retried, alerted on, and fallen back from. A source that fails silently — an open socket with no frames, a None inside a successful response, HTML that still parses into wrong numbers — needs a validator you write yourself, and that validator is usually more code than the fetch. Price that in.

Measure your own set

Here is the harness. It is deliberately provider-agnostic: give it callables, get a CSV of wall-clock and outcome class per call. Run it against your real symbol list, on your real schedule, for a week.

import csv, socket, time
from datetime import datetime, timezone

def classify(exc: Exception) -> str:
    """Whose problem is it? Local, shared, or theirs."""
    s = str(exc)
    if "NameResolution" in s or isinstance(exc, socket.gaierror):
        return "local_dns"
    if "Read timed out" in s or "ReadTimeout" in s:
        return "shared_timeout"
    if "429" in s or "rate limit" in s.lower():
        return "provider_throttle"
    if any(c in s for c in ("500", "502", "503", "504")):
        return "provider_5xx"
    return "other"

def probe(name, fn, expect_fields=()):
    """One call. Records elapsed, outcome, and how many expected fields came back null."""
    t0 = time.perf_counter()
    row = {"ts": datetime.now(timezone.utc).isoformat(), "source": name}
    try:
        result = fn()
        row["outcome"] = "ok"
        row["null_fields"] = sum(
            1 for f in expect_fields if result.get(f) in (None, "", 0)
        )
    except Exception as e:
        row["outcome"] = classify(e)
        row["null_fields"] = len(expect_fields)
    row["elapsed_ms"] = round((time.perf_counter() - t0) * 1000, 1)
    return row

def run(probes, path="probe_log.csv"):
    rows = [probe(*p) for p in probes]
    with open(path, "a", newline="") as f:
        w = csv.DictWriter(f, fieldnames=["ts", "source", "outcome", "null_fields", "elapsed_ms"])
        if f.tell() == 0:
            w.writeheader()
        w.writerows(rows)
    return rows

What to read out of the CSV after a week, in order of how much it will change your mind:

1. null_fields by hour of day. This is the coverage axis, and it is the one that surprises people. A source is not “covered” or “not covered” — it is covered at 14:00 and null at 06:00.
2. outcome grouped into local / shared / provider. If local_dns dominates, as it did for me, stop evaluating providers and go fix your host.
3. Total elapsed for a full cycle, against your slot length. Not p50 — the sum, including throttle sleeps. That single number decides whether the schedule holds.
4. p99, not p50. The slow tail is what overruns the slot; the median never does.

Every one of these is a number about your workload. That is the whole point — a comparison table written by someone with a different symbol set, a different schedule, and a different network is telling you about their constraints, not yours.

What follows

The cluster posts under this one go deeper on individual sources and failure modes. Two are already up: the WebSocket reconnect asymmetry for streaming feeds, and look-ahead bias in multi-timeframe backtests for what happens after the data lands. The rest — free-library gotchas, broker API comparisons, and the cost arithmetic in more detail — get the same treatment: measured on something I actually run, or not published.