JSON streaming explained
October 18, 2026 · 13 min read
JSON streaming means processing data incrementally instead of loading one massive value into memory. For log pipelines, ETL, and large exports, streaming turns an O(file size) memory spike into bounded buffers. It is not a different JSON spec - it is a consumption pattern plus formats like NDJSON that line up with streams.
Why stream instead of parse all at once
A 3 GB JSON array of objects cannot fit in a typical laptop’s heap for JSON.parse. Streaming emits or consumes one record at a time, enabling backpressure, early filtering, and parallel workers per line.
NDJSON (JSON Lines)
Each line is a complete JSON value - usually an object. Unix tools (grep, wc -l) work line by line. Producers append; consumers tail. Validate a sample line with the JSON validator before processing millions of lines in production.
{"id":"a1","level":"info","msg":"start"}
{"id":"a2","level":"error","msg":"timeout"}Streaming a JSON array
Some servers write [ then objects separated by commas without wrapping the full array in memory. Clients use event-based parsers (stream-json, ijson, JSONDecoder.raw_decode loops) to read one element at a time. Standard pretty-printed arrays are not stream-friendly until split or re-exported as NDJSON.
Tools for developers
For small excerpts, format one object at a time in the JSON formatter. Compare before/after pipeline transforms with JSON compare on representative lines, not whole files.
API and export design
Prefer cursor pagination for public REST JSON. Offer application/x-ndjson or gzipped NDJSON for bulk export jobs. Document that clients must not assume a single JSON value for multi-million-row downloads.
FAQ
- Is NDJSON valid JSON?
- The file is not one JSON value; each line is valid JSON. Parsers are line-oriented.
- Can fetch stream JSON in browsers?
- Yes with ReadableStream and parsers; many apps should paginate instead for simplicity.
- JSON streaming vs WebSockets?
- WebSockets deliver messages over time; NDJSON is file/log oriented. Both avoid one giant parse.
Related: Large JSON file debugging · Large JSON responses