Large Data and Performance

XY keeps exact canonical columns in Python and chooses a rendered representation for each trace. The goal is to keep transport and draw work bounded by what the viewport can distinguish while retaining exact rows for readout and refinement.

The current defaults begin M4 line decimation above 10,000 rows and automatic scatter density above 200,000 points. Density grids default to 512×384 cells, and very large density traces can build a lazy multiresolution pyramid for viewport queries. These are pre-1.0 policy thresholds, not API guarantees; write code against the behavior and recorded tier, not a hard-coded count.

Polar traces are an explicit exception to this ladder. They always use the direct tier because Cartesian M4 and density reductions do not preserve cyclic polar geometry. Polar line, scatter, and area traces have a hard 200,000-point ceiling; building a larger point trace raises ValueError instead of silently drawing an unbounded payload or applying an unsuitable reduction. Heatmap and contour grids are not rejected merely because their cell count exceeds that point ceiling. Polar heatmaps inverse-sample at the requested browser/export surface, so output work is pixel-bounded even though source ingest and validation still depend on the source grid.

• density=None chooses automatically and is the normal large-data setting.
• density=True requests a density overview explicitly.
• density=False forces direct points. Above the soft ceiling XY warns about fill-rate and allocation risk, but honors the explicit opt-out.

Density aggregation is computed by the native core and rendered as a compact WebGL texture. Calling it “GPU density” would blur those two stages: the GPU draws the result, while the source-row binning happens before that draw.

Reduction changes visible geometry, not the canonical source store. Hover, selection, pick(), and drilldown resolve source rows when the active tier has an exact mapping. When a dense view cannot support exact per-marker semantics, XY exposes its aggregate/sample representation rather than pretending that a bin is an individual row.

Line decimation preserves extrema for the visible buckets, but it does not claim that every intermediate vertex is drawn. A narrow view is re-decimated against its own window.

There are two different scaling regimes:

1. Ingest, validation, range scans, binning, and decimation inspect source data and therefore retain row-dependent work.
2. The resulting wire payload, WebGL geometry, and static SVG/native-PNG scene are bounded by the chosen viewport representation.

The output side after reduction scales with pixels; the complete data-to-pixels pipeline still includes row-dependent ingest and reduction work.

Tier decisions are also recorded in the built payload rather than being silent. Use the memory report for many-chart dashboards and long-running append workloads, and use the committed benchmark harness before publishing a performance comparison.

See the benchmark snapshot for a measured example, or Interactions and selections for exact readout behavior.