Choose columns and compress blob storage exports
Niklas Semmler, PhDPick which field groups are written to each row and enable gzip compression in scheduled S3, GCS, and Azure exports. Shrink files and drop fields you don't want to land in your warehouse.
Two new controls let you reduce the size of your scheduled blob storage exports: column selection and gzip compression. Both are configurable per integration in Project Settings → Integrations → Blob Storage.
Column selection. Eleven groups cover the enriched observations row — toggle off the ones you don't need:
- Drop
metadatafor privacy. Keep user data out of your warehouse without filtering downstream. - Drop
ioto shrink files. Inputs and outputs are usually the largest columns; deselecting them produces dramatically smaller exports for cost or latency analytics. - Drop
toolsandpromptwhen your downstream consumer only needs traces, timings, and cost.
The core group (id, trace_id, start_time, end_time, project_id, parent_observation_id, type) is required and always exported. The other ten groups — basic, time, io, metadata, model, usage, prompt, metrics, tools, trace_context — are individually toggleable. Existing integrations continue to export all groups; no action needed unless you want to narrow the schema.
Field groups are only available with the new enriched observations export, which is based on the denormalized v4 data model. The legacy export source uses a fixed column set and is unaffected.
Gzip compression. New integrations compress exported files with gzip by default, producing .csv.gz, .json.gz, or .jsonl.gz files readable by most data warehouses and pipeline tools without a manual decompression step. Toggle Gzip Compression off if your consumer expects plain files. Existing integrations are unaffected — they were backfilled as uncompressed so nothing breaks for current consumers.
Both controls are available on the REST API: exportFieldGroups and compressed on GET/PUT /api/public/integrations/blob-storage.