Enhance Log Exports with Cloudflare Logpush Transformers for Privacy and Efficiency

Key takeaways

• Cloudflare's Logpush Transformers let users filter, reshape, and redact log data before export using SQL queries.
• This feature is now generally available on all Cloudflare self-serve plans, simplifying log management.
• Transformers run inside the Logpush pipeline, reducing data volume and exposure of sensitive information.
• Businesses exporting logs to SIEMs or analytics platforms can normalize data and improve privacy compliance.
• Setup and management are done via Cloudflare dashboard or API, with monitoring and version control.

What are Logpush Transformers?

Logpush Transformers is a new feature integrated into Cloudflare's Logpush service that allows users to process log data before it leaves Cloudflare's infrastructure. Instead of exporting raw logs and then filtering or modifying them externally, users can now define SQL queries that filter out unwanted records, redact sensitive fields, and reshape the data structure. This pre-export transformation helps reduce storage costs, enhance privacy, and tailor logs to specific analytics needs. The feature is available on all self-serve plans, including Free, Pro, Business, and Enterprise.

What changed with Cloudflare Logpush Transformers?

Previously, teams exporting Cloudflare logs had to deploy separate processing services to filter and transform logs after export, which added complexity, latency, and cost. Cloudflare’s Transformers embed SQL query execution directly into the Logpush export pipeline, allowing filtering and reshaping to happen in real time before data reaches external destinations.

This shift means:

• Logs can be filtered using SQL WHERE clauses to exclude health checks or internal traffic.
• Sensitive data like client IP addresses can be redacted within the export pipeline.
• Output schemas can be customised by renaming, nesting, or removing fields.
• Data volume sent to SIEMs or analytics platforms is reduced, lowering ingestion and storage costs.

By running transformations in-process, Cloudflare maintains high throughput and reliability without introducing network dependencies or extra failure points.

Who benefits from using Transformers?

Businesses and teams that export Cloudflare logs to security information and event management systems (SIEMs), analytics platforms, or object storage will find Transformers particularly useful. Key beneficiaries include:

• Observability teams managing logs from multiple CDN providers who want to normalize schemas for consistent analysis.
• Organizations concerned about data privacy that need to redact sensitive fields before export.
• Cost-conscious users who want to filter out unnecessary records such as health checks or internal traffic to reduce data volumes.

For example, a company collecting logs from various sources can add a static "provider" field and reshape records to a common format, simplifying downstream analytics.

How do Logpush Transformers work technically?

Transformers execute SQL queries on log batches within the Logpush pipeline using Apache DataFusion and Apache Arrow technologies. The process involves:

1. Batch Processing: Logpush receives compressed batches of logs, decompresses them, and converts them into Apache Arrow RecordBatches — a columnar in-memory format optimized for analytics.

1. SQL Execution: The SQL query defined by the user is applied to each batch using Apache DataFusion, a Rust-based query engine. This allows filtering (WHERE clauses), field selection, renaming, nesting, and redaction.

1. In-Process Execution: Running Transformers inside the Logpush delivery service avoids network calls that could introduce latency or failures.

1. Output Format: Transformed logs are exported in NDJSON format, compressed, and sent to the configured destination.

This architecture supports high throughput (millions of rows per second) and predictable resource use, ensuring reliable log export.

How to set up and manage Logpush Transformers?

Users can create and manage Transformers through the Cloudflare dashboard's Transformer Studio or via API. The typical workflow involves:

1. Selecting the Logpush dataset to transform.
2. Writing a SQL query to filter, redact, or reshape data.
3. Previewing the transformation on sample data to verify results.
4. Saving the Transformer, which creates a versioned record for audit and rollback.
5. Attaching the Transformer to a matching Logpush job.

Cloudflare provides monitoring tools to track transformation success, failures, data volume changes, and performance metrics. If a Transformer fails, Logpush retries the transformation and disables jobs that consistently fail, protecting overall system reliability.

What should businesses do now?

Businesses currently exporting Cloudflare logs should review their log export workflows to identify opportunities for filtering and redaction using Transformers. Practical steps include:

• Removing health check and internal traffic records before export to reduce noise and costs.
• Redacting sensitive fields such as client IP addresses to improve privacy compliance.
• Normalizing log schemas when aggregating data from multiple CDNs or sources.
• Monitoring Transformer health regularly and updating SQL queries as needed to maintain efficiency and compliance.

Using features like Short links, Branded domains, and Analytics alongside Transformers can help businesses build more secure, efficient, and privacy-conscious digital workflows.

Sources

• Filter, reshape, and redact logs before export with Transformers — Cloudflare, 12 October 2026