Stream MongoDB to
BigQuery

Real-time CDC streaming from MongoDB to BigQuery for analytics. Automatic schema mapping, nested JSON handling, and incremental sync.

Start streaming free →View docs

Real-Time CDC Streaming

Capture every insert, update, and delete from the MongoDB oplog and stream to BigQuery in real time. No batch windows — data arrives as it changes in production.

Sub-second latency

Nested JSON → BigQuery STRUCT

MongoDB nested documents and arrays are automatically mapped to BigQuery STRUCT and ARRAY types. Maintain nested data structures for rich analytical querying.

STRUCT & ARRAY

Schema Evolution

When your MongoDB documents gain new fields, BigQuery tables adapt automatically. Add nullable columns or use BigQuery's flexible JSON type for schemaless fields.

Auto-evolve schema

Partitioned Tables

Tables are automatically partitioned by event time or ingestion time. Partition pruning keeps your queries fast and your storage costs low. Configurable partition granularity.

Time-partitioned

Incremental & Full Load

Run a full historical backfill to seed your BigQuery tables, then maintain freshness with CDC. No need to pause writes or take your app offline during initial load.

Backfill + streaming

Service Account Auth

Connect to BigQuery via Google Cloud service account. IAM-based access control ensures only authorized jobs write to your datasets. No static credentials stored.

IAM auth

BSON to BigQuery type mapping

ObjectIdSTRINGHex-encoded 24-character string

StringSTRINGUnlimited-length Unicode string

NumberIntINT64BigQuery uses 64-bit integers natively

DoubleFLOAT64IEEE 754 double precision

ISODate / DateTIMESTAMPUTC timestamp with microsecond precision

BooleanBOOLNative boolean type

ArrayARRAY<T> / JSONTyped arrays of primitives or flexible JSON arrays

Object / nested docSTRUCT / JSONNested documents as typed STRUCT or flexible JSON

Binary / BinDataBYTESBase64-encoded binary data

NullNULLABLEAll BigQuery columns are nullable by default

Common questions

How does CDC work with BigQuery?

NoSQLSync tails the MongoDB oplog and streams changes to BigQuery using the Storage Write API for low-latency, high-throughput inserts. Changes are committed within seconds.

Can I query BigQuery as data streams in?

Yes. BigQuery's streaming buffer makes data available for querying immediately. Keep in mind that the streaming buffer has a short delay (typically 90 seconds) before data appears in standard tables.

How do you handle schema changes?

Our schema evolution engine detects new or changed fields and adds nullable columns to the BigQuery table automatically. You can opt for strict mode (reject unknown fields) or flexible mode (add columns).

What about BigQuery costs?

We optimize for BigQuery's pricing model: batch writes use load jobs (free), CDC uses the Storage Write API. We batch small changes to minimize API calls and provide partition filtering to reduce query costs.

Can I choose specific datasets and tables?

Yes. You specify the target BigQuery dataset during setup, and each MongoDB collection maps to a table within that dataset. You can customize table names and add _synced_at metadata columns.

Does this work with MongoDB Atlas?

Yes. Atlas clusters expose the oplog through the MongoDB driver, which NoSQLSync uses for CDC. Atlas Data Lake is also supported as a source for one-time backfills.