CDC Data Replication Made Easy

Keboola's Change Data Capture (CDC) solution is designed specifically to address the pain points of traditional database replication techniques. Leveraging advanced CDC technology, Keboola ensures that your database replication is fast, efficient, and hassle-free.

Understanding CDC: The Basics

Change Data Capture is a process that detects changes in your database systems and enables downstream systems to act based on those changes. Unlike traditional database queries that fetch current database states, CDC works by directly reading the transaction logs generated by your databases.

This method provides key advantages:

• No load on your database CPU: CDC uses transaction logs instead of executing resource-intensive queries.
• Captures deletes and updates: Track what has been changed, added, or removed in your database, something traditional extractors cannot easily do.
• No need for special timestamp columns: Unlike typical incremental loading, CDC doesn't require specific timestamp columns like "updated_at".

Why Choose Keboola's CDC Solution?

Keboola's CDC approach addresses several common challenges faced by businesses:

• Ease of Setup: Traditional CDC tools are complex and costly. Keboola offers a simple, managed solution that saves you time and lowers your TCO.
• Near Real-Time Replication: Keboola offers replication every five minutes, ensuring your data analytics stay current.
• Schema Shift Handling: Easily manage schema changes without breaking replication, including adding, renaming, or deleting columns.
• Massive Data Handling: Keboola CDC is capable of replicating up to 1 million rows per minute, making it ideal for large-scale data environments.

Technical Insights into Keboola CDC

Keboola CDC utilizes leading open-source technologies such as Debezium and DuckDB, integrated seamlessly into Keboola's data platform. This combination provides robust, enterprise-grade performance with minimal setup.

Supported Databases

Our CDC solution currently supports popular databases including:

• MySQL
• MariaDB
• PostgreSQL

Additionally, Keboola is positioned to quickly launch support for Oracle, MongoDB, SQL Server, and Cassandra based on customer demand.

Performance Benchmarks

In recent benchmarks, Keboola replicated a 225 million record database, including 20 million changes, in just 22 minutes. This puts Keboola in the top-tier of CDC solutions globally, delivering superior speed at a competitive price.

Setting Up CDC with Keboola

Getting started with CDC in Keboola is straightforward:

1. Enable transaction log generation (binlog for MySQL, WAL for PostgreSQL).
2. Allow Keboola's IP access to your database.
3. Enter database credentials in Keboola and configure your replication settings using our intuitive UI.

Detailed documentation and step-by-step guides are available to help you configure your database quickly and easily.

Advanced Replication Features

Keboola CDC offers advanced options to enhance your replication process:

• Column Masking: Protect sensitive data by masking or hashing columns such as email addresses or personal identifiers.
• Include/Exclude Tables & Columns: Selectively replicate only the data you need.
• Flexible Loading Types: Choose between incremental load, full load, or deduplicated loads to suit your analytical needs.

Incremental vs. Full Load: Which to Choose?

Keboola provides multiple load options:

• Incremental Load (Deduplicated): Maintains a near-real-time mirror of your database, ideal for ongoing analytics.
• Full Load (Deduplicated): Captures only changes since the last run, ideal for event tracking and auditing.

Practical Examples

Imagine running an e-commerce business. Keboola CDC can instantly notify your warehouse and logistics teams whenever a new order is placed or an existing order is updated. This ensures faster order fulfillment and improved customer satisfaction.

Similarly, if you operate in industries requiring regulatory compliance, CDC ensures you maintain an accurate, timestamped log of all database changes for audits and compliance reporting.

Keboola's Commitment to Zero Maintenance

Keboola CDC requires minimal maintenance, as it is fully managed and monitored. Should replication failures occur, Keboola enables quick database restoration and automatic recovery to avoid any data loss or downtime.

Cost-effective Pricing

At a competitive pricing model, Keboola CDC is significantly more affordable than comparable solutions, starting at just $1,300/month for unlimited CDC replication. Volume discounts and flexible pricing options are available for larger deployments.

Conclusion

Whether you're looking to replicate databases for analytics, compliance, or operational workflows, Keboola's CDC solution offers unparalleled ease of use, speed, and reliability. With near-real-time capabilities and comprehensive data handling features, Keboola simplifies your data replication needs, enabling your teams to focus on driving business value.