Large language models (LLMs) now simplify code development by generating functional code from natural language prompts. For Megaladata’s JavaScript component, custom prompt templates enable rapid script...
120 GB from Kafka to ClickHouse in 4 minutes
Megaladata performance test on a real-world ETL workflow
Recently, we conducted a performance test of Megaladata using a realistic ETL workflow – without synthetic benchmarks, simplified datasets, or artificial optimizations.
Test setup
- Data source: Apache Kafka
- Data volume: 120 GB
- Message format: JSON
- Target database: ClickHouse
- Operating system: Linux
- Megaladata Server: 24 CPU cores | 64 GB RAM
- Platform version: Megaladata 7.3.0
Workflow
The workflow is as close to a real production scenario as possible:
- JSON messages consumed from Apache Kafka
- Parsing into a flat table
- Lightweight data transformations
- Statistics calculation
- Loading results into ClickHouse
In other words, a typical stream → transform → analytical storage (ETL) pipeline that many teams run every day.
Result
- Total execution time: 4 minutes 2 seconds
120 GB of data was processed from Kafka to ClickHouse – including parsing, transformations, and calculations.
No Spark cluster. No manual JVM tuning. No complex code.
A video demonstrating the full execution of the test is available below.
Why this matters
ETL platform performance is often tested:
- On small datasets,
- Without real transformations,
- Or in laboratory conditions far from production.
Here, this is a realistic, honest scenario that can be easily transferred to real systems:
- Streaming analytics
- Logging
- Telemetry
- Event processing
- Transactional data streams
- IoT
- Finance
What this shows in practice
- Megaladata comfortably processes tens and hundreds of gigabytes of data
- Suitable not only for analytics, but also for heavy ETL / streaming workloads
- Low-code does not mean “slow”
- The platform scales vertically and uses hardware efficiently
Conclusion
Megaladata is not just a “visual Excel for analysts.” It is a full-featured, high-performance data processing platform that can compete with traditional ETL stacks while remaining simple to use.
If you need to quickly ingest data from Kafka, transform it, calculate metrics, and load it into ClickHouse or another analytical database, this test speaks for itself.
A video of the test execution:
See also
AI code generation for the Javascript component in Megaladata
Megaladata at Plug and Play Armenia Batch 3 Kick-Off Event
On April 17, 2026, Megaladata participated in the International Pre-Acceleration Program in Armenia Kick-Off Event, marking the official launch of Plug and Play Armenia Batch 3.
Model Degradation in Machine Learning
After deploying a machine learning model, its accuracy often declines over time: a phenomenon called model degradation or AI aging. This occurs due to changes in business processes and the data they...
About Megaladata
Megaladata is a low code platform for advanced analytics
A solution for a wide range of business problems that require processing large volumes of data, implementing complex logic, and applying machine learning methods.
GET STARTED!
It's free