People often ask me why I call myself the BigData Pilot.It started as a metaphor but over time it became my way of working, thinking and leading projects in the world of data and AI. Checklists over ego In aviation, even a captain with 30,000 flight hours still uses a checklist. Not because they don’t […]
Why I’m the BigData Pilot Weiterlesen »
Ask any AI leader how their last project went and you’ll likely hear about accuracy. F1 scores. AUC. The model “performed well.” Ask them how it’s doing in production, and things get quieter. The truth is uncomfortable but necessary:Most AI doesn’t fail in training. It fails in deployment. Accuracy Is Not a Success Metric In
Most AI Fails in Deployment Weiterlesen »
There’s a disconnect we don’t talk about enough.Data systems have gone real-time.Enterprise planning hasn’t. While Kafka pipes millions of events per second and Flink runs stateful computations in motion, most enterprises are still operating on a roadmap that looks like a spreadsheet and moves like a barge. This isn’t just a pace mismatch—it’s a mindset
Real-Time Data Is Real—Your Enterprise Roadmap Isn’t Weiterlesen »
Why real-time systems aren’t luxury infrastructure—they’re how smart businesses stay ahead. Let’s get one thing out of the way:Batch is fine—for laundry. Not for decisions. Most companies still move data the same way they moved it in 2005: extract, load, wait, analyze, repeat. It’s comfortable. It’s familiar. But it’s also a few hours—or days—behind what’s
Kafka Isn’t Just a Queue. And Flink Isn’t Just a Buzzword. Weiterlesen »
Expectations Have Changed—Permanently We live in a world that expects everything now. People track their parcels obsessively. They refresh flight apps every five minutes to check for gate changes. They want instant payment confirmations, real-time fraud checks, and status updates before they even think to ask. The modern customer is impatient—not because they’re difficult, but
Batch Is Fine—for Laundry. Not for Business Decisions Weiterlesen »
In today’s digital world, real-time data streaming is no longer a luxury—it’s a necessity. Businesses and AI-driven applications thrive on instant insights, requiring robust data pipelines that can process and react to events as they happen. Apache Kafka and Apache Flink form the backbone of modern real-time data processing, enabling seamless data flow, low-latency analytics,
As we have explored in the previous parts of this series, high-quality and real-time data are essential for AI and ML applications. Now, let’s take a deeper look into how to implement real-time data products effectively using Apache Kafka and Apache Flink. This part focuses on two crucial features of Flink that enable reliable and
Implementing Real-Time Data Products with Apache Kafka and Apache Flink (Part 3) Weiterlesen »
Building and maintaining data products for AI and ML is not just about collecting data—it is about ensuring data quality, scalability, and accessibility. Without addressing these challenges, AI models will produce unreliable results, and organizations will struggle to use data effectively. Two of the biggest challenges in this area are data quality and scalability. Ensuring
Challenges in Building and Maintaining Data Products for AI and ML (Part 2) Weiterlesen »
I still remember the days in the early 2010s, when the term „big data“ was widely discussed in the tech industry. Companies were encouraged to collect as much data as possible, seeing it as a key resource (or the new oil as we called it those days) for the digital economy. However, focusing only on
What Are Data Products and Why Do They Matter for AI and ML? (Part 1) Weiterlesen »
Financial markets generate one of the densest streams of real-time data we can observe today. Price ticks, order submissions, cancellations, executions, and settlement instructions all occur at millisecond scale. Within that torrent of activity, regulators and trading firms need to detect suspicious behavior: wash trades, spoofing, layering, or coordinated account activity. The traditional approach—batch analysis
Trade Monitoring and Pattern Matching with Flink and Kafka Weiterlesen »