Adam Bellemare

Featured in: Favicon

medium.com Favicon

flipboard.com Favicon

infoworld.com Favicon

dzone.com

thenewstack.io Favicon

confluent.io

Articles

The End of the Bronze Age: Rethinking the Medallion Architecture

2 months ago | infoq.com | Adam Bellemare |Thomas Betts

Write for InfoQ Feed your curiosity. Help 550k+ global senior developers each month stay ahead.Get in touch Operational and analytical use cases are not able to access relevant, complete, and trustworthy data reliably.
How Apache Iceberg Won the Open Table Wars

Dec 3, 2024 | bigdatawire.com | Alex Woodie |Adam Bellemare

Apache Iceberg has recently emerged as the de facto open-table standard for large-scale datasets, with a thriving community and support from many of the leading data infrastructure vendors. But why did Iceberg emerge as the preferred format? And what should you know before you wade in? Iceberg is a high-performance table format that brings the reliability and simplicity of SQL tables to large-scale data analytics.
How to Design Event Streams, Part 2

Dec 2, 2024 | dzone.com | Adam Bellemare

In Part 1, we covered several key topics. I recommend you read it, as this next part builds on it. As a quick review, in part 1, we considered our data from the grand perspective and differentiated between data on the inside and data on the outside. We also discussed schemas and data contracts and how they provide the means to negotiate, change, and evolve our streams over time. Finally, we covered Fact (State) and Delta event types.
How to Design Event Streams, Part 1

Oct 28, 2024 | dzone.com | Adam Bellemare

Event streaming is becoming increasingly common in the world today. An event is a single piece of data that describes, as a snapshot in time, something important that happened in your business. We record that data to an event stream (typically using an Apache Kafka topic), which provides the basis for other applications and business processes to respond and react accordingly — also known as event-driven architecture (EDA). Event-driven architectures (EDA) rely extensively on events.
Headless Data Architecture, Part 2: How to Build It

Oct 25, 2024 | confluent.io | Adam Bellemare

The headless data architecture is the formalization of a data access layer at the center of your organization. Encompassing both streams and tables, it provides consistent data access for both operational and analytical use cases. Streams provide low-latency capabilities to enable timely reactions to events, while tables provide higher-latency but extremely batch-efficient querying capabilities. You simply choose the most relevant processing head for your requirements and plug it into the data.
Headless Data Architecture, Explained for Developers

Oct 17, 2024 | confluent.io | Adam Bellemare

The headless data architecture is an organic emergence of the separation of data storage, management, optimization, and access from the services that write, process, and query it. With this architecture, you can manage your data from a single logical location, including permissions, schema evolution, and table optimizations.
Preventing and Fixing Bad Data in Event Streams, Part 2

Oct 11, 2024 | confluent.io | Adam Bellemare

Alright, I’m back. Time for part 2. In the first part, I covered how we handle bad data in batch processing. In particular, cutting out the bad data, replacing it, and running it again. But this strategy doesn’t work for immutable event streams as they are, well, immutable. You can’t cut out and replace bad data like you would in batch processed data sets.
How to Handle Bad Data in Event Streams

Oct 8, 2024 | thenewstack.io | Adam Bellemare

A recent Gartner survey found that poor data quality costs organizations an average of $12.9 million annually and can increase the complexity of data ecosystems. “Bad data” is defined as corrupted or malformed data that doesn’t conform to developers’ expectations. It can create outages and other corrupting effects for data scientists, analysts, machine learning, AI and other data practitioners. Apache Kafka topics are immutable.
Preventing and Fixing Bad Data in Event Streams

Oct 4, 2024 | confluent.io | Adam Bellemare

At a high level, bad data is data that doesn’t conform to what is expected. For example, an email address without the “@”, or a credit card expiry where the MM/YY format is swapped to YY/MM. “Bad” can also include malformed and corrupted data, such that it’s completely indecipherable and effectively garbage. In any case, bad data can cause serious issues and outages for all downstream data users, such as data analysts, scientists, engineers, and ML & AI practitioners.
Preventing, Fixing Bad Data in Event Streams: Pt 2

Aug 21, 2024 | dzone.com | Adam Bellemare

Alright, I’m back — time for part 2. In the first part, I covered how we handle bad data in batch processing; in particular, cutting out the bad data, replacing it, and running it again. But this strategy doesn’t work for immutable event streams as they are, well, immutable. You can’t cut out and replace bad data like you would in batch-processed data sets.