Mikkel Dengsøe

2 months ago | towardsdatascience.com | Niklas Lang |Roger Noble |Mikkel Dengsøe |Rohan Paithankar

As we have already seen with the basic components (Part 1, Part 2), the Hadoop ecosystem is constantly evolving and being optimized for new applications. As a result, various tools and technologies have developed over time that make Hadoop more powerful and even more widely applicable. As a result, it goes beyond the pure HDFS & MapReduce platform and offers, for example, SQL, as well as NoSQL queries or real-time streaming.

2 months ago | towardsdatascience.com | Niklas Lang |Roger Noble |Mikkel Dengsøe |Rohan Paithankar

Nowadays, a large amount of data is collected on the internet, which is why companies are faced with the challenge of being able to store, process, and analyze these volumes efficiently. Hadoop is an open-source framework from the Apache Software Foundation and has become one of the leading Big Data management technologies in recent years. The system enables the distributed storage and processing of data across multiple servers.

Feb 13, 2025 | towardsdatascience.com | Richie Bachala |Roger Noble |Mikkel Dengsøe

As data continues to grow in importance and become more complex, the need for skilled data engineers has never been greater. But what is data engineering, and why is it so important?

Feb 11, 2025 | towardsdatascience.com | Eirik Berge |Roger Noble |Mikkel Dengsøe

Introduction — Purpose and ReasonsDatasets, Tasks, and SettingsResultsConclusionsWrapping UpSpeed is important when dealing with large amounts of data. If you are handling data in a cloud data warehouse or similar, then the speed of execution for your data ingestion and processing affects the following:Cloud costs: This is probably the biggest factor. More compute time equals more costs in most billing models.

Jan 31, 2025 | towardsdatascience.com | Bernd Wessely |Roger Noble |Bennett Meares |Mikkel Dengsøe

Now that we understand the business requirements, we need to check if the current Data Architecture supports them. If you’re wondering what to assess in our data architecture and what the current setup looks like, check the business case description.