Roger Noble

2 months ago | towardsdatascience.com | Niklas Lang |Roger Noble |Mikkel Dengsøe |Rohan Paithankar

As we have already seen with the basic components (Part 1, Part 2), the Hadoop ecosystem is constantly evolving and being optimized for new applications. As a result, various tools and technologies have developed over time that make Hadoop more powerful and even more widely applicable. As a result, it goes beyond the pure HDFS & MapReduce platform and offers, for example, SQL, as well as NoSQL queries or real-time streaming.

2 months ago | towardsdatascience.com | Niklas Lang |Roger Noble |Mikkel Dengsøe |Rohan Paithankar

Nowadays, a large amount of data is collected on the internet, which is why companies are faced with the challenge of being able to store, process, and analyze these volumes efficiently. Hadoop is an open-source framework from the Apache Software Foundation and has become one of the leading Big Data management technologies in recent years. The system enables the distributed storage and processing of data across multiple servers.

2 months ago | towardsdatascience.com | Mateus Trentz |Piero Paialunga |Roger Noble

Privacy Preference CenterWhen you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies.

2 months ago | towardsdatascience.com | Lee Vaughan |Roger Noble |Dhruv Matani

If you’re an Anaconda user, you know that conda environments help you manage package dependencies, avoid compatibility conflicts, and share your projects with others. Unfortunately, they can also take over your computer’s hard drive. I write lots of computer tutorials and to keep them organized, each has a dedicated folder structure complete with a Conda Environment. This worked great at first, but soon my computer’s performance degraded, and I noticed that my SSD was filling up.

2 months ago | towardsdatascience.com | Sarah Lea |Shreya Rao |Roger Noble

“I train models, analyze data and create dashboards — why should I care about Containers?”Many people who are new to the world of data science ask themselves this question. But imagine you have trained a model that runs perfectly on your laptop. However, error messages keep popping up in the cloud when others access it — for example because they are using different library versions.