Jelani Harper

2 weeks ago | thenewstack.io | Jelani Harper

Today, data warehouse giant Snowflake announced a significant increase in its support for Apache Iceberg tables. The expanded integration with the open table format lets Snowflake customers access Iceberg data as though it were no different than other data contained in the popular cloud data platforms. As a result, there’s now a host of Snowflake-enabled features that work with Iceberg tables, making the latter more secure, easier to share, and more performant for certain workloads.

3 weeks ago | datasciencecentral.com | Jelani Harper |Vincent Granville |Dan Wilson

The quantum leap forward in natural language technologies attributed to foundation models, LLMs, and modern vocal applications of AI is due in no small part to the mastery of the concept of attention. When training and deploying the aforementioned models, attention mechanisms account for several things. One of the most valuable is allowing models to look back at different parts of a conversation or text and determine how that context relates to present inputs (or lines of text).

1 month ago | thenewstack.io | Jelani Harper

The recently introduced Apache Kafka 4.0 comes with a number of upgrades across nearly every aspect of the open source distributed event streaming platform. The recent release has numerous Kafka Improvement Proposals (KIPs) — which provide new functionality from the open source community — across Kafka Streams, Kafka Connect, and Kafka brokers, consumers, producers, and more.

1 month ago | thenewstack.io | Jelani Harper

A recent research report from analysis firm Trail of Bits highlights some of the key differences — representing critical considerations for contemporary information retrieval — between OpenSearch and Elasticsearch. OpenSearch and the Open Search Project were created by Amazon; OpenSearch’s search and analytics platform was forked from Elasticsearch. The offerings were evaluated with the OpenSearch Benchmark, which compares solutions according to various workloads.

1 month ago | datasciencecentral.com | Jelani Harper |Dan Wilson |Alan Morrison

There’s a really good reason almost every credible vector database—or enterprise application of this technology—incorporates re-ranking models, or re-rankers. It’s not just because these deep neural networks score vector retrieval engines’ results so they’re more useful for search and Retrieval Augmentation Generation (RAG). Quite simply, it’s because for most vector database use cases, no one has actually fine-tuned the embedding model on an organization’s specific, or proprietary, data.