-
2 weeks ago | 
databricks.com | Ali Ghodsi |Reynold Xin |Adam Conway |Daniel Weeks
We are excited to announce the Public Preview for Apache IcebergTM support in Databricks, unlocking the full Apache Iceberg and Delta Lake ecosystems with Unity Catalog. This Preview introduces two new features to Unity Catalog. First, you can now read and write Managed Iceberg tables using Databricks or external Iceberg engines via Unity Catalog’s Iceberg REST Catalog API.
-
Oct 13, 2024 | 
dx.doi.org | Zheng Ma |Jiazhen Chen |Lei Xin |Ali Ghodsi
-
Jul 19, 2024 | 
biorxiv.org | Benyamin Jamialahmadi |Mohammad Kohandel |Ali Ghodsi |Mahmood Chamankhah
AbstractAntibodies, or immunoglobulins, are integral to the immune response, playing a crucial role in recognizing and neutralizing external threats such as pathogens. However, the design of these molecules is complex due to the limited availability of paired structural antibody-antigen data and the intricacies of structurally non-deterministic regions. In this paper, we introduce a novel approach to designing antibodies by integrating structural and sequence information of antigens.
-
Jun 13, 2024 | 
databricks.com | Matei Zaharia |Ali Ghodsi |Reynold Xin |Arsalan Tavakoli-Shiraji
We are excited to announce that we are open sourcing Unity Catalog, the industry’s first open source catalog for data and AI governance across clouds, data formats, and data platforms. Here are the most important pillars of the Unity Catalog vision:Open source API and implementation: It is built on OpenAPI spec and an open source server implementation under Apache 2.0 license. It is also compatible with Apache Hive's metastore API and Apache Iceberg's REST catalog API.
-
Jun 6, 2024 | 
databricks.com | Kobie Crawford |Jonathan Frankle |Ali Ghodsi |Naveen Rao
Businesses are making remarkable progress on their data and AI journeys. They’re advancing from a few pilot projects confined to use cases likely deemed “non-mission-critical,” to deploying applications in real-world operations across thousands of users that are enhancing employee productivity and improving efficiency. However, building and scaling these next-generation systems is a complex, resource-intensive combination of engineering challenges up and down the stack.
-
Jun 4, 2024 | 
databricks.com | Ali Ghodsi |Reynold Xin |Arsalan Tavakoli-Shiraji |Adam Conway
We are excited to announce that we have agreed to acquire Tabular, Inc, a data management company founded by Ryan Blue, Daniel Weeks, and Jason Reid. This acquisition brings the original creators of Apache Iceberg™ and those of Linux Foundation Delta Lake, the two leading open source lakehouse formats, together. As one, we are going to lead the way with data compatibility so that you are no longer limited by which lakehouse format your data is in.
-
May 3, 2024 | 
databricks.com | Josh Howard |Jonathan Frankle |Ali Ghodsi |Naveen Rao
Moving generative AI applications from the proof of concept stage into production requires control, reliability and data governance. Organizations are turning to open source foundation models in search of that control and the ability to better influence outputs by more tightly managing both the models and the data they are trained on. Databricks has assisted thousands of customers in evaluating use cases for generative AI and determining the most appropriate architecture for their organization.
-
Apr 16, 2024 | 
databricks.com | Has Been Prescribed |Jonathan Frankle |Ali Ghodsi |Naveen Rao
IntroductionIn this blog post we dive into inference with DBRX, the open state-of-the-art large language model (LLM) created by Databricks (see Introducing DBRX). We discuss how DBRX was designed from the ground up for both efficient inference and advanced model quality, we summarize how we achieved cutting-edge performance on our platform, and end with some practical tips on how to interact with the model.
-
Mar 27, 2024 | 
databricks.com | Jonathan Frankle |Ali Ghodsi |Naveen Rao |Hanlin Tang
Databricks’ mission is to deliver data intelligence to every enterprise by allowing organizations to understand and use their unique data to build their own AI systems. Today, we are excited to advance our mission by open sourcing DBRX, a general purpose large language model (LLM) built by our Mosaic Research team that outperforms all established open source models on standard benchmarks.
-
Feb 28, 2024 | 
arxiv.org | Ali Ghodsi
