Ritendra Datta

Oct 2, 2024 | databricks.com | Linqing Liu |Matthew Hayes |Ritendra Datta |Matei Zaharia

IntroductionApplying Large Language Models (LLMs) for code generation is becoming increasingly prevalent, as it helps you code faster and smarter. A primary concern with LLM-generated code is its correctness. Most open-source coding benchmarks are designed to evaluate general coding skills. But, in enterprise environments, the LLMs must be capable not only of general programming but also of utilizing domain-specific libraries and tools, such as MLflow and Spark SQL.

Jul 24, 2024 | databricks.com | Yi Liu |Matei Zaharia |Ritendra Datta |Justin Kim

Evaluating long-form LLM outputs quickly and accurately is critical for rapid AI development. As a result, many developers wish to deploy LLM-as-judge methods that work without human ratings. However, common LLM-as-a-judge methods still have major limitations, especially in tasks requiring specialized domain knowledge. For example, coding on Databricks requires understanding APIs that are not well-represented in the LLMs’ training data.