Dongyu Ru

Oct 29, 2024 | amazon.science | Xiangkun Hu |Dongyu Ru |Tianhang Zhang |Zheng Zhang

Large Language Models (LLMs) have shown impressive capabilities but also a concerning tendency to hallucinate. This paper presents REFCHECKER, a framework that introduces claim-triplets to represent claims in LLM responses, aiming to detect fine-grained hallucinations. In REFCHECKER, an extractor generates claim-triplets from a response, which are then evaluated by a checker against a reference.

Mar 27, 2024 | amazon.science | Shuai Zhang |Huijun Yu |Xiangkun Hu |Dongyu Ru

We introduce a prompt pre-training method POMP, which fisrt enables prompt learning on large-scale datasets like ImageNet-21K with over twenty-thousand classes. POMP is memory and computation efficient. Compared with previous methods like CoOp, it achieves comparable accuracy on ImageNet-1K with only 19% GPU memory and 50% training time. POMP achieves new SOTAs on various open-vocabulary visual recognition datasets and tasks.

Jan 25, 2024 | amazon.science | Michael Kearns |Aaron Thomas Roth |Xiangkun Hu |Dongyu Ru

Project DescriptionThis is a test framework for the bias bounties project. Getting Started as a Bounty HunterIf you are interacting with this codebase as a "bounty hunter", you'll need to have a way to run Jupyter notebooks. The easiest way to do this is to download Anaconda, which will also manage all of your python packages for you. See here for installation instructions: https://docs.anaconda.com/anaconda/install/index.html.

Jan 23, 2024 | amazon.science | Yu Liu |Huijun Yu |Xiangkun Hu |Dongyu Ru

This package contains the Hierarchical Bayesian model to predict sample size for online activity. The bang package (https://cran.rstudio.com/web/packages/bang/index.html) was used and accelerated by modifying it to use sufficient statistics, and to only simulate from the posterior over the hyper-parameters. "misc.R", "beta_prior.R", "binom_beta.R", "hef.R" and "set_and_check_prior.R" are source files from bang packages (https://cran.rstudio.com/web/packages/bang/index.html).

Jan 17, 2024 | amazon.science | Xiangkun Hu |Dongyu Ru |Tamer H.M. Soliman

For all their remarkable abilities, large language models (LLMs) have an Achilles heel, which is their tendency to hallucinate, or make assertions that sound plausible but are factually inaccurate. Sometimes, these hallucinations can be quite subtle: an LLM might, for instance, make an assertion that’s mostly accurate but gets a date wrong by just a year or two.