[AI Seminar Series] Seminar by Prof. Xiaoqian Liu, Friday October 30th, 12-1pm, MRB Seminar Room

[Vassilis Tsotras]

The next AI Seminar will be on Friday October 30th, 12-1pm, in the MRB
Seminar Room (1st floor).

*** Pizza and refreshments will be provided ****

To keep track of the number of attendees, please *register* at:
https://www.eventbrite.com/e/ai-seminar-series-tickets-2003899932642

The talk will be given by *Prof. Xiaoqian Liu*, Department of Statistics,
UCR

TITLE: Beyond Average Accuracy: Interpretable Evaluation of Large Language
Models at Scale

ABSTRACT:
The rapid proliferation of large language models (LLMs) has created an
increasing demand for reliable and efficient evaluation methods. Yet
conventional approaches typically rank LLMs based on average accuracy,
treating every question as equally informative while revealing little about
the benchmarks themselves. Item response theory (IRT), a statistical
framework long used in psychometrics, offers a principled alternative: it
models each response as a random outcome governed by the model’s latent
ability and the question’s difficulty and discrimination. However, standard
IRT methods can be computationally expensive and numerically unstable at
the scale of modern LLM leaderboards. To address these challenges, we
develop an interpretable and scalable framework for LLM evaluation based on
the majorization–minimization principle. Our approach recasts IRT
estimation as a sequence of constrained matrix factorization problems that
admit simple least-squares updates, with theoretical guarantees for
identifiability and convergence. Experiments on the Hugging Face Open LLM
Leaderboard, involving over 2,000 models and more than 21,000 questions
across six benchmarks, demonstrate 41–86× speedups over existing methods.
The estimated model abilities are consistent with established scaling laws,
and the estimated question difficulties agree with human annotations. Our
results also reveal that accuracy-based rankings can be misleading when
benchmark questions differ substantially in their ability to distinguish
models. Finally, I will briefly discuss our ongoing work on reducing
redundancy in benchmarks, paving the way toward more efficient and reliable
LLM evaluation.

Bio:
Xiaoqian Liu is an Assistant Professor in the Department of Statistics at
UC Riverside. She received her Ph.D. in Statistics from NC State University
in 2022 and was a postdoctoral fellow at the UT MD Anderson Cancer Center
from 2022 to 2024. Her research lies at the intersection of statistical
machine learning, computational statistics, and numerical optimization,
with applications in artificial intelligence, bioinformatics, and cancer
biology. Her recent research focuses on developing interpretable and
scalable methods for evaluating large language models, as well as robust
methods for transferring knowledge across heterogeneous data sources.

------------------------------------
Sponsored by the RAISE at UCR Institute, the AI Seminar Series presents
speakers working on cutting edge Foundational AI or applying AI in their
research. The goal of these seminars is to inform the UCR community about
current trends in AI research and promote collaborations between faculty in
this emerging field. These seminars are open to interested faculty and
graduate/undergraduate students. Please forward this email to other
colleagues or students in your lab that may be interested. After the seminar a
discussion will follow for questions, open problems, ideas for possible
collaborations etc.

Sincerely,
Vassilis Tsotras
Professor, CSE Department
co-Director, RAISE at UCR Institute

Amit Roy-Chowdhury
Professor, ECE Department
co-Director, RAISE at UCR Institute
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ucr.edu/pipermail/raise-seminar/attachments/20261025/a3f54e86/attachment.htm>