[AI Seminar Series] REMINDER: Seminar by Prof. Xiaoqian Liu, tomorrow Friday October 30th, 12-1pm, MRB Seminar Room
Vassilis Tsotras
vassilis.tsotras at ucr.edu
Thu Oct 29 09:34:59 PDT 2026
Please see below; register with the link if you plan to attend.
V. Tsotras
------------------------------------
On Sun, Oct 25, 2026 at 7:11 PM Vassilis Tsotras <vassilis.tsotras at ucr.edu>
wrote:
> The next AI Seminar will be on Friday October 30th, 12-1pm, in the MRB
> Seminar Room (1st floor).
>
> *** Pizza and refreshments will be provided ****
>
> To keep track of the number of attendees, please *register* at:
> https://www.eventbrite.com/e/ai-seminar-series-tickets-2003899932642
>
> The talk will be given by *Prof. Xiaoqian Liu*, Department of Statistics,
> UCR
>
> TITLE: Beyond Average Accuracy: Interpretable Evaluation of Large
> Language Models at Scale
>
> ABSTRACT:
> The rapid proliferation of large language models (LLMs) has created an
> increasing demand for reliable and efficient evaluation methods. Yet
> conventional approaches typically rank LLMs based on average accuracy,
> treating every question as equally informative while revealing little about
> the benchmarks themselves. Item response theory (IRT), a statistical
> framework long used in psychometrics, offers a principled alternative: it
> models each response as a random outcome governed by the model’s latent
> ability and the question’s difficulty and discrimination. However, standard
> IRT methods can be computationally expensive and numerically unstable at
> the scale of modern LLM leaderboards. To address these challenges, we
> develop an interpretable and scalable framework for LLM evaluation based on
> the majorization–minimization principle. Our approach recasts IRT
> estimation as a sequence of constrained matrix factorization problems that
> admit simple least-squares updates, with theoretical guarantees for
> identifiability and convergence. Experiments on the Hugging Face Open LLM
> Leaderboard, involving over 2,000 models and more than 21,000 questions
> across six benchmarks, demonstrate 41–86× speedups over existing methods.
> The estimated model abilities are consistent with established scaling laws,
> and the estimated question difficulties agree with human annotations. Our
> results also reveal that accuracy-based rankings can be misleading when
> benchmark questions differ substantially in their ability to distinguish
> models. Finally, I will briefly discuss our ongoing work on reducing
> redundancy in benchmarks, paving the way toward more efficient and reliable
> LLM evaluation.
>
> Bio:
> Xiaoqian Liu is an Assistant Professor in the Department of Statistics at
> UC Riverside. She received her Ph.D. in Statistics from NC State University
> in 2022 and was a postdoctoral fellow at the UT MD Anderson Cancer Center
> from 2022 to 2024. Her research lies at the intersection of statistical
> machine learning, computational statistics, and numerical optimization,
> with applications in artificial intelligence, bioinformatics, and cancer
> biology. Her recent research focuses on developing interpretable and
> scalable methods for evaluating large language models, as well as robust
> methods for transferring knowledge across heterogeneous data sources.
>
> ------------------------------------
> Sponsored by the RAISE at UCR Institute, the AI Seminar Series presents
> speakers working on cutting edge Foundational AI or applying AI in their
> research. The goal of these seminars is to inform the UCR community about
> current trends in AI research and promote collaborations between faculty
> in this emerging field. These seminars are open to interested faculty and
> graduate/undergraduate students. Please forward this email to other
> colleagues or students in your lab that may be interested. After the
> seminar a discussion will follow for questions, open problems, ideas for
> possible collaborations etc.
>
> Sincerely,
> Vassilis Tsotras
> Professor, CSE Department
> co-Director, RAISE at UCR Institute
>
> Amit Roy-Chowdhury
> Professor, ECE Department
> co-Director, RAISE at UCR Institute
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ucr.edu/pipermail/raise-seminar/attachments/20261029/1f5e86b6/attachment.htm>
More information about the raise-seminar
mailing list