2023-2024 Winner

Handwritten Math Benchmark for LLMs

An open source visual benchmark dataset designed to assess LLM's understanding of handwritten student work in math

United States of America
Track:
Learning Science Research

Prize Level:

Catalyst Prize

Project Description

Teaching Lab will develop an open source benchmark dataset made up of images of handwritten student math work, with the goal of improving the ability of multi-modal LLMs to understand and interact with handwritten student work—making it easy to measure the performance of leading models on highly relevant student math work. This project is a collaboration between Teaching Lab, ASSISTments and the Allen Institute for AI.

Meet Our Team

Ryan Knight

Head of Data & AI Practice, Insource Services, Inc.

Alice Ng

Consultant, Teaching Lab

Kyle Lo

Lead Scientist, Allen Institute for AI

Lucy Li

PhD Candidate, University of California, Berkeley

Sami Baral

Phd Candidate, Worcester Polytechnic Institute

Phase II of the 2025 Tools Competition is open. Proposals are due Jan 16, 2025.