STACKAI Math Educator

Problem & context

Problem

Help mathematics educators investigate error patterns, grades and student groups through natural-language questions without allowing a language model to invent or calculate analytical quantities.

Intended users

Mathematics educators and researchers analysing STACK assessment exports with Newman's Error Analysis categories.

Role and contributions

Research architecture, analytical methods, benchmark design and full-stack implementation

Dataset / inputs

The verified benchmark dataset contains 2,820 retained error responses from 615 represented students across six questions, with an educator-supplied complete cohort size of 618.

Methods & architecture

A language model selects a typed analytical function and extracts its arguments. Convex then performs the calculation deterministically, returns structured evidence with explicit population scope, and constrains the model's explanation to computed values. Conversation state supports grounded follow-up questions.

Implementation

1. Upload and validate prepared STACK error-response datasets
2. Visualise grades, error categories and assessment indices
3. Expose deterministic analytics through closed tool schemas
4. Ground every reported rate in an explicit numerator, denominator and population
5. Preserve question, category and student references across follow-up turns
6. Maintain a scored educator-question benchmark with frozen ground truth

Current evidence

The current implementation verifies the project foundation below; broader outcome claims will be added after evaluation.

Limitations & next steps

Synchronise the validated local architecture to GitHub, run educator usability studies and report benchmark performance alongside human evaluation.

Attribution & licence

Apache License 2.0, according to the local repository.