Research

The research behind Jielewe

With AI tools, students can produce working code without understanding it, so traditional programming tests say less than they used to. Jielewe started as a master's thesis on assessing understanding instead: what you know, how you reason, and how sure you are.

Master's thesis · August 2026

Redefining Evaluation Using Adaptive AI-Generated Parsons Problems

Mercy Jerop Tum · Master of Science (MS) in Software Engineering, University of Michigan-Dearborn

Advisor: Professor Bruce Maxim · Committee: Professor DeLean Tolbert Smith, Professor Zhiwei Xu

DOI: 10.7302/dspace/30871View on Google Scholar

How it works

Question generation
GPT-4.1, checked against a JSON schema and rule-based quality checks
Question format
Four-tier diagnostic test: answer, confidence, reasoning, confidence
Scoring
Modified Elo rating mapped to a CEFR scale from A1 to C2
Study
Quasi-experimental, mixed-method; 8 students and 4 expert reviewers

What the study found

  • Jielewe picked up meaningful patterns in what students understood, beyond whether their code was right.
  • Its CEFR levels ran higher than the expert reviewers' judgements, so the level thresholds needed calibrating.
  • Misconceptions needed more weight in the score, and a larger, more balanced dataset is needed for full validation.

Cite this work

APA
Tum, M. J. (2026). Redefining evaluation using adaptive AI-generated Parsons problems [Master's thesis, University of Michigan-Dearborn]. Deep Blue Documents. https://doi.org/10.7302/dspace/30871
BibTeX
@mastersthesis{tum2026redefining,
  author = {Tum, Mercy Jerop},
  title  = {Redefining Evaluation Using Adaptive {AI}-Generated {Parsons} Problems},
  school = {University of Michigan-Dearborn},
  year   = {2026},
  doi    = {10.7302/dspace/30871},
  url    = {https://hdl.handle.net/2027.42/204753}
}