NoteAid-AQG: project overview

Draft · November 17, 2026

The problem

If a chatbot or a clinician explains a clinical note to a patient, how do we know the patient understood it? Generic health-literacy tests measure general reading skill, not whether this patient knows what their own note says about their medications, their self-care, or the warning signs to watch for. Writing a personal quiz for every patient by hand is not realistic.

What I am trying to solve

I want to generate, automatically, a short multiple-choice quiz from each patient's own note that checks two things: whether the patient can find and state the facts the note gives (for example, the dose of a new medication), and whether they can put the note's medical language into plain words. The questions must be correct, fair, and safe for patients to answer.

How

  • What to ask about. Questions cover the topics a patient needs most: medications, self-care to do, things to avoid, and warning signs, plus surgery and follow-up when the note has them.
  • Note-grounded questions. A language-model pipeline first selects the relevant parts of the note, then writes each question from them. The patient answers with the note in front of them.
  • Wrong answers you can check. Every wrong option is built from the same note, so the note itself proves it wrong. A common type is the incomplete answer, which mirrors the real mistake of following only part of the instructions.
  • Feedback and review. Patients see the correct answer right away, so a quiz never leaves them with a wrong idea. Question quality is checked by an automatic judge, then by physicians, then with patients.