Can Lectora assist with clinical case-based exam answers?

Lectora can draft scores and feedback on written clinical case answers using the supplied rubric and examiner guidance. The published comparison concerns score agreement on long-form written exams; it does not establish diagnostic competence or validate every feedback statement. A qualified examiner reviews the clinical reasoning and approves the grade.

How can a rubric assess differential diagnosis prioritization?

The examiner guidance can specify which alternatives an answer should discuss and how the student should justify their order. Lectora uses that guidance to draft an assessment. The examiner checks whether the student's reasoning and the proposed credit are appropriate; the draft is not clinical authority.

How can Lectora assist with answers about test ordering?

A rubric can ask whether a proposed test is justified by the written case, whether the student explains the sequence, and whether the answer addresses the question. Lectora drafts against those expectations. The examiner checks the clinical interpretation and corrects errors rather than relying on an automatic over-ordering flag.

Can Lectora assist with written history-taking answers?

It can draft an assessment of written answers describing the history a student would take, using the case and examiner checklist. The examiner checks which relevant questions and findings the student addressed. The published score comparison does not establish performance on live OSCE stations.

Does Lectora support Norwegian medical course material?

Norwegian case questions, examiner guidance and relevant course material can inform the draft. The examiner must check clinical and legal references against the supplied guidance and applicable sources. The study does not establish correctness across every Norwegian medical curriculum.

How can ethical and legal reasoning be assessed in a written case?

The rubric can distinguish naming a relevant concept from explaining and applying it to the case. Lectora uses the provided criteria to propose scores and feedback. A qualified examiner checks the interpretation and decides the credit.

PILOT CASE STUDY · MEDICINE

What written medical exams at UiB show about AI-assisted assessment

Q: Has Lectora been evaluated on medical exams?

Yes. On long-form written exams at the Faculty of Medicine, University of Bergen, Lectora versus the course teacher had R² = 0.85, compared with R² = 0.64 between two human graders on the same material. R² describes a fitted linear relationship, not a percentage of correct grades. A new course requires its own evaluation.

At the Faculty of Medicine, University of Bergen, Lectora was run blind on long-form written exams that examiners had already scored. Its draft agreed with the course teacher at R² = 0.85, compared with R² = 0.64 between two human graders on the same material. In a retrospective simulation of targeted manual scoring, where examiners calibrate the model on their own sample and every uncertain case is scored in full, about 70% fewer tasks needed manual scoring at one pass/fail boundary. The simulation did not measure time saved, and a qualified examiner still approves every grade.

About the UiB Medicine validation

The comparison covers long-form written exams at the Faculty of Medicine, University of Bergen, that examiners had already scored. Lectora was run blind against the same papers and rubric, without access to either grader's scores. Its draft agreed with the course teacher at R² = 0.85; two human graders agreed with each other at R² = 0.64 on the same material.

The written clinical questions address areas such as differential diagnosis, history-taking, treatment-plan justification, test ordering and ethical or legal reasoning. The rubric and examiner guidance establish what each answer should demonstrate. The study measures score agreement; it does not separately validate every type of clinical reasoning or the quality of the generated feedback.

Course codes, exam identities and candidate-level data are not published. The separate medical assessment example is a fictional teaching case, not an extract from this dataset.

How does targeted manual scoring keep an examiner in the loop?

Scoring every free-text answer in full is the most expensive part of a written exam. Targeted manual scoring keeps a human assessment for every candidate, but concentrates detailed scoring where the pass/fail decision needs it.

Lectora first drafts a score for each answer. Examiners then score a small sample without seeing the drafts, which shows how Lectora's scores relate to the marking standard of that particular exam. Submissions whose pass/fail outcome is in doubt go to full manual scoring. Examiners review the remaining drafts, and a qualified examiner approves every grade before it is published.

HUMAN IN THE LOOP · HOW THE SCORING WORK IS ALLOCATED

Lectora
Draft every submission
Lectora drafts a score for each answer from the rubric and the examiner guidance.
Examiners
Calibrate on a human sample
Examiners score a small sample without seeing the drafts, and Lectora's scores are aligned to their marking standard for this exam.
Examiners
Score the uncertain cases in full
Every submission whose pass/fail outcome is in doubt goes to full manual scoring.
Examiners
Approve every grade
Examiners review the remaining drafts, and a qualified examiner approves every grade before it is published.

In a retrospective simulation on earlier medical exams at the University of Bergen, this allocation needed about 70% fewer manually scored tasks at one pass/fail boundary. These are task counts, not measured time.

The calibration is local: it comes from the examiners' own scores on that exam, not from a general model setting. The detailed calibration and selection rules are shared with participating institutions rather than published, while the workflow is verified prospectively in pilots.

What did the retrospective simulation find?

Re-run on earlier exams that examiners had already scored in full, the method needed about 70% fewer manually scored tasks than a full manual pass at one pass/fail boundary. The saving varied between exams: smaller cohorts, and exams with many candidates close to the boundary, saved less.

These are simulated task counts, not minutes. They do not measure the time needed to read, review or approve an assessment, and they apply to one pass/fail boundary rather than a letter-grade scale. Within the simulation, candidates outside the full-scoring group used calibrated estimates; that modelling choice is not a claim that they can skip review in practice.

How can Lectora support assessment of written clinical reasoning?

The examiner guidance can distinguish the student's conclusion from the reasoning offered for it. Differential diagnosis, history-taking, treatment-plan justification, test ordering and ethical or legal reasoning can each contribute to an assessment where the rubric specifies them. Lectora drafts scores and written explanations from the supplied material; the examiner judges whether those proposals match the answer and the course's standards.

For differential diagnosis, an answer might need to compare alternatives and justify a priority using the case information. The examiner determines which alternatives are acceptable and whether an unusual but well-supported argument earns credit. The AI draft can help organize that review, but it does not establish the clinical validity of an answer.

For a written history-taking question, the guidance can specify which questions, observations and missing information the student should address. An examiner can use the draft to examine omissions and the student's explanation of relevance. This does not extend the published evidence to live clinical performance, patient care or OSCE stations.

Ethical and legal questions can distinguish naming a principle from applying it to the given facts. The supplied rubric governs the proposed credit. Clinical and legal references still need examiner checking; the model is not an independent authority on the applicable rules.

What does the comparison with human graders establish?

On long-form written exams at the Faculty of Medicine, Lectora versus the course teacher had R² = 0.85, compared with R² = 0.64 between two human graders on the same material. That is a stronger fitted linear relationship in this comparison. R² alone does not measure the size of individual score differences, prove that a grade is correct or establish a general advantage over human graders.

The scope matters when considering another course. The medical setting makes the comparison relevant to written clinical assessment, but it does not supply one accuracy figure for all clinical curricula, feedback quality or assessment formats. A new evaluation should use the institution's own questions, rubric and reference scores, with explicit checks at its decision boundaries.

See the validation methodology for what the score comparison measures and where its limits are.

Lectora drafts a score for every submission. Examiners score a small sample without seeing the drafts, and Lectora's scores are aligned to their marking standard for that exam. Every submission whose pass/fail outcome is in doubt goes to full manual scoring, and examiners review the remaining drafts. A qualified examiner approves every grade. The detailed calibration and selection rules are shared with participating institutions rather than published, while the workflow is verified in pilots.

Request access →