MED-RAG

01 / INSTALLATION & USE

Installation and current use

In October 2025, I installed the system on the personal computer of a professor treating breast cancer at Korea University Anam Hospital. The professor uses it during clinical work and as needed, and I continue to provide remote updates.

The system retrieves cases and guidelines and exposes the sources behind answers. The delivered code and material remain private; published performance metrics come from a separate Vertex port and synthetic-data experiments.

02 / USE & EXPERIMENTS

Personal-PC use and public experiments

Current use and support are reported by the project owner as of Sep 15, 2026. The installation is scoped to the professor’s personal PC, supported through remote updates. Separate public experiments evaluate retrieval, answer quality, and tuning choices.

03 / SYSTEM

Architecture

Cases are retrieved first, then supported by guidelines. The system checks clinical similarity beyond age and sex and surfaces uncertainty when evidence is insufficient. The public Vertex port uses a two-stage research-and-review agent.

1. 01Question

   Case and evidence needs

2. 02Retrieve

   Chroma · cases / guidelines

3. 03Research

   Draft with source evidence

4. 04Review

   Grounding and uncertainty

04 / EVIDENCE

Evaluation

The original judge never saw the retrieved documents. Including them changed the answer pass rate from 24/24 to 20/24 and exposed invented quotations and unsupported claims.

Retrieval and answer quality are measured separately: finding the right document did not guarantee a grounded answer.

05 / FAILURE ANALYSIS

A retrieved source does not guarantee a grounded answer

In q14, the correct source was retrieved, but the judge failed a quotation absent from the evidence it was given. All four failed cases, q06, q14, q22, and q23, retrieved a gold document. A retrieval-recall gate alone would miss these errors.

Conversely, q08 missed the gold source but passed the judge through appropriate abstention. Retrieval misses, unsupported claims, and appropriate abstention therefore need separate outcomes.

1. 01Retrieval check

   Compare gold source filenames

2. 02Evidence to judge

   Include retrieved document text

3. 03Answer check

   Claims · quotes · abstention

4. 04Inspect failures

   Read per-case reasons

06 / ITERATION

QLoRA experiments and model selection

QLoRA training on 154 examples reduced fabricated quotes from two to zero, but the Qwen2.5-7B model began refusing answerable questions.

Eight of 37 abstention examples (about 22%) were mislabeled. Repairing them improved v2, but it still trailed the base model. The 22% denominator is the abstention subset, not the full training set.

This comparison supports recommending the base model with a strict prompt and review stage. It is a public-experiment recommendation, not evidence that the hospital delivery changed models.

• The same 30 validation cases were reused for Base, v1, and v2. These are not results on an untouched final test set after iteration.
• Initial local inference on RTX 5090 and recorded QLoRA training on RTX 4090 are separate experiments.

07 / SERVING LAB

MED-RAG Serving Lab

Public code implements quality gates, pinned model and data versions, streaming latency measurement, and manual rollback procedures. Thirty-one CPU tests and 14 real-loopback rehearsal contracts pass. The evidence explorer shows a good fixture passing and defective candidates and edited summaries blocked before release preparation.

GPU performance and recovery time have not yet been measured. Results will be added after testing in an environment separated from the active RTX 4090 meeting service.

1. 01Version

   Code · model · data

2. 02Evaluate

   Citation errors · over-refusal

3. 03Serve

   Latency · queues · GPU

4. 04Recover

   Previous artifact

ENGINEERING DECISIONS

Engineering decisions