Handwritten prescriptions, filed by agents.
My dad is a doctor. When a patient arrives, he checks their previous prescriptions and writes a new one by hand. After every consultation, one assistant packs the medicines while the other manually types the handwritten prescription into an old-school patient software. As the day goes on the chits pile up - and that backlog can take days or even weeks to clear. So we built the fix: take a photo of the prescription, and AI enters it into the patient software the clinic already uses. No new app, no switching software, and every past patient record stays exactly where it has always been.
Minutes
To file a day's backlog (was days)
0
New apps for the clinic
20+ yrs
Patient history retained
100%
Entries human-approved
The hackathon track · main aim
Build a web-based Enterprise Agent that grounds its decisions in documents.
That is the Vultr track brief, and it is explicit about what does not count: a single retrieve-then-answer call is not enough. The track asks for a multi-step workflow where the system plans, retrieves more than once when it needs to, calls tools, makes decisions, and produces an outcome a real enterprise team could actually use - transforming agentic operations in industries like Healthcare, Telecommunications, Finance, and Hospitality. One hard constraint sits underneath all of it: every LLM workload must run on Vultr Serverless Inference.
patient-information-software-ai is our answer: a healthcare enterprise agent that reads real handwritten prescriptions and files them into a real, 20-year-old clinic system. Here is the brief, requirement by requirement, against what the project actually does:
01
“The keyword is agent” - a single retrieve-then-answer call is not enough
Five CrewAI agents run a genuine multi-step workflow per script: an OCR reader extracts the handwriting, an intake agent normalizes it, a records agent verifies identity, a pharmacist grounds every medicine, and a composer assembles the final entry. Each stage reasons over the previous stage's evidence and can change the plan.
02
Plans, and retrieves more than once when it needs to
The Patient Records Agent looks up every plausible RegNo reading against the live clinic database. If no record carries the patient's name, it plans a second retrieval round - digit-confusion variants of the handwriting (this doctor's 2 reads as 9, his 8 as 6) - until a name-consistent record is found, or it honestly reports not-found.
03
Grounds its decisions in documents
Every handwritten medicine line is scored against a homeopathy remedy corpus using VultronRetriever rerank on Vultr. The pharmacist may only canonicalize a remedy the corpus evidence supports - each accepted line carries its corpus citation, and unsupported lines are flagged for a human instead of guessed.
04
Calls tools and makes decisions
The agents call real tools: vision OCR, live PIS patient lookup through the Vultr relay, VultronRetriever rerank, and the clinic submission queue. The composer then decides - ready for entry, or needs review with explicit reasons - and never invents a value that is not in the evidence.
05
An outcome a real enterprise team could actually use
The output is not a chat answer. It is a completed entry inside the clinic's actual 20-year-old Patient Information System - the same Access database and WinForms software the assistants already use, with every past patient record intact. Healthcare is one of the track's named industries.
06
All LLM workloads on Vultr Serverless Inference
Agent reasoning runs on DeepSeek-V4-Flash and document grounding on VultronRetrieverPrime-Qwen3.5-8B, both on Vultr Serverless Inference. The agent backend and the clinic relay API are deployed on a Vultr compute instance. Only perception (reading the photo) uses a vision model - every agentic decision is made on Vultr.
The agent crew · CrewAI on Vultr
Five agents, one script at a time.
Each photographed script runs through the whole crew before the next one starts. Every agent hands structured evidence to the next - OCR alternates, live database records, corpus citations - and the composer can only use what the evidence contains. Perception reads the photo; every agentic decision runs on Vultr Serverless Inference.
Perception
Script OCR Reader
Reads the handwritten chit: RegNo, patient name, shorthand medicine lines, duration, and amount - with alternates for every ambiguous digit, because the RegNo is the one field the clinic cannot afford to get wrong.
GPT-4o Vision
Normalization
Script Intake Agent
Turns the raw reading into structured fields, drops cancelled text the doctor struck through, and never fills a blank with a guess - missing data stays null and gets flagged.
DeepSeek-V4-Flash · Vultr Serverless Inference
Identity · live retrieval
Patient Records Agent
Verifies the patient against the live clinic database in real time. When no record matches the name, it retries digit-confusion RegNo variants until the identity is name-consistent - or reports not-found for human attention.
DeepSeek-V4-Flash · Vultr Serverless Inference
Document grounding
Homeopathy Pharmacist
Grounds every medicine line in the remedy corpus via rerank. Accepts only what the evidence supports, cites the corpus line it used, and flags the rest for review.
VultronRetrieverPrime-Qwen3.5-8B · Vultr rerank
Decision
Entry Composer Agent
Assembles the final PIS entry with confidences and citations from every stage, then makes the call: ready for entry, or needs human review - with the exact reasons listed.
DeepSeek-V4-Flash · Vultr Serverless Inference
From paper to patient record
The full pipeline, end to end.
Run-of-show
photo → agents → human review → one-button import
Photograph the day's scripts
An assistant photographs the handwritten prescriptions - one photo per script, straight from the phone. One upload per batch.
The crew processes each script
OCR, intake, live identity verification, remedy grounding, entry composition. Each script runs through the whole crew before the next begins, and every agent step streams to the screen over SSE.
A human reviews every entry
A swipe deck shows each composed entry next to its original photo. Approve, edit, or skip - nothing moves forward without a person's decision.
One button inside the old software
Approved entries wait in the Vultr queue. Inside PIS, the assistant clicks Get New Data from App and they import into the patient records - backup first, transaction-protected, duplicate-safe.
Why this is an enterprise agent
Not a demo against a fake API.
Most hackathon agents talk to a mock backend. This project integrates with the clinic's actual Patient Information System - a two-decade-old Access database behind a .NET 2.0 WinForms app on Windows 7 - and treats it as the system of record it really is.
Legacy software, untouched workflow
The clinic keeps its .NET 2.0 WinForms software and Access database. We patched one new button into the existing executable - no migration, no retraining, no new app to learn.
A live two-way bridge on Vultr
A relay API on a Vultr compute instance lets the agents query the live patient database in real time and queue approved entries back - even though the clinic machine is a Windows 7 box that cannot speak modern TLS.
Safety as a hard boundary
Database backup before every import, transaction rollback on failure, duplicate-safe job IDs, and one strict rule across all agents: missing data is never invented - it stays null and gets flagged.
The best enterprise agent is invisible: the clinic keeps its software, its data, and its habits - only the backlog disappears.
Try it yourself
Watch the crew read a real script.
Pick one of the doctor's real handwritten prescriptions - or upload your own photo - and watch every agent decision stream live: OCR alternates, live database lookups, corpus citations, and the final composed entry.