Vibe-coded Prototypes

Problem: Most agent-evaluation tools answer “did v1.9 beat v1.8?” — not the question a non-technical population-health manager actually has: “Can I let this agent act on my members on its own — and if not, for whom and how far?” Scores don't translate into a defensible deployment decision.

Approach: I designed evaluation as trust infrastructure, not a reporting screen. Verdicts are expressed in what the agent is allowed to do (an Observe → Assisted → Partially → Fully Autonomous ladder), not F1 scores. Checks are layered (deterministic first, LLM-judge second) under one mental model, and everything is cohort-sliced so sub-population bias surfaces instead of hiding behind a strong average.

Solution & trade-off: The headline trade-off was objectivity vs. nuance. I kept the rollup strictly objective — rules → cases → cohorts → capability → autonomy verdict, with no weighted averages or mid-way tags — so any verdict can be walked down to the failing case and the agent's step-by-step reasoning. V1 stayed tight (pre-built packs, deterministic comparison, simple dashboard); AI-generated evaluators, cohort value-maximization, and RBAC were pushed to V2/V3.

Problem: In virtual care, patients seek second opinions or stockpile — and can end up with duplicate prescriptions under different drug names. Rule-based checks flag exact matches but miss dose titration, naming variation, and cross-system data. The risk spans patient safety, payer clawbacks, regulatory exposure, and platform trust.

Approach: A six-step pipeline that separates deterministic work from AI reasoning: normalize the pending Rx, normalize medication history, filter candidates, compute overlap (all deterministic), then classify intent and surface a finding (AI reasoning). The agent surfaces evidence and a severity tier; the clinician always decides. Mindset: assist, never replace.

Solution & trade-off: The core trade-off is safety vs. alert fatigue. I tuned for it with a three-tier escalation model (Low/Medium/High by confidence and potential harm) and a deliberately minimal clinician UI showing only the matched med, strength, fill date, days supply, pharmacy, overlap, and a clear action. Key decision: don't let the LLM do math — anchor date/overlap logic in code and use the model only where genuine ambiguity exists.

Problem: Operational teams in care delivery juggle scattered tools and signals to triage, route, and resolve work — slowing decisions and making it hard to see what needs attention now versus later.

Approach: I prototyped a single operational console that consolidates the queue and surfaces the next best action, applying the same principle from my agentic work: make the system explicit and measurable — clear states, traceable decisions, and a human firmly in control of consequential actions.

Solution & trade-off: The decision call was breadth vs. depth. Rather than a sprawling do-everything console, I scoped a focused surface that proves value on the highest-volume operational paths first, with deterministic guardrails around any automated step and obvious manual override — leaving richer workflows for a later iteration.

Problem: Clarity is a pediatric care company, and almost every patient arrives because a pediatrician referred them. The referral form itself works fine — the drop-off happens after it. The family gets stuck on booking, insurance, or paperwork, never completes the first visit, and the pediatrician never finds out what happened. The numbers are blunt about the cost: when that first referred family actually completed a visit, the practice referred again 66% of the time. When they didn't, only 22%.

Approach: Don't rebuild the referral form. Add an activation layer right after it. The copilot figures out why a referred family is stuck, clears the easy barriers on its own (reminders, insurance check, paperwork help, language), hands the hard cases to a person, and then sends the pediatrician a short “care handoff receipt” of what happened — so the referral stops feeling like a patient disappearing into a black box.

Solution & trade-off: Two sequenced steps — family scheduling support first, then closing the loop back to the practice. What I deliberately left out: teaching practices about the conditions beyond ADHD (a real opportunity, but worthless while the first referral still fails), and chasing the ~1,000 practices that never referred at all (cause unknown, expensive to move). Fix the loop first; everything else compounds on top of it.

Problem: Abby Care pays family members to care for Medicaid patients at home — elderly people with complex conditions, and people with disabilities. Roughly 80 staff nurses across 8 states are supposed to keep an eye on all of them. Two useful streams of information already exist: insurers hand over claims data (ER visits, hospital stays, missed medication refills), and Abby Care's own app collects daily check-ins from the family caregivers. Nobody puts the two together. So a coordinator starts a shift with no list of who is getting worse — they find out a patient was in the ER weeks after it happened.

Approach: Fuse the two feeds into one ranked worklist and, crucially, always show the evidence. Each patient gets a risk score and an urgency tier — Urgent, This Week, or Monitor — and the payer-side and caregiver-side signals are listed separately rather than blended into one opaque number, so a coordinator can see why someone jumped to the top and act on it immediately.

Solution & trade-off: A coordinator opens the queue, sees who to call first, drills into a patient to read the drivers and the 6-week trend, then logs the outreach. Two awkward cases are handled on purpose: when the insurer feed is delayed, the patient is flagged stale rather than quietly rendered as low-risk; and when the two feeds disagree, the disagreement is shown as a warning instead of averaged into a falsely reassuring score. Deliberately left out: an enrollment queue for patients who don't yet have an Abby Care caregiver (valuable, but that's acquisition, not triage), plus care-plan editing, in-app calling, and analytics dashboards.