There is a kind of labor at the center of medicine that rarely appears in a chart. It does not sit in the problem list or the billing code. It unfolds in conversation, often quietly, as a patient tries to give shape to something real but not yet defined. They reach for words that are approximate — tired, off, not quite right. The words are not false; they are insufficient. What is being described is not a diagnosis but an experience, and experience resists compression.
In clinical practice, this work lives in a specific place: the history of present illness, or HPI. The HPI reconstructs what has happened to a person over time — how symptoms emerged, evolved, interacted with the physical world, and were perceived. It precedes examination. It precedes testing. It is where medicine begins.
The physician’s task in the HPI is not transcription but interpretation. We ask what was happening when the symptom appeared, whether it arose with exertion or at rest, whether recovery changed, whether confidence shifted before function did. We test meanings against timelines and refine language against physiology, gradually aligning what was said with what can be understood clinically, because the lived details of onset, progression, and functional change materially alter the pre-test probability of disease. A laboratory value or imaging finding does not carry the same meaning in every patient; its significance is conditioned by the story that precedes it.
In 15 compressed minutes, we attempt to distill months of lived experience into something coherent and usable. But memory is a leaky vessel. Language is imprecise. Details collapse into summary; long stretches of normalcy disappear; moments of discomfort loom larger than their duration. We are asking patients to reconstruct a life they were busy living, not recording — to compress time into narrative, and narrative into signal. We turn to a second narrator — a partner or family member — not for authority but for calibration. Is this new? Has it truly changed? What else have you observed at home? The aim is not agreement, but clarity.
Consider a grandmother who presents to clinic and insists she is “fine.” She speaks warmly about her grandchildren, her routines unchanged. Only with repeated questioning does a pattern emerge: She now walks more slowly; she rests where she once did not; she attributes breathlessness to age. A daughter recalls that she paused walking from a parking lot to a restaurant. A husband with mild cognitive decline thinks she once walked farther. Piece by piece, what began as “fine” reveals itself as acute disease.
This exchange — centered in the HPI — is investigative, interpretive, and irreducibly human. It depends on training, attention, and judgment. It is also one of the enduring constraints on how far medicine could scale — and a central constraint on how far artificial intelligence can extend medicine beyond the clinic.
For centuries, the most consequential data in health care has lived inside this fragile conversation. Before laboratory panels, before imaging, before algorithms, physicians attempted to understand how the world presses upon a particular body over time. Yet that understanding has always been bounded by presence: A clinician must be there; the patient must remember; the story must be assembled after the fact. What happens between encounters — the daily negotiation between biology and environment — largely vanishes.
The world is now looking to artificial intelligence to address its most intractable crises of health care access and outcomes. Society is building toward a future that lies somewhere on a spectrum — from an augmented clinician made more productive by AI to systems so capable they may not require a human in the loop at all.
Yet the ceiling for any of these futures is dictated by the data they consume. If AI relies solely on episodic, compressed reconstructions from the clinic, its impact will plateau. To truly transform care, we must unlock the dataset of lived experience — stubbornly resistant to scale.
Artificial intelligence has progressed dramatically, extracting signal from imaging, synthesizing longitudinal clinical records, anticipating deterioration, and assisting in clinical decisions at scale. But even as systems have grown more capable, they remain constrained by the data they can access. They can analyze biomarkers, images, and transcripts; what they cannot access is lived experience as it actually occurred.
Patients sense this gap. Increasingly, they upload laboratory results and medical records into AI systems, asking for interpretation that once required a clinician. But in doing so, they are feeding these systems the same episodic snapshots they already have — not the lived context that would make those snapshots interpretable. Interpretation without context is fragile. Laboratory values do not speak for themselves. Clinical meaning rarely resides in the number alone; it is constructed at the intersection of diagnostics and lived experience.
The inflection point is not simply that artificial intelligence has grown more powerful. It is that it has become multimodal and world-facing. AI-enabled glasses and other world-facing wearables, equipped with cameras and microphones, are now positioned to capture clinically meaningful moments in a person’s day — during exertion, in moments of stress, or in the ordinary negotiations of daily life. Even limited preservation of those moments begins to shift what can be known. In effect, these systems introduce a kind of silent narrator to the clinical story — one less dependent on memory alone.
For the first time at meaningful scale, it becomes possible to preserve the core elements of the HPI closer to how they unfolded, rather than reconstruct them later. Clinical AI has made impressive strides in reasoning over complex clinical data. Yet without access to lived experience as it actually occurred, it was reasoning over fragments. It could see the biology; it could not see the life surrounding it. That epistemic constraint defined the ceiling of what medicine could accomplish at scale.
As that gap narrows, AI cannot just read the chart; it must begin to understand the life that produces it. If lived experience can be preserved and understood, it becomes a foundational dataset — not a single biomarker or imaging modality, but a longitudinal account of how a person pushes on the world and how the world pushes back. A patient’s gradual decline in walking pace, captured passively by a wearable device, would become as legible to her cardiologist as her last echocardiogram — not as a replacement for the clinical encounter, but as the context that makes it meaningful.
But making lived experience visible does not solve the problem. It reveals the depth of it.
The question is no longer only whether medicine can see life between visits. It is whether the system is capable of holding it — of organizing it, interpreting it, and acting on it. It is about architecture. Where should the data of a human life live?
Today, that data is scattered across institutions. Hospitals hold clinical records; laboratories hold diagnostic results; consumer devices capture fragments of behavior. Each system stores its own isolated piece of the story. Medicine has learned to interpret these fragments, but it has never solved how they should be assembled. A patient moves through the health care system like a traveler through airports: Each stop generates a new record, stored locally, rarely connected to the rest of the journey. The result is a medical history that exists everywhere and nowhere at once.
Consider something as simple as exertional capacity — the very thing that revealed the grandmother’s disease. A patient may gradually take longer to climb the same staircase, pause more frequently on a daily walk, or breathe differently while gardening. These subtle shifts are often the earliest signals of disease. Yet they rarely appear in institutional records until long after the pattern has emerged in daily life. The signals that matter most for understanding health often exist outside institutional walls — in the life that unfolds between encounters.
Over the past decade, initiatives like the 21st Century Cures Act and the Trusted Exchange Framework and Common Agreement have pushed health systems toward standardized data exchange. The plumbing for institutional interoperability is finally emerging. These efforts make institutional data movable — but they do not address the absence of the data that never enters the system to begin with.
As we enter the era of AI, we are discovering that exchanging institutional records is only the first step. Because the system was designed primarily for institutional exchange, entire companies have emerged to stitch fragmented data together. These middleware platforms translate formats and aggregate records so applications can retrieve data from multiple systems at once.
Health care has seen this pattern before. When systems become sufficiently fragmented, intermediary layers emerge to make them function. Pharmacy benefit managers arose to navigate the tangled relationships of drug pricing. Clearinghouses emerged to route billing data between providers and insurers. These organizations often begin as pragmatic solutions to structural problems, but over time they become permanent, expensive fixtures that add complexity without resolving it.
Middleware companies risk becoming the next such layer — essential navigators of fragmentation that remind us the underlying architecture was never designed to work as a whole. Middleware does not solve this problem. It makes it tolerable — because the architecture of health data was built for episodes of care, not for the continuity of life.
One possible path forward is to rely on increasingly capable AI agents to navigate this landscape. Acting on behalf of a patient, an agent could reach into multiple hospital systems, retrieve records, and assemble a coherent view of a person’s health history in real time. This mirrors what clinicians already do manually — pulling together notes, lab results, and patient recollections to reconstruct a narrative.
But AI agents can only traverse what exists. They can assemble records, but they cannot create continuity where none was ever captured. If the architecture only records isolated encounters, even the most capable agents are left reconstructing the same incomplete story clinicians face today.
In other words, agents can navigate fragmentation; they cannot abolish it.
The alternative would be more ambitious: to rethink the architecture itself. Instead of organizing health data around institutions, we might organize it around individuals. In such a model, the patient becomes the root node — the master copy and central hub — of their own health information. Clinical records, diagnostic results, and contextual signals from consumer devices flow into a shared layer of data governed by the individual but accessible to clinicians. The record would follow the person rather than the facility.
Conceptually, such an architecture solves the continuity problem. Artificial intelligence could reason over a person’s longitudinal history rather than isolated snapshots. Clinicians could see changes in function across environments rather than reconstruct them from memory. The record of a life would begin to look less like a collection of documents and more like an individual story unfolding across time.
But the challenge is obvious. Architectural revolutions in health care are rare, and institutions do not easily relinquish their position at the center of the record.
This leaves a third possibility: a hybrid model where a new architecture and intelligent agents evolve together. In practice, this means building a patient-centered layer capable of capturing lived experience — the life between visits — while allowing AI agents to bridge the gap, reaching back into legacy institutional systems to retrieve records and integrate them into a patient’s longitudinal story. But this layer cannot exist outside medicine; it must connect back into it.
Making this happen requires a shift in how we define a “medical record.” Today policy frameworks are designed primarily to move institutional data. To move beyond this, we need technical standards that treat signals from daily life — the pause on the stairs, the shift in heart rate — with the same clinical rigor as a laboratory blood test. We also need a shift in the duty of care: If patients provide high-fidelity context about their lives between visits, the health care system must have the incentives and the legal framework to ingest and act on it.
Signals from daily life must flow into clinical systems where physicians and clinical AI can interpret them. That interpretation must then return to the patient’s record as guidance, alerts, or interventions. The goal is not simply to observe life between visits, but to close the loop between the clinic and the world, making those signals clinically actionable.
Yet transforming lived experience from transient memory into actionable data introduces profound friction. To capture the interaction between biology and environment is to blur the historically protected boundary between clinical data and everyday life. It forces a reckoning with agency: The HPI has always been a story told by the patient — a curated act of sharing. Augmenting that story with objective environmental context shifts the power dynamic of the clinic. It raises questions of privacy, security, ownership, and consent; it forces a reckoning with signal and noise — specific instances of ordinary existence must be filtered to isolate subtle clinical truth — and it demands clarity around integration and liability so that targeted data empowers care rather than overwhelms it.
Data, however granular, is not destiny. To matter, it must be responsibly integrated into clinical reasoning and health care delivery. Architecture determines what information exists; agents determine how that information moves and how it is synthesized. Together they allow medicine to see not just isolated clinical events, but the life that connects them.
Across many industries, intelligent systems emerged only after shared infrastructure made the underlying data legible. The modern internet became possible once common protocols allowed information to move freely; global trade accelerated once standardized shipping containers allowed goods to move predictably across ships, ports, and rail. Innovation did not begin with smarter agents navigating fragmented systems; it began with a common architecture.
Health care may now be approaching a similar moment. If we succeed, the middleware companies that currently exist to stitch our fragments together will begin to disappear — their obsolescence serving as a proxy for a functioning system.
The first era of digital health focused on digitizing medical records. The next era will be defined by something more difficult: deciding how the data of a human life should be structured, governed, and interpreted. Artificial intelligence will transform how medicine reasons, but the systems that organize the data will determine how far that transformation can go.
For centuries, medicine has scaled what it could measure and relied on clinicians to supply what could not be measured. That division is ending. Whether this moment becomes incremental or transformative will depend not only on the power of our tools, but on the discipline with which we build around them.
Freddy Abnousi, M.D., is vice president of health technology at Meta, an interventional cardiologist, and an adjunct professor at the Stanford University School of Medicine. Celina Yong, M.D., is associate professor of medicine at the Stanford University School of Medicine and director of interventional cardiology at the Veterans Affairs Palo Alto Health Care System. AI-assisted tools were used for document organization, version management, and limited copy-editing suggestions during the editorial process. All substantive writing, arguments, and intellectual content are entirely the authors’ own.
