Apache Kafka in Hospital Systems: Event-Driven Architecture for Clinical Interoperability in Venezuela

A mid-sized hospital in Venezuela typically has between 4 and 8 systems that need to find out, in real time, about the same things: that a patient was admitted, that a lab result is ready, that an invoice needs to be issued. Every time that communication is solved with point-to-point integrations, each new system multiplies the connections that have to be maintained: specific interfaces, format transformations, cron jobs, and scripts someone wrote "quickly" years ago.

Apache Kafka proposes flipping the model: instead of systems talking directly to each other, they all write to and read from the same central event stream. It's an architecture already used by several healthcare organizations as an integration backbone for HL7 and FHIR data between medical records, laboratory, and billing, as described by Confluent's documentation on healthcare interoperability.

This article assumes you already understand the underlying problem — we cover it in detail in why healthcare interoperability projects fail in Venezuela. Here we go straight to the concrete architecture: how it's designed, which decisions matter most in a Venezuelan clinical context, and when Kafka is the wrong tool.

What problem does Kafka solve that point-to-point integration doesn't?

With point-to-point integrations, connecting 5 systems requires up to 10 distinct connections (n×(n-1)/2), each with its own format, its own error handling, and its own point of failure. With Kafka, each system connects to the event stream once — it publishes what it owns and consumes what it needs. Going from 5 to 8 systems doesn't multiply the connections; it keeps them practically constant.

In the interoperability project we built for a Venezuelan hospital network, this was exactly the starting point: every new integration between HIS, LIS, and billing was built from scratch, with its own sync script and its own duplicate-record bugs. Migrating to a centralized event stream eliminated that multiplication and let future systems (BI, SAP, new clinical modules) connect to the same event plane without rewriting anything that already existed.

How is clinical data organized into topics and partitions?

Each clinical domain should live in its own topic — not a single topic for "the whole hospital." The typical separation that works in an environment with HIS, LIS, and billing:

The content of each message is typically encoded in HL7 v2.x or FHIR (for example, FHIR resources Encounter, Observation, MedicationRequest); Kafka's job is limited to guaranteeing reliable transport, ordering by key, and decoupling between producers and consumers. This separation by domain simplifies message versioning and onboarding new systems — a clinical rules engine, for example, only needs to subscribe to resultados-laboratorio and ordenes-medicas, not the hospital's entire event stream.

Within each topic, partitions are assigned by a clinically meaningful key — typically the patient ID or the encounter ID (episode of care). This guarantees that every event for a given patient arrives in the order it occurred to any system consuming it, because Kafka only guarantees ordering within a partition, not across different partitions. Choosing the wrong key can produce dangerous situations, like a lab result arriving before the medical order that triggered it. In hospitals that work by episode of care, using the encounter ID as the key tends to be the most robust option, because it groups admission, orders, results, and billing for the same clinical case.

Why the replication factor is non-negotiable in a hospital

In most business systems, a replication factor of 2 is acceptable to tolerate a server going down. In healthcare, the minimum standard should be 3, no exceptions: a lab result or a medical order that gets lost isn't a lost business record — it's a clinical decision that didn't arrive in time.

With replication.factor=3 and min.insync.replicas=2, the cluster tolerates the complete loss of a node without stopping accepting or serving data, and without risking the durability of an already-confirmed event. This carries a real infrastructure cost — three copies of every event, three minimum nodes in the cluster — that's often underestimated when budgeting the project. It's worth stating this bluntly during the diagnostic phase, before committing a budget that later falls short of the minimum redundancy required.

"Exactly-once": what it means and why it matters twice as much for lab results

Exactly-once semantics (EOS) guarantees that an event is processed exactly one time, even in the face of network failures or producer retries. Without this guarantee, Kafka's default mode is "at-least-once": an event can be processed more than once after a transient failure.

In billing, a duplicate event generates a double charge — annoying, but fixable. In lab results, a duplicate event can generate a repeated clinical alert or a misinterpretation if the consuming system doesn't deduplicate by result ID. Kafka solves this with two mechanisms, as detailed in Conduktor's technical guide on exactly-once semantics:

• Idempotent producers (enable.idempotence=true, enabled by default since Kafka 3.0): every message carries a per-partition sequence number, and Kafka discards duplicate retries without rewriting them.
• Transactional producers (a unique transactional.id per instance): they let the read-process-write pattern — reading from one topic, processing, and writing to another — be an atomic, all-or-nothing operation.

On the consumer side, isolation.level=read_committed ensures only messages from committed transactions are read, discarding aborted or in-flight writes. And in streaming pipelines (for example, a clinical rules module over lab results), processing.guarantee=exactly_once_v2 in Kafka Streams extends this guarantee across the whole pipeline without reimplementing deduplication in every consumer.

This guarantee isn't free: transactional coordination adds a few milliseconds of latency and reduces throughput compared to "at-least-once." That's an acceptable cost for lab results or billing; it isn't always acceptable for high-volume, low-risk streams, like equipment telemetry.

How to keep every downstream system from reprocessing the entire history

A common mistake is that every new consuming system (a Business Intelligence module added two years later, for example) starts reading the entire topic from the beginning, generating unnecessary load and duplicating work other consumers already did. The correct pattern is to use independent consumer groups per downstream system: each group maintains its own offset and advances at its own pace, without interfering with the others or reprocessing what another system already consumed.

This is what allows billing, the BI module, and a future integration with SAP in Venezuelan hospitals — if the hospital ever reaches that level of complexity — to consume the same event stream at different speeds, without one blocking or duplicating the other's work.

When is Kafka the wrong tool?

Not every hospital needs Kafka. If a clinic has 2-3 systems and a low daily event volume, a well-built point-to-point integration — or even a simpler sync layer — solves the problem with less infrastructure to maintain. Kafka starts to make sense when there are 4 or more systems that need to learn about the same events in real time, when the hospital already knows it will keep adding systems in the coming years, or when there are clear auditing and traceability requirements over clinical events.

Putting Kafka into a small clinic out of technology fashion is the same mistake, in reverse, as sticking with point-to-point integrations in a large hospital network.

Frequently asked questions

Does Apache Kafka replace the HL7/FHIR interoperability engine?

No. Kafka is the event transport layer; HL7/FHIR remains the standard for structuring the clinical content of each message. In practice they complement each other: the interoperability engine validates and transforms, Kafka guarantees transport, ordering, and resilience.

How long does it take to implement Kafka in a hospital that already has HIS and LIS running?

It depends on how many existing integrations need to be migrated, but a pilot with two connected systems (HIS and LIS, for example) is usually feasible in 4-6 weeks if those systems already expose some event-export mechanism.

Do I need a dedicated data team to maintain a Kafka cluster?

Not for a mid-sized hospital. A well-configured 3-node cluster, with basic monitoring and clear upgrade procedures, can be maintained within an existing IT team, as long as someone has prior experience with distributed systems.

What about the security and privacy of clinical data in Kafka?

Kafka integrates with encryption in transit (TLS), authentication, and per-topic authorization, and can coexist with pseudonymization strategies for clinical data within the messages. The specific design must align with the hospital's information security policy.

If your hospital or clinic already runs 3 or more systems that are synced by hand today or through fragile integrations, this is exactly the kind of problem we solve at Code by Meléndez — with direct experience from the real healthcare data interoperability case we cited above.