🔄 Event Sourcing: The Complete Guide to Storing Facts, Not State

🔄 Event Sourcing: The Complete Guide to Storing Facts, Not State

Every application needs to track state. A user's balance, an order's status, a shopping cart's contents — these are the lifeblood of software. But how you store that state fundamentally shapes what your system can do. Most developers reach for traditional CRUD (Create, Read, Update, Delete), overwriting old values with new ones. Event Sourcing takes a radically different approach: instead of storing the current state, you store every event that ever happened and derive the current state by replaying those events. Think of it as a Git history for your domain data — you never lose information, and you can always reconstruct any point in time.

In this guide, we'll explore event sourcing from the ground up: what it is, how to design an event store, how to rebuild state, how snapshots keep things performant, and when you should (and shouldn't) use this pattern. If you're building systems where auditability, scalability, and temporal queries matter, event sourcing deserves a place in your architecture toolkit.

📦 What Is Event Sourcing?

Event Sourcing is an architectural pattern where state changes are captured as an immutable sequence of events. Rather than mutating a row in a database, you append a new event that describes what happened. The current state of any entity (called an aggregate) is derived by replaying all of its events from the beginning.

Each event is a fact — something that already happened. Events are named in the past tense: OrderPlaced, PaymentReceived, ItemShipped. They are immutable. You never update or delete an event. If a mistake was made, you append a compensating event (e.g., OrderCancelled) rather than rewriting history.

This mirrors how many real-world systems already work. Your bank doesn't store "balance = $500." It stores every deposit and withdrawal, and your balance is the sum of all transactions. Accounting ledgers, medical records, and legal systems all follow this principle — and for good reason.

⚔️ Traditional CRUD vs Event Sourcing

Understanding the difference between CRUD and event sourcing is crucial before diving deeper. Here's a side-by-side comparison:

With CRUD, when you update an order's status from "pending" to "shipped," the old value is gone forever. With event sourcing, you have OrderPlaced, PaymentProcessed, and OrderShipped — a complete narrative of what happened and when. This distinction becomes critical in domains where understanding how you got to a state matters as much as the state itself.

🗄️ Designing the Event Store

The event store is the heart of an event-sourced system. It's an append-only database that persists events in the order they occurred. Each event belongs to a stream (typically one stream per aggregate instance). Here's a practical schema:

CREATE TABLE events (
    event_id        UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    stream_id       VARCHAR(255) NOT NULL,
    event_type      VARCHAR(255) NOT NULL,
    event_data      JSONB NOT NULL,
    metadata        JSONB DEFAULT '{}',
    version         INTEGER NOT NULL,
    created_at      TIMESTAMP DEFAULT NOW(),
    UNIQUE (stream_id, version)
);

CREATE INDEX idx_events_stream ON events (stream_id, version);
CREATE INDEX idx_events_type ON events (event_type);
CREATE INDEX idx_events_created ON events (created_at);

Key design decisions in this schema:

• stream_id groups events by aggregate (e.g., order-12345). Each aggregate has its own stream.
• version ensures ordering within a stream and provides optimistic concurrency control. Before appending, check that the expected version matches the latest stored version.
• event_data holds the event payload as JSON, making the schema flexible for different event types.
• metadata captures cross-cutting concerns: correlation IDs, user context, causation chains.

The UNIQUE(stream_id, version) constraint is critical — it prevents concurrent writes from corrupting event ordering, acting as your concurrency guard. For high-throughput database scenarios, dedicated event store solutions like EventStoreDB or Marten offer optimized implementations.

🔁 Rebuilding State from Events

To get the current state of an aggregate, you load all its events and replay them in order. Each event modifies the aggregate's in-memory state through an apply method. Here's a complete example of a bank account aggregate:

class BankAccount:
    def __init__(self, account_id: str):
        self.account_id = account_id
        self.balance = 0
        self.status = "active"
        self.version = 0
        self._pending_events = []

    def apply(self, event: dict):
        event_type = event["event_type"]
        data = event["event_data"]

        if event_type == "AccountOpened":
            self.balance = data["initial_deposit"]
            self.status = "active"
        elif event_type == "MoneyDeposited":
            self.balance += data["amount"]
        elif event_type == "MoneyWithdrawn":
            self.balance -= data["amount"]
        elif event_type == "AccountClosed":
            self.status = "closed"

        self.version += 1

    def deposit(self, amount: float):
        if self.status != "active":
            raise ValueError("Cannot deposit to a closed account")
        if amount <= 0:
            raise ValueError("Deposit amount must be positive")
        event = {
            "event_type": "MoneyDeposited",
            "event_data": {"amount": amount}
        }
        self.apply(event)
        self._pending_events.append(event)

    def withdraw(self, amount: float):
        if self.balance < amount:
            raise ValueError("Insufficient funds")
        event = {
            "event_type": "MoneyWithdrawn",
            "event_data": {"amount": amount}
        }
        self.apply(event)
        self._pending_events.append(event)

    @classmethod
    def load_from_events(cls, account_id: str, events: list):
        account = cls(account_id)
        for event in events:
            account.apply(event)
        return account

The load_from_events class method is the key: it creates a fresh aggregate and replays every event to reconstruct the current state. The deposit and withdraw methods enforce business rules before creating events — an event represents something that has already been validated and accepted.

📸 Snapshots for Performance

Replaying thousands of events every time you load an aggregate gets expensive. Snapshots solve this by periodically saving the aggregate's state at a given version. On the next load, you start from the snapshot and only replay events that occurred after it.

class SnapshotStore:
    def __init__(self, db_connection):
        self.db = db_connection

    def save_snapshot(self, stream_id: str, version: int, state: dict):
        self.db.execute(
            """INSERT INTO snapshots (stream_id, version, state, created_at)
               VALUES (%s, %s, %s, NOW())
               ON CONFLICT (stream_id)
               DO UPDATE SET version = %s, state = %s, created_at = NOW()""",
            (stream_id, version, json.dumps(state), version, json.dumps(state))
        )

    def load_aggregate(self, stream_id: str):
        snapshot = self.db.query(
            "SELECT version, state FROM snapshots WHERE stream_id = %s",
            (stream_id,)
        )

        if snapshot:
            aggregate = BankAccount.from_snapshot(stream_id, snapshot.state)
            start_version = snapshot.version + 1
        else:
            aggregate = BankAccount(stream_id)
            start_version = 1

        events = self.db.query(
            "SELECT * FROM events WHERE stream_id = %s AND version >= %s ORDER BY version",
            (stream_id, start_version)
        )

        for event in events:
            aggregate.apply(event)

        if aggregate.version - (snapshot.version if snapshot else 0) > 100:
            self.save_snapshot(stream_id, aggregate.version, aggregate.to_dict())

        return aggregate

A common strategy is to take a snapshot every N events (50–200 is typical). This bounds the replay cost while keeping the performance characteristics predictable. Some teams snapshot on every write; others snapshot lazily when the replay count exceeds a threshold.

📊 Projections: Building Read Models

Event sourcing naturally pairs with projections — derived read models built by processing the event stream. While the event store is the source of truth, projections give you query-friendly views optimized for specific use cases.

class OrderDashboardProjection:
    def __init__(self, read_db):
        self.db = read_db

    def handle(self, event: dict):
        event_type = event["event_type"]

        if event_type == "OrderPlaced":
            self.db.execute(
                """INSERT INTO order_summary
                   (order_id, customer, total, status, placed_at)
                   VALUES (%s, %s, %s, 'placed', %s)""",
                (event["stream_id"], event["event_data"]["customer"],
                 event["event_data"]["total"], event["created_at"])
            )
        elif event_type == "OrderShipped":
            self.db.execute(
                """UPDATE order_summary
                   SET status = 'shipped', shipped_at = %s
                   WHERE order_id = %s""",
                (event["created_at"], event["stream_id"])
            )
        elif event_type == "OrderCancelled":
            self.db.execute(
                """UPDATE order_summary
                   SET status = 'cancelled', cancelled_at = %s
                   WHERE order_id = %s""",
                (event["created_at"], event["stream_id"])
            )

Projections can be rebuilt from scratch at any time by replaying the entire event stream. This means you can add entirely new read models retroactively — a capability that CRUD systems simply cannot match. This is where event sourcing truly shines for microservices architectures where different services need different views of the same data.

🤝 Integration with CQRS

Event sourcing and CQRS (Command Query Responsibility Segregation) are natural partners. CQRS splits your application into a write side (commands that produce events) and a read side (projections optimized for queries). Event sourcing provides the mechanism that connects them — events flow from the write model to the read model via projections.

The write side validates commands, produces events, and appends them to the event store. The read side subscribes to those events and updates denormalized read models. This separation lets you scale reads and writes independently and optimize each side for its specific workload. For a deeper dive into this synergy, see our guide on CQRS pattern design.

Without CQRS, event sourcing is still useful (you can derive state from replayed events for both reads and writes). But the combination of both patterns unlocks the full potential: write-optimized event storage with read-optimized query models, connected by a stream of immutable facts.

📐 Event Versioning and Schema Evolution

Events are immutable, but your domain evolves. When event schemas change, you need a strategy for handling old events alongside new ones. There are three common approaches:

• Upcasting: Transform old events into the new format on read. A version-aware deserializer converts OrderPlaced_v1 to the current shape before the aggregate processes it.
• Weak Schema: Use flexible formats (JSON with optional fields). New fields default to null; deprecated fields are ignored. This is the simplest approach but can accumulate technical debt.
• Copy-Transform: Migrate the entire event store to a new schema. This is the nuclear option — expensive but provides a clean slate. Only use for major structural changes.

class EventUpcaster:
    def upcast(self, event: dict) -> dict:
        event_type = event["event_type"]
        version = event.get("schema_version", 1)

        if event_type == "OrderPlaced" and version == 1:
            event["event_data"]["currency"] = "USD"
            event["event_data"]["shipping_method"] = "standard"
            event["schema_version"] = 2

        return event

Upcasting is the most common and least disruptive approach. It keeps old events untouched in storage while presenting them in the current format to application code. For guidance on managing schema changes across services, check our API versioning strategies guide.

✅ Pros and Cons

🎯 When to Use (and When NOT to Use)

Use event sourcing when:

• Auditability is required: Finance, healthcare, legal, and compliance-heavy domains where you must prove what happened and when.
• Complex domain logic: Systems with rich business workflows where understanding the sequence of state transitions matters (order processing, claims management).
• Temporal queries: You need to answer questions like "What was the portfolio value on March 15th?" or "What was the inventory count before this shipment?"
• Event-driven architecture: You're already publishing domain events — event sourcing makes the event stream your source of truth.
• Analytics on behavior: You want to analyze user journeys, conversion funnels, or process bottlenecks from historical data.

Do NOT use event sourcing when:

• Simple CRUD is sufficient: A basic content management system or a settings page doesn't benefit from event history.
• Your team is unfamiliar: Event sourcing has a steep learning curve. Adopting it without team buy-in leads to accidental complexity.
• High-frequency updates on single entities: An IoT sensor sending 1000 updates per second to one aggregate will create an unmanageable event stream. Consider time-series databases instead.
• Strict consistency on reads: If your users cannot tolerate any read delay (eventual consistency), the projection model adds unacceptable latency.

🌍 Real-World Examples

Banking and Financial Services: Every transaction (deposit, withdrawal, transfer, fee) is an event. Account balances are derived by summing all transactions. Regulators can audit the complete history. Fraud detection systems analyze event patterns in real-time. This is arguably the most natural domain for event sourcing — it mirrors how accounting has worked for centuries.

E-Commerce Order Management: An order's lifecycle flows through OrderPlaced → PaymentAuthorized → ItemsPicked → OrderShipped → OrderDelivered. Each event triggers downstream processes: payment capture, warehouse picking, shipping label generation, customer notifications. If a customer disputes a charge, you have the complete timeline of every action taken.

Healthcare Systems: Patient records as event streams capture every diagnosis, prescription, lab result, and procedure. Clinicians see the current state, but the full history is always available for treatment decisions. Regulatory compliance (HIPAA, GDPR) benefits from the immutable, auditable nature of event logs.

Supply Chain and Logistics: Tracking goods through receiving, storage, picking, packing, and shipping. Each state transition is an event. When something goes wrong, you can trace exactly where in the chain the issue occurred and what actions led to it.

❓ Frequently Asked Questions

How does event sourcing handle deleting data for GDPR compliance?

Since events are immutable, GDPR's "right to erasure" requires special handling. The most common approach is crypto-shredding: encrypt personal data in events with a per-user key, and delete the key when erasure is requested. The events remain in the store, but the personal data becomes unreadable. Alternatively, you can use event transformation to replace sensitive fields with anonymized values, though this technically modifies the event store and should be done with strict controls and audit trails.

What's the difference between event sourcing and event-driven architecture?

Event-driven architecture is about using events for communication between components — services publish and subscribe to events. Event sourcing is about using events as the primary persistence mechanism. You can have event-driven architecture without event sourcing (publish events but store state via CRUD), and you can have event sourcing without full event-driven architecture (store events but communicate via synchronous APIs). They complement each other beautifully, but they're distinct patterns solving different problems.

How do I handle failures when projecting events to read models?

Track the last processed event position for each projection. If a projection fails, it can resume from its last checkpoint rather than reprocessing the entire stream. Use idempotent event handlers so that re-processing an event doesn't cause duplicate side effects. For critical projections, implement dead-letter queues to capture and retry failed events without blocking the entire pipeline.

What databases work best for event stores?

EventStoreDB is purpose-built for event sourcing with built-in stream management, subscriptions, and projections. PostgreSQL with a well-designed events table works excellently for most teams — it's familiar, reliable, and supports JSONB for flexible event payloads. Apache Kafka can serve as an event store for high-throughput scenarios, though it requires careful topic partitioning and retention configuration. Marten (for .NET) and Axon Framework (for Java) provide full event sourcing frameworks with built-in stores.

Can I adopt event sourcing incrementally in an existing system?

Yes, and this is often the smartest approach. Start by identifying one bounded context where event sourcing provides clear value — such as order processing or payment handling. Implement event sourcing there while keeping the rest of your system on CRUD. Use an anti-corruption layer at the boundary to translate between the event-sourced and CRUD worlds. As your team gains experience, expand event sourcing to other contexts where it makes sense. You do not need to rewrite your entire system.