The Decide, Evolve, React pattern in Ruby

The Decide, Evolve, React pattern provides a unified mechanism for expressing, handling and reacting to state changes in a system.

It optionally provides a lossless audit trail of changes, and the seeds of a pub/sub sub-system, by tying together events to the state changes they cause.

It’s a generalisation of the Event Sourcing pattern, but it doesn’t require a log of events or a replay mechanism, and can be readily leveraged by traditional CRUD systems.

Terminology

• State The current state of an entity in the system. A shopping cart, a user profile, a bank account.
• Command A request or intent to change the state of an entity. Add an item to the cart, update the user's email, transfer money.
• Event A record of a change in state. Item added to cart, email updated, money transferred.

1. Decide

The decide step takes and validates input in the form of a command, fetches any necessary data to fulfil the command, and decides how to update the state based on all that. The state changes are expressed as a set of events.

Let’s say we’re building a shopping cart system. We have a cart object that we can add and remove items from. This is our state.

cart = Cart.new
cart.add_item(id: 1, name: 'Apples', price: 100, quantity: 2)
cart.add_item(id: 2, name: 'Oranges', price: 200, quantity: 1)
cart.total # => 400

We don’t update the cart directly in our app. Instead, we define a command that describes the change we want to make to the cart.

AddItemToCart = Data.define(:cart_id, :product_id, :quantity)

Now, given a new command sent by the client…

command = AddItemToCart.new(cart_id: 'cart-123', product_id: 3, quantity: 1)

… And a cart instance (fetched from the database, or a new instance)

cart = DB.find_cart(command.cart_id) || Cart.new(cart.id)

We feed the cart and the command to the decider function (or class, module, etc). Its job is to evaluate the current state of the cart, the command, and decide whether an item can be added to the cart.

# @param cart [Cart]
# @param command [Command]
# @return [Array<Event>]
def decide(cart, command)
  case command
    when AddItemToCart
      decide_add_to_cart(cart, command)
    else
      raise ArgumentError, "Unknown command #{command.class}"
  end
end

def decide_add_to_cart(cart, command)
  # 1. Is there an actual product with this ID?
  product = DB.find_product(command.product_id)
  raise "Product not found" unless product

  # 2. Is the product in stock?
  raise "Out of stock" unless product.inventory >= command.quantity

  # 3. Return events to add the item to the cart
  [
    ItemAddedToCart.new(
      item_id: product.id, 
      name: product.name, 
      price: product.price, 
      quantity: command.quantity
    )
  ]
end

So that’s it: given a command, we expect one or more events. We can replicate the pattern to handle more commands.

case command
  when AddItemToCart
  # ...
  when UpdateItemQuantity
  # ...
  when RemoveItemFromCart
  # ...
end

2. Evolve

The event classes are also just structs.

ItemAddedToCart = Data.define(:item_id, :name, :price, :quantity)
ItemRemovedFromCart = Data.define(:item_id)

Once the decide function evaluates a command and returns events, we iterate the events and “evolve” the state of the cart accordingly.

# @param cart [Cart]
# @param events [Array<Event>]
# @return [Cart]
def evolve(cart, events)
  events.each do |event|
    case event
    when ItemAddedToCart
      cart.add_item(
        id: event.product_id, 
        name: event.name, 
        price: event.price, 
        quantity: event.quantity
      )
    when ItemRemovedFromCart
      cart.remove_item(event.item_id)
    end
  end

  cart
end

That’s evolve done. Given a piece of state and one or more events, update the state.

3. React

React takes the new state and generated events, and triggers side-effects. This could be sending an email, updating a database, or publishing a message to a queue.

In most cases this is where you’ll actually persist the results of the steps above. Ie. updating the new version of the shopping cart back to the database, and saving the events if you want to keep them around for auditing or replaying.

def react(cart, events)
  DB.transaction do
    DB.save_cart(cart)
    DB.save_events(events)
    Emails.send_cart_updated(cart)
    Webhooks.notify_cart_updated(cart, events)
  end
end

Implementation will vary, but I’m wrapping the above in a transaction to ensure that all the steps succeed or fail together. This is important to keep the system in a consistent state. Events are the source of truth in this system, so you want to make sure they are persisted along with the state.

Note that saving state and events in an ACID transaction is only possible if both are persisted to the same database. In some cases you'll be using an event store that is separate from your main database, and you'll have to ensure consistency between the two systems by other means, one being the Outbox pattern.

Regardless of the implementation details, this interlocking of events and decision logic together is what gives you a lossless audit trail of changes in your system.

• 1. 2024-09-16 11:28:46 cart-123 2x Apples added to cart
• 2. 2024-09-16 11:28:59 cart-123 1x Apples removed from cart
• 3. 2024-09-16 11:29:10 cart-123 3x Oranges added to cart

In many cases, you’ll want your react step to initiate new command flows. For example by scheduling a new command that is then picked up by a background worker and fed back into the decide step.

def react(cart, events)
  DB.transaction do
    # DB.save_cart(cart)
    # DB.save_events(events)
    # Emails.send_cart_updated(cart)
    # Webhooks.notify_cart_updated(cart, events)
    CommandJob.perform_later(UpdateInventory.new(cart_id: cart.id))
  end
end

Putting it all together

There’s many ways to put this all together, but this is one option given the examples above.

class CartDomain
  def run(command)
    # Fetch or initiate the shopping cart
    cart = DB.load_cart(command.cart_id) || ShoppingCart.new(command.cart_id)
    # Run the decide function and get the events
    events = decide(cart, command)
    # Run the evolve function and get the updated cart
    evolve(cart, events)
    # Run the react function to persist the changes
    # and trigger side effects
    DB.transaction do
      react(cart, events)
    end
  end

  # Decide, Evolve, React functions and any other helpers down here
end

In these examples I'm assuming the shopping cart is a mutable object. For example evolve is assumed to update the cart instead of returning a new copy. In functional implementations all three steps may be pure functions that return new versions of the cart and events, without modifying the originals.

Making it nicer

In real code you might want to abstract some of the implementation into more reusable helpers. For example if you plan to use this pattern for several entities in your system. You will also very probably want to facilitate validating command attributes to make sure they’re valid before handling. Below is an example of the type of internal APIs I’ve come up with in the past.

module Commands
  # Define commands with structural validation of attributes.
  # For example using ActiveModel::Validations, Dry-Types, etc.
  class AddItemToCart < Command
    attribute :cart_id, String
    attribute :product_id, Integer
    attribute :quantity, Integer

    validates :cart_id, :product_id, :quantity, presence: true
  end

  # etc.
end

# A class to encapsulate the full lifecycle of a shopping cart
class CartDomain < StateHandler
  entity ShoppingCart

  # Command handlers. A.K.A. "deciders"
  decide Commands::AddItemToCart do |cart, command|
    # validate command, fetchh product, etc
    # Return new events
    [ItemAddedToCart.new(product_id: product.id, ...)]
  end

  decide Commands::RemoveItemFromCart do |cart, command|
    # ...
  end

  # Event handlers. A.K.A. "evolvers"
  evolve Events::ItemAddedToCart do |cart, event|
    cart.add_item(...)
  end

  evolve Events::ItemRemovedFromCart do |cart, event|
    cart.remove_item(...)
  end

  # Side effect handlers. A.K.A. "reactors"
  # Make sure to persist events and cart
  react :any, PersistEventsAndEntity

  # Here we can react to specific events
  react Events::OrderPlaced do |cart, event|
    Emails.send_order_confirmation(cart)
    # Here we can send a command to manage a separate domain or entity
    CommandJob.perform_later(Commands::UpdateInventory.new(cart_id: cart.id))
  end
end

And then you use this wherever you handle user input in your app (controllers, background workers, CLIs, etc).

# POST /carts/:id/items
command = Commands::AddItemToCart.new(params[:command])
# Do something if the command is invalid
# respond_with_error(command.errors) unless command.valid?

CartDomain.run(command)

State machines

You’ll already have noticed that this basically describes a state machine. The fact is that any change to state in any app is a state machine, whether you think of it that way or not. This pattern makes it explicit and consistent for all state mutations in your app.

Note that you can still model specific “business states” on top of this. The following example adds an order_status field to shopping carts, and events to track the transition from an open shopping cart to a placed order.

decide Commands::PlaceOrder do |cart, command|
  raise "Cart already placed" if cart.order_status == 'placed'
  [OrderPlaced.new(cart_id: cart.id)]
end

evolve Events::OrderPlaced do |cart, event|
  cart.order_status = 'placed'
end

react Events::OrderPlaced do |cart, event|
  Emails.send_order_confirmation(cart)
end

You can go back to previous event handlers in your domain and encode business rules, such as forbidding mutations of closed carts.

decide Commands::AddItemToCart do |cart, command|
  raise "Cart is closed" if cart.order_status == 'placed'
  # ...
end

Side effects as commands

In many cases you’ll want side effects in the react step to be themselves logged as events, and the logic to run those effects might require its own decision process. In this scenario it makes a lot of sense to have react just issue new commands, which in turn will run the side effects and log events with the result.

We can modify our API so that react has the signature #react(state, events) => Array<Command>. Any new commands returned by react will be scheduled to run next. In this way, all side effects are encoded as just another command -> events -> react workflow.

# When an order is placed, schedule a new command to send an email
react Events::OrderPlaced do |cart, event|
  [Commands::SendConfirmationEmail.new(cart_id: cart.id)]
end

# Then handle the new command and send the email and log events
decide Commands::SendConfirmationEmail do |cart, command|
  raise "could not send email for #{cart.id}" unless Mailer.send_confirmation_email(cart)

  [Events::EmailConfirmationSent.new(cart_id: cart.id)]
end

# optional handle the EmailConfirmationSent event if you need to update the cart
evolve Events::EmailConfirmationSent do |cart, event|
  cart.email_confirmation_sent = true
end

This “flattens” the implementation even further by making all operations and their side-effects part of the same protocol, and we get a more complete audit trail to boot.

• 1. 2024-09-16 11:28:46 cart-123 2x Apples added to cart
• 2. 2024-09-16 11:28:59 cart-123 1x Apples removed from cart
• 3. 2024-09-16 11:29:10 cart-123 3x Oranges added to cart
• 4. 2024-09-16 11:40:16 cart-123 Order placed
• 4. 2024-09-16 11:40:28 cart-123 Confirmation email sent

Also, nothing stops you from saving commands to history as well as events, which can give you a direct cause-and-effect view of you user’s actions.

Errors as events

So far I’ve been using exceptions to catch errors and bail out of command handling. But in many cases you’ll want to capture some errors as domain events that allow the system to react or recover from them.

decide Commands::AddItemToCart do |cart, command|
  product = DB.find_product(command.product_id)
  # Missing product is a genuine exception
  raise "Product not found" unless product
  # Out of stock is expected, and can be handled by the domain
  # Return an event that captures the error
  if product.inventory < command.quantity
    return [Events::ItemOutOfStock.new(cart_id: cart.id, product_id: product.id, quantity: command.quantity, available: product.inventory)]
  end

  [Events::ItemAddedToCart.new(product_id: product.id, ...)]
end

# We can now react to Events::ItemOutOfStock and show a notice to the user, 
# offer them alternatives or discounts, etc.
react Events::ItemOutOfStock do |cart, event|
  # ...
end

And, once more, we get a fuller picture of the cart’s history, for free:

• 1. 2024-09-16 11:28:46 cart-123 2x Apples added to cart
• 2. 2024-09-16 11:28:59 cart-123 1x Apples removed from cart
• 3. 2024-09-16 11:29:10 cart-123 3x Oranges added to cart
• 4. 2024-09-16 11:40:16 cart-123 Oranges out of stock: only 2 available

Testing

By segregating the handling of input (commands), mutations (events) and side-effects (reactions) you can test each part in isolation.

domain = CartDomain.new(id: 'cart-123')
event = Events::ItemAddedToCart.new(cart_id: cart.id, product_id: 1, ...)
cart = ShoppingCart.new
domain.evolve(cart, event)
expect(cart.items.size).to eq(1)

Documentation

An interesting outcome of this pattern is that it also “flattens” your domain’s internal API into a list of actions (commands), a list of known state changes (events), and a list of side effects. Together they form a comprehensive “protocol” of what your app can do. For example you could list your commands and generate documentation for your API.

It can also make the code itself more cohesive and self-documenting.

Drawbacks

Nothing is free, unfortunately. This pattern is more complex than just updating state directly, and it can be overkill for simple CRUD systems. It also requires a bit of boilerplate to set up, and can be hard to understand for developers unfamiliar with it. It may also conflict with data-management libraries that want to own side effects triggered by state changes. For example ORM callbacks. It’ll take some discipline to avoid using those and registering side effects as explicit reactions instead.

Event-sourced vs state-stored systems

As mentioned at the start, this pattern is a super-set of Event Sourcing. The version illustrated here uses a “state-stored” implementation where the current state of entities is fetched from a regular database, and later persisted back to it. This plays well with traditional CRUD systems that just want to consolidate state management and auditing.

A slightly different “event-sourced” implementation first replays past events to reconstruct the current state of entities, and then applies new events to update it.

# Event-sourced version
# 1. fetch past events and evolve them into an initial state
cart = ShoppingCart.new(id: command.cart_id)
historical_events = DB.load_events_for(cart.id)
evolve(cart, historical_events) # replay past events to get current state

# 2. decide, evolve, react
new_events = decide(cart, command)
evolve(cart, new_events)
react(cart, new_events)

Sources

• Functional Event Sourcing Decider
• f { model }