Using DynamoDB Streams Vs Updating Data In A Transaction

When designing your DynamoDB queries one choice you will often face is whether to update data in transaction or let a change stream event update the data asynchronously.

Take the following scenario:

A user adds a comment to a post and sends a notification item to the author. 

In this case, you can go about implementing this feature in 2 ways:

1. Use a transaction to insert a comment item and insert a notification item to your table(s).

2. Insert a comment item to your table and have a DynamoDB stream trigger a function to insert a notification item to your table.

Both approaches are valid. Both have their pros and cons. 

Understanding when to use each approach is important for performance, scalability and maintainability.

The DynamoDB Streams Approach

DynamoDB streams enable event-driven architectures by capturing change events (inserts, updates, deletes) at the table level and sending these events to consumers like Lambda.

This allows you to decouple write operations from downstream updates.

For example, our comment and notification feature above is a typical use case for a DynamoDB stream. It also offers resilience and scalability by processing the asynchronous write in isolation. 

However because Streams only support eventual consistency, they may not always be the right fit, especially when multiple items need to be updated atomically.

Also if the downstream operation fails, you must manually handle retries to ensure consistency of the data.

This approach is acceptable when the consequences of a slight delay isn’t critical or when loose coupling and scalability matter more than atomicity

The transaction approach

On the other hand, DynamoDB transactions allow you to group multiple read and write operations atomically.

This makes sure that all operations either succeed or fail together, which is essential for strong consistency.

Transactions are useful for use cases like recording a product sale and updating the product’s stock count together. They guarantee that no partial writes occur and keep data consistent.

However, transactions come with tradeoffs. 

They are more expensive and are not as scalable as streams in high-throughput workloads.

Transactions are best used when the operations are tightly coupled, and the correctness of your system depends on atomic updates.

When to use which approach?

Using DynamoDB Streams

You should choose DynamoDB streams when you need to scale the downstream processes independently. 

For example, if your database in getting a lot of adding comments write requests, using DynamoDB streams may be a better choice than transactions. Especially when being notified for every comment isn’t critical.

DynamoDB streams are also a good choice when eventual consistency is affordable or if you need to decouple core writes from side-effects like logging or notifications. 

Using Transactions

On the other hand, transactions will be a good fit when you require atomicity across multiple items.

Aggregate counts or product inventory counts are good examples when transactions are critical, even to the point where you are sacrificing scalability. 

It wouldn’t usually make sense to have an incorrect product stock count; customers would be placing orders on products that are out of stock and your system wouldn’t be able to prevent that.

Other fits for transactions are when the delays or retries of Streams would break business logic. 

Oftentimes a hybrid approach works best. Use transactions for critical consistency requirements and Streams for everything else. 

This will give you data consistency when it is needed and scalability and resilience when data consistency is not needed.

Summary

In this article we compared DynamoDB streams with transactions when updating related data like creating comments and notifying users. 

We outlined when to favor scalability and decoupling of event-driven updates with Streams versus the strong consistency and atomicity which transactions provide.

Finally we suggest how a hybrid approach works best by balancing both based on requirements.

👋 My name is Uriel Bitton and I hope you learned something in this edition of Excelling With DynamoDB.

📅 If you're looking for help with DynamoDB, let's have a quick chat.

🙌 I hope to see you in next week's edition!