Supercharge ⚡️ your DynamoDB single-table design pattern with TypeDORM

The TypeDORM is an Object Relational Mapper library running in Node,js and written in Typescript. TypeDORM is a feature-rich DynamoDB ORM to help you write high-quality, loosely coupled, scalable, and maintainable applications with highly relational data in the most productive way.

What this article is about

In this article, we will be looking at different challenges that may arise while developing with DynamoDB using a single-table-design pattern and how TypeDORM can help us avoid them.

What this article is NOT about

We will not be covering why and who should use DynamoDB single-table design as Alex DeBrie has done an excellent job at explaining it here.
Once you understand how single-table design works with DynamoDB, you would want to try it out with a real-life example. There is already a very popular article by Forrest Brazeal that can be found here, which you should first look at before continuing with this article.

What you should know before moving on

What is a single-table-design pattern

How it works with a real-life relational dataset

Once you are familiar with the single-table design pattern and have also looked at how it can be used to model highly relational data in DynamoDB, let's talk about why it isn't a very viable way to work with data and what can we do to make it better.

When first introduced to single-table design pattern, there are some common questions and concerns developers start to ponder:

I can't use a single-table design, I need operation time "types safety".

All the attributes and their names just don't make sense anymore, how do I reliably interact with them?

As I keep adding more indexes, how do I make sure that all the attributes created for modeling don't end up returning to clients?

If I choose to keep the original attribute value and its index-referenced attribute separate, how do I make sure they are always in sync?

How do I keep the key schema up to date with a matching pattern for each entity that I add?

How do I confidently interact with all entities without having to remember to provide correct condition expressions for common operations (i.e for unique attributes, making sure to always include condition expressions with attribute_not_exists)?

With a single table design, I can't use data mapper libraries.

The resulting code is hard to read as there are too many attributes/symbols representing different meanings.

My colleges may know how to work with the DynamoDB but this particular way is very unfamiliar.

I want to use the single-table design but I don't want to manage each entity schemas in a separate spreadsheet.

There might be many more questions but most of them are pointing to one concern in particular:

"Do I really want to trade off development ease, developer experience, code quality over this new way of working with DynamoDB?"

The answer is: you don't have to. There is a better option.
Let me introduce you to TypeDORM: Strongly typed ORM for DynamoDB - Built for the single-table-design pattern. Let's first have a quick look at what it is and how it can help us to solve all the problems stated earlier.
Later, we will also have a look at how we can leverage TypeDORM to better structure entities (continuing from the example from this great article).

Introduction to TypeDORM

TypeDORM is an ORM built from the ground up using TypeScript to provide an easy entry point when doing complex, highly relational data modeling in DynamoDB. Its syntax might be familiar since the decorator-based syntax is highly influenced by TypeORM - one of the very popular ORM for working with SQL databases.

Let's take a quick look at some of the many features that TypeDORM provides:

Single table design first-class support

DataMapper development pattern

Full type safety enabled using advanced typescript generics

Declarative relational schema

Multiple connections support

Operation based managers per connection, includes Entity manager, Transaction manager, Batch manager, and Scan Manager

Code follows all possible best practices and provides first-class support for some of the very popular patterns used in the single-table-design pattern

Additional higher-level features like, such as the ability to declare non-key attribute as unique, automatic retries

Auto-Generated values for attributes

Auto-update attributes and its referenced indexes on UPDATE operations

Dynamic & Static default values for attributes

Complex update, key condition, and condition expression all made easy to work with

Powerful expression builder to auto-generate expressions from typesafe input

Getting Started with TypeDORM

Installing

Let's first install all the modules that we will need.

npm install @typedorm/common @typedorm/core --save

We will also need to install reflect-metadata shim that TypeDORM relies on and configure it so that metadata's are correctly detected by TypeDORM

npm install reflect-metadata --save

Enable emitting metadata at runtime

If you are using TypeDORM with TypeScript, make sure you also have the below options enabled in your tsconfig.json

Once this is done we need aws-sdk which TypeDORM uses to communicate with the DynamoDB table.

npm install aws-sdk --save

We are all set now, next we will need to configure the connection. If you have ever worked with TypeORM, this setup will look very familiar.

In the next section, we will have a look at the relational data set that we will be working with. You can always refer back to Forrest Brazeal's article any time you feel lost.

Getting to know our data

Here are some access patterns that Brazeal has listed:

Get employee by employee ID

Get direct reports for an employee

Get discontinued products

List all orders of a given product

Get the most recent 25 orders

Get shippers by company name

Get customers by contact name

List all products included in an order

Get suppliers by country and region

Entities that we will be working with

These are all the entities we will need to create with a given signature to fulfill our access patterns. Here is a slightly modified version of the Entity chart.

The process of selecting what goes under PK and SK remains the same as described here.

If you were following this closely, you will notice few differences between the above entity chart and one presented by Brazeal, Which are:

Each attribute is only ever referenced in a single index

Each entity PK now has a prefix of STATIC entity type - this helps us better identify items where they are all put together in the same table as well as allows us to use incrementing ids

Our access patterns

Here is the list of all access patterns and what attributes we will be querying on. In case you didn't follow this article and came up with different access patterns, that's perfectly okay too.

Hereafter building on top of Brazeal's access patterns, this is what it looks like now.

You will still have to follow all the steps required in a single-table design pattern, with one slight change being that all the entities need to have a SQL-like entity structure (of course, you can still add more attributes to entities later).

Understanding how to build an entity schema would be a different article on its own - our main focus is how we can reliably work with the above entities using TypeDORM.

Developing with TypeDORM

This section will focus on getting connections, entities, tables configured for TypeDORM that can effectively parse data.

Declare a Table

Since we are doing a single-table design we really only need a single DynamoDB table with a single Global Secondary Index (GSI). We declare that by creating a new instance of Table:

Define entities

Now it's time to covert our entity chart into an actual TypeDORM entity, we do this by annotating our model with @Entity decorator, exported by @typedorm/common. Product and Employee entities are the only ones listed here, but you got the overall idea.

Product Entity

Employee Entity

You will need to define all entities in a similar way to the above examples.

Establishing a connection

Once we have all the entities in, the next thing is to initialize a DynamoDB connection. We do this by creating a connection instance:

If all of your entities are grouped under the same directory, you can also provide a relative path to entities option and TypeDORM will auto load all the entities from a given path.

Now since we have everything in place, we are ready to work with a single-table designed DynamoDB table in a type-safe way.

Querying records

Once everything is set up, it's will be super easy to perform all kinds of operations like Create, Delete, Put, Update, all in a type-safe way.

For example, let's have a look at what our queries will look like for the first two access patterns:

P.S. You can provide additional options to filter/log queries.

For the most up-to-date list of available options and entity have a look at API Spec.

Even though we have only covered querying items with TypeDORM here, there are many more use-cases where TypDORM proves its worth. Experiment with different TypeDORM APIs and tell us what you think in the comments.

More about TypeDORM

TypeDORM is committed to delivering new features every month until all the major document client APIs are covered, meanwhile if you want to access the feature before the release we also publish a beta version.

To find out more about what new features are on the road map, check out the TypeDORM public roadmap.

If there are any other features you would like to be included, open a new issue here.

Special thanks to Zak Barbuto for proofreading this article.