TypeScript Monorepos with Yarn

In a past article in this monorepo series, we’ve discussed setting up CI/CD for JavaScript packages using Yarn Workspaces. This time, we will figure out the same for TypeScript. We’ll learn how to build and test TypeScript projects at scale with Yarn and Semaphore.

At the end of the tutorial, we’re going to have a continuous integration pipeline that builds only the code that changes.

Uniting Yarn and TypeScript

TypeScript extends JavaScript by adding everything it was missing: types, stricter checks, and a deeper IDE integration. TypeScript code is easier to read and debug, helping us write more robust code.

Compared to JavaScript, however, TypeScript saddles us with one more layer of complexity: code must be compiled first before it can be executed or used as a dependency. For instance, say we have two packages, “child” and “parent”. The child is easy to compile since it has no other dependencies:

$ npm install -g typescript
$ cd child
$ tsc

Yet, when we try to do the same with the parent that depends on it, we get an error since the local dependency is not found.

$ cd parent
$ tsc

src/index.ts:1:20 - error TS2307: Cannot find module 'child' or its corresponding type declarations.

1 import { moduleName } from 'child';

Found 1 error.

Without specialized tooling, we have to build and link packages by hand while preserving the correct build order. Yarn Workspaces already solves problems like these in JavaScript. Fortunately, with a bit of tweaking, we can extend it to TypeScript.

Setting up Workspaces in Yarn

Fork and clone the following GitHub repository, which has a couple of packages to experiment with.

TomFern / semaphore-demo-monorepo-typescript

Yarn Workspaces + TypeScript monorepo

Monorepo TypeScript Demo

A hello world type monorepo demo for TypeScript and Yarn Workspaces.

Before Yarn Workspaces

Withouth workspaces, you have to build and link each project separately. For instance:

$ npm install -g typescript
$ cd shared
$ tsc

This builds the shared package. But when we try to do the same with sayhi, we get an error since the local dependency is not found:

$ cd ..
$ cd sayhi
$ tsc

src/sayhi.ts:1:20 - error TS2307: Cannot find module 'shared' or its corresponding type declarations.

1 import { hi } from 'shared';
                     ~~~~~~~~
Found 1 error.

Yarn workspaces help us link projects while keeping each in its own separate folder.

Configure Yarn Workspaces and TypeScript

To configure workspaces, first install the latest Yarn version:

$ yarn set version berry

This creates .yarn and .yarnrc.yml

Initialize workspaces, this creates the packages folder…

View on GitHub

We’re going to build a TypeScript monorepo made of two small packages:

shared: contains a few utility functions.

sayhi: the main package provides a “hello, world” program.

Let’s get going. To configure workspaces, switch to the latest Yarn version:

$ yarn set version berry

Yarn installs on .yarn/releases and can be safely checked in the repo.

Then, initialize workspaces. This creates the packages folder, a .gitignore, and the package.json and yarn.lock.

$ yarn init -w

You can add root-level dependencies to build all projects at once with:

$ yarn add -D typescript

Optionally, you may want to install the TypeScript plugin, which handles types for you. The foreach plugin is also convenient for running commands in many packages at the same time.

Next, move the code into packages.

$ git mv sayhi shared packages/

To confirm that workspaces have been detected, run:

$ yarn workspaces list --json

{"location":".","name":"semaphore-demo-monorepo-typescript"}
{"location":"packages/sayhi","name":"sayhi"}
{"location":"packages/shared","name":"shared"}

If this were a JavaScript monorepo, we would be finished. The following section introduces TypeScript builds into the mix.

TypeScript Workspaces

Our demo packages already come with a working tsconfig.json, albeit a straightforward one. Yet, we haven’t done anything to link them up — thus far, they have been completely isolated and don’t reference each other.

We can link TypeScript packages using project references. This feature, which was introduced on TypeScript 3.0, allows us to break an application into small pieces and build them piecemeal.

First, we need a root-level tsconfig.json with the following contents:

{
  "exclude": [
    "packages/**/tests/**",
    "packages/**/dist/**"
  ],
  "references": [
    {
      "path": "./packages/shared"
    },
    {
      "path": "./packages/sayhi"
    }
  ]
}

As you can see, we have one path item per package in the repo. The paths must point to folders containing package-specific tsconfig.json.

The referenced packages also need to have the composite option enabled. Add this line into packages/shared/tsconfig.json and packages/sayhi/tsconfig.json.

{
  "compilerOptions": {
     "composite": true

     . . .

  }
}

Packages that depend on other ones within the monorepo will need an extra reference. Add a references instruction in packages/sayhi/tsconfig.json (the parent package). The lines go at the top level of the file, outside compilerOptions.

{
  "references": [
    {
      "path": "../shared"
    }
  ]

  . . .

}

Install and build the combined dependencies with yarn install. Since we’re using the latest release of Yarn, it will generate a zero install file that can be checked into the repository.

Now that the configuration is ready, we need to run tsc to build everything for the first time.

$ yarn tsc --build --force

You also can build each project separately with:

$ yarn workspace shared build
$ yarn workspace sayhi build

And you can try running the main program.

$ yarn workspace sayhi node dist/src/sayhi.js
Hi, World

At the end of this section, the monorepo structure should look like this:

├── package.json
├── packages
│   ├── sayhi
│   │   ├── dist/
│   │   ├── src/
│   │   ├── package.json
│   │   └── tsconfig.json
│   └── shared
│       ├── dist/
│       ├── src/
│       ├── package.json
│       └── tsconfig.json
├── tsconfig.json
└── yarn.lock

That’s it, Yarn and TypeScript work together. Commit everything into the repository, so we’re ready to begin the next phase: automating testing with CI/CD.

$ git add -A
$ git commit -m "Set up TS and Yarn"
$ git push origin master

Building and testing with Semaphore

The demo includes a ready-to-work, change-based pipeline in the final branch. But we’ll learn faster by creating it from zero.

If you’ve never used Semaphore before, check out the getting started guide. Once you have added the forked demo repository into Semaphore, come back, and we’ll finish the setup.

We’ll start from scratch and use the starter single job template. Select “Single Job” and click on Customize.

The Workflow Builder opens to let you configure the pipeline.

Build Stage

We’ll set up a TypeScript build stage. The build stage compiles the code into JavaScript and runs tests such as linting and unit testing.

The first block will build the shared package. Add the following commands to the job.

sem-version node 14.17.3
checkout
yarn workspace shared build

The details are covered in-depth in the starter guide. But in a few words, sem-version switches the active version of Node (so we have version consistency), while checkout clones the repository into the CI machine.

Scroll down the right pane until you find Skip/Run conditions. Select “Run this block when conditions are met”. In the When? field type:

change_in('/packages/shared/')

The change_in function is an integral part of monorepo workflows. It scans the Git history to find which files have recently changed. In this case, we’re essentially asking Semaphore to skip the block if no files in the /packages/shared folders have changed.

Create a new block for testing. We’ll use it to run ESLint and unit tests with Jest.

In the prologue, type:

sem-version node 14.17.3
checkout

Create two jobs in the block:

Lint with the command: yarn workspace shared lint

Unit testing: yarn workspace shared test

Again, set the Skip/Run conditions and put the same condition as before.

Managing dependencies

We’ll repeat the steps for the sayhi package. Here, we only need to replace any instance of yarn workspace shared <command> with yarn workspace sayhi <command>.

Now, create a building block and uncheck the Dependencies section. Removing block dependencies in the pipeline makes blocks run in parallel.

Next, set the Skip/Run Condition on the new block to: change_in('/packages/sayhi/').

To finish, add a test block with a lint job and a unit test job. Since this package depends on shared, we can add a block-level dependency at this point. When done, you should have a total of four blocks.

The Skip/Run Condition, in this case, is different because the test block should run if either sayhi or shared change. Thus, we must supply an array instead of a single path in order to let change_in handle all cases correctly:

change_in(['/packages/sayhi', '/packages/shared'])

Running the Workflow

Click on Run the Workflow and then Start.

The first time the pipeline runs, all blocks will be executed.

On successive runs, only relevant blocks will start; the rest will be skipped, speeding up the pipeline considerably, especially if we’re dealing with tens or hundreds of packages in the repo.