HTTP API Lean Workflow: from contract to documentation

How do we rely on open source software to achieve collaborative API design and produce our documentation?

TL:DR;

A contract-first approach enables high collaboration and removes overhead.

Versioning contracts with the code since they share the same lifecycle.

Stoplight.io DX is a powerful API IDE for OpenAPI.

Enriching the OpenAPI contract with markdown reduces the cost of maintenance.

Rely on Redoc to expose the contract as documentation with a standard UX & UI.

Context

At Golem.ai, we are maintaining a platform and several services following the SOA concepts. Our protocol between these services relies on REST API level 2.

These API are either:

consumed programmatically (internally and by our customers)

used by several web applications (addressing both our business analysts and customers)

Contract-first approach

There are only two hard things in Computer Science: cache invalidation and naming things.
-- Phil Karlton

Designing API is a tough job. It's a highly structuring action where we define the boundary of "something". These boundaries exposes our knowledge and understanding about this "something": a service, a micro-service, a bounded context.

At Golem.ai, API design is not only about decision, responsibility & technicality. We advocate continuous collaboration to guarantee the quality & value of our APIs.

Adopting the dogfooding philosophy, we consume our own API. For every new feature, fix or improvement that impacts an endpoint, we follow this 3-step process:

Discovery: provider & consumer (example: backend & frontend) engineers sit together and discuss their technical challenges. Exploring code, questioning the shape (specification) and referring to the REST approach.

Agreement: ultimately, engineers will come out with a pragmatic and relevant solution. They update with Stoplight.io and version the OpenAPI contract with git. Usually, this contract has been drafted during the discovery by both parties.

Implementation: TDD is cheaper, our OpenAPI contracts include examples that provide a free mock for the client and free test assertion & server-side validation.

Adjustment: nothing is perfect and near perfection takes time and iteration. Developers can adjust an endpoint but they have to go again through steps 1, 2 & 3. Hopefully, adjustments are usually small and the whole process is short, 5 minutes tops: a slack message, a meeting using screen sharing and a quick discussion will lead to a git commit updating the OpenAPI contract.

Contract as Documentation

We needed more than a contract or a reference to on-board, assist, and guide developers using our API.

Documentation needs maintenance and its lifecycle is tightly bound to the implementation.

After assessing solutions on the market, we realised that most of the option would either involve changing our workflow or adding complexity to sync the documentation.

We finally decided to use our OpenAPI contract as documentation with Redoc (read more below).

The final workflow is simple, involving only the OpenAPI contract, versioned with the code. Our regular pipeline deploy the code and the documentation.

Side effects:

The code review validates everything: contract, code & documentation.

We didn't increase the complexity and the cost of our API.

Everything, from modelling to documenting, is tied to the lifecycle of the code: ideation, testing, release & deployment.

We ended up maximizing the value of every step of our API management, reducing the overhead for our developers, and removing conflict during integration. Collaboration versioned in an OpenAPI contract ensure predictable, meaningful, and enjoyable API development.

Tools: our API Stack

Stoplight.io

We use the desktop application, so we can locally load our contract from our git repository into the editor.

Next level DX: none of our developers were familiar with OpenAPI and they didn’t have to be with stoplight: WYSIWYG for discovery, linter, direct edition in yaml/json, preview... Engineers can focus on the "what" & "why" rather than the "how".

Not intrusive: the desktop app lets you define your own convention, standard, directory structure, and workflow. It adapts to your needs and your organization doesn't have to fit the tool.

Provides a mock server locally. We can even quickly prototype our implementation: our local docker container fetches generated data complying to the defined contract.

Redoc

As we maintained and documented several services, we needed to aggregate all our contracts/documentations under a single url (developers.golem.ai). This Fork by Volbrene solved this problem for us.

Once again, the stoplight studio desktop app is bliss, offering markdown edition & preview:

Git & Gitlab perk

API contracts are versioned in the same git repository as the related base code at /doc/OpenAPI.

Gitlab offers a nice perk based on a naming convention. Adding "openapi" in your contract file name enables a swagger-like preview in the gitlab code explorer.