Understanding the Lambda Logs API

In one of my previous blog posts, I've talked about the AWS Lambda Runtime API. There we've learned how to create our custom runtimes and interact with the Runtime API.

This time, I will focus on the Lambda Logs API. We will be looking at how the Logs API fits in the grand scheme of various AWS Lambda APIs and how to use it.

Let's dive in!

What the Logs API is

In the context of AWS Lambda, the Logs API is a publisher of logs produced by various elements of the AWS Lambda execution environment. The API will push desired logs to a given destination given a subscriber - either an HTTP endpoint or a TCP address.

The AWS Lambda Logs API is usable only from within a registered extension. It does not interact with your code in any shape or form as a custom runtime would.

Here is an illustration depicting my mental model of how this API fits in the grand scheme of various AWS Lambda APIs (some parts were omitted for brevity).

Publisher / subscriber

As I eluded earlier, the relation between your Lambda extension code and the Lambda Logs API is that of a publisher and a subscriber.
It's your responsibility to spin up resources (most likely an HTTP server) that listen to incoming requests produced by the Lambda Logs API.

For more information about the subscription options that are available to you, please refer to the official AWS documentation.

Worth noting is the fact that the Lambda Logs API does not work locally. AFAIK no emulator exists that would allow you to test your implementation locally. This fact was a bit surprising to me as I could not find such information within the documentation. The AWS-provided Node.js and Go examples seem to be hinting at this through log messages.

The main use case

The primary use case that Lambda Logs API enables is relatively hassle-free instrumentation of your lambda functions. Most third-party observability vendors now offer a special Lambda extension that does just that.

Previous implementations included streaming CloudWatch logs contents to the provider for parsing or sending the metrics directly after your code finished its execution. Both of the approaches would result in additional AWS charges not related to the observability offering of your choice - not ideal.

For even more inspiration related to the Logs API, checkout the AWS-provided samples. Some examples include batching the logs and delivering them to S3 or a template to get you started.

Summary

The Lambda Logs API role can be greatly underestimated in the grand scheme of things. But, I find the functionality it provides is valuable for observability purposes, especially for third-party vendors.

I encourage you to give it a try!

You can find me on Twitter - @wm_matuszewski

Thank you for your time.