Exploring Redis II. Redis Stream.

Introduction

This is a continuation of my previous post about Redis. In this case, I'm going to cover briefly the use of a new Redis data type called Stream, well maybe not so new but very useful.

Prerequisites

Same as before

Streams, naive description

Have used logs in your applications? Yep, those messages that you write in console or in a file, to keep a track of the behavior of the application. Can be very useful when trying to know what happened in a certain time of the application life cycle. Well streams in Redis, behave like this, they are like a record of all the events that you decide to append on the stream. I'm not saying that you should stop using logs, and start using Redis for this. No, what I'm telling you is that it behaves very similar. In both cases you have a stream of data, that you want to access any time, and you could access multiple times and from multiple clients.

Well, in the sketch it seems that the consumers only can get the data sequentially, but actually they can request any range of data in the stream. It's like saying, "give me the date from 1 to 10", you can literally request a data in the stream being very specific about what you want to consume. After you consume the data, other consumers could request this same data, so a request for a data doesn't "pop" or "delete" it from the stream. Let's see this in action.

Setup

Almost same setup as in the previous tutorial, only difference you just need one instance of the redis-cli.

Adding elements to the stream

Let's split this syntax, that at first could be a little confusing. The first part is the XADD command, that allow you to append data to the stream. The twitt_feeds is the name of the stream, the other incoming character *, well this is a way to say to Redis that you don't want to specify an ID of the data by yourself, instead you are requesting to the server to assign this ID for you. And the last part date 01-07-21 username Gealber, represents the actual data that you want to store, this data comes in pairs, the first part in this case is the date 01-07-21, and the other one is username Gealber.

Now go and try it by yourself, you will notice that the server will return you the generated ID. This ID generated had the following format:

Generally you don't want to supply this by yourself, but you can, in case you want to 🤷‍♂️.

Querying the data

Let's say that you want all the data in the stream, how could we request that? Take into account that maybe this is not so smart, it depends on the size of your stream, which you could also request, we'll see how to do that too.

Mmmm... - and +, does these characters have a special meaning? Well in this particular case you are saying, "give me the data from the first data in the stream to the last one", so basically you are requesting all the data.

Now let's say you want to fetch the last 3 data in the stream,

In this case you are saying, "give me from the last data to the first data in the stream, but only 3 of them", so the "last three elements". Pretty cool, right?

You could also request from a specific time stamp, given the format of the IDs, remember <millisecondsTime>-<sequenceNumber>. You could request elements in a range of time, for example

These two long numbers represents the milliseconds on the ID, here you are requesting for data between 1625161017441 and 1625162015154. In a real application this is pretty useful, you could request info that happened in a range of time.

Requesting size of stream

There are cases when you may want to know the size of your stream, which could be pretty convenient in case you ask all the data in your stream and the stream is pretty large. Knowing its size beforehand, could make you give up on this idea of "I want it all", instead request in a smart way, but that's just me, if you want it all you could get it all. Anyway, here it is:

This should return you the number of elements in your stream.

Redis Stream in action, a good resource to look at

I recently discovered these series, Redis Stream in action, and is fascinating how this guy applies this simple idea into a practical app, definitely something to look at.

Conclusion

I won't elaborate more on this, because I think there's better documentation on this part that this article could supply. Really there's a lot of information about this feature of Redis, my purpose was to make you aware of it. Now go and implement some pretty cool application using this, so you won't forget about it. I will also provide you some documentation, that you could consume and learn from it. In the next part I will use Redis as a reliable queue, awesome, right? 😎.

Bibliography

Stream Intro

How to use Redis for real-time stream processing

How to use Redis Stream