Generating Twitter Lists with Python

Generating Twitter Lists with Python and Tweepy

The finished project can be seen here

His Twitter activity keeps me entertained and informed, spanning from discussions about air quality significantly impacting life expectancy to interesting discussions he has with his kids.

The air pollution discussion this morning motivated me to want to see his feed and all of the interesting things a man like Paul sees. I started to follow the accounts he follows and after about 30 or so clicks I decided to investigate Twitter's API for a way to automate this.

This is where I discovered Twitter Lists.

Twitter Lists are a great way to create custom feeds

Viewing a List timeline will show you a stream of Tweets from only the accounts on that List.

Twitter Lists are the perfect solution. It gives me the ability to create custom feeds.

If I feel like delving into Paul Grahams feed, it's a click away. Same with Elon Musk or any of the other influential users on Twitter.

Using Tweepy to automatically generate our Twitter Lists

Tweepy is an easy-to-use Python library for accessing the Twitter API.

Tweepy provides built in methods to easily interact with the Twitter API.

In this project, I use the following from Tweepy:

.get_users_following()

.add_list_members

Authenticating with Twitter's API using Tweepy

This tutorial will assume you already have a Twitter developer account. If you do not, I went into detail here on how to generate the proper authentication keys needed to access Twitter's API.

Tweepy provides the helper function .OAuthHandler that requires you to pass it your consumer key and consumer secret initializing the auth object, you then must call its method .set_access_token() which requires your access token and access token secret given to you when creating your developer account and generating your app.

import tweepy

auth = tweepy.OAuthHandler(os.getenv("consumer_key"), os.getenv("consumer_secret"))
auth.set_access_token(os.getenv("access_token"), os.getenv("access_token_secret"))

Initializing Tweepy's API and Client objects

We can now utilize our auth variable to create API and Client objects using the following code:

api = tweepy.API(auth)
client = tweepy.Client(
    bearer_token=os.getenv("bearer_token"),
    consumer_key=os.getenv("consumer_key"),
    consumer_secret=os.getenv("consumer_secret"),
    access_token=os.getenv("access_token"),
    access_token_secret=os.getenv("access_token_secret"),
    wait_on_rate_limit=True,
)

With these objects, we can now access every Twitter endpoint.

Creating a List using Tweepy's API

Tweepy methods used:

.create_list(): Takes three parameters: name, mode, and description. Name and description are self-explanatory and mode can either be "public" or "private" to define the visibility status of the List.

The response from a successfully List creation returns the List data. I will be using this list later on, so I decided to extract it's id into the variable list_id.

list_name = "Paul Grahams's Feed"
list_description = "A list of everyone Paul Graham follows"

twitter_list = api.create_list(name=list_name, description=list_description)
list_id = twitter_list._json["id"]

Getting a User and the accounts they follow

Tweepy methods used:

.get_user()

.get_users_following()

By passing client.get_user() a twitter handle - in this case, Paul Graham - I can get all of the public data Twitter provides regarding that user.

I then use that user's data with client.get_users_following() alongside max_results. The argument max_results defines how many user objects Twitter will pass back. In this case, I used the max of 1000. The default is 100.

twitter_handle = "paulg"
user = client.get_user(username=twitter_handle)

followers = client.get_users_following(id=user.data.id, max_results=1000)

Automatially adding users to the list

Tweepy methods used:

.add_list_members

Now that we have the List we want to populate and the followers we want populate it with, we use the api.add_list_members() method.

This method takes a list of up to 100 Twitter User ID's. The code snippet below chunks out follower data into sections of 100 Users and generates a list of ids to pass.

We are also passing this method the list_id that was saved when we created the list.

for i in range(0, len(followers.data), 100):
    ids = [follower["id"] for follower in followers.data[i : i + 100]]
    api.add_list_members(list_id=list_id, user_id=ids)

Things to keep in mind when working with Twitter's API

Twitter rate limits just about all of their endpoints. Here is Twitter's documentation on what the rate limits are. Keep this is mind when developing Twitter-based applications because you may find yourself with an odd error.

Thank you for reading! I hope you enjoyed it.