28
loading...
Python API Client Using Magic Methods
Python "magic methods" allow for some very fun and powerful code.
Magic methods refers to all those special methods in Python classes that start and end with
__. The documentation for Special method names says:
A class can implement certain operations that are invoked by special syntax (such as arithmetic operations or subscripting and slicing) by defining methods with special names.
For instance the
__init__ method is called when a new instance of a class is instantiated. __str__ is a method that returns a string representation of the class. As stated in the docs, these are not methods that generally get invoked directly on the class. For example, you don't call my_object.__str__() but __str__ would be called by print(my_object).In this post, I'm going to demonstrate using the
__getattr__ and __call__ magic methods to build a dynamic API client.Let's say you're building a client for an API with endpoints like:
There are a lot of ways to build a client for this. One might look like:
import logging
import requests
class APIClient:
BASE_URL = "https://some-api.net"
def make_request(method, url, params=None, data=None):
response = requests.request(
method,
f"{self.BASE_URL}/{url}"
params=params,
json=data
)
response_body = {}
try:
response_body = response.json()
except ValueError:
log.warning("Unexpected Response '%s' from '%s'",
response.content, response.url)
return response_body
def get_user(self):
return self.make_request("GET", "user")
def get_user_permissions(self):
return self.make_request("GET", "user/permissions")
def create_article(self, data):
return self.make_request("POST", "article", data=data)
def get_article(self, id):
return self.make_request("GET", f"article/{id}")We would then use the client:
client = APIClient()
user = client.get_user()
permissions = client.get_user_permissions()
new_article = client.create_article({"some": "data"})
existing_article = client.get_article(123)And this is fine.
But what if the API is changing often? Or if there are dozens of endpoints and you don't know which you need to call? If later on you discover you need to call
GET https://some-api.net/user/roles or GET https://some-api.net/some_other_thing?some_param=foo, you'll need to go back to you client and add matching methods.It would be nice to have things a bit more dynamic, and one way to do that is with magic methods. Specifically, I'll be using:
__getattr__ which is "Called when the default attribute access fails with an AttributeError" (see the docs). This means that if you call my_object.some_missing_attr, then __getattr__ will get invoked with "some_missing_attr" as the name parameter.__call__ which is "Called when the instance is "called" as a function" (see the docs), eg. my_object()
Using these, we can build something like:
import logging
import requests
log = logging.getLogger(__name__)
class APIClient:
BASE_URL = "https://some-api.net"
def __init__(self, base_url=None):
self.base_url = base_url or BASE_URL
def __getattr__(self, name):
return APIClient(self._endpoint(name))
def _endpoint(self, endpoint=None):
if endpoint is None:
return self.base_url
return "/".join([self.base_url, endpoint])
def __call__(self, *args, **kwargs):
method = kwargs.pop("method", "GET")
object_id = kwargs.pop("id"))
data = kwargs.pop("data", None)
response = requests.request(
method,
self._endpoint(object_id),
params=kwargs,
json=data
)
response_body = {}
try:
response_body = response.json()
except ValueError:
log.warning("Unexpected Response '%s' from '%s'",
response.content, response.url)
return response_bodyThis allows us to call the API like so:
client = APIClient()
user = client.user()
permissions = client.user.permissions()
new_article = client.article(method="POST", data={"some": "data"})
existing_article = client.article(id=123)How does this work?
Well in the case of
client.user.permissions(), .user is an attribute and since the client doesn't have that attribute, it calls __getattr__ with name="user", which then returns an APIClient instance with the name appended to the endpoint URL. The same happens when .permissions is invoked on the APIClient instance that was returned by .user, in turn giving us another APIClient instance, now with a path of https://some-api.net/user/permissions. Finally this APIClient instance is called by (), which invokes the __call__ method. This method makes the actually HTTP call to the constructed URL based in any parameteres passed in, but defaults to a GET request.In the case of the article calls,
client.article(id=123) works much the same way, but the APIClient is called with an id parameter. This id gets appended to the url by the same internal _endpoint() method that __getattr__ uses, which results in a GET call to https://some-api.net/article/<id>.For
client.article(method="POST", data={"some": "data"}), we override the method and add a data payload to the POST.As you can see, the client itself can handle a large variety of API calls without any significant changes, allowing the utilizing application a lot of flexibility.
In the example of the newly required calls,
GET https://some-api.net/user/roles and GET https://some-api.net/some_other_thing?some_param=foo, no changes to the client are needed:roles = client.user.roles()
thing = client.some_other_things(some_param="foo")28