Elixir background processes synchronization with GenServer and DynamicSupervisor

In this post we are gonna see how to handle background processes synchronization using powerful Elixir tools such as GenServer and DynamicSupervisor. Using these tools we can see real concurrent capabilities of Elixir.

For the example application, we are gonna sync GitHub Mentions for a specific user in a given GitHub repository. This sync should be made every X period of time to make sure we have Mentions up to date. We will be using GitHub's REST API to fetch all Pull Request comments on the given repository, and then filter these comments by Mentions for the given user.

We have to make sure we have Mentions up to date, that's why we are gonna use a GenServer to implement the background synchronization. Also, we need to spawn a new GenServer every time a new client wants to sync mentions, that's why we are gonna use DynamicSupervisor to spawn a new GenServer for each client.

Finally, in this example, we are gonna use Phoenix LiveView as the client that's going to request Mentions, display those Mentions, and receive updates when new Mentions are found. Since Phoenix also uses DynamicSupervisor to spawn a process every time a client requests a route (LiveView process in this case), many users can sync Mentions at the same time.

Let's go to the code...

Implementing the Mentions GenServer

defmodule GithubPrMentions.Mentions do
  use GenServer

  ...

  def child_spec(lv_pid) do
    %{
      id: {__MODULE__, lv_pid},
      start: {__MODULE__, :start_link, [lv_pid]},
      restart: :temporary
    }
  end

  def start_link(lv_pid) do
    GenServer.start_link(
      __MODULE__,
      lv_pid,
      name: via(lv_pid)
    )
  end

  def via(lv_pid) do
    {:via, Registry, {GithubPrMentions.Registry.Mentions, lv_pid}}
  end

  def init(_opts) do
    state = %{
      id: nil,
      interval: :timer.seconds(60),
      timer: nil,
      prs_curr_page: %{},
      base_url: nil,
      username: nil,
      token: nil
    }

    {:ok, state}
  end

  ...
end

Here lays the magic behind starting the GenServer. We need a child_spec/1 callback function to specify an identifier of the child process (the GenServer), the starting function, and the restarting policy. This function is going to be executed every time our DynamicSupervisor spawns a new GenServer.

We need also a start_link/1 callback to start the new GenServer linked to its parent supervisor (the DynamicSupervisor, in our case). This function is going to be executed by child_spec/1 passing it the arguments provided by this function as well. There, we name the GenServer via a Registry to have each GenServer process registered and identified by a LiveView PID, for later calling purposes. This is a handy way for identifying a GenServer process without having to remember its PID.

Then, to start syncing Mentions we need to dynamically create a new GenServer that will do the job. Each GenServer is going to be spawn every time the public API function get_mentions/4 is called because that function calls the DynamicSupervisor.start_child/2 function, creating a new GenServer process for the given LiveView PID:

def get_mentions(repo_url, username, token, lv_pid) do
    DynamicSupervisor.start_child(
      GithubPrMentions.Supervisor.Mentions,
      {__MODULE__, lv_pid}
    )

    GenServer.cast(via(lv_pid), {:set_initial_state, repo_url, username, token, lv_pid})
    GenServer.cast(via(lv_pid), {:fetch_pulls, 1})
end

That's all we need to start spawning GenServers dynamically. For the rest of the logic of fetching Pull Request comments, you can see the repo GitHub PR Mentions. But there are 2 important GenServer function callbacks and 1 private function that are a key part of the background processes synchronization: handle_cast({:set_initial_state, repo_url, username, token, lv_pid}, state), handle_info({:DOWN, ref, :process, pid, reason}, state) and schedule_refresh(state).

def handle_cast({:set_initial_state, repo_url, username, token, lv_pid}, state) do
    Process.monitor(lv_pid)

    initial_state = %{
      base_url: GitHub.get_base_url(repo_url),
      username: username,
      token: token,
      id: lv_pid
    }

    state = Map.merge(state, initial_state)

    {:noreply, schedule_refresh(state)}
end

This handle_cast callback perform actions that involves the other 2 functions. The first one starts a process monitor for the current LiveView PID (lv_pid). This process monitoring is going to invoke a handle_info callback when the current associated LiveView PID has died:

def handle_info({:DOWN, _ref, :process, _pid, _reason}, state) do
    {:stop, :normal, state}
end

With the help of this notification, we can then stop the current GenServer spawned specifically for the LiveView that has just died. That way, we make sure that we kill background processes that are no longer required.

The second key action is the one that invokes schedule_refresh(state), making sure that every 60 seconds we refresh mentions:

defp schedule_refresh(state) do
    pid = GenServer.whereis(via(state.id))

    %{state | timer: Process.send_after(pid, :refresh_mentions, state.interval)}
end

Finally, we also need a Registry, a DynamicSupervisor, and the Mentions GenServer in the list of the main app's Supervisor (supervision tree), to be treated as children and be securely restarted when an eventual crash happens:

defmodule GithubPrMentions.Application do
  ...
  def start(_type, _args) do
    children = [
      ...
      {Registry, [name: GithubPrMentions.Registry.Mentions, keys: :unique]},
      {DynamicSupervisor, [name: GithubPrMentions.Supervisor.Mentions, strategy: :one_for_one]},
      GithubPrMentions.Mentions
    ]

    ...
  end
  ...
end

Implementing the LiveView Client

For the LiveView client is just as simple as calling Mentions.get_mentions/4 when the LiveView is going to be mounted to start the background sync.

Once the background sync for Mentions is started, it's going to eventually send messages to its specific LiveView process. Having this in mind, we need to add a handle_info callback which is going to receive mentions that the GenServer sends and start displaying them.

defmodule GithubPrMentionsWeb.MentionsLive do
  use GithubPrMentionsWeb, :live_view

  alias GithubPrMentions.Mentions

  @impl true
  def mount(%{"repo" => repo, "username" => username}, %{"current_user" => current_user}, socket) do
    Mentions.get_mentions(repo, username, current_user.access_token, self())

    {:ok,
     socket
     |> assign(prs: [], username: username)
     |> put_flash(:info, "Searching mentions..."), temporary_assigns: [prs: []]}
  end

  @impl true
  def handle_info({:new_mentions, pr_number, mentions}, socket) do
    send_update(GithubPrMentionsWeb.ShowMentions, id: pr_number, mentions: mentions)

    {:noreply,
     socket
     |> assign(:prs, [pr_number])
     |> put_flash(:info, "Looking for more mentions...")}
  end
end

For the logic of displaying those Mentions, you can see the rest of the implementation in the repo GitHub PR Mentions, or I even recommend you to take a look at the Phoenix LiveView Comment and Reply article to have a better understanding of how this works (btw, it's a fantastic demonstration of LiveView's capabilities).

Happy Coding!

To see the full implementation you can visit the repo: