Build a web traffic monitor with Django and IPStack

In today's online world, it is important to know where your website traffic comes from in order to help website owners understand their visitors better. In this tutorial, we build a very basic traffic monitor, the traffic monitor will display details about visitors on the website. For every visitor, there will multiple columns such as time of visit, continent, country and city. This article was originally posted on my blog, The Pixel District.

Getting started with IPStack API

The documentation outlines in detail the API features, available options and integration guides in different programming languages.

Get a free API access key to proceed using this API.

Sample API Response

IPStack API responses come with comprehensive location-related data, currency-related data, timezone-related data, connection-related data and security-related data. Have a look at the example below.

Please note: For illustration purposes we have included both Hostname Lookup and the Security Module in the above API response. Please refer to the sections Hostname Lookup and Security Module for details.

{
  "ip": "134.201.250.155",
  "hostname": "134.201.250.155",
  "type": "ipv4",
  "continent_code": "NA",
  "continent_name": "North America",
  "country_code": "US",
  "country_name": "United States",
  "region_code": "CA",
  "region_name": "California",
  "city": "Los Angeles",
  "zip": "90013",
  "latitude": 34.0453,
  "longitude": -118.2413,
  "location": {
    "geoname_id": 5368361,
    "capital": "Washington D.C.",
    "languages": [
        {
          "code": "en",
          "name": "English",
          "native": "English"
        }
    ],
    "country_flag": "https://assets.ipstack.com/images/assets/flags_svg/us.svg",
    "country_flag_emoji": "🇺🇸",
    "country_flag_emoji_unicode": "U+1F1FA U+1F1F8",
    "calling_code": "1",
    "is_eu": false
  },
  "time_zone": {
    "id": "America/Los_Angeles",
    "current_time": "2018-03-29T07:35:08-07:00",
    "gmt_offset": -25200,
    "code": "PDT",
    "is_daylight_saving": true
  },
  "currency": {
    "code": "USD",
    "name": "US Dollar",
    "plural": "US dollars",
    "symbol": "$",
    "symbol_native": "$"
  },
  "connection": {
    "asn": 25876,
    "isp": "Los Angeles Department of Water & Power"
  },
  "security": {
    "is_proxy": false,
    "proxy_type": null,
    "is_crawler": false,
    "crawler_name": null,
    "crawler_type": null,
    "is_tor": false,
    "threat_level": "low",
    "threat_types": null
  }
}

Prerequisites

Following Python best practices, we will create a virtual environment for our project, and install the required packages.

First, create the project directory.

$ mkdir djangoapp
$ cd djangoapp

Now, create a virtual environment and install the required packages.

For macOS and Unix systems:

$ python3 -m venv myenv
$ source myenv/bin/activate
(myenv) $ pip install django requests

For Windows:

$ python3 -m venv myenv
$ myenv\Scripts\activate
(myenv) $ pip install django requests

Setting Up Your Django Application

First, navigate to the directory djangoapp we created and establish a Django project.

(myenv) $ django-admin startproject mainapp

This will auto-generate some files for your project skeleton:

mainapp/
    manage.py
    mainapp/
        __init__.py
        settings.py
        urls.py
        asgi.py
        wsgi.py

Now, navigate to the directory you just created (make sure you are in the same directory as manage.py) and create your app directory.

(myenv) $ python manage.py startapp monitor

This will create the following:

monitor/
    __init__.py
    admin.py
    apps.py
    migrations/
        __init__.py
    models.py
    tests.py
    views.py

On the mainapp/settings.py file, look for the following line and add the app we just created above.

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'monitor', #new line
]

Ensure you are in the monitor directory then create a new directory called templates and a new file called urls.py. Your directory structure of monitor application should look like this

monitor/
    __init__.py
    admin.py
    apps.py
    migrations/
    templates/
        __init__.py
    models.py
    tests.py
    urls.py
    views.py

Ensure your mainapp/urls.py file, add our monitor app URL to include the URLs we shall create next on the monitor app:

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    #path('admin/', admin.site.urls),
    path('', include('monitor.urls')),#monitor app url
]

Now, on the monitor/urls.py file, add our website there:

from django.urls import path, include
from . import views

urlpatterns = [
    path('', views.home, name="home"),
    path('monitor/', views.monitor, name="monitor"), #system monitor view to be created next
]

Create our database models which we shall use to store our website visitors information. On the monitor/models.py file:

from django.db import models

# Create your models here.
class Monitor(models.Model):
    continent = models.CharField(max_length=50, blank=True, null=True)
    country = models.CharField(max_length=50, blank=True, null=True)
    city = models.CharField(max_length=50, blank=True, null=True)
    capital = models.CharField(max_length=50, blank=True, null=True)
    datetime = models.DateField(max_length=50, blank=True, null=True)
    ip = models.CharField(max_length=50, blank=True, null=True)

    def __str__(self):
        return self.ip

Now, on the monitor/views.py file, add the lines below:

from django.shortcuts import render, redirect
import requests
import time
import psutil
import os
from datetime import datetime
from .models import Monitor
from urllib.parse import urlparse
from django.core.paginator import Paginator

#traffic monitor
def traffic_monitor(request):
    dataSaved = Monitor.objects.all().order_by('-datetime')
    # Getting loadover15 minutes 
    load1, load5, load15 = psutil.getloadavg()
    cpu_usage = int((load15/os.cpu_count()) * 100)
    ram_usage = int(psutil.virtual_memory()[2])
    p = Paginator(dataSaved, 100)
    #shows number of items in page
    totalSiteVisits = (p.count)
    #find unique page viewers & Duration
    pageNum = request.GET.get('page', 1)
    page1 = p.page(pageNum)
    #unique page viewers
    a = Monitor.objects.order_by().values('ip').distinct()
    pp = Paginator(a, 10)
    #shows number of items in page
    unique = (pp.count)
    #update time
    now = datetime.now()
    data = {
        "now":now,
        "unique":unique,
        "totalSiteVisits":totalSiteVisits,
        "cpu_usage": cpu_usage,
        "ram_usage": ram_usage,
        "dataSaved": page1,
    }
    return render(request, 'traffic_monitor.html', data)
#home page
def home(request):
    x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
    if x_forwarded_for:
        ip = x_forwarded_for.split(',')[0]
    else:
        ip = request.META.get('REMOTE_ADDR')
    response = requests.get('http://api.ipstack.com/'+ip+'?access_key=GETYOURACCESSKEY') #change from HTTP to HTTPS on the IPSTACK API if you have a premium account
    rawData = response.json()
    print(rawData) # print this out to look at the response
    continent = rawData['continent_name']
    country = rawData['country_name']
    capital = rawData['city']
    city = rawData['location']['capital']
    now = datetime.now()
    datetimenow = now.strftime("%Y-%m-%d %H:%M:%S")
    saveNow = Monitor(
        continent=continent,
        country=country,
        capital=capital,
        city=city,
        datetime=datetimenow,
        ip=ip
    )
    saveNow.save()
    return render(request, 'home.html')

In the code above on the home view, the code starts by getting the users IP address. On getting the visitors IP, we then make a request to IPStack with the IP address above appended to the request and store the results to the database. If you wish to collect more information, just add them on the models and add the variable like so above with the respective data you are collecting eg country.

For the traffic monitor view, the code starts buy getting all data on the Monitor table and renders the data on the HTML page which we shall create below. The code also collects the cpu and ram usage of the system. Let us now proceed to rendering the data above on our webpage.

On the monitor/templates folder, create traffic_monitor.html and home.html web page and add the lines below:

monitor/templates/traffic_monitor.html file:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <meta name="description" content="">
    <meta name="author" content="">
    <title>ALienx &#8211; Connections</title>
    <link rel="canonical" href="https://getbootstrap.com/docs/4.0/examples/album/">
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css">
    <!-- Bootstrap core CSS -->
    <link href="https://getbootstrap.com/docs/4.0/dist/css/bootstrap.min.css" rel="stylesheet">

    <!-- Custom styles for this template -->
    <link href="https://getbootstrap.com/docs/4.0/examples/album/album.css" rel="stylesheet">
  </head>

  <body>

    <header>
      <div class="collapse bg-dark" id="navbarHeader">
        <div class="container">
          <div class="row">
            <div class="col-sm-8 col-md-7 py-4">
              <h4 class="text-white">About</h4>
              <p class="text-muted">Add some information about the album below, the author, or any other background context. Make it a few sentences long so folks can pick up some informative tidbits. Then, link them off to some social networking sites or contact information.</p>
            </div>
            <div class="col-sm-4 offset-md-1 py-4">
              <h4 class="text-white">Contact</h4>
              <ul class="list-unstyled">
                <li><a href="#" class="text-white">Follow on Twitter</a></li>
                <li><a href="#" class="text-white">Like on Facebook</a></li>
                <li><a href="#" class="text-white">Email me</a></li>
              </ul>
            </div>
          </div>
        </div>
      </div>

      <div class="navbar">
        <div class="container d-flex justify-content-between">
          <a href="#" class="navbar-brand d-flex align-items-center">
            <i class="fa fa-desktop" aria-hidden="true"></i> <strong> Site Visitors</strong>
          </a>
          <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarHeader" aria-controls="navbarHeader" aria-expanded="false" aria-label="Toggle navigation">
            <span class="navbar-toggler-icon"></span>
          </button>
        </div>
      </div>
    </header>

    <main role="main">

      <section class="jumbotron text-center">
        <div class="container">
          <h1 class="jumbotron-heading"><i class="fa fa-desktop" aria-hidden="true"></i> Site Visitors and Resources Monitor</h1>
          <p class="lead text-muted">Contact <a href="#">developer</a> if CPU & RAM USAGE exceeds 80%</p>

        </div>
      </section>


      <div class="album py-7 bg-light">
            <div class="content-wrapper">
                <div class="container-fluid">

                  <ol class="breadcrumb">
                    <li class="breadcrumb-item">
                      <a href="#">Dashboard</a>
                    </li>
                    <li class="breadcrumb-item active">Site Traffic & Resources Monitor</li>
                  </ol>
                  <br>

                  <!-- Icon Cards-->
                  <div class="container">
                    <div class="row">
                        <div class="col-sm-3 block">
                            <div class="circle a">


                                <p><b>CPU Usage</b>
                                    <br>
                                    <b>{{ cpu_usage }}%</b>
                                </p>

                            </div>
                        </div>
                        <div class="col-sm-3 block">
                            <div class="circle b">

                                <p><b>RAM Usage</b> 
                                    <br>
                                    <b>{{ ram_usage }}%</b>
                                </p>

                            </div>
                        </div>

                        <div class="col-sm-3 block">
                            <div class="circle c">


                                <p><b>Total Site Visits</b> 
                                    <br>
                                    {{ totalSiteVisits }}
                                </p>

                            </div>
                        </div>
                        <div class="col-sm-3 block">
                            <div class="circle d">
                                <p><b>Unique Page Views</b>
                                    <br>
                                    {{ unique }}
                                </p>
                            </div>
                        </div>


                    </div>
                </div>
                <style>
                  .block {
                                    bottom: 30px;
    text-align: center;
    vertical-align: middle;
}
.circle {
    font-size: 15px;
    background: white;
    border-radius: 200px;
    color: white;
    height: 170px;
    font-weight: bolder;
    width: 170px;
    display: table;
    margin: 20px auto;
}
.circle p {
    vertical-align: middle;
    display: table-cell;
}
                    .a {
                        background-color: #39c0dc;
                    }
                    .b {
                        background-color: darkorange;
                    }
                    .c {
                        background-color: #5c6dfd;
                    }
                    .d {
                        background-color: #f14c6e;
                    }
                    .g {
                        background-color: greenyellow;
                    }
                    .f{
                        background-color: fuchsia;
                    }
                </style>

                  <!-- Example contextTables Card-->
                  <div class="card mb-3">
                    <div class="card-header">
                      <b style="color: #0664c9;"><i class="fa fa-table"></i> User Sessions</b></div>
                    <div class="card-body">
                      <div class="table-responsive">
                        <table class="table table-bordered" id="contextTable" width="100%" cellspacing="0">
                          <thead>
                            <tr>
                              <th>Time</th>
                              <th>IP Address</th>
                              <th>Continent</th>
                              <th>Country</th>
                              <th>City</th>
                            </tr>
                          </thead>
                          <tfoot>
                            <tr>
                              <th>Time</th>
                              <th>IP Address</th>
                              <th>Continent</th>
                              <th>Country</th>
                              <th>City</th>
                            </tr>
                          </tfoot>
                          <tbody id="customer-table">
                            {% for data in dataSaved.object_list %}
                            <tr>
                                <td>{{ data.datetime }}</td>            
                                <td>{{ data.ip }}</td>
                                <td>{{ data.continent }}</td>
                                <td>{{ data.country }}</td>
                                <td>{{ data.city }}</td>
                            </tr>
                            {% endfor %}
                          </tbody>
                        </table>
                      </div>
                      <br>
                      <nav aria-label="...">
                        {% if dataSaved.has_other_pages %}
                        <ul class="pagination justify-content-center">
                            {% if dataSaved.has_previous %}
                          <li class="page-item">
                            <a class="page-link" href="?page={{ dataSaved.previous_page_number }}" tabindex="-1">Previous</a>
                          </li>
                          {% endif %}

                          {% for i in dataSaved.paginator.page_range %}
                             {% if dataSaved.number == i %}
                          <li class="page-item active">
                              <a class="page-link" href="#">{{ i }}<span class="sr-only">(current)</span></a>
                           </li>
                          {% endif %}
                          {% endfor %} 
                          <li class="page-item">
                            {% if dataSaved.has_next %}
                            <a class="page-link" href="?page={{ dataSaved.next_page_number }}">Next</a>
                            {% endif %}
                          </li>
                        </ul>
                        {% endif %}
                    </nav>
                    </div>
                    <div class="card-footer small text-muted">Updated at <span id="session-update-time">{{ now }}</span></div>
                  </div>
                </div>

      </div>

    </main>

    <footer class="text-muted">
      <div class="container">
        <p class="float-right">
          <a href="#">Back to top</a>
        </p>
        <p>Need help? <a href="#">Contact site developer</a> or the <a href="#">site admin</a>.</p>
      </div>
    </footer>

    <!-- Bootstrap core JavaScript
    ================================================== -->
    <!-- Placed at the end of the document so the pages load faster -->
    <script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script>
    <script>window.jQuery || document.write('<script src="../../assets/js/vendor/jquery-slim.min.js"><\/script>')</script>
    <script src="https://getbootstrap.com/docs/4.0/assets/js/vendor/popper.min.js"></script>
    <script src="https://getbootstrap.com/docs/4.0/dist/js/bootstrap.min.js"></script>
    <script src="https://getbootstrap.com/docs/4.0/assets/js/vendor/holder.min.js"></script>
  </body>
</html>

On the monitor/templates/home.html file:

<!doctype html>
<html lang="en">
  <head>
    <!-- Required meta tags -->
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">

    <!-- Bootstrap CSS -->
    <link href="https://cdn.jsdelivr.net/npm/[email protected]/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-EVSTQN3/azprG1Anm3QDgpJLIm9Nao0Yz1ztcQTwFspd3yD65VohhpuuCOmLASjC" crossorigin="anonymous">

    <title>Hello, world!</title>
  </head>
  <body>
    <h1>Hello, world!</h1>

    <!-- Optional JavaScript; choose one of the two! -->

    <!-- Option 1: Bootstrap Bundle with Popper -->
    <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/js/bootstrap.bundle.min.js" integrity="sha384-MrcW6ZMFYlzcLA8Nl+NtUVF0sA7MsXsP1UyJoMp4YLEuNSfAP+JcXn/tWtIaxVXM" crossorigin="anonymous"></script>

    <!-- Option 2: Separate Popper and Bootstrap JS -->
    <!--
    <script src="https://cdn.jsdelivr.net/npm/@popperjs/[email protected]/dist/umd/popper.min.js" integrity="sha384-IQsoLXl5PILFhosVNubq5LC7Qb9DXgDA9i+tQ8Zj3iwWAwPtgFTxbJ8NT4GN1R8p" crossorigin="anonymous"></script>
    <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/js/bootstrap.min.js" integrity="sha384-cVKIPhGWiC2Al4u+LWgxfKTRIcfu0JTxR+EQDz/bgldoEyl4H0zUF0QKbrJ0EcQF" crossorigin="anonymous"></script>
    -->
  </body>
</html>

Testing if it Works!

Migrate, then run the server by running the commands below then navigating to the respective port:

(myenv) $ python manage.py makemigrations
(myenv) $ python manage.py migrate
(myenv) $ python manage.py runserver

Below is a snapshot of my implementation:

Thanks for reading!

I hope you enjoyed this blog as much as i did, incase of any question, feel free to ask on the comment section below. Note, you can play with this data as you like, like adding a date range filter to show visitors by date or showing the visitors by country.