Explore how countries produce energy for MongoDB Atlas Hackathon

Overview of My Submission

The climate change is an important topic we should all care to protect our planet. So, I wanted to create a simple web application to explore how countries produce energy, if they rely on renewable energy or fossil fuels.

To visit my web application click here! Similarly to the TV show Our Planet, I wanted to call my app Our Energy because it is a resource we all should be care of!

Below you can see the homepage:

You can look the amount of MegaWatts of energy produced by a country. To find the countries in the Mongo Database I used Atlas Search service. Below you can see the values for my country, Italy:

Moreover, from the homepage you can explore the top 5 countries producing energy using a particular renewable source. You can choose among solar, wind, waves and tidal, and hydro. Below there is a screenshot for the wind energy:

Data preparation

The original dataset was in .csv format. Firstly I removed many columns that were not necessary to my scope to reduce the size. Then I converted it to a JSON file which is easy to be uploaded to MongoDB Atlas.

Connecting to Atlas Cloud is as easy as connecting to any local database thank you to this official guide. To upload the dataset onto Mongo Cloud I used literally one function: insert_many(). You can explore the full code here.

Atlas Search

To search data by country I had to match the input string with the country_long field of the dataset. I used the Atlas UI to create an index with static mapping for this field. You can refer to this guide to create one.

Aggregation pipeline

The dataset consists of many power plants located in many countries. To retrieve a summary information for a certain country I exploited the power aggregation pipeline feature of MongoDB. Look at this code snippet that I use to find the 5 countries producing the biggest quantity of energy from a particular fuel (oil, gas, wind, solar, etc..., renewable energy sources are considered fuels as well):

result = collection.aggregate([
      {
         '$match': {
            'primary_fuel': fuel
         }
      },
      {
         # Accumulate capacity by fuel type
         '$group':
         {
            '_id': '$country_long',
            'totCapacity': {
               '$sum': {'$toDecimal': '$capacity_mw'}
            }
         }
      },
      {
         '$sort': {'totCapacity': -1}
      },
      {
         '$limit': 5
      }
   ])

A pipeline is made of many stages that performs different operations to extract the data we want from the database. The output of a stage is the input of the next stage.

The first step find all the entries whose field 'primary_fuel' matches the string fuel:

{
    '$match': {
        'primary_fuel': fuel
    }
},

This step groups the output of the previous step by country (using the field country_long, the country name) and accumulate the energy production of a power plant. Up to now, we know the amount of wind energy produced by each country.

{
     '$group':
     {
        '_id': '$country_long',
        'totCapacity': {
           '$sum': {'$toDecimal': '$capacity_mw'}
        }
     }
},

We sort the output in descending order

{
    '$sort': {'totCapacity': -1}
},

Eventually, we limit the output to only 5 results

{
     '$limit': 5
}

And in the web app we can see the Top 5 countries producing that kind of energy. For instance, if you want to discover who are the 5 biggest producer of energy from waves and tydal are:

Conclusion

It was exciting to produce a simple web application from scratch to production using Python and MongoDB.

If you want to contribute to my project or propose some improvements, visit the GitHub repository.

Submission Category:

Choose Your Own Adventure

GitHub repository

andregri / OurEnergy

Climate Care is Flask application made for the MongoDB Atlas Hackathon 2021

Climate Care :earth:

How to contribute

Preparation of the dataset on MongoDB

Environment variables:

export MONGODB_URI='<your connection string to MongoDB'
export GPPDB_PATH='path/to/dataset/csvfile' # Do not write the extension '.csv'

For example my GPPDB_PATH env variable for the file ./data/global_power_plant_database.csv is:

export GPPDB_PATH='./data/global_power_plant_database'

Run Flask app

FLASK_ENV=development FLASK_DEBUG=true FLASK_APP=app flask run

Deploy to production

Create a .env file with the following variables:

export MONGODB_URI='<your mongo uri>'
export SECRET_KEY='<your secret key>'

You can generate a secret key using python:

python -c 'import secrets; print(secrets.token_hex())'

'<your secret key>'

Use waitress as production server:

waitress-serve --call 'app:create_app'

View on GitHub

Resources

Global Power Plant Database

Categories:

Python Mongodb

Tags:

Atlashackathon Python Flask Mongodb

NodeJS vs Python

App Development Practices To Follow For Your Business In 2021

Creating a Podcast of book summaries and articles in PDF in Portuguese with AI in Python!

Getting started with Docker & Flask.

21 must-read books for kids, beginner, advanced & interview with PDF 🐍

Weekly Challenge 140

Logistic Regression

OOP in Python