14
loading...
Just Another (Average) Full Stack Web Project
Back in 2019 when I first started delving into web development out of my own interest, I never had the opportunity (or rather, I was massively procrastinating) to start my own full stack web project. My past personal projects were always frontend-focused, using technologies like React or Bootstrap. I always found frontend development to be more enjoyable and captivating, as it allowed me to visualize my work and explore my creative side when designing interfaces.
But it was time to get out of my comfort zone.
I started to dabble in backend technologies - ranging from MongoDB to Express - during the summer of 2021. I used this as an opportunity to learn more technology frameworks and tools, mainly through YouTube tutorials from Traversy Media and Full Stack Junkie. It definitely brought me enjoyment, finally able to explore a whole different side to web development.
It was not until my final year in National University of Singapore (NUS), August 2021, that I had the opportunity to create my very own full stack web project.
As Computer Engineering undergraduates in NUS, we had to complete the infamous CG4002 Capstone module. Why did I say infamous? Well, because this was our first project where we had to utilise all the knowledge and skills that we picked up over the past 3 years of our degree program. Embedded systems? Yup. Socket programming? Of course. Machine learning? You bet. Vivado? Absolutely hate it, but yes.
I think you get the point.
It was a huge project with so many different components interacting with one another but luckily, we worked in groups, where each one of us played different roles, and became experts in our different fields. Of course, I picked the web development role. Why would I be writing this post if there wasn't any web development. For just a bit of context, the problem statement of our project was:
A wearable system that detects and coaches dance moves of a dance group
We were given total design freedom, albeit a few guidelines to follow. We could decide the direction of the project and how we wanted to execute it. For my web development component, also known as the dashboard, I had a few requirements to fulfil:
How will the dashboard store, analyze, and display the data from
the dancers in real-time? What analytics can you provide to coach the dancers? How can the dashboard handle many simultaneous users? How will you assess user feedback of the dashboard?
Simply put, dancers would have wearables (with sensors) strapped to their wrists. The data will be used to predict what dance move a dancer is performing. I had to create a dashboard that would display the result of the machine learning model and also provide real-time data visualisation and analytics while the dancers performed their moves. For now, let's not worry about the details of the data pipeline between the other components and mine.
It's finally time to delve into the web development part that you've all been waiting for.
I opted for the MERN tech stack for my project. Let me briefly go through the different components in MERN and why I chose them. It has to be said that the entire web application, both the client and server, was run locally ob my own device.
MongoDB
MongoDB is one of the most famous NoSQL database. Unlike traditional databases, MongoDB represents its information in a schemaless series of JSON-like documents, as opposed to the table and row format of relational systems such as PostgreSQL or MySQL. I picked it for 2 main reasons - (1) it allowed me to watch for real-time changes in the database using Change Streams and (2) I had a complicated love-hate relationship with SQL.
Express
Being a smart (and hard worker) I wanted to achieve my goals without having to write API functions or routes from scratch. Express comes to the rescue as a minimal web application framework that provides me with a myriad of HTTP utility methods and middleware, to ease the creation of APIs.
React
Of course I picked React. What else would I have chosen? React is obviously the most superior frontend tool. All jokes aside, I was most comfortable with React as compared to other technologies like Vue and Angular, due to my extensive exploration of the library back when I was developing my personal web portfolio. (sorry for the shameless plug.)
Node
Using Node was essentially a no brainer for me. Node would allow me to run JavaScript outside of the browser, hence, I am able to use it server side as well.
Using Node was essentially a no brainer for me. Node would allow me to run JavaScript outside of the browser, hence, I am able to use it server side as well.
Having complete design freedom was definitely exciting, but it can be challenging because it may seem directionless at times. Thankfully, I tried to develop a proper workflow so that I could achieve an extraordinary end result.
I first started to look for inspiration on this amazing website, Dribbble. They had TONS of user-submitted designs and animations which I then used to generate my own ideas.
Next, I sent out a simple preliminary survey to my group mates, friends and family. I would then use these (un)overwhelming responses to develop use cases, user stories, and feature lists, all centered around the users.
Last step was the exciting part, which involved the designing of the UI. Thankfully, I had paid for the entire Adobe Creative Suite which included Adobe Xd, which allowed me to create simple mockups.
Considering it was my first full stack web project, I was met with quite a few challenges, which was to be expected from a newbie like me. Although I had Teaching Assistants and Professors that could guide my process, I wanted to be as independent as possible. This meant spending countless of sleepless nights debugging, re-designing and scouring the depths of StackOverflow. Here are just a few challenges I faced during the development of the application and I how I tried to overcome them or simply just ignore them (ignorance is bliss).
MongoDB provides a cloud-based solution, MongoDB Atlas, which handles all the complexity of deploying, managing, and handling of deployments on a cloud service provider of our choice. All I thought I had to do was to just set up the endpoints on my server that would connect to the cloud database and I would be good to go. Boy was I wrong. Unfortunately, there were troubles connecting to the cloud service on the FPGA that we deployed to run ML predictions and send results to the database. The FPGA was sitting in a lab in campus (we had to connect to it remotely) throughout the course of the project and was connected to the campus network. The network apparently had a certain firewall which I never managed to get around. However, a workaround was to set up the MongoDB server locally on my laptop. Since I was staying on campus, my laptop and the FPGA would be on the same network!
I definitely had troubles displaying real-time graphs on screen. Despite having MongoDB's Change Streams and Socket.io to aid in my real-time streaming, I still faced troubles. The issue was that, my database was receiving data at frequency of 30Hz and my server would then vomit out the data at the same rate to the frontend, resulting in major latency issues. An easy fix I thought of was to sample the incoming data, reducing the frequency to only about 5Hz. On deciding the sampling rate, it was about finding the sweet spot between having minimal latency, and still meeting the requirement of 'real-time'.
The biggest obstacle I faced was the lack of time. It was nearing the final evaluation but I had submissions for other modules to focus on too. My dashboard was mostly done, except for the sign up, login and logout functionalities. It was a shame as I spent quite a considerable amount of time learning about authentication using JSON Web Tokens (JWT), and Redux to manage and keep track of the state of the application. Since this was originally a 'Nice to Have', I decided to forgo the functionality and simplify the authentication process with just simple conditional checks and password encryption with bcrypt. I was definitely not proud of it, but considering it was not even a project requirement, and just something I wanted to have fun with, I was not too bummed for long.
After almost 4 months of painstaking hard work, it was finally done. Our final evaluation went very smoothly, with the dashboard performing as it should. Since I had to show only a small part of my website during the evaluation, I thought maybe I could shamelessly show off my masterpiece here.
Enjoy
Here are just some of the tools and npm packages I used in making some of the components of my dashboard:
Was I happy with my work? Absolutely. Could I have done more? Maybe? When all is said and done, I believed that it was a great experience finally doing a deep dive on the different aspects of full stack web development. From exploring my misconceptions in React to understanding how HTTP requests work, I definitely learned a lot throughout this journey. This project showed me that backend development was definitely not my cup of tea and I still had a very soft spot for frontend development. I would like to give a shoutout to my lovely groupmates Andrew, Billy, Jess, Jiayi and Alex for being such great teammates and for making my CG4002 journey memorable. To others, this project may seem trivial and simple, but it was definitely a stepping stone for me in becoming a proficient web developer.
Here's to many more projects!
14