zack hoang

Scope

Overview

MyDrive is a cloud storage for your images. Written using ExpressJS and its middlewares ecosystem, it contains most of the basic features you can expect from a file uploader: Authentication, CRUDable folders and files upload, download and delete.

Tools

• ExpressJS: JavaScript backend framework
• EJS: genereate HTML with JavaScript
• Express Validator: sanitize and validate user-provided input
• Multer: handle processing uploaded files
• PostgreSQL: database
• Prisma: ORM
• PassportJS: NodeJS authentication middleware
• PassportJS Local Strategy: authentication strategy with username and password
• Cloudinary API: image storage

Features

• Authentication: with usernames and passwords
• CRUDable folders: Create, Read files, Update folders' name, and Delete folders. These are records that lies inside database that gets queried based on its ID and its author
• Upload files: upload images and store a record of upload information in the database, while image is stored in Cloudinary
• View file's details: file details can be viewed in a seperate route. Details include: file name, upload time, file size

Challenges

• Setting Up Initial Schema: setting up the database during the early stage took me the most amount of time and also a lot of migrations to get to the final results. I ended up writting relations that I did not need at all until I realized I could make the app work with just querying scalar fields, with a small cost of more codes. However, I did not find this to be much of a problem at all given this project's scope.


• Folder deletion: deleting one folder means to also delete all of its descendants and files that contains within itself and its descendants. One could solve this problem recursively by deleting folders and files with the given ID of a parent. However, I settled for a simple Breadth First Search approach since it made the most sense to me at the time. This is how it works:
  1. Given a folder's ID, insert that ID into an array called queue
  2. Start the search, searching for folders that has its ID parent equal to the ID that we are trying to search
  3. If the search does not return any result, we know that folder does not have anymore descendant. We mark them as visited by insert that ID into another array called visitedFoldersID, dequeue, and search the next item in the queue
  4. Once the queue is empty, meaning the search is done, visitedFoldersID should now be an array of ID strings, which could be iterated through to look for folders to delete on the database. For each iteration, we also delete all files that are contained in each folder.


• File Type Checking: I wanted to restrict users to only upload images. This is a small security check I implemented so that Cloudinary's storage does not grow too quick, since we can assume images files are quite small, ranging around a few kilobytes to megabytes and making sure users does not upload any suspicious or malicious files. Initially, file checking was only limited to checking its MIME type and extension, provided by the Multer middleware. While it seems reasonable on the first glance, changing the extension will also change the MIME type, which renders this solution ineffective. The solution then, is to check the magic number on the image's buffer using the file-type package.

Takeaway

This has been my most challenging project so far and it has given me some ground on how to operate with Prisma. On top of being fun, it has given me some insights on how a common cloud storage works on the surface.