Heatmap Service

Problem Statement

Video content administrators wanted a way to view the geographical distribution of clips around the world so that they could figure out if the business was able to deliver on content licensing requests from tourist boards and travel companies. For example, if a content licensing request for a specific country was received, the team needed to be able to see at a glance if the business had enough video content in that location to meet the request.

The company's existing admin portal made this difficult as, while it was possible to use text-based searches across a number of video attributes, there was no way to visualise the spread of data on a map. Furthermore, the company had technical concerns about any solution that queried the database directly as its dataset was growing rapidly at the time and there were fears that the data layer wouldn't be able to handle the increased load.

Solution

We came up with an asynchronous solution to the problem using an eventually consistent data model whereby data was periodically extracted from the database using an AWS Lambda invoked through an Amazon EventBridge scheduled rule. Given that the company's dataset grew rather slowly proportionate to its overall size, eventual consistency was an acceptable trade-off to meet the business's need to minimise database read load. Upon extraction, the data was then persisted as a private JSON BLOB in an encrypted Amazon S3 bucket.

In the company's admin portal, a UI component queried a REST endpoint to retrieve a secure presigned URL to view the JSON data, which was rendered on an interactive map using a clustering library.