Bluesky

Case Study: Warehouse Explorer

About Bluesky: A startup company in seed stage, it is a full platform solution for data cloud efficiency, focusing on Snowflake data use & cost savings. Bluesky provides data-driven organizations with intelligent workload optimization and cost governance tools that help them innovate while keeping costs under control.

My role: I worked closely alongside the senior product manager in this project, meeting together with customers and white-boarding ideas throughout the duration of the project. The result of these sessions was a series Figma prototypes for testing and engineering implementation.

Project Definition & Goals

  • User data in Snowflake is processed through one or more data warehouses depending on query volume size. Costs can increase when a large warehouse is processing a small number of queries or a small warehouse is processing an excess load of queries.

  • Warehouses may also experience spikes in data processing that can result in unexpected costs. The user will want to know whether this spike was temporary or indicative of a longer-term trend, requiring them to adjust how their data is handled. A tool was needed to help the user quickly identify these issues.

  • Goal: Create a tool enabling the user to examine multiple warehouses within varying time frames (e.g. month, half a day, a few minutes, etc). This tool should help the user decide whether to change the size of a warehouse or merge multiple warehouses into a single warehouse.

Research / Discoverability

  • For a company without a high level of traffic and users, considering it’s in a early seed round stage, there was no meaningful amount of traffic data to glean deep enough insights, as typical usage was less than one hundred logins or less per week.

  • The product manager and myself relied on input from investors and early adopters of the Bluesky platform. Additional feedback was provided in contract negotiations from potential large-scale customers during their research phase of the application. This meant regular weekly meetings with design partners to gage their use and expectations on the feature.

  • This feedback as well as input from engineering and executive teams were a series of Figma mocks and prototypes.

Earlier versions of the Warehouse Explorer Tool

Potential Obstacles

  • While it was easy enough to provide the user with a list of warehouses to compare against each other, there were some limitations in terms of the time frames we could show to the user. We estimated that we had roughly 900px available to display data by the hour/minute/second on a desktop screen. This limited us to 30 days at a time. If the user wanted to see a wider window of time, they needed to switch to the next or previous month. The breakdown went as following:

    • 60 seconds x 15 minutes = 900px (seconds over a quarter hour)

    • 60 minutes x 12 hours = 720px (minutes over half a day)

    • 24 hours x 30 days = 720px (hours over one month)

  • Some users were requesting to view warehouse performance by microseconds, but we were unable to do this in the initial iteration.

  • Another potential drawback to showing this graph was load time as there were up to 900 points of data being brought in from the backend to populate the graph. This could be problematic as some users might not be patient enough if it took, say, 10 seconds or more for the page to load.

  • We limited ourselves to show up to 4 warehouses at a time to compare against. More than that we felt made it looks cluttered. But it was easy enough to switch out different warehouses.

  • Considering that a customer

One month (720 hours, 1px per hour)

Half a day (720 minutes, 1px per minute)

One quarter hour (900 seconds, 1px per second)

Development & Release

  • In developing this feature, I worked closely with the lead data warehouse engineer, the lead front-end engineer and the product manager.

  • We met regularly with customers and senior Bluesky execs to gather their feedback and ideas throughout the process.