Graduation Data & Dashboards

In 2017 I was running Udacity's alumni community, and I wanted to know whether any of it was actually working. The questions were simple. What percentage of our graduates joined? What percentage actually participated? Which channels were growing? Every one of those questions leans on the same number, how many graduates we had.

That led to a stark realization. Nobody knew. Literally no one in the entire company had an exhaustive list of who had graduated from our programs, which explained a lot of inconsistencies I'd run into administering the community. It turned out the way I'd been verifying graduates by hand was already the best method available. In the early days nobody had thought to keep a durable record. Students were told they'd graduated, got their certificate, and went their merry way, and no one had asked whether we'd ever need to check back later. There wasn't even an agreed answer to the underlying question. What gate actually counted as a graduation?

With help from the engineering team I pieced the story together. The data existed, but it was scattered across three different systems and buried in a monorepo that had grown into a rats' nest of legacy code. So I built the missing piece. Pipelines that blended those sources into one dependable answer, and a React interface on top, so verifying a batch of members went from an afternoon of digging to a quick check.

With a real list of graduates under me, the reporting I'd wanted could finally exist. An ETL pipeline moved data out of Google App Engine into Amazon Redshift, and dashboards in Chartio and Jupyter sat on top, tracking the numbers a community lives or dies by. What percentage of alumni joined. What percentage actually participated. How fast each channel was growing. The dashboards showed me where the community was carrying itself and where I was still the one pushing.

Then something I didn't expect happened. Other teams started leaning on the graduation answer. Customer support came first. Udacity refunded half your tuition if you finished your Nanodegree on time, and every one of those refunds began with proof the student had actually graduated. Supporting that meant landing a commit in the monorepo itself, my first time putting code into a codebase I didn't entirely control. The thing I'd built to feed my own dashboards had quietly become infrastructure, running whether or not I was watching, doing work for people I'd never met.