Data Warehouse & Dashboards

A professor watched me operate in his classroom and decided I was the kind of person who could talk to anyone and technical enough to figure out the rest. He recruited me into MISOM, a consulting firm that built data warehouses and business intelligence dashboards for coal mines. They needed someone to do site work and help bridge the gap between technical staff and onsite operations.

MISOM's core business was building data warehouses, and the purpose of those warehouses was to get people the information they needed, largely through scorecards and dashboards, plus ad-hoc queries against the OLAP cubes that fed both.

A working mine produces numbers constantly: dispatch logs, equipment telemetry, production tallies. Even more never gets digitized at all. Safety records live on paper. Targets live in one planner's spreadsheet. A clerk knows figures that the people in operations and engineering don't. I often say that data in an operation moves at the speed of a haul truck, about ten miles an hour, carried on paper when it could, and should, move at the speed of light. MISOM's developers could build dashboards on any of it. They just needed someone to tell them what data was there and what the sites needed, and I turned out to be pretty damn good at that. The frontline people know which numbers matter and where they live, but they don't speak schemas. My job was to fill that gap.

MISOM sent me to the sites, about ten of them. I sat with dispatchers, engineers, clerks, shift supervisors, and just about anyone else you can imagine on a mine site, and diagrammed how information actually moved: every hop, every form it took, every person who touched it. A typical visit turned up 40 to 50 data sources nobody had catalogued. The surprising ones were never in a database. They were on paper, or in some obscure spreadsheet passed hand to hand like a long game of telephone.

It sounds easy to surface this kind of information, but the human element makes it harder than it sounds. Anyone who has ever sat with a consultant looking over their shoulder, talking about process improvements, has felt the same thought pass through their mind. Am I about to be part of a "workforce reduction"? You're asking people about their jobs while they're worried about keeping them. I've lived a very blue-collar life and carry a very white-collar level of information. That mix is my superpower. I got people to show me things other technical visitors never saw because I don't smell like a "bullshitter." I have a genuine curiosity about people's day-to-day, and they can feel that. What came back was an inventory worth warehousing, and business processes documented well enough that a developer could build without setting foot on a mine.

The method had one rule. Don't upturn anyone's life. No new ERP. We pulled data from wherever it already lived. We'd restructure a spreadsheet slightly so it could be scraped, or ask one person to save one file to one folder. Keeping that spreadsheet might feel inefficient in a world of apps, but that spreadsheet is sometimes somebody's entire work world. They've spent years working on it. They know how it works. It meets their needs. So we made data-collection interfaces in Excel and Access, tools the mines already trusted. We abstracted the complexity onto the backend, away from the users, and met the users where they already worked.

So the mines had these scorecards, and anyone who's worked with a scorecard knows what gets measured gets managed. What you don't measure gets pushed aside. Focus only on tons, and safety compliance suffers. So you had better find the right numbers. My favorite chain: a mine kept running out of blasted inventory. Why? Plans weren't ready. Why? Engineering was late. Why? Geotech information came in late. Why? Permits weren't getting signed. The metric that landed on the dashboard was almost comically small. Was every permit needed today signed within an hour? Once people watched it, the downstream numbers moved. That's the craft. Chase a lagging indicator upstream through a complex system until you find the leading one somebody can act on today.

I also vividly remember getting an email from a department that had been berated for years over its budget. The cost number kept looking off on the scorecard, so they dug into the OLAP cube behind it. In their own data they found that a merger years earlier had tangled two operations' cost codes. Their Canadian site had been quietly absorbing expenses for a mine in Alabama. No analyst had caught it. The people being blamed caught it, the first time anyone handed them their own numbers. That's why good organizations democratize information.

The production warehouses and dashboards were the developers' build, not mine. That division of labor was the point. My time counted most in the field and at the whiteboard. Theirs counted most in the codebase. MISOM's founder ran change management from the top, getting a mine's leadership bought in so the mandate rolled downhill. Years later at Kennecott I'd do it the opposite way, winning users one demonstration at a time with no authority at all. I'm glad I learned both. The metrics moved quickly. And because everything stayed simple and modular, many of those mines still run the reporting today. I picked up my first real SQL, full-stack work, and wireframing along the way, all while carrying a full engineering course load.

I've had better titles since, and bigger budgets. But a surprising amount of my career has been this same job at different scales. Go to where the work happens, learn what the numbers mean, and carry that meaning to the people who can build with it. This is where I learned the job existed.