It’s always the same story. Beginning of the week/month/quarter, and time for a new round of reporting. It always feels like this time it’s going to be easy. Finally time to cash in on the hard work you’ve put in to building up reporting systems. But stakeholders have a way of looking at your hard work, thanking you for (if you’re lucky) and then asking for exactly the new view that will be hardest to implement. How do you build out a report that won’t become tangled mess as all of these requests pile up? I’ve highlighted some critical principles below.
The first thing to understand is that you can’t predict the future. You, an analyst, should understand this better than anyone in the organization. You spend all day every day trying to help decision makers predict the future, and the impact of their decisions will have on it. You know how many factors can throw off carefully laid plan, and need to have the humility to incorporate flexibility into your own project.
Part of that need for human flexibility is that the information is going to be interpreted by humans, and as Alex Castrounis points out, “People see and understand data differently, which can lead to different levels of insight. Some people prefer numbers, words, and tabular data, while others better understand visualizations and charts. Insights and actionability are best delivered with a combination that’s designed to deliver the greatest impact!” The most important thing you can do is go into planning with an understanding that things will change. So doing it by hand first time isn’t inefficiency – it’s a rational response to the exploratory nature of your first round of reporting. Think of it another way: Make careful investments in building out an optimized data pipeline, until you’ve manually collected and tested some of its water.
This is where it gets controversial
My next piece of advice, after “do it by hand first time” is even more controversial – do it by hand the SECOND time as well! And third, if you can get away with it. The reason is that your stakeholders don’t know what they want until they see it, and it’s only when they are regularly reviewing your results without additional requests that it makes sense to start setting the pipeline in stone. This is a natural extension of the fact that “Data visualization is an instrument of storytelling. Look for outliers in data, surprising patterns that explains why your story is important”, as explained by Uldis Leiterts.
During these first two rounds by hand will be painful, since at least half of the work will probably be exactly the same. A famous advertising executive once said “half of my money is wasted, I just don’t know which half”. Your situation is a little bit different: Half of your effort is wasted, and you can’t PREDICT which half. Fortunately, once you have a few examples under your belt, you will have a much better understanding of what the stakeholder will need each reporting cycle. That is how your manual slog through the first few iterations will pay off in spades, when you build your automated pipeline right the first time.
And remember that your different stakeholders will each need to get their own conclusions out of the same visualization. As Cam Davidson-Pilon points out “Similar to a scientific theory, if they can make a (or many!) hypothesis and find an answer, then it’s a good data visualization.”
Where do you go from here?
The critical first step is taking a first step. Whichever report you’re most dreading, just try it. Do it whichever way is easiest, and send over the results. You’ll feel great, your stakeholders will much prefer something decent now to something polished later (or never), and I guarantee that the second round of refinement will bring you dramatically more clarity than staring at a whiteboard.