Want to see a return on your analytics initiative? You might be asking all the wrong questions!

Most people think that the biggest challenge in analytics is needing the right data or the right tools to answer a question. In my experience, it’s actually knowing the right questions to ask.

I’m used to hearing a few frequently asked questions from business stakeholders:

Let’s break these down.

“Is there anything that stands out or that’s interesting in the data?”

There’s a lot of ambiguity in this question. The analyst may have zero or little context around what’s considered interesting to the stakeholder. This also doesn’t provide any boundaries around what data the analyst should be looking at or including in their analysis.

“Can you answer this very specific business question for me?”

With this question, typically, the analyst goes down a rabbit hole of their interpretation of what you mean by the specific question you asked. Again, very little context about the question and why it’s important. They may spend days — maybe weeks — getting access, pulling data and eventually chasing down their best version of an answer. By the time they get back to you, your question is not relevant anymore and you ask them to look up something else. In some cases, they may even feel pressured to tell you what they think you want to hear. This analyst then turns into your ad hoc reporting minion. I’ve seen entire teams dedicated to this exact task. This is an inefficient use of resources (expensive resources at that).

In both scenarios, there’s little to no structure in the communication between the business owner and the analyst. In the case where a business owner needs an answer to a one-off question, typically there’s a larger question to answer. If the process forces everyone to keep peeling back the layers to the fundamental question, the time spent answering it will be a much better investment. If you are asking these questions without context, it’s no wonder your return on investment for analytics isn’t what you want it to be.

This post will walk through identifying candidates for analytics as a solution and what you, as a business stakeholder, can do to ensure you get the return you’re looking for.

Typically, any person managing a line of business understands the financial structure. They have expectations on revenue, growth, and costs. It’s their job. They may also need to identify new sources of revenue. Where the communication typically breaks down is the impact or affect data can have on the financial structure.

Can data help to reduce costs? Can it increase revenue? Can it increase customer retention? Can it increase profit margins? Can it increase quality? If so, where? How?

As a business owner, it’s important to understand that there’s a distinct difference between identifying an opportunity for improvement and finding the best solution. Going through the process to clearly define the opportunity and understand the current state both take priority over evaluating solutions for those opportunities. It’s tempting to want to build a complex and highly sophisticated solution-using data. Resist the temptation.

Clearly defining the opportunity, prioritizing it over other opportunities, and going through exercises to understand the current state of that opportunity are all necessary steps prior to ideating about solutions. There’s an entire process of solution feasibility and exploration that comes next. Once you and your team sit down and go through the problem statement, your team will set off to find and sift through all available and relevant data. They’ll go through an iterative exploration process to understand context and to clean the data. Then, they’ll start to understand if any meaningful relationships exist. This process can absolutely take time and consume resources. They’ll need to continue checking their assumptions and exploring additional data. This is why it’s important to spend the time prioritizing opportunities.

This is the point where those who understand the process will thrive and those who don’t will start to lose hope that analytics will ever have the impact all of those white papers promised. If your success metric in this process assumes that a particular solution will be feasible, you’re setting yourself up for failure. Instead, think of the solution feasibility process as an early phase of learning that will inform your next move. It’s so important to be honest and real in these conversations with your team. Many times, I’ve seen such a potential promise and case for analytics to have an impact, only to find out that not enough data exists or that there’s no significance in the data available.

Unfortunately, you can’t manufacture relationships in your data. It’s either related, or it’s not. Simply put, certain data can be indicators for metrics you care about (for example, time spent on a website might be a good indicator of a sale), and if no such indicators exist, then you must continue the search for new data or change directions. Data scientists and analysts aren’t magicians — or, in other words, don’t shoot the messenger.

It’s critical to understand the potential outcomes of an analytics project — or, as I like to say, the phases of an analytics project. There are a few dimensions you can use to evaluate potential solutions. Simplicity and effectiveness are two important ones. Simple solutions ensure you are not over investing resources, especially when the solution is not yet proven to have an impact. This is what some people refer to as an MVP (minimum viable product) or a baseline solution. This gives you a comparison point to evaluate other solutions against. Obviously, measuring the effectiveness of the solution to your bottom line will tell you how much “bang for your buck” you got. How much does the solution influence your key metrics? What does “good” mean and how do you measure it? And, of course, you must be able to credit your solution with that influence, and not external factors. This is typically the hardest part.
It’s important to note that it’s completely reasonable for an outcome to be an answer to a question and not the answer to the question. If you’re hoping to identify key characteristics of a lead who’s most likely to buy your product, one question you may start with is, “Do we have enough relevant data to determine this confidently?” The answer might be no. Be prepared for your return to initially be answers to questions you should ask, but often don’t. Your team may run through a series of experiments that expose what you need in order to be able to answer the question of interest.

Justify areas of focus that will potentially have the largest impact on the business. Analysts and data scientists most likely don’t have access to this information. They typically don’t sit in the board meetings or financial meetings.

Boil the objective of the project down into specific metrics and goals and identify areas of uncertainty. This will define scope for the initial data exploration process. Are you focused on higher sales conversion rates? What is it now? What’s your goal? Why is that your goal? Have you identified where the process breaks down or where the inefficiencies are? This will expose enough context to create a meaningful starting point for your team to go looking for relevant data. Where does the data exist that shows historical conversions and historical behaviors of your buyers? Can you see how those ad hoc questions start to get answered throughout this process?

Identifying potential areas of impact is a precursor to the feasibility conversation. Familiarize yourself with the typical questions to ask during the feasibility or exploration phase. Do you have access to relevant data? Is there enough data to draw a conclusion? What does your baseline solution look like? You then have a decision to make: either make the investment to increase feasibility of completing that analytics project, or pick another one off your prioritized list.

Related posts:

January 2018 Goals. Finish Python Crash Course (I just need to finish the building a Django app project to get through all the fundamentals I wanted to cover….

Tenho algumas lembranças da escola no Ensino Médio, e entre elas está a minha professora de Geografia. Sua presença era forte, suas mãos e voz, grossas, e seu jeito de falar fazia com que qualquer…

When the heart burns, submerged in love, drowning in the pangs of absence, your face disperses a rainbow through the prism of my tear. I long for the sun to shine through the cracks into the…