Data best practices vlog with BI specialist, Joseph Hobbs. Season 3 Episode 2 Hobbs talks about the importance of testing hypothesis instead of correlating KPIs based on a hunch.

Video Transcript

Hello everyone, welcome back to TNT. My name is Hobbs, I’m your host this evening and we're going to be talking about the difference between a hypothesis and a hunch as it relates to business intelligence problems.

Welcome back everyone, my name is Hobbs. I would like to tell you a story. I was working with a client a few years ago and they had a KPI (key performance indicator) that they absolutely, as a company, at the highest levels, in front of stakeholders, swore by. They said it was the best way to measure the effectiveness and profitability of their company. I won't tell you anymore details than that but suffice it to say they have this KPI that they loved. And everyone assumed that this KPI was tightly correlated with profitability. So then I was brought in and another analyst was brought in and we began working on a variety of BI projects, trying different things out. And one of the days I had a little bit of free time on my hands and so I sat down, and I said, ‘I wonder if that's true? What if instead of treating this as kind of like a hunch that everyone thinks this is true, what have we test it?’

So, I checked. I checked individual transactions and then clients as a whole, and I compared their profitability and this other KPI and there was no correlation between these two things. This company had built all of their analytics, that’s a bit of an exaggeration, but most of their analytics, around this concept that was based on a hunch. The hunch that A was connected to B. And it was only after we really looked at the data that we discovered that they weren't.

So, today as a best practice, what I would like to propose to you is focus on hypotheses not hunches. And let's define those two terms. In your brain, probably when you think about hypothesis you think of the scientific method, and that's where I’d like you to go. A hypothesis is something that is testable. You could run a test and at the end of that test say either it's true or it's not true or you know its partially true, somewhere in the middle. A hunch is not something we inherently think of as testable. We just have this feeling that something is accurate and correct.

So, what I'd like you to do in the next BI project that you're involved with, keep an ear out for people's hunches. Pay attention to the way they talk about their data, and their business in particular. As they talk about that business, are there any places where they're just assuming that this one thing matters or is important or connected to this other thing? And whenever you encounter that, as an analyst, one of the best things you can do is say, ‘let's check that’, right? ‘Let's test this hypothesis.’ And through that, that's where you can have some really transformative moments for companies. Where they can realize that the way they're running their business may not be the way they should be running their business, based on the actual facts of the case.

So, to recap that really quickly, what I'm asking of you as my audience, right? As analysts or managers or companies interested in business intelligence, if you have a hunch about the way your business runs, test it. Take that hunch and transform it into a hypothesis so that you can truly be making sure that the data drives your decision making. And you don't end up in a spot where the primary KPI you're reporting to your stakeholders is one that isn't actually connected with your success.

Thank you for watching everyone, I hope you enjoy this video. If you've got any comments for me, by all means leave them here. Find me on LinkedIn, come and check out our website or our other blogs. If you would like to us to come in and do some analysis for you, will be very happy to come alongside and partner with you in any way that we can. Otherwise, I will see you next week.