Data best practices vlog with BI specialist, Joseph Hobbs. Season 3 Episode 3 Hobbs talks about when it's time to rely on computers versus your own brain power to figure out what questions to ask.

Video Transcript

Hello everyone, welcome back to TNT. My name is Hobbs, this is Valorem Reply, and we're going to be talking today about business intelligence and best practices. Specifically, about when it is you should focus on forming hypotheses and when you should let the computer do the work for you.

Welcome back everyone. As I said, my name is Hobbs. In our prior video, I talked about the importance of moving away from hunches, which we consider mostly untestable, and instead thinking in terms of hypothesis. A thing which can be tested and found true, false, or somewhere in the middle. And I'd like to advance that idea a little further forward. I got a chance with a client recently, to begin working on their data. When they brought me in, my typical approach to a client is to say, ‘look, you're the expert in your business, in your data. You know what things you want to test and don't test. I'm here to help you find the answers to your questions.’ And I had an interesting response this time. They said, ‘look, we don't actually know what questions we should be asking. So, why don't you find interesting things and come and tell us about them?’ It was a really interesting challenge and I enjoyed it quite a bit, because it required me to shift my mindset. Instead of thinking of a list of finite hypotheses, I was required to step out and say, ‘OK, what if we just took the data and looked for every conceivable correlation, and then saw what came to the surface out of those results?’

So, I began experimenting with machine learning and what you can do through that engine, or rather that concept. So, the way I did it, in particular, [when] I'm working, I spent a lot of my time in [Microsoft] Power BI, in the [Microsoft] Power Platform. Power BI has some analytical tools where you can ask it to explain a result, or you can put in a metric of some kind. Let's [say for example] profitability, or efficiency, or whatever number you're looking to increase. Drop in all of the conceivable variables that might impact this and say, ‘tell me what pops out on the other side.’ And it will take your data model, all of the relationships involved in it, and it will begin looking for statistically significant correlations between A and B through Z, right?

And it was a fascinating process for me because I got to go back to the client and say, ‘alright, I'm going to present you with 40 things that I think are interesting results of this. Probably twenty of these things are going to be meaningless,’ right? As cost goes down and revenue goes up, profit goes up. Well of course those two things are correlated, right? But in many cases, they were shocked to see what was coming out. They would say ‘why is this business segment so closely correlated with profitability? What causes that to be the case?’ And it required me, as an analyst, to grow and to expand. And to be willing to step out of a comfort zone that I had been used to of form a hypothesis and test the hypothesis. To say let's not rely on my brainpower, let's not rely on me to be creative enough to think of the right question to ask. What if we just ask every question? What if we let the computer ask 100,000 different questions of the data and then only return results where certain criteria have been met?

I would encourage you, as you begin working with your data; especially once you've got a good data model built out and you feel confident of the quality of it and the relationships involved, your data integrity; use some of these machine learning tools to see if you can let the computer do the hard work. And then you go through, and you determine what's meaningful from what is produced there. Don't rely on your brainpower, right? Rely on the computers and let them do what they do best. Which is asking all of these questions and not having to worry about the time and energy that it would take a person to do the same.

Thank you for joining me for this video today. It was a really interesting experience for me to get to do this, and I was glad that I was pushed to grow in this area. If you're interested in having us come in, Valorem Reply come in alongside you, partner with you, help you through some of your analysis, your data problems, a project you might have, we would absolutely love to partner with you. Otherwise follow us on social media, leave comments on these videos, find me on LinkedIn, and we would be happy to connect. I'll see you next time.