Chatting with a colleague recently, I was struggling to explain how to make the most of a data scientist. Working with analysts and actuaries, who are used to manipulating spreadsheets, downloading data and extrapolating trends, it’s not so easy to explain the wondrous skills of data science. I wracked my brains and reflected on the skills of my favourite data scientist, Angeliki, and then it struck me! Data scientists are just like detectives, and Angeliki is just like Miss Marple. And here’s why…
Unconventional thinking: Data scientists know that the answer is always there, but how you find the answer is often not the way you might think. If I ask, “why do customer behave in a certain way”, the data scientist will need to look at patterns that fit AND patterns that don’t fit to give a definitive answer. Instead of following convention, data scientists follow many random paths to find the most reliable answer. Like a good detective, data scientists must thinks differently to get the best result.
Taking a different view point: We all have bias, be it conscious or unconscious. As human beings, we are pre-programmed to complete a thought process and fill in the blanks. This may not always be with correct information, because we form views based on our own experience. Data scientists need to consider many different angles when addressing data questions – you can’t just stick with a safe assumption, you have to look above, below and round the side to get a full view of your data. All good detectives see a crime through the eyes of the victim, witnesses and perpetrator, and thus data scientists must also see the data from many different perspectives.
Never making assumptions: The old adage goes that assuming makes an ASS- out of –U- and –ME. Cheesy cliché, for sure, but it’s a very true statement. Data scientists can never make assumptions. As they approach a data problem, they must keep a clear head, and avoid being led down a path that may result in key information being overlooked. The best detectives clear their mind of all preconceptions, and in the same way, data scientists must approach each new data challenge with an fresh, untainted perspective.
Solving the puzzle, even without all the piece: I expect my data scientist to struggle with the wacky and abstract requests that I make. I know what I want, and why I need the answer, but it’s not always easy for the data scientist to complete the puzzle with the data available. Unless they are extremely fortunate, there will often be missing data, be that because it was not captured, or because it can’t be validated. This doesn’t stop data scientists, but merely encourages them to seek alternate ways to answer the question. A good detective rarely has every piece of the puzzle, but like data scientists, through thorough investigation, a mystery can be solved!
To conclude my story, I explained to the colleague that if you came across a murder victim, you wouldn’t dream of gathering evidence, assessing the crime scene and taking samples from those present before contacting the police. Instead, you’d report the matter to the authorities, and leave it to them to investigate. They’d ask you and everyone else questions, investigate the evidence and then tell you “whodunnit”. In the same way, rather than trying to master data, and work out what information we need to answer a question, it’s far better to leave it to the expert data scientists. They’ll probably ask you some questions to clarify what you are trying to do, then they will go away, crunch the numbers and come back to you with the “whodunnit” answer. This really helped the colleague see how to get the best from the data scientist. The only problem is, every time I think of my favourite data scientist, Angeliki, I can’t help thinking of one of my childhood heroines, Miss Marple!