“Raw data is both an oxymoron and a bad idea; to the contrary, data should be cooked with care.” – Bowker, 2005

The number of analytics platform providers has grown substantially over the past decade, from a few niche players servicing specialised clientele to many ‘mass market’ providers servicing a range of businesses.

A large sales force with fancy suits, armed with carefully constructed dashboards, make it seem like these solutions are a ‘silver bullet’ to solve the world’s problems, however they often require expert users to develop the same standard of work.

Many of these platforms targeted at the lower end of the market are marketed as being ‘DIY’ or ‘easy to use’, however the promise is often met with implementation problems, frustration and most importantly, more dollars spent from the budget.

Why ignoring the need for understanding ends in disaster

Automation and ready-made analysis make these tools seem attractive, as many businesses, particularly in financial services, do not employ statisticians, data scientists, or mathematicians within their business. Often they employ staff from an IT background as their base role involves specialist knowledge of data architecture or other IT systems.

Data architects are vital members of the team and should not be underestimated, however, many lack the analysis skills required for business intelligence. These ready-made analysis tools seem like a cost effective way of filling a skill deficiency, however, lack of understanding of the results and analysis being performed is fraught with danger.

Easy to use but hard to master

Many of these tools are easy to use at a base level. However, performing basic functions very intuitively to get the full utility out of these tools requires an inhouse expert. In particular, visualisation tools are often easy to use, allow many people to access and manipulate data, and have allowed many businesses to take the first step towards true analytics sophistication.

The issue is, many businesses fail to realise is there is a direct trade off between functionality and ease of use. Tools which claim to be easy to use are often restrictive in the type of visualisation and analysis one can conduct, whereas tools which claim to be robust and can perform multiple tasks are often more difficult to use.

This is not necessarily a bad thing; tools are just that, ‘tools’ that help us get to where we need to go. The problem is that businesses fail to recognise the true constraints of the tools they are purchasing and using.  

AI needs more specialists, not less

A lot of the talk surrounding artificial intelligence (AI) focuses on the narrative that jobs will be replaced, but fails to acknowledge the potential for job creation. Generally speaking, for every job technology takes away, new jobs are created to service that technology. One example is bank tellers; ATMs replaced a huge amount of bank tellers, but in turn created many new jobs. New positions became available, such as technicians, designers, credit fraud specialists, security guards and actually more tellers. Since ATMs made it cheaper to operate a branch, due to a reduction of labour costs, banks opened more branches, needing more bank tellers to service advanced banking needs.

The same is true for the AI revolution; it will replace tasks previously completed by certain roles, but the role might not necessarily be replaced. In the case of AI, remedial tasks will be replaced, for example, the task of taking meeting minutes may eventually be replaced by natural language processing, an application of AI. However, the person taking the meeting minutes might not necessarily be out of a job, but instead tasked with higher value work, such as analysis or report writing.

For all of the advantages of AI, we will still require humans to operate and understand the world, and provide a deeper ethical understanding of the complex world we face. AI is a tool that humans can use to make effective decisions, streamline processes, and increase the productivity of our workforce. However, it can never replace humans, as it is in effect, trying to copy the way humans learn. I was once told that applying a data solution to a people problem will end in disaster, and for all of the promise of AI, it’s hard to believe there will ever be a data solution that can solve a people problem.