AI Data Preparation: Why First-Time Users Still Get Stuck
Making data tools accessible starts with the questions we ask and the work we expect users to do.
Lately, I have been talking with our engineers about whether we need to make the product easier to use. The team thinks it is already straightforward enough. I am less certain that someone opening it for the first time would agree. People who build a product every day know different things from people arriving with a problem to solve.
Imagine someone building an AI feature for a small company on their own. They open the product because they want to know which customers are likely to leave. The screen offers starting paths labelled model, agent and source data. Those categories are familiar to us. For this user, working out which category their task belongs in is already another problem.
In a larger company, that difficulty may be less visible. A data team can prepare the files, make the technical choices and hand the results to the person doing the business task. The experience can feel easy because someone else has already done the difficult work. A solo builder or a small team handling client projects may have nobody in that role.
A different kind of question
That is why I am thinking first about the questions we ask, rather than simply reducing the number of menu items. A question that someone familiar with the business can answer is different from one that requires a data specialist. Putting both into a dropdown does not make them equally easy.
Get Minchan Chung’s stories in your inbox
Join Medium for free to get updates from this writer.
For example, asking “Which method should we use to create more examples of customers who signed up?” requires the user to understand data generation methods. Asking “Are you choosing whom to contact before a call, or analysing records after the call?” lets them explain their work. We need that answer to decide whether call duration can be used. Before the call, the information does not exist yet.
Our team’s bank data experiment shows the specialist work that follows. The software selected fields for training, added examples of the less common customers who signed up, and compared the results with training on the original data. The run left us with the prepared training data, the results and code for rerunning the comparison.
On the same original records, it correctly identified 25 more customers who had signed up, but also incorrectly flagged 23 more customers who had not. Because the inputs included call duration, this was a classification of records from completed calls. It did not show an increase in actual subscriptions or better selection of people to contact.
The judgement the user needs to make becomes clearer here. Does analysing completed calls fit their purpose? Is this increase in errors acceptable in exchange for finding more subscription records? Answering those questions requires knowledge of the work. Whether the user should also have to write the code to add training examples before seeing the results is a problem for us to solve.
What we are changing
We are now changing the product so that it starts with what the user wants to do and the data they have. The idea is to ask for the business requirements, use those answers to guide the preparation and show the results. Users should understand why a step is needed and be able to judge its outcome without first having to learn every technical method involved.
This new experience is still being designed. We have confirmed that the software can perform the preparation and comparison. We still need to establish whether a person without specialist knowledge can complete the process on their first visit. The existence of a feature does not establish that it is easy to use.
I want to reduce the points where a customer has explained their work but still has to learn our product’s way of classifying it. Being able to build an AI feature does not give a small team a data specialist. The product needs to take on the preparation work in between before what feels easy to us can feel easy to the customer.
I’m Minchan Chung, co-founder and co-CEO of CUBIG. I write about the data work behind AI.
