One terrific aspect of my job is that I more often than not get called into discussions where everyone has an opinion about how a product or algorithm decision should be made. Each of these discussions quickly turns into a debate (sometimes a heated one) where everyone is trying to justify the pros of their approach and the cons of others’ approaches. This article talks about why it is important for companies not only to seek expert opinions but also to experimentally evaluate the success of new strategies.

An experiment is defined as “a scientific procedure undertaken to make a discovery, test a hypothesis, or demonstrate a known fact.”

For a data scientist, experiments are like their bread and butter, the more experiments that are done, the more hypotheses can be assessed and a robust solution can be arrived at. Even at school or college, everything we learned was through experimentation.

In this article, I will introduce Ludwig and comet.ml and build models on them using Google Colab notebook.

Ludwig

Ludwig is a toolkit that allows you to train and test deep learning models without the need to write any code. While there might be many solutions that allow you to do automated machine learning, Ludwig is open-source and completely customizable. One can add their architecture of deep learning models, pre-processing steps and much more that they think is suitable in their context.

Building your first DeepLearning Model using Ludwig

You can visit the Ludwig website to read in detail about the benefits of using the library. Their documentation is quite detailed and will walk through a bunch of user cases from multiple domains that will help you get started.

Comet.ml

Comet is a meta machine learning platform designed to help AI practitioners and teams build reliable machine learning models for real-world applications by streamlining the machine learning model lifecycle. By leveraging Comet, users can track, compare, explain and reproduce their machine learning experiments.

Running your First Experiment through Comet.ml

Installing comet is as simple as installing Ludwig and starting your first experiment is even simpler. And with Ludwig’s comet integration, you need not do anything else.

All of this is tied up in this Google Colab notebook which should help you get started.

The world doesn’t have to wait for Tony Stark to create J.A.R.V.I.S to start having conversations that are useful.

Today, the same fundamental pattern recognition techniques are able to produce state-of-the-art results in natural language understanding, face recognition, speech recognition and so on. It is possible to show a computer many examples of something so it can recognize or extract meaning out of it accurately. Just like the AI that Stark built for himself, it should be possible to teach an assistant how to perform specific tasks. But before teaching it to perform tasks that might be useful, can it be taught to talk to a human instead.

Conversations are powerful. We can express ideas, information, knowledge, thoughts, and feelings, as well as understand what is expressed by others. Effective two-way conversation requires one to understand what is being said. Understanding the intent is fundamental to an effective conversation. Machine learning approaches to understand human language and to handle contextual dialogues are constantly improving.

Although, it is still quite common that AI assistants commit errors. However, instead of pre-empting these errors and providing extensive conversational data to train the assistant to handle all eventualities, there should be a mechanism to hand-hold it to recover from errors and get back on track. In a way, an assistant should be able to learn from the feedback provided and get better at having a conversation.

Vision for Human-Machine Interaction

Humans have been using machines to interact with them for quite some years now. It started off by configuring machines to answering a few simple structured commands which evolved into asking assistants to fetch information from a bunch of predefined questions and answers (FAQs). Enabling these assistants to have engaging conversations would require them to handle context in a conversation. Eventually, the AI assistant should be able to leverage user data to tailor conversation according to one’s preferences and offer an ultimate, personalized experience. At this level, an AI assistant will learn when it’s a good time to get in touch and proactively reach out based on this context. It could mean blurring the gap between an actual human assistant and artificial intelligence.

Blurring the Gap between Human and Machine - The Turing Test

Turing proposed that a computer can be said to possess artificial intelligence if it can mimic human responses under specific conditions. The Turing Test has been criticized, in particular, because the nature of the questioning ("Yes" or "No" answers or pertained to a narrow field of knowledge) had to be limited in order for a computer to exhibit human-like intelligence.

As an antithesis of this idea of testing whether the computer program could successfully fool the questioner, we could help evolve the program to respond in a way that would convince the questioner that the responder is human.

Learning through Interaction

Interactive Learning is a great way to train an AI assistant and generate training data while conversing with the chatbot. During the conversation, one can provide feedback for every prediction that was made - intent classification, entity extraction, context identification/management as well as response prediction. These conversations, where explicit feedback is given and corrections are made to the assistant's responses, can be used to train the multiple components that are used behind the scenes. Using interactive learning you can design paths that an AI assistant can follow to eventually lead to a solution. These can be seen as happy paths where the user is cooperating well and the system understands the user input correctly.

To give you an example, using this technique, we can train the assistant to answer the question “how’s the weather?” by fetching information about the current weather from an API and respond accordingly. However, the questions “what’s the weather like outside?” and “how’s the weather?” are both asking the same thing. The question “what’s the weather like outside?” can be asked in hundreds of ways. People can say identical things in numerous ways, and they may make mistakes when writing or speaking. They may use the wrong words, write fragmented sentences, and misspell or mispronounce words. Can the assistant be made to generalize and understand each of these intents instead of training it again and again? The answer is Natural Language Understanding (NLU). NLU helps in inferring what was meant from language even when the written or spoken language is flawed by leveraging AI algorithms to recognize attributes of language such as sentiment, semantics, context, and intent. It enables computers to understand the subtleties and variations of the language.

Key Capabilities to a Contextual Assistant

While understanding and deciphering a user message is crucial, there are some key capabilities that should be considered while designing a conversational AI assistant.

• Gracefully Handle Any User Goal: Assistants won’t have super intelligence overnight but it is still very important that they can handle any user goal gracefully. This can either mean that they actually handle a full conversation end-to-end because they were trained to do so or are able to generalize from previous conversations. Alternatively, this could also mean to hand-off to a human directly (e.g. directly in this conversation) or indirectly (e.g. customer service contact)
• Read/Write Contextual Data: It is important for the AI assistant to be able to read existing or write new contextual data to secure data stores e.g. through APIs. This lets conversations change data in existing systems and ensure that returning users don’t start from scratch.
• Clear Understanding: Human language usually contains a lot of ambiguity - that’s why it is important to remove uncertainty.
• Change of Context: The user might at any point in time change their mind. This can affect any type of contextual data from user response (e.g. typo “2016” instead of “2017”) to a change of user goal (e.g. now want to cancel an order)
• Business Logic: Assistants should be able to follow different rules about what to ask or what content of the message to show

What’s in the Context?

Understanding the Context of the conversation is key as it can either dramatically shorten conversations or make it possible to deal with ambiguous user input.

Different types of context with different contextual data that impact the flow of the conversation in our case

• User Profile: Data about a user can vary a lot based on the use case or industry, e.g. age, name, address or which insurance policy they purchased.
• User Goal: A user goal is usually the starting point of a conversation and can be, for example, a specific problem. All user goals are intents but not all intents are user goals (e.g. yes/no is not a user goal)
• Conversation History: What has been said before contains a lot of important contextual data, especially to clarify what the user meant. For example, a pronoun like “it” might refer to a noun before like “MacBook” (coreference).
• User Response: A user’s response can be directly connected to the context it was asked in. For example, a “yes” answer in itself doesn’t mean much but needs to be connected to a specific question before.
• Knowledge about Medicine and Concepts: This is a tricky one as it is very broad. It could be anything from knowing the location of the user so that “downtown” means “Manhattan” if the user is in New York or “rangers” means “New York Rangers” because the conversation is about ice hockey. It could also include data from sensors or feeds (e.g. the prevailing weather or fuel level in a car)

A contextual assistant can handle any user goal gracefully and help accomplish it as best as possible. This doesn’t mean that the assistant can answer everything but it at least handles the conversation to help the user, for example by handing it over to a human.

Making it Possible

Building a contextual AI assistant which can converse and learn simultaneously is possible. The challenge is to design a machine learning framework to enable Collaborative Learning or Crowd-Sourced Learning. Why should the assistant not be trained by each one of us to solve our own personal use-cases and be shared with others? If this is made possible, the opportunities this can open are endless - Software developers would be teaching assistants to write code, Data scientist would be enabling them to train predictive models, Doctors would be using these assistants to triaging and personalized patient management and much much more.

At 1mg, we process a large number of prescription orders every day. These typically come attached with a picture of the prescription. A central feature of 1mg’s order processing system is our digitization system wherein a trained pharmacist codifies the attached prescription image using a smart software system. Once digitized, the system decides eligibility and duration for dispensation of medicines and also quantities that can be dispensed. The system is built to maximize patient safety and eliminate dispensation errors.

Given that 1mg handles tens of thousands of orders each day, and the core nature of the digitization system, we have always been keen to improve automation in the digitization system without compromising safety OR accuracy. This article looks at the challenges in creating ‘a digital prescription ecosystem’, and our approach which does not require any change in a doctor’s behavior.

Challenges in creating a Digital Prescription Ecosystem

Urban India generates an estimated 3 billion prescriptions each year. And almost all of them are on paper! A typical medical prescription contains doctor and patient identifiers, diagnosis, tests required, medication, dosage, and other details. It is in an unstructured format and is mostly written in individual handwriting styles on letterheads. Challenges in digitizing the large swath of prescriptions include:

Firstly, doctors in India deal with high patient loads and as a result low consultation time per patient in most hospitals and clinics. This is largely due to a low number of doctors present per 1000 patients in India. This means that doctors spend a very small amount of time with each patient and cannot afford to type prescriptions.

Secondly, the process of manually digitizing the prescription image is time-consuming. A typical prescription has 4–6 medicines and takes a pharmacist some time to digitize as it involves extracting the doctor and patient identifiers, appointment or visit dates and medicines along with their strengths and dosages.

Thirdly, the hard to understand handwriting of doctors adds to the complexity of the process by making it time-consuming and error-prone.

Early Approaches for Medicine Suggestions

As we understand from the above discussion, the bottleneck in speeding up the digitization process is finding, typing and searching for medicines from a prescription image. Accurate medicine suggestions to the pharmacist digitizing the prescription can circumvent this activity and make the process of adding medicines faster. To achieve this, we initially resorted to the following techniques.

Optical Character Recognition (OCR)

We thought that generating medicine suggestions was possible simply by detecting the text in a prescription. A widely used algorithm to detect text in documents that we used was Optical Character Recognition (OCR), in which we solved a pattern recognition problem to classify individual characters or symbols. To perform OCR on a document, we cropped the image into multiple word images. Each of these images was then split horizontally into symbols and each symbol was classified to a letter in the English alphabet. OCR provided by Google Vision APIs gave reliable results for computerized text and legible handwriting. Once the text was extracted, we matched it with medicine brands from our product catalog using various fuzzy matching techniques to get relevant suggestions. This worked well for some computer-generated prescriptions with printed medicines but failed miserably for a majority of prescriptions where the medicine names were handwritten. The extracted text after performing OCR on prescription was extremely inaccurate which either could not be matched to any medicine or matched with wrong brands and gave false positives.

Multi-Instance Learning (MIL)

Our manual digitization process generates weakly annotated data, where the region corresponding to each medicine is not explicitly annotated and only the entire image is associated with all medicines found in the image. Even though we could extract word images, matching them to a medicine brand was challenging. To overcome this issue, we employed a Multi-Instance Learning (MIL) approach.

Attention-based Multi-Instance Learning is a variation of supervised learning where a single class label is assigned to a bag of instances. In other words, our hypothesis was that given enough prescriptions containing a particular medicine brand, a MIL model would be able to identify which words have a similar pattern and thus, correspond to the particular medicine brand. Even though these set of experiments showed slightly better results, the accuracy of medicine suggestions was not good enough for deployment in production forcing us to go a step further.

Present Approaches using Labeled Prescriptions

As we moved further into our investigation to build better and more accurate models for suggesting medicine brands and doing more, we recognized that multiple bounding boxes and labels on prescription images like those for medicine brands, doctor credentials, appointment dates were necessary. There are a few tools that are used to annotate images, however, none of them provided essential features like the ability to rotate images. Moreover, they could only be used on local machines rather than be deployed on cloud instances. These constraints forced us to build our own image annotation tool to mark regions in a prescription and label them appropriately.

Once we started generating annotations on prescription images, it opened up a plethora of avenues:

• Object Detection using convolutional neural networks (CNN) was one of the approaches we used. It helped detect instances of semantic objects of a certain class in digital images without specifically defining features. You only look once (YOLO) is a state-of-the-art, real-time object detection system which applies a single neural network informed by the global context in the image, making it extremely fast and accurate. We trained multiple YOLO models using labeled annotations on prescriptions to segment various regions in the image like bounding boxes for doctor details, patient name, medicines and medicine brands.

• Word level handwriting classification model was also one of the techniques which trained on labeled word images where labels corresponded to medicine brands. It helped us extract medication information from prescription images and infer medicine brand labels among all the word images found in a prescription.

Taking Help of Intelligent Heuristics

In India, it is a common practice among doctors to prescribe branded generics since they have come to trust these companies and their brands for manufacturing quality drugs over time. Also, Indian branded generic companies have been innovative in terms of drug delivery systems to improve absorption, reduce side-effects, thereby increasing the efficacy of the drug. This has led to an extremely large catalog of medicines that are found in medical prescriptions causing problems for any algorithmic approach focused on finding the medicines in the prescription. This is because classifying bounding boxes on text found in images or matching the extracted text to such a huge number of unique medicines is not feasible.

Simple heuristics and rule-based machine learning help in reducing the search universe for each prescription image.

• Finding the Doctor or their Specialisation — Even though there are a large number of medicines that can be prescribed, a single doctor typically specializes in one of 18–20 domains of medicine like cardiology, gynecology, etc., and usually prescribe medicines in only a few therapeutic and pharmacological classes. A valid prescription image always contains the doctor’s credentials along with their specialization which can be used to reduce the candidate set for searching for medicines.
• Association rule mining is a data mining process of finding the rules that may govern associations and causal objects between sets of items. We leverage a large number of digitized prescriptions by 1mg to observe frequent patterns of co-prescribed medicines to generate accurate suggestions as soon as a single medicine is added.

It is 3.7 times more likely that ‘Duphaston’ occurs in a prescription if ‘Ecosprin’ is present, than otherwise

Towards a Faster and Scalable Rx Digitization

At 1mg, we have developed models to classify valid prescription images, segment regions in the image where medicines are written, extract bounding boxes for brand and drug keywords and build hierarchical classifiers to identify medicines. This has enabled us to set up a process to obtain relevant medicine suggestions that go beyond what can be provided by using the best OCR technologies leading to speeding up the prescription digitization process with much higher levels of automation.

References

https://economictimes.indiatimes.com/industry/healthcare/biotech/healthcare/india-facing-shortage-of-600000-doctors-2-million-nurses-study/articleshow/68875822.cms

Accelerating Rx Digitisation using Medicine Suggestions was originally published in Tata 1mg Technology on Medium, where people are continuing the conversation by highlighting and responding to this story.

Henceforth, we began with a vision to personalise the products for each user shown in a list on the 1mg mobile app homepage. We wanted to build a recommendation system that can leverage all users’ short and long term view and purchase history and predict what they would like in future.

The Idea was ‘Collaboration’

Collaborative filtering models are fundamental to recommendation systems which are based on assumption that people like things similar to other things they like, and things that are liked by other people with similar taste.

The underlying assumption of the collaborative filtering approach is that if a person A has the same opinion as a person B on an issue, A is more likely to have B’s opinion on a different issue than that of a randomly chosen person

- Wikipedia

CF is a general concept and there are several algorithms to implement it. One of the popular approaches are based on low-dimensional factor models (model based matrix factorisation).

The idea behind matrix factorisation models is that preferences of a user can be determined by a set of hidden factors. We can call these factors as Embeddings. Intuitively, we can understand embeddings as low dimensional hidden factors for products and users. Each element in the product embedding vector could represent different characteristics about the product and likewise the elements in the user embedding matrix might represent different characteristics about the user. A higher number from dot product of user-X and product-A matrix means that product-A is a good recommendation for user-X.

Collecting Clickstream Data

Our internal data pipeline collects and stores request response logs for all API calls in compressed hourly files in our data warehouse in Amazon S3. This way we have been able to gather users’ clickstream activity on the Android and Web. These files are read and processed using Spark deployed on a small clusters and then stored back in columnar data format which serves as our Tracking Data Store.

Matrix Factorisation — Implicit Preferences and ALS

A typical user might specify their preference for a product by giving it a high rating if they like it or a low rating if they dislike some aspect about the product. Collaborative filtering models are typically built using explicit preferences, where available. We had no explicit ratings for products and hence, we had to look for alternative methods to assess a user’s preference towards a product. In some ways, a user’s purchase of a product can also be seen as an implicit indicator of their preference. We started with a fairly simple implicit preference function which has become more nuanced over time (takes weighted sums of a user’s views/add to carts/purchases of products). A discerning reader can tell that preference functions can be constantly tweaked and improved.

Matrix Factorisation works on a dataset with users, products and user’s preference for the product. We used our data pipeline as described above, to find which user interacted with which product and extracted aggregated interactions for each user-product combination for a period of nine months.

An out-of-the-box implementation of Alternating Least Squares (ALS) technique in MLlib library in Apache Spark has been used widely for solving CF problems. Using a three instance cluster with Spark and Spark’s high-level API in Python, we trained an ALS model with implicit ratings. We evaluated the model by measuring the root-mean-square error of rating prediction.

Ensemble Models

ALS has a number of hyper parameters like dimensions of latent factors, iterations, regularisation etc. which can be tuned to improve the evaluation metric. Through our experiments, we found that an ensemble of matrix factorisation models (labelled as MF — Bagging) was better than individual models.

Deep Learning

The exploration of deep neural networks on recommender systems has received relatively less scrutiny. We came across Neural Collaborative Filtering (NCF) where we trained a Generalised MF and Multi-Layer Perceptron to learn separate embeddings, and combined the two models by concatenating their last hidden layer. This model combines the linearity of MF and non-linearity of DNNs for modelling user–item latent structures. We evaluated the NCF model using Hit Ratio (No. of correct predictions / No. of total predictions) and Normalised Discounted Cumulative Gain (nDCG).

Personalisation in Production

The final output of the modelling exercise was 20 product recommendations for all the users in the dataset. We pushed the new list into production using an AB Framework and appropriate fallbacks to the static list. Both the metrics that we used to track — clicks through rate (CTR) and conversion in same session, jumped up. These initial results were very promising and have pushed us further into improving the personalisation that we have been able to achieve.

As we moved ahead, some new questions came up

• Were users satisfied with the list, or in other words, were users clicking the recommendations that appear first in the list more often now?
• Were users able to discover more items using this list?
• Was the list decaying with time?
• Were the recommendations changing when we periodically recomputed the recommendations?

Deep Diving into Evaluation

These questions forced us to identify better and more appropriate metrics that would answer these questions and more. Below, we describe some metrics we were able to calculate.

Positions of Products in the List

We took a course on metrics for recommender systems and came up with two metrics which could tell us more about the positions where the clicks were happening -

• Mean Reciprocal Rank (MRR) — Reciprocal rank is defined as the inverse of the rank of the first item in the list with which the user interacts (clicks/purchase).
• Discounted Cumulative Gain (DCG) — looks at the utility of the item at each position in the list and discounts this utility or gain by position in the list. So we place more emphasis on the items at the top of the list and then we normalise by comparing the discounted cumulative gain of the list the recommender produced versus the discounted cumulative gain of a perfect recommendation list.

Do the Recommendations Change with Time?

Lathia’s Diversity talks about How temporal diversity is an important factor in recommender systems . How the recommendations change when new data is introduced with respect to time.

From a Business Standpoint

• Revenue Per Mille (RPM) could tell us how the price value of converted items from the list is varying
• Conversions Per Mille (CPM) could tell us how the number of converted items from the list is varying

Do the Clicks on the List Decay with Time?

We analysed the New Users vs Old Users who click on the list on daily basis. We observed that the list was not decaying as there was a set of new users who had never interacted with the list clicking on the products in the list.

What about Cold Start?

Collaborative filtering does not provide any recommendations for new users with less or no tracking history and also does not include new products in the recommendations for users. In our case, we show a static list of products where we do not have any recommendations from the CF model. This problem is popularly known as the Cold Start Problem. One of the ways it can be addressed is by relying on content based vectors for products and profile based vectors for users and using them alongside the embedding vectors in the model to learn preferences of all users for all products.

The Future still holds promise

As we start to dwell further on personalising product recommendations for users, we have many exciting paths to tread. Working out better solutions for tackling implicit ratings for products, addressing cold start for products and users and updating recommendations in realtime could also lead to better click through and conversion and thereby better personalisation.

A List on the Homepage was originally published in Tata 1mg Technology on Medium, where people are continuing the conversation by highlighting and responding to this story.