Textual data augmentation.
Techniques to Intelligently create artificial similar samples of existing data, for boosting performance on text classification tasks.
You’re probably familiar with data augmentation with CNN models, as it increases image data size by making small changes such as random shearing, flipping, rotating, blurring, cropping, zooming, etc.
But when it comes to NLP tasks, textual data augmentation is not that easy, it’s a much more challenging task!
But why do we need data augmentation?
- Training on a small-size data increases the chances of overfitting.
- It’s often used for tasks where the model expects a large amount of data, but we’ve limited access to the data.
- Data augmentation reduces the costs of collecting and labeling data.
It Improves model prediction accuracy by preventing data scarcity for better models and resolving the data imbalance problem in the classification task.
In this article, we’ll get engaged with different techniques including Easy Data augmentation (EDA) techniques in addition to back translation and generative models. how data augmentation can be performed on text data to boost the performance of the text classification task.
Back Translation.
This approach uses machine translation to paraphrase a text while retaining the meaning, through:
Translate sentence/s to another language
Get the output sentence/s back to original language
Check if the new sentence/s is different from the original sentence/s.
If it is, use this new sentence as an augmented version of the original text.
In case the sentence is still the same you can drop it or take advantage of several intermediate languages.
Easy Data Augmentation Techniques.
The 2019paper explores 4 simple but powerful text augmentation techniques serving as a good baseline to augment text data:
Synonym Replacement (SR)
Random Insertion (RI)
Random Swap (RS)
Random Deletion (RD)
Let’s find out how each of the above-mentioned techniques works.
1) Synonym Replacement (SR)
Word Embedding-based Replacement:
- Randomly pick up n-words from the sentence “excluding stop-words” and replaces those words with their synonyms chosen at random.
- Pretrained word embedding like GloVe, Word2Vec, fastText can be used to find the nearest word vector from embedding space as a replacement in the original sentence.
- Contextual Bidirectional embedding like ELMo, BERT can be used for more reliability as its vector representation is much richer.
Get AI & Data Science 604’s stories in your inbox
Join Medium for free to get updates from this writer.
Lexical based Replacement.
- Wordnet is a lexical database for English that has meanings of words, hyponyms, other semantic relations, etc. Wordnet can be used to find synonyms for the desired token/word from the original sentence that needs to be replaced. NLTK, Spacy is NLP python packages can be used to find & replace synonyms from the original sentence.
Original Sentence: Probably my all-time favorite movie, a story of selflessness, sacrifice, and dedication to a noble cause.
Updated Sentence: Probably my all-time dearest movie, a story of selflessness, renunciation, and dedication to a noble cause.
2) Random Insertion (RI):
Similar to synonym replacement technique, but in this case, the synonyms of the randomly picked n-words are inserted at a random position without removing the original word.
For the below-mentioned sample sentence, we randomly pick n=1 word (comedy) and insert its synonym word at a random position.
Original Sentence: Probably my all-time favorite movie, a story of selflessness, sacrifice, and dedication to a noble cause.
Updated Sentence: Probably my all-time favorite movie, a story of selflessness, sacrifice, and chronicle dedication to a noble cause
3) Random Swap (RS):
The random swap technique chooses any two words from the sentence at random and swaps their position. This technique can be performed n number of times for n-pair of words.
For the below-mentioned sample sentence, we randomly pick n=1 pair of words (the, roads) and insert its synonym word at a random position.
Sentence: Probably my all-time favorite movie, a story of selflessness, sacrifice, and dedication to a noble cause.
Updated Sentence: Probably my all-time favorite movie, a noble of selflessness, sacrifice, and dedication to a story cause.
4) Random Deletion (RD):
Randomly removes each word in the sentence with the probability ‘p’.
Sentence: Probably my all-time favorite movie, a story of selflessness, sacrifice, and dedication to a noble cause.
Updated Sentence: Probably my all-time favorite movie, a of selflessness, sacrifice, and dedication to a noble cause.
In the coming post we’re going to take about generative-based techniques.
References:
[1] Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. https://arxiv.org/abs/1901.11196
