<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arthur F. S. Neto on Medium]]></title>
        <description><![CDATA[Stories by Arthur F. S. Neto on Medium]]></description>
        <link>https://medium.com/@arthurflor23?source=rss-13120c388894------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*_oHsY_7eqkGJyL3Y0kWnKA.png</url>
            <title>Stories by Arthur F. S. Neto on Medium</title>
            <link>https://medium.com/@arthurflor23?source=rss-13120c388894------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 16:51:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@arthurflor23/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Handwritten Text Synthesis and Recognition]]></title>
            <link>https://arthurflor23.medium.com/handwritten-text-synthesis-and-recognition-cf5bcf2aaeb8?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf5bcf2aaeb8</guid>
            <category><![CDATA[handwriting-recognition]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[handwriting-synthesis]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Wed, 15 Apr 2026 17:10:54 GMT</pubDate>
            <atom:updated>2026-04-15T17:10:54.790Z</atom:updated>
            <content:encoded><![CDATA[<h4>Can an AI learn to write before it reads?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H0Hwn9mo0Rle8OeCSm9xwQ.png" /><figcaption>Escola Politécnica de Pernambuco - Universidade de Pernambuco - Recife, Brasil.</figcaption></figure><p>After several years and gray hairs, a lot has happened. I completed my master&#39;s degree, my doctorate, got married, moved houses, and relocated to a new country. Today, I&#39;m happy to share that I have finally published my final PhD contribution.</p><p>Over the years, I often imagined when and how this post would be written. I questioned if the creative blocks and the lines of code would ever be made public. In this text, I will share the journey of this doctoral project, detailing its stages, core ideas, and reflections. While looking back at the major milestones, I will also dive into the technical aspects.</p><p>So, here it is. The paper titled “<strong><em>HTSR-Pollen: Handwritten Text Synthesis and Recognition System to Overcome Data Scarcity</em></strong>” presents an overall view of this project. For those interested, here are the links to the paper and to the repository, which is the same repository from my <a href="http://medium.com/r?url=https%3A%2F%2Farthurflor23.medium.com%2Fhandwritten-text-recognition-using-tensorflow-2-0-f4352b7afe16">first post</a>:</p><ul><li><strong>Paper:</strong> <a href="https://ieeexplore.ieee.org/document/11475723"><em>HTSR-Pollen: Handwritten Text Synthesis and Recognition System to Overcome Data Scarcity</em></a></li><li><strong>Code: </strong><a href="https://github.com/arthurflor23/handwritten-text-recognition"><em>https://github.com/arthurflor23/handwritten-text-recognition</em></a></li></ul><p>The project aimed to develop a method for generating synthetic handwriting images to use during the training of an offline Handwritten Text Recognition (HTR) system, in order to improve its recognition accuracy. I can say that the final result went well beyond that initial scope. <strong><em>So, grab a coffee and enjoy the read.</em></strong></p><p>One of the first stages of the doctoral program was to conduct and publish a systematic literature review in my research area. It was the first time I had worked with a well-defined search protocol, and it was a lot of work.</p><ul><li><strong>Paper:</strong> <a href="https://link.springer.com/article/10.1007/s42979-023-02583-6"><em>Data Augmentation for Offline Handwritten Text Recognition: A Systematic Literature Review</em></a></li></ul><p>The review aimed to identify related works focused on data augmentation methods applied to handwritten text recognition in data-scarce scenarios. It is worth mentioning that the scope was intentionally strict, since we focused only on data augmentation applied to text recognition within a low-volume data context. Everything outside that scope was excluded.</p><p>In this context, we used the following scientific databases: ACM Digital Library, IEEE Digital Library, Science Direct, Scopus, and Springer. The timeframe covered from January 2012 to December 2024. Initially, the window was set for 10 years; however, as the project progressed, we had to update it several times, extending the range to include 2023 and 2024. I believe it ended up being much more work than I expected.</p><p>Following the defined evaluation criteria and a peer-review process, we selected 43 works from thousands of papers. From this, a few key perceptions stood out. We observed a clear publication trend related to the theme, starting in 2016 and continuing through 2024, the volume grew from 1 to 10 publications per year. <strong><em>I believe this number is even higher today.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gjeF84Nm-0bYacqzE9s1kA.png" /><figcaption>Distribution of the Selected Papers by Publication Year</figcaption></figure><p>We also noticed that most papers focused more on word images than on other recognition levels, such as characters, lines, or paragraphs. In general, the works tend to use word-level images to generate new samples, which can later be expanded to lines and then paragraphs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xy6FLL_NUCpGIc1FVA6KQQ.png" /><figcaption>Proportion of Recognition Levels Used by the Papers</figcaption></figure><p>Among the datasets used, researchers prioritize IAM and RIMES, as they are the most established and serve as the primary benchmarks for text recognition. Another frequently used base is the CVL-Database, which is common in studies focused on writer identification. Other datasets appear less often, usually in research involving specific languages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aStW-K8FksenSSKtyHbiYg.png" /><figcaption>Proportion of Datasets Used by the Papers</figcaption></figure><p>Regarding approaches, many of these works use Digital Image Processing (DIP) alongside the optical model, as it is a flexible and low-cost technique, though limited by text structure. Transfer Learning (TL) is also explored, but on a smaller scale, facing challenges when adapting between datasets with very different patterns. Finally, the use of deep learning for Data Synthesis (DS) has seen growing application in recent years.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*voUMRGsb36Fg-EYz8gGS-Q.png" /><figcaption>Proportion of Approaches Used by the Papers</figcaption></figure><p>Within the context of deep learning for Data Synthesis, the main approaches included Transformers, Generative Adversarial Networks (GANs), and Diffusion models. It was interesting to see the early papers applying Transformers, then gradually moving toward GANs, and finally toward Diffusion. At that time, only a few works used Diffusion, but I believe its use has increased in recent years.</p><p>By the end of the systematic review, I had a clearer picture of how the literature on handwriting synthesis was evolving. The field started with word generation and then moved toward line-level generation, varying textual content and applying methods to interpolate writing style, which I found very interesting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CgGRXgLaAQfzVlHWX8eoaQ.png" /><figcaption>Advances in Handwritten Text Synthesis</figcaption></figure><p>At that point, you might think “<em>That&#39;s it, the path and solution are clear. Integrate a synthesis model with a recognition model and everything will work</em>”. <strong><em>To be honest, I thought the same.</em></strong></p><p>One of the main goals behind the proposed integration was for the synthesis model to generate images that were already ready for the recognition stage, eliminating the need for post-processing or intermediate save-and-load cycles. However, a deeper look into the available papers and code revealed a mismatch with those requirements.</p><p>The synthesis models analyzed were built to the word level, which required post-processing to compose lines and paragraphs. The images also went through a preprocessing step for width normalization, since those models were trained to generate images of a fixed-size per character.</p><p>Furthermore, training with line images became impractical because of the GPU memory usage. While word images of size 32 × 256 pixels were already enough to fully occupy a 24 GB GPU with a batch size of 4 samples, processing line images of size 128 × 1024 pixels with a batch size of 8 represented an even greater technical challenge for the available hardware.</p><p><em>It hit me like a bucket of cold water.</em> The works that previously seemed like a solution became only a theoretical foundation. Instead of using synthesis models from the literature, I decided to develop my own. The new goal was to create a synthesis model according to my needs: a compact architecture, able to operate directly at the text-line level, without pre- or post-processing. All this in a 24 GB GPU.</p><h3>BRESSAY: A Brazilian Portuguese Dataset for Offline Handwritten Text Recognition</h3><p>Along the way, I had an unexpected but very rewarding <em>spin-off</em>. During the systematic review, we realized that there was no Brazilian Portuguese dataset among the related works. At the same time, we had a collection of 1,000 essay images stored away, but without the proper labeling for recognition. That&#39;s when the opportunity arose.</p><p>I had the pleasure of working with an incredible research group, including students from several universities in Pernambuco (Brazil), to organize and label everything from scratch. The goal was to present a dataset as a paper at the International Conference on Document Analysis and Recognition (ICDAR 2024).</p><p>I admit that joining that project at that moment was difficult and risky. I was already halfway through the PhD timeline, still without concrete results from the synthesis model, and under a lot of pressure. Fortunately, with a strong team and a generous dose of exhaustion, we made it happen.</p><p>We even created our own annotation tool, and the effort resulted in two papers: one presenting BRESSAY and another describing the recognition competition we launched at the conference. It was hard, but it was worth it.</p><ul><li><strong>Paper:</strong> <a href="https://link.springer.com/chapter/10.1007/978-3-031-70536-6_19"><em>BRESSAY: A Brazilian Portuguese Dataset for Offline Handwritten Text Recognition</em></a></li><li><strong>Paper: </strong><a href="https://link.springer.com/chapter/10.1007/978-3-031-70552-6_21"><em>ICDAR 2024 Competition on Handwritten Text Recognition in Brazilian Essays - BRESSAY</em></a></li><li><strong>Dataset:</strong> <a href="https://zenodo.org/records/11637681"><em>A Brazilian Portuguese Dataset for Offline Handwritten Text Recognition (BRESSAY)</em></a></li></ul><p>Academic performance in Brazil, especially through essay writing, can be a turning point in a student&#39;s life. The National High School Exam (<em>Exame Nacional do Ensino Médio</em> - ENEM) is a good example of this, serving as the main gateway to higher education. With the advancement of HTR systems, the potential to make the evaluation of these essays faster and more efficient is enormous, but the path is full of challenges. The nuances of handwriting, such as corrections, strike-throughs, and highly varied writing styles, make automatic recognition a very complex task.</p><p>The dataset, as well as the competition, focused on recognition at three levels (line, paragraph, and page), using 1,000 essay images from different authors. Working with this data brought real-world challenges, as the images have varying sizes and resolutions depending on their source. Another point was the labeling of errors, erasures, and out-of-position text. We decided to label everything using a special tagging system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pewUVe3g61yhJ0_TeEKxeQ.png" /><figcaption>Examples of Handwritten Pages from BRESSAY</figcaption></figure><p>The experience of organizing a competition at an international conference was incredible. First, because you get to meet the authors you have spent years reading and citing. Second, because of the weight of the contribution: <em>providing a Brazilian Portuguese dataset for the research area</em>.</p><p>After months of balancing all these demands, the BRESSAY cycle came to an end. It was time to return to the PhD project and finally finish it. <strong><em>However, after so much effort, finish what exactly?</em></strong></p><h3>HTSR-Pollen: Handwritten Text Synthesis and Recognition System to Overcome Data Scarcity</h3><p>With full focus on the synthesis model and the requirements defined, the development took a few months, but the learning lasted much longer. Today, part of that study is already consolidated in a paper, with a more polished and concise approach.</p><ul><li><strong>Paper:</strong> <a href="https://ieeexplore.ieee.org/document/11475723"><em>HTSR-Pollen: Handwritten Text Synthesis and Recognition System to Overcome Data Scarcity</em></a></li></ul><p>Along the way, I remember feeling stuck on concepts and code many times. Every time I finally unlocked one stage, another knot would appear. It&#39;s that feeling that the more you learn, the more you realize you know nothing. Each new discovery about a topic made me realize that my previous understanding was incomplete. <strong><em>It was a real emotional roller coaster.</em></strong></p><p>Overall, the idea was consolidated into three stages within the same workflow: (<em>i</em>) <strong>Generative</strong>, where the synthesis model creates new images based on input features or random noise to increase diversity; (<em>ii</em>) <strong>Optical</strong>, responsible for extracting visual features from the synthetic images and decoding them into text; and (<em>iii</em>) <strong>Language</strong>, which acts on refining the final text output of the system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/976/1*DbNncZZ4AYResr0edatCVw.png" /><figcaption>HTSR-Pollen: Handwritten Text Synthesis and Recognition System</figcaption></figure><p>Reaching the final version of the synthesis model was, without a doubt, the greatest challenge. After a lot of trial and error, I identified some of the main reasons why synthesis models tend to be so expensive in this context:</p><ul><li><strong>Redundancy and Scale:</strong> the use of large and generic deep learning models in parallel within the same pipeline. In many cases, these models are not specialized and end up introducing redundancy.</li><li><strong>Excessive Depth:</strong> models with too many layers and an too many number of filters for feature extraction at each level.</li><li><strong>Spatial Self-Attention</strong>: applying self-attention to spatial features consumes a lot of GPU memory due to the image size.</li><li><strong>Diffusion Models</strong>: besides being slow to train and infer, these models process the image at its full size throughout the entire generation, pushing GPU consumption to the limit.</li><li><strong>Flow Conflicts (Horizontal vs. Vertical)</strong>: many recognition models transpose images to extract features, while synthesis models operate horizontally. Combining both approaches in the same training ends up duplicating content in memory.</li><li><strong>Processing Inefficiency:</strong> the need for pre- and post-processing to adjust both input and output. Since many models work with fixed widths per character, everything has to be readjusted so that the handwriting looks natural rather than a rigid grid.</li></ul><p>That said, it wasn&#39;t all smooth sailing, and each topic required a technical choice. To overcome these bottlenecks, my decisions were:</p><ul><li><strong>Redundancy and Scale:</strong> instead of using generic models, I developed a new and more efficient optical model (FlorV2). Also, I reused its compact encoder as the base for the other auxiliary models.</li><li><strong>Excessive Depth:</strong> I adopted a <em>minimum of the minimum</em> philosophy. I started with the smallest possible number of layers and filters, planning to increase complexity only if the results were unsatisfactory.</li><li><strong>Spatial Self-Attention</strong>: I replaced self-attention with a residual convolutional layer with gated blocks. This allowed me to highlight spatial features without the restrictive memory cost.</li><li><strong>Diffusion Models</strong>: even though they are the current trend, I opted for Conditional Generative Adversarial Networks (CGANs). They are lighter, faster, and delivered the performance I needed.</li><li><strong>Flow Conflicts (Horizontal vs. Vertical): </strong>I standardized the models to work with horizontal images only.</li><li><strong>Processing Inefficiency: </strong>I removed both pre- and post-processing. To make the handwriting look natural, I created a resizing layer inside the synthesis model itself. It uses a mask to guide the content resizing, avoiding the fixed character grid in the output images.</li></ul><p>In addition, I changed the standard line size to 64 × 1024 pixels instead of the traditional 128 × 1024. It may seem like a small detail, but together with the other changes, it was what made it possible to reach the performance I was looking for. The result was the training of full text lines with batch sizes of 8 samples on a single 24 GB GPU, using around 2× fewer trainable parameters than GAN-based synthesis models and even 10× fewer than Diffusion-based models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*82SUFAvQK3V3QTx7oKGvUw.png" /><figcaption>Overview of the Proposed Generative Module</figcaption></figure><p>Following the GAN concept, I used a <strong>Discriminator</strong> model to distinguish between real and synthetic images, and a <strong>Generator</strong> based on the <a href="https://arxiv.org/abs/1809.11096">BigGAN</a> approach. The idea here is to start the generation from a small patch and, at each convolutional block, gradually expand it until the target image size.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Pvc3y3ZGZKs56a1RkkUWg.png" /><figcaption>Overview of the Proposed Handwritten Text Generator and Discriminator Models</figcaption></figure><p>The Generative module, however, doesn&#39;t work alone; it requires a few auxiliary models during training. The <strong>Handwriting Recognition</strong> model (HTR) is responsible for transcribing the text, while the <strong>Writer Identification</strong> model (WID) uses the style extractor to capture handwriting style features. These two models are essential, because the HTR guides the synthesis toward generating readable text, while the WID guides the preservation of cursive style.</p><p>Finally, there is the <strong>Style Encoder</strong>. It acts as a bridge, taking the style features extracted from the images and projecting them into a latent space that follows a normal distribution, inspired by the concepts of Variational Autoencoders (VAEs). This is a key point, because it allows the generator to also produce cursive styles randomly, creating new handwriting patterns without depending on a reference image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gnzf4T_Th26hTofypDCUig.png" /><figcaption>Overview of the Proposed Handwritten Text Recognition, Writer Identification, and Style Encoder Models</figcaption></figure><p>For the training strategy, I decided to simplify what the literature often treats in five stages into only two. Instead of training each model separately, all models are trained together from scratch. To keep this combination of influences from becoming a mess, I used a dynamic weight balancing strategy: the content and cursive-style reconstruction losses adjust according to the training gradient, while recognition and writer identification follow an uncertainty-based weighting approach. In other words, at the beginning of training, when the model is still lost, these tasks carry less weight, and as learning progresses, they begin to guide the synthesis more strongly. This fine adjustment is what helps maintain stability without requiring an eternity of processing time.</p><blockquote>In case you haven&#39;t realized it, we are still in the synthesis module. Once trained, we can integrate the generator model only into the optical module and use real-time synthesis to train new optical models.</blockquote><p>In the optical module, the roles are reversed. If before the recognition model was only an auxiliary component helping the synthesis process, here it becomes the protagonist. The full focus is to train the optical model from scratch to predict text as accurately as possible. The interesting part is that the synthesis generator enters as a training partner, generating diverse handwriting samples to feed the optical model. This provides a good amount of flexibility, since we can combine synthetic data with traditional data augmentation, allowing the recognition model to learn from a variety of writing styles that would be very difficult to obtain from real data alone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uljiR6QL03ZLzJpL_hFfDA.png" /><figcaption>Overview of the Proposed Optical Module</figcaption></figure><p>To complete the workflow, there is the Language module, which works as a spelling reviewer for the final text. Instead of training another model from scratch or relying on external dictionaries, I used a GPT model as a post-processing tool. The idea is simple, the recognized text is sent to the model, which receives a restricted instruction to correct only obvious mistakes.</p><p>With the workflow defined, it was time to plan the experiments, and I confess I might have gone a bit overboard here. I decided to test the proposal across different datasets, training variations, optical models, and methods for data scarcity, such as data augmentation and transfer learning. The result? More than 35,000 GPU hours and almost two years of constant concern. <strong><em>It was a real stress test.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*zpaW66lCwckTuFO1WKajzA.png" /><figcaption>Overview of the Experimental Design</figcaption></figure><p>Regarding the results, I first performed a grid search, a brute-force exploration, across parameters and transformation methods for the data augmentation pipeline. What surprised me the most was the impact of perspective variations. I had never explored this method deeply in previous projects, and the result was impressive. By itself, it reduced the Character Error Rate (CER) by 16.1% and the Word Error Rate (WER) by 13.2%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RI0-7hEC_lC-SBGpJv0shw.png" /><figcaption>Average Error Reduction by Augmentation Method (Higher Is Better)</figcaption></figure><p>At the end of this exploration, we reached an ideal combination of methods, which guaranteed a total reduction of 22.9% in CER and 17.7% in WER. The recipe involved elastic transformations, perspective, shearing, scaling, and rotation, each with its parameters. Since the project is a few years old and the code has evolved, the exact application has changed, but the lesson remains, the benefit of these transformations holds up, even with minor variations in parameter preferences.</p><p>Moving on to the synthetic images and results by dataset. One of the major tests for the project was the <strong>BRESSAY</strong> dataset. The challenge here was the noise of the real images (typical of essay sheets) and the labels for strike-throughs. Even by focusing only on the text and ignoring all strike-through labels, the result was still positive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EBJRb_ncAXkvqzHgyVJduA.png" /><figcaption>Examples of Generated Images and Comparison with Related Works on BRESSAY (Lower Is Better)</figcaption></figure><p>Another challenge was the <strong>CVL-Database</strong>, which tested the optical models against diverse and varied handwriting styles. The synthesis generator felt the weight of that diversity when creating new samples, but even so, the use of synthetic data during training still helped improve accuracy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rv4XZn85DyABT69s-WEMUg.png" /><figcaption>Examples of Generated Images and Comparison with Related Works on CVL-Database (Lower Is Better)</figcaption></figure><p>For <strong>IAM</strong>, which is the most referenced dataset in the field, the challenge was to reproduce punctuation and spacing patterns. Transfer learning once again stood out, and at this point, you may be wondering “<em>Will transfer learning alone yield the best results?</em>” Not always. When only a minimal amount of training data is available, transfer learning-based models may show less stable convergence. In this scenario, data synthesis proved to be a valuable complement, leading to more consistent results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*icbCf5eHD3EjgZNGUCf7SQ.png" /><figcaption>Examples of Generated Images and Comparison with Related Works on IAM (Lower Is Better)</figcaption></figure><p>In the case of the <strong>Parzival</strong> dataset, the main challenge was dealing with a medieval writing style. Since it is a historical corpus in archaic German, it was necessary to remap some characters to support both image generation and recognition. For the generator, the difficulty was reproducing uncommon characters while preserving of the original writer style. In addition, because of the vocabulary and older spelling conventions, the Language module was not applied in this context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Csp4EwRh4B23CTZEzaDSJg.png" /><figcaption>Examples of Generated Images and Comparison with Related Works on Parzival (Lower Is Better)</figcaption></figure><p>The <strong>RIMES</strong> dataset presents a large variety of French handwriting styles, but with relatively clean and uniform image quality. This consistency made the generation of synthetic data easier, which directly contributed to recognition performance. As the second most commonly used dataset in handwritten text recognition research, RIMES served as an important benchmark for validating the system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WsaRbwIIPmUvKStF8GAU4w.png" /><figcaption>Examples of Generated Images and Comparison with Related Works on RIMES (Lower Is Better)</figcaption></figure><p>For the <strong>Saint Gall</strong> dataset, the context involved Latin texts with uniform handwriting and a limited character set, without punctuation or accents. This structural simplicity made it easier for the generator to create synthetic images with consistent variations. However, the archaic language still posed a challenge when generating unseen characters while following the same cursive style as the original images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HycDTybum3SvdsuNlPSAuw.png" /><figcaption>Examples of Generated Images and Comparison with Related Works on Saint Gall (Lower Is Better)</figcaption></figure><p>To close the dataset analysis, the <strong>Washington</strong> dataset presented a particular challenge because it had the smallest amount of data among all the evaluated datasets. This kind of scarcity is a natural obstacle for deep learning models, which required a stronger reliance on data augmentation and data synthesis techniques to obtain consistent improvements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oP8xe9JqxL8nDVhEBvRslQ.png" /><figcaption>Examples of Generated Images and Comparison with Related Works on Washington (Lower Is Better)</figcaption></figure><p>Looking at everything that was tested, it is clear that each technique plays a specific role depending on the scenario. Data augmentation helped where a reasonable volume of data already existed by providing the necessary visual variability, though this approach requires adjustments for each new context. On the other hand, transfer learning showed improvements by adapting knowledge from pre-trained models to smaller databases. However, this technique faced convergence difficulties in cases of extreme data scarcity, which is a common challenge in deeper architectures that still require a minimum amount of information to learn with stability.</p><p>Data synthesis was a positive highlight, especially with historical manuscripts. By generating additional samples, it was possible to stabilize training from the start and improve generalization where real data was limited. It is true that some generated images show certain visual inconsistencies, but this lack of perfection actually helped the recognition model become more resilient, forcing it to focus on the textual content. In the end, line-level synthesis proved to be not just a valid strategy, but a competitive approach for bypassing the lack of real-world examples.</p><p>Finally, it is worth mentioning that the generator model was trained for only 50 epochs due to time constraints, suggesting that a longer exploration could lead to even better results. In fact, since the conclusion of the project, several adjustments and improvements have been implemented in both the synthesis and recognition modules. Thus, although <strong>the</strong> <strong>current results already surpass</strong> <strong>those documented here</strong>, I chose to keep the original records to preserve the original analysis. <strong><em>Enough experiments!</em></strong></p><blockquote>So, <strong>can an AI learn to write before it reads? </strong>Well, not exactly. In practice, the recognition model (reading) must first <strong>learn enough</strong> in order to guide the generator model (writing), even though <strong>both can evolve together</strong>.</blockquote><h3>Closing Thoughts</h3><p>If you have followed along this far, whether by reading every technical detail or jumping straight to the conclusions, <em>my sincere thanks</em>. To be honest, there were many moments when I didn&#39;t believe this material would ever see the light of day. My last major public contribution was in 2019, about my master&#39;s project that began back in 2018. Today, I&#39;m finally publishing this work, which started in 2021, and closing a cycle that demanded as much as any other great challenge I have ever faced.</p><p>Looking back, this project was built through many important moments and creative breakthroughs, sometimes with a level of tiredness that felt close to exhaustion. Every small win in the code happened alongside real life. There were hard moments, excitement, boredom, and satisfaction, all mixed into the journey. A lot has happened since the first line of code, and each technical step forward also became a personal milestone.</p><p>None of this would have been possible without the support of the people who were with me. I&#39;m deeply grateful to my wife, Sarah, for her support during all these years. To my advisors, Byron and Alejandro, who not only guided me, but also trusted me enough to follow these crazy ideas. Lastly, thank you to Cubricks; starting to work with them at the beginning of my doctorate gave me experience that went far <strong><em>beyond borders</em></strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf5bcf2aaeb8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting Started with ETL]]></title>
            <link>https://arthurflor23.medium.com/getting-started-with-etl-9c2accef66cb?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c2accef66cb</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-integration]]></category>
            <category><![CDATA[dataiku]]></category>
            <category><![CDATA[etl]]></category>
            <category><![CDATA[azure-data-factory]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Thu, 06 Nov 2025 11:29:17 GMT</pubDate>
            <atom:updated>2025-11-06T11:29:17.756Z</atom:updated>
            <content:encoded><![CDATA[<h4>Key components in Azure Data Factory and <strong>Dataiku</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ttLObGyqjDPPp8ga.jpg" /></figure><p>After some years, it feels good to write again and share experiences from my journey. In recent years, I have worked in consulting for multinational companies, supporting teams and helping them onboard different platforms for data integration and automation.</p><p>Among all the tools I have worked with, two have become essential parts: <strong>Azure Data Factory (ADF)</strong> and <strong>Dataiku</strong>.</p><p>This post is a short overview of their key components, based on the training sessions I conducted. It is not an advanced tutorial, but a guide to help new professionals understand where to start and how each platform can be used to build strong and reliable data workflows.</p><h3>Understanding ETL</h3><p>Before diving into the platforms themselves, it is important to understand what <strong>ETL</strong> means and why it matters.</p><p>ETL stands for <strong>Extract, Transform, and Load</strong>, which represents the three main steps in moving data from one place to another in a structured and meaningful way.</p><ul><li><strong>Extract</strong> is about collecting data from different sources such as databases, APIs, cloud storages, or spreadsheets.</li><li><strong>Transform</strong> involves cleaning, enriching, or modifying this data so that it becomes consistent and usable.</li><li><strong>Load</strong> is the final step, where the prepared data is stored in its destination, often a data warehouse or another database for analysis.</li></ul><p>In general, ETL processes are essential for data projects because they ensure that teams work with clean, organized, and reliable data. Without this process, information would remain scattered across systems, and generating reports would become a complex and time-consuming task.</p><p>A well-designed ETL process brings <strong>automation</strong>, <strong>consistency</strong>, and <strong>traceability</strong>. It helps data engineers, analysts, and decision makers rely on accurate and up-to-date information to drive their work. This is where platforms like Azure Data Factory and Dataiku play a fundamental role.</p><p>It is also worth mentioning that there are variations of ETL, such as <strong>ELT (Extract, Load, Transform)</strong>, where the data is first loaded into a storage system and then transformed inside it. The idea behind ELT is to take advantage of the computational power of cloud data warehouses.</p><p>Regardless of the approach, ETL and ELT share the same goal: enabling teams to handle data efficiently and turn it into something valuable.</p><h3>Azure Data Factory (ADF)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GmW-Nq5rzBqp_0H0inzidQ.png" /><figcaption><a href="https://azure.microsoft.com/en-us/products/data-factory">https://azure.microsoft.com/en-us/products/data-factory</a></figcaption></figure><p>When I first explored ADF, I realized it was more than a tool to move data. It is an orchestration service that allows you to visualize, automate, and monitor data pipelines from multiple sources in one single place.</p><h4>Datasets and Linked Services</h4><p>To connect ADF with external systems, you use <strong>Linked Services</strong> and <strong>Datasets</strong>.</p><p><strong>Linked Services</strong> define the connection details to your data <em>sources</em> or <em>destinations</em>, while <strong>Datasets</strong> describe the data structures you want to use.</p><p>These two components work together to define what you are working with and how you connect to it.</p><h4>Pipelines and Activities</h4><p><strong>Pipelines</strong> are the main structure in ADF. They represent <strong>workflows</strong> that organize the sequence of steps involved in an integration. Each step is an <strong>Activity</strong>, which can copy data, run a stored procedure, call an API, etc.</p><p>By combining <strong>pipelines</strong> and <strong>activities</strong>, you can build processes visually, without the need for extensive coding. This structure makes ADF flexible for collaboration and simple to maintain as projects grow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*orskFghr8nQI17suwY2IDA.png" /><figcaption>Sample Pipeline in an ADF project</figcaption></figure><h4>Data Flows</h4><p><strong>Data Flows</strong> are where the actual data transformation happens.</p><p>They provide a visual way to apply joins, filters, and transformations through an intuitive interface. This feature is especially useful when introducing new team members, since they can easily see how data moves and changes across steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SCPeIq9PuLZLz5Tcmye3Kg.png" /><figcaption>Sample Data Flow in an ADF project</figcaption></figure><h4><strong>Monitoring</strong></h4><p>ADF also provides a Monitoring Hub where you can track pipeline runs, check execution times, and investigate failed tasks. Having a clear view of what happened and when it happened helps ensure consistency and reliability in production.</p><p>Once you understand these key components, you already have the foundation needed to start creating integration pipelines with ADF.</p><h3>Dataiku</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R6nGTIIrFxsld32dePrDbw.png" /><figcaption><a href="https://www.dataiku.com/">https://www.dataiku.com</a></figcaption></figure><p>If ADF is the orchestrator, <strong>Dataiku</strong> is the collaborative space where data transformation, exploration, and automation come together.</p><p>It provides a shared environment where engineers, analysts, and data scientists can all work together in the same visual interface.</p><h4>Projects and Flow</h4><p>Every project in Dataiku starts with a <strong>Flow</strong>. This visual layout shows how D<strong>atasets</strong> connect to each other through <strong>Recipes</strong>, which are the <em>transformation steps</em>. All pipelines and recipes are separated by <strong>Zones</strong>, which separates processes and transformations.</p><p>In addition, recipes can be built visually (<em>predefined transformation functions</em>) or through custom code using Python, R, SQL or Shell languages. It is a flexible environment that welcomes both technical and non-technical users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yuw4jiVD2FmbfeYxktDwtg.png" /><figcaption>Sample Zone in a Dataiku project</figcaption></figure><h4>Scenarios</h4><p>Scenarios bring automation into the project. You can schedule data refreshes, trigger workflows when new files arrive, or even automate model retraining.</p><p>This feature allows teams to maintain their data pipelines without manual execution, improving efficiency and control.</p><h4>Collaboration</h4><p>Dataiku includes tools for <strong>version control</strong>, <strong>documentation</strong>, and <strong>permissions management</strong>. In larger projects, these features are essential to maintain traceability and governance across different teams.</p><p>Through onboarding sessions, I have seen how quickly teams adapt to Dataiku&#39;s Flow structure. Once they understand the relationship between datasets and recipes, they can start building their own data workflows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WPUjQvxwb6RwH4F2XQjVOA.png" /><figcaption>Example of a Dataiku project</figcaption></figure><h3>Closing Thoughts</h3><p>ETL development may seem complex due to its various components and possibilities, but with the right understanding, it becomes an easy process for moving, transforming, and organizing information.</p><p>It is funny how time passes so fast. I still remember the feeling of starting my first data engineering project, five years ago. My role was simple: learn and create structured data integrations that could scale into many others. The questions were endless. <em>How do I start? What are the best practices? Is there even a standard?</em> Most of the answers came through experience, long projects, many revisions, and hands-on learning.</p><p>Then, one day, I found myself giving training sessions on the very platforms I had once needed to learn. Leading people from different countries how to follow the simple and organized patterns that I discovered through trial and error. <em>Every challenge eventually becomes a story to share.</em></p><p>If you made it this far, thank you for reading. I&#39;m already working on my next post, where I will share more about my PhD project: a <strong>Handwritten Text Synthesis and Recognition System</strong>. Coming soon :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c2accef66cb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MLOps in Industry 4.0]]></title>
            <link>https://arthurflor23.medium.com/mlops-in-industry-4-0-17f5bda964e3?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/17f5bda964e3</guid>
            <category><![CDATA[digital-twin]]></category>
            <category><![CDATA[experience]]></category>
            <category><![CDATA[industry-4-0]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Fri, 30 Dec 2022 21:28:27 GMT</pubDate>
            <atom:updated>2022-12-30T21:28:27.579Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Roadmap for Machine Learning on a Digital Twins Platform</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1015/1*7k-PHyt6on_qu2BT9rckjw.png" /><figcaption>Source: <a href="https://roboticsimulationservices.com/how-oil-and-gas-industry-operators-benefit-from-digital-twin-technology-digital-twin/">How Oil and Gas Industry Operators Benefit from Digital Twin Technology</a></figcaption></figure><p>In the last two years, I have worked on a Research and Development (R&amp;D) project. The objective was to build a Minimum Viable Product (MVP) of a Digital Twins platform for the energy industry, using machine learning models and data stream.</p><p>In the beginning, it wasn’t easy to understand all the concepts at once. So after a long time learning about it, I focused on the development of a <em>tool</em> that helps in the setup and self-management of the lifecycle of machine learning models (training, evaluation, deployment). That is a small part of the whole project, and I would say it is more like an <em>interface</em> between the platform and the models developed by the team.</p><h4>Some Concepts</h4><p>I read several concepts about Digital Twins two years ago, and I felt completely lost. Asking me things like: <em>how do I make one? — where do I start? — what is necessary? — what platforms or tools help me to create one?</em></p><p>Still far from the best definitions, today I can try to explain on my own. <strong>Industry 4.0</strong> is the transformation of products into services through technological environment within companies. Roughly speaking, it’s composed of massive data ingestion, cloud processing and machine learning models. <strong>Digital Twin</strong> is the way to use that technological environment. Since Internet of Things (IoT) allows data mapping through sensors, why not map all the main information about equipment, sector, industry, or even a city? After that, all the data is used by machine learning models to cross-information and simulate reality with predictions. Finally, <strong>MLOps</strong> (Machine Learning Operations) refer to the steps required to train machine learning models, select the best model already trained, and make it available for use.</p><p>That said, I would like to share my understanding of some important pillars to start developing a digital twin project. It’s worth mentioning that it was a private work, so there will be no mention of code or sensitive information.</p><h3>Real Life</h3><p>I considered Real Life as a pillar because it’s a crucial factor in the development of a project. Here you will find the daily challenges and facilities. Not to mention that each scenario will have its specificities.</p><p>As expected, some challenges arose due to my project requirements. Among them, I can highlight that the production environment should have a simplified deployment pipeline, different from the development server. The goal was to create an isolated and lightweight production environment with only models trained in the development server.</p><p>Another point was the modularization of all components through parameters. This made it possible to reuse and recombine various functions in different model pipelines. Furthermore, the <em>tool</em> should be able to manage sets of pipelines per client. All that in an automated and self-managing way.</p><p>Thus, I was responsible for building this part of the project. Figure 1 highlights the mentioned component within the project overview.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/378/1*5dfdq2t9yuOFwiZHcXAF5A.png" /><figcaption>Figure 1: Project overview</figcaption></figure><h3>Continuous Integration/Continuous Delivery (CI/CD)</h3><p>CI/CD is responsible for automating the lifecycle process of an application in a target environment. This process runs new builds and makes them available on servers (development or production, for example).</p><p>This process can be configured through repositories and Git pipelines, which trigger actions with certain conditions (such as commits or tags). In other words, the objective is to create an automatic mechanism for code repositories to be processed, configured, and made available on the server with each triggering action.</p><h4>Container Service</h4><p>After Integration and Delivery, you need to know how you will make your application available on the server. For our context, we used the simple Docker and Docker Compose to build and configure containers with isolated environments for each module.</p><p>Container is a software that packages up code and all its dependencies. It’s a lightweight, standalone, executable software that includes everything needed to run an application. So, the objective is to create isolated and scalable environments for each service that will be made available. In such a way, with each code update, only the referring container will be updated, instead of the entire project.</p><p>At this point, the code is already linked <em>end-to-end</em> between the local machine, git repository, and server. Thus, the development of modules will be updated in the git repositories, which will trigger actions. These actions will take the new code and throw it on the target server, and then it will package and make it available in a container.</p><h3>MLFlow and Airflow</h3><p>Here is the most important part, through two complementary frameworks. Basically, MLFlow is responsible for training, selecting, and serving the best machine learning models. Airflow is responsible for scheduling and triggering pipelines in a synchronous and dependent way, which concerns the lifecycle of machine learning models in our context. It’s important to remember that at this point, the data has already been ingested, processed, and stored for use.</p><p>The details and motivations of the code are very extensive. So I separated the main challenges and goals to be achieved concerning this module (MLFLow + Airflow).</p><p><strong>First</strong> — standardize the environment variables between the two frameworks. This was necessary because MLFlow configures and uses its folder (<em>mlruns</em>) in a relative path (inside python script folder). On the other hand, Airflow configures and uses its folder (<em>airflow</em>) in an absolute path (inside user folder). In this way, some environment variables were configured, centralizing the artifacts generated by the two frameworks. Thus, it was possible to match the same environment between localhost and container; in addition, it was possible to isolate environments between clients (<em>did you already forget about this requirement?</em>).</p><p><strong>Second </strong>— add an abstraction layer to simplify commands. Many pipelines need an extensive command line; or perhaps partially run through code snippets with a specific setup. In any case, it is important to provide an execution abstraction layer. So, If previously we unified the target folder of artifacts, here we unify the command entries.</p><p><strong>Third</strong> — parameterize everything. It’s necessary to have good flexibility and integration between the scripts. Thus, the chain of parameters enables different setups with dynamic imports of modules and the reuse of parts of the code. Some examples of what to parameterize: client, data source, models, hyperparameters, pre- and post-processing functions.</p><p><strong>Fourth </strong>— architect and organize all the code. The goal here is to keep the code readable to make it easy for another developer to join the team and maintain it. So, I recommend creating and maintaining development standards within the team, as well as, good code documentation with small tutorials in Jupyter Notebook.</p><p>Overall, the code was modularized and resolved our issues. Figure 2 shows the main code components.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/909/1*tNT9pTnbAGA4wDps7oAJHQ.png" /><figcaption>Figure 2: Main components overview</figcaption></figure><h4>Dashboards</h4><p>The icing on the cake is the dashboards that MLFlow and Airflow provide. Through them, it’s possible to monitor the status of models and pipelines in real-time. Figure 3 shows a sample of the MLFlow (top) and Airflow (bottom) dashboards.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L8yU6WkH9OWBNgOySxJIMg.png" /><figcaption>Figure 3: MLFlow (top) and Airflow (bottom) dashboards</figcaption></figure><p><strong>Bonus — </strong>I also took the opportunity to integrate custom data analysis models into the training and deployment process, and provide interactive web pages. So, using <em>Pandas Profiling</em> and <em>Streamlit</em>, it was possible to generate web pages to analyze the data that are being used in the training of machine learning models. Figure 4 shows Pandas Profiling (left), which provides simpler analysis, and Streamlit (right), which provides customized analysis focused on time series.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iVxycIQHk7DBwYMGm_iH4g.png" /><figcaption>Figure 4: Pandas Profiling (left) and Streamlit app (right) dashboards</figcaption></figure><h3>Conclusion</h3><p>The two-year-long project was proposed to develop a Digital Twins platform MVP for the energy industry. Within this huge project, I was responsible for developing a machine learning model orchestration tool, which is responsible for the self-management of models in containers. In addition, I also had the objective of simplifying the development of new pipelines, through modularization and code reuse, which enables scalability for <em>N</em> clients and <em>N</em> machine learning models — <em>it was an amazing and powerful tool that made me proud for sure.</em></p><p>In this post, I described some knowledge acquired along the journey, in an attempt to organize and structure the ideas, and who knows, help someone. It was really a challenge to learn and apply everything, but it certainly helped me to evolve professionally.</p><p>Finally, after many challenges and obstacles (<em>stress and demotivation too</em>); the feeling that remains is happiness and mission accomplished. Today, I can say for the last time — <em>There are 0 weeks left to the end of the project</em>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17f5bda964e3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Prediction of Late Payments Using Decision Tree-Based Algorithms]]></title>
            <link>https://arthurflor23.medium.com/prediction-of-late-payments-using-decision-tree-based-algorithms-ce72a2fbccab?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce72a2fbccab</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[payments]]></category>
            <category><![CDATA[decision-tree]]></category>
            <category><![CDATA[data-mining]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Thu, 25 Nov 2021 21:33:43 GMT</pubDate>
            <atom:updated>2023-03-02T14:45:11.453Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Divide and conquer</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wlf4Y50JQwXry27kpLd_nQ.png" /></figure><p>Here I am, another post on decision trees. Unlike my first experience, this project was exciting and full of ideas throughout its process, and of course, more motivating. I’m currently at the beginning of my PhD, and this project was carried out in a class at the beginning of the year (2021), but it is worth mentioning it in a post. If you are interested, the link to the project and paper:</p><ul><li><a href="https://github.com/arthurflor23/invoice-payment-prediction">https://github.com/arthurflor23/invoice-payment-prediction</a></li><li><a href="http://revistas.poli.br/index.php/repa/article/view/1746">http://revistas.poli.br/index.php/repa/article/view/1746</a></li></ul><h3>What are Decision Trees?</h3><p>Just to have an overview, a decision tree is a data structure in which the combination of conditions (nodes) leads to a decision (leaf node). So you use machine learning to build the best decision tree based on the data you have.</p><h3>What is Invoice-to-Cash?</h3><p>The project context refers to late payments in the business area. One of the processes I had the opportunity to read was about Invoice-to-Cash. This process involves the actions of a company to receive payment. Thus, it is essential for the financial stability of any company, given the collection of accounts receivable as its main activity. So imagine we have the following questions:</p><blockquote>How long will the customer take to pay me? Will the customer pay me after the due date or sooner? Will the customer pay me?</blockquote><p>With that in mind, it’s common for managers to want some answer and then take action involving the company’s capital.</p><h3><strong>Dataset</strong></h3><p>Unfortunately, the dataset is private. However, I can mention that it brings information about customer purchases, such as dates, values, customers, and related companies. In total, there were 2.7 million records initially, considering January 2018 to February 2021.</p><p>The step I consider most important is the processing and transformation of data (Fig. 1). Here you can adapt/improve according to your scenario.</p><p>First, we have the dataset and data extraction. In this step, I made some filters related to the business rule and removed features that didn’t add to the project context.</p><p>Second, outlier detection. It is necessary to carefully analyze possible outliers and, at the same time, validate with the business team.</p><p>Third, grouping the data. This step is dataset-specific (<em>I think</em>), as a customer can make multiple purchases on the same day. So we want to group customer invoices by day.</p><p>Fourth, historical data. At this point, I generate features related to customer history for each purchase day. In other words, new features related to his past were added for every purchase. This step is essential because we will add relevant information, such as the customer’s financial situation, average customer delay, purchase frequency, etc.</p><p>Fifth, time-related information. Finally, the date features are converted to numerical values to find gaps between the purchase and due date, days left until the end of the month, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7dWZNwyVPe6qzgkzS_m4gg.png" /><figcaption>Fig. 1: Pre-processing steps</figcaption></figure><h3>Experiment</h3><p>As a proposed solution for the project, I split the problem into 3 steps (Fig. 2). The first is to identify the invoices that will be paid late. The second is to identify the invoices that will be paid after the due month. Third, estimate how many days are late after the due month. In this way, we were able to bring more information to the results, improving decision-making.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/1*-ANCmXPd2Xq4Tg5g45MiOg.png" /><figcaption>Fig. 2: Steps for the experiment</figcaption></figure><p>We can say that for each defined step, the complete experiment was carried out: selection of features, search for hyperparameters (grid search), training with cross-validation, testing, and evaluation (Fig. 3).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*3tb4voEfD4IT4VK-k-LciA.png" /><figcaption>Fig. 3: Overview of the experiment performed</figcaption></figure><p>Finally, 8 decision tree models were used. Here, the important thing is to realize that: classification models were used for steps 1 and 2; and step 3, regression. The 8 models were: (<em>i</em>) Bagging; (<em>ii</em>) Balanced Bagging; (<em>iii</em>) Random Forest; (<em>iv</em>) Balanced Random Forest; (<em>v</em>) AdaBoost; (<em>vi</em>) Gradient Boosting; (<em>vii</em>) RUSBoost; e (<em>viii</em>) XGBoost.</p><h3>Results</h3><p>In step 1, a total of 13,123 records were tested (Fig. 4). Just to remind you, step 1 refers to predicting the invoices that will be paid late. In a way, the models got pretty close results, but XGBoost did better throughout the experiment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/509/1*AuNmRYbHg02K53-ARXJTvg.png" /><figcaption>Fig. 4: Step 1 results (P = <em>Precision</em>, R = <em>Recall</em>, F = <em>F1-Score</em>)</figcaption></figure><p>In step 2, a total of 5693 records were tested (Fig. 5). So about 85% of invoices paid after the due month were identified. Here, Balanced Random Forest was the one with the best result. It’s worth mentioning that AdaBoost achieved greater precision but a low recall.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*cqMAwu3z6xwsShKPKe8VUg.png" /><figcaption>Fig. 5: Step 2 results (P = <em>Precision</em>, R = <em>Recall</em>, F = <em>F1-Score</em>)</figcaption></figure><p>Finally, in step 3, a total of 1493 records were tested (Fig. 6). So for this step, the models would have to predict the number of days late after the due month.</p><p>Well, this step has one comment: the metric. Initially, I didn’t know how to handle the regression results for people outside the computing context. I know MAE, MSE, RMSE… but it didn’t make sense to other people. So I organized the results as follows: accuracy under predictions and day variations. That is, for variation 0, the model has to predict the exact number of days. In variation 1, the model has to predict the number of days with up to 1 variation… and so on. I know… I learned the meaning of the MAPE and MAAPE metrics a while after this project.</p><p>In general, AdaBoost stood out a lot in this step (between 60–84%), even achieving low results compared to the steps before.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*HD4NuR7BBiRIlOuqF-pDVg.png" /><figcaption>Fig. 6: Step 3results (A = <em>Accuracy, </em>± Day variation)</figcaption></figure><h3>Conclusion</h3><p>In the end, I saw that dividing the problem into ´subproblems´ helped to achieve the proposed objective. We have more information in the results, which only adds to who will use the models daily.</p><p>I would like to mention that the project is not the secret formula but only a possible way to solve the problem addressed. Also, there were several points where I would improve currently. Here I refer directly to step 3. For example, we could have approached it as a sorting step and divided the late days into ranges.</p><p>It’s funny to see old projects from another perspective, with more knowledge and self-criticism. For anyone who has read my post about decision trees and signatures, well, I changed my mind about decision trees. Anyway, I hope this journey helps someone and also me in the future. <em>haha</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce72a2fbccab" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Spelling Correction using TensorFlow 2.x]]></title>
            <link>https://arthurflor23.medium.com/spelling-correction-using-tensorflow-2-x-a063f428c106?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/a063f428c106</guid>
            <category><![CDATA[spelling-correction]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Sun, 01 Nov 2020 13:49:57 GMT</pubDate>
            <atom:updated>2023-03-02T14:44:55.885Z</atom:updated>
            <content:encoded><![CDATA[<p>Study and application of Spelling Correction in offline Handwritten Text Recognition Systems</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6lG-4g_jIIco25FPyRa1AQ.png" /></figure><p>Welcome back, my dear reader. In this post, I bring a continuation of my master degree project, the Spelling Correction. You can find the first part <a href="https://medium.com/@arthurflor23/handwritten-text-recognition-using-tensorflow-2-0-f4352b7afe16">here</a>.</p><p>Firstly, I would like to thank everyone who contacted me and sent feedback on the others posts/projects, thank you! This makes me motivated and now, I present a new project.</p><p>This post will present my journey in this area of Natural Processing Language, then, no codes here, but only my experience, some concepts that I learned, some results and conclusions.</p><p>Just to highlight, the focus of spelling correction here is to use it in conjunction with Handwritten Text Recognition, so the goals and metrics may differ from the conventional ones (Grammatical Error Correction or Machine Translation, for example).</p><p>If you’re a code lover, here the project with tutorial Jupyter Notebook and python files:</p><ul><li><a href="https://github.com/arthurflor23/spelling-correction">https://github.com/arthurflor23/spelling-correction</a></li></ul><h3>Introduction</h3><p>Beginning with my mission here. My goal was study and present some alternatives to post processing text in the <a href="https://medium.com/@arthurflor23/handwritten-text-recognition-using-tensorflow-2-0-f4352b7afe16">HTR</a>, because currently in this area of research, the <a href="https://kaldi-asr.org/">Kaldi</a> toolkit with <a href="http://www.speech.sri.com/projects/srilm/">SRILM</a> is used in HTR output through the statistical approach, thus improving the text recognized by the optical model. ~<em>This was all information that I had~</em></p><p><em>Text Correction</em> or <em>Spelling Correction</em>. What do you think? The first search that I did, I found some statistical approaches, such as <a href="http://norvig.com/spell-correct.html">Norvig</a> and <a href="https://github.com/wolfgarbe/SymSpell">SymSpell</a> techniques and several “trivial” doubts.</p><blockquote>What is token? n-gram? filtering? lemmatization? Language Modeling? Where is Kaldi and SRILM? <em>And more later,</em> &lt;EOS&gt;? &lt;SOS&gt;? encoders? decoders? attention? multi-head attention?</blockquote><p>In this way, I found this excellent post <a href="https://machinelearnings.co/deep-spelling-9ffef96a24f6">Deep Spelling</a> and this <a href="https://github.com/sebastianruder/NLP-progress/blob/master/english/grammatical_error_correction.md">GitHub repository</a>. So, I saw that Neural Networks have been growing in this area too. ~C<em>heers</em>~</p><p>In fact, I had more doubts that I expected, but on the other hand, I knew I would use character-level <strong><em>tokens</em></strong> and HTR metrics (such as Character Error Rate and Word Error Rate), instead word-level <strong><em>tokens</em></strong> and GER metrics (such as Precision and BLEU). Finally, my goal would be simpler too, unlike the Machine Translation and Grammar Error Correction approaches.</p><p>At the end of a long research and study, I summarized the existing techniques that I would use in the project (there is more, of course), they are:</p><ul><li><strong>Statistical</strong>: <em>Similarity (n-gram)</em>, <em>Norvig</em>, <em>Symspell</em>, <em>Kaldi + SRILM.</em></li><li><strong>Neural Network</strong>: <em>Seq2Seq</em> (<em>Bahdanau and Luong attention</em>), <em>Transformer.</em></li></ul><p>To an expert, these methods are “common” and nothing new. I agree. Newer approaches already deal with <a href="https://arxiv.org/abs/1901.02860">Transformer-XL</a> and <a href="https://arxiv.org/abs/1901.11117">Evolved Transformer</a>, but I preferred to go basic and simple in this task. Then, I’ll try to quickly explain each of them, along with the terms and methods used.</p><h3>Statistical Approach</h3><p>It is interesting, but it has many statistical approaches to spelling correction. I will only mention a few that I saw and used. However, first is necessary explain what is Tokens and N-grams, since they are common terms in this area.</p><p>Just to mention, the explanations here were from what I understood from readings and articles, after all the work, code, and various errors that I went through. So, I just detail for you, <em>my point of view</em>, since some works just skip over these subjects, or just explain in general, assuming we already know.</p><h4>Tokens and N-grams</h4><p>Well, Tokens are pieces of our context work, this is, texts. Simple, right? <strong>No! </strong>What <em>pieces </em>of the text<em> </em>means? And now, we have two approaches: can be words pieces (word level), or characters pieces (char level).</p><p>So, before coding, we have to know what context we want to work. For instance, statistical methods can use word level, making a word dictionary of their occurrences, in this case, the words become tokens, and we can combine with <em>n-gram</em> approach to make new tokens, if we want. However, only pure char level can be hard to correct, and we can’t make a simple char in a token (<em>you can see why?</em>). We need first add some <em>information</em> and in statistical scenarios, we must combine with <em>n-gram</em> approach to make our char level tokens.</p><p>But now, what is n-gram approach? Imagine how we can partition (segment) our text into several sections of the same size without leaving any combination out. Yeah, this is n-gram, where the <em>N </em>is the size that we want to partition. I really like the example figure of the N-gram definition from <a href="https://en.wikipedia.org/wiki/N-gram">Wikipedia</a>, is simple, objective and intuitive:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DI1HYvz3l-2tyf3kzdZkvg.png" /></figure><p>You can see the underscores “_” right? Remember that partition must be the same size. So, we can use underscore or any other “special character” to fill.</p><h4>Similarity</h4><p>This technique is the more simple from all. It just makes n-grams from our dataset and use similarity approach to search the possible “correct text”. <a href="https://pdfs.semanticscholar.org/4aee/94c2a0da3245335ec3378b255873e863dad6.pdf">Similarity</a> approach will just check how many n-grams is shared between the texts and the most shared is the potential correction.</p><p>I used N-gram python module. Simple and have fuzzy search to performance:</p><blockquote>PyPI: <a href="https://pypi.org/project/ngram/">https://pypi.org/project/ngram</a></blockquote><blockquote>Github: <a href="https://github.com/gpoulter/python-ngram">https://github.com/gpoulter/python-ngram</a></blockquote><h4>Norvig</h4><p><a href="http://norvig.com/spell-correct.html">Norvig</a> is already better known in research, but from what I understand, it is already obsolete. However, it is still interesting and uses a different approach. It uses word level and <a href="https://nymity.ch/sybilhunting/pdf/Levenshtein1966a.pdf">Levenshtein Distance</a> (Edit Distance) to compute the scores for the potential correction. In addition, it compares all permutations (insertions, exclusions, substitutions and transpositions) with the words in the dataset to arrive at that score. However, with these permutations, have a computational cost and time to process, so it’s the slowest.</p><p>I used Pyspellchecker python module:</p><blockquote>PyPI: <a href="https://pypi.org/project/pyspellchecker/">https://pypi.org/project/pyspellchecker/</a></blockquote><blockquote>Github: <a href="https://github.com/barrust/pyspellchecker">https://github.com/barrust/pyspellchecker</a></blockquote><h4>SymSpell</h4><p>This is my surprise, SymSpell was the first technique that I found, with this awesome posts: <a href="https://towardsdatascience.com/symspellcompound-10ec8f467c9b"><em>1000x faster Spelling Correction</em></a> and <a href="https://towardsdatascience.com/symspell-vs-bk-tree-100x-faster-fuzzy-string-search-spell-checking-c4f10d80a078"><em>SymSpell vs. BK-tree: 100x faster fuzzy string search &amp; spell checking</em></a><em>. </em>Yeah, by the titles you may have noticed that it is fast.</p><p>The interesting part is that it uses the symmetric delete algorithm to reduce the processing complexity (different from that presented by Norvig) and makes use of the Damerau-Levenshtein distance for the scores. I really don’t understanding how can be so fast. ~<em>haha</em>~</p><p>Just for you to see a speed comparison, this image from author Wolf Garbe:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/0*eMNzVQG7GaFYE_k4.png" /></figure><p>Anyway, I used the Symspellpy python module (you can find in several other languages as well):</p><p>PyPI: <a href="https://pypi.org/project/symspellpy/">https://pypi.org/project/symspellpy/</a></p><p>Github: <a href="https://github.com/mammothb/symspellpy">https://github.com/mammothb/symspellpy</a></p><p>Original GitHub: <a href="https://github.com/wolfgarbe/SymSpell">https://github.com/wolfgarbe/SymSpell</a></p><h4>Kaldi + SRILM</h4><p>OK, so… this is the approach that I least understood, the most complicated that I found, however, the most used in HTR (at least that’s what <em>they</em> told me), but faster (of course, the only one in C++) and with better results.</p><blockquote>Let’s do it by steps.</blockquote><p><a href="https://kaldi-asr.org/">Kaldi</a> is a speech recognition toolkit (<em>yeah, speech recognition</em>) and <a href="http://www.speech.sri.com/projects/srilm/">SRILM</a> is a toolkit for building and applying statistical Language Models (LMs).</p><p>So, we need to download and compile the source code of the two tools and integrate them (Kaldi uses SRILM). After that, we will need to execute some Kaldi commands to decode the output of the optical model (now the list of prediction matrices, that is, the model output without going through the CTC decoder). In general, Kaldi will decode the predictions, create the Language Model using SRILM, create a statistical structure using Hidden Markov Models, create lexical structures, correct the predictions and finally, decode into text using the dataset’s character dictionary. Uff !!!</p><p>Just to you know, Kaldi is a very good choice, but I think that is <em>workaround </em>to HTR context (sorry about that). Why? You ask me. I don’t know indeed, just I feel that <em>break</em> the process…</p><p>Well, I make this project <a href="https://github.com/arthurflor23/handwritten-text-recognition">Handwritten Text Recognition</a> with TensorFlow 2 and we can run all the project through browser. The output is already the final text. So, when I had to use the Kaldi’s approach, I had to create another alternative output, in which I literally cut the end of the decoding process and with that, we will no longer have the final text.</p><p>This output is a series of standardized files to make easy in the Kaldi input (you can get this files through <em>kaldi_assets </em>parameter :]).</p><p>And finally, I used a script in the <a href="https://github.com/arthurflor23/spelling-correction">Spelling Correction</a> project, to get this standardized files and the Kaldi + SRILM toolkits (already compiled by yourself) to make an automatic execution/correction. ~<em>Did you think I would do it manually?</em>~</p><p>I know this is a huge and complicated process, but I just tried to automate this integration, no matter how much it went out of the project pattern.</p><h3>Neural Network Approach</h3><p>Recently (2014 to today), I saw an increase in the use of Neural Network for Machine Translation and Grammatical Error Correction. The results really are incredible, but with gigantic models, datasets of billions of sentences and with days or weeks of training. Yes, I know, kind of demotivating for those just starting out. So I tried to go with a simple approach, with the smallest model possible and with hours or days of training (<em>not ideal yet, but ok</em>).</p><p>Just to you know, the process of tokens is the same, just forget n-grams for now, because here we have to feed the model through numbers instead of texts. So, I will complement this subject with One Hot Encoding and Embeddings.</p><p><strong>Encode/Decode Tokens</strong></p><p>To work with Neural Network, we must work with numbers. Then, if we work with char level tokens, we need use One Hot Encoding. Instead, word level tokens can be better using Embeddings approach.</p><p>One Hot Encoding, briefly, is transform the text into binary matrix, mapping the tokens. For isntance:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/1*nwRMReaFWG33FOFGrhTfrA.png" /></figure><p>Where ABC = [[1, 0, 0], [0, 1, 0], [0, 0, 1]]</p><p>Embeddings, briefly, is transform the text into numbers and then, use Embedding Layer into the model to compute more information from texts. For isntance:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*44iKifJpkCpAt8Ojyju8Mw.png" /></figure><p>Where ABC = [1, 2, 3]</p><p>I really, really, simplified, but this is the concept indeed. In addition, both scenarios you have <em>special </em>tokens, predefined and mandatory in this context. Special tokens can be any unique token in the entire dataset, so:</p><p><strong>Start sentence:</strong> to indicate the begin of sentence. We can use &lt;SOS&gt; or &lt;B&gt; to word level or some special character to char level.</p><p><strong>End sentence: </strong>to indicate the end of sentence. We can use &lt;EOS&gt; or &lt;E&gt; to word level or some special character to char level.</p><p><strong>Space</strong>: to indicate a space in the sentence. We can use &lt;SPACE&gt; or &lt;S&gt; to word level only.</p><p><strong>Padding: </strong>to indicate a sentence filling (remember that in the neural network, the input is matrices of encoded texts). We can use &lt;PAD&gt;, or PAD, or &lt;P&gt; to word level or some special character to char level. Another interesting information is that there is a <em>standardization</em> of always using this token in the first index, that is, index 0.</p><p><strong>Unknown: </strong>to indicate an unexpected element, never seen inside the dataset. We can use &lt;UNK&gt;, or UNK, or &lt;U&gt; to word level or some special character to char level.</p><p>To make it clearer, here are some examples for both levels, encoding “I code” and “You code” sentences.</p><p>First, for one hot encoding (char level), we will use a character dictionary that macthes all character possibilities: [¶, t, d, I, Y, , u, e ,c ,o ,« ,¤ , »]. ~<em>can you see the special tokens?</em>~</p><p>So, what we really encode is:</p><blockquote>«I code»¶¶ → 10 tokens</blockquote><blockquote>«You code» → 10 tokens</blockquote><p>Then:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*2ioRsk-t6Jp85c7oJX85lw.png" /><figcaption>One hot encoding (char level) example</figcaption></figure><p>Now, for embeddings (word level), we will use a word dictionary that macthes all words possibilities: [&lt;P&gt;, I, You, &lt;S&gt;, code, &lt;B&gt;, &lt;U&gt;, &lt;E&gt;] ~again, <em>can you see the special tokens?</em>~</p><p>So, what we really encode is:</p><blockquote>&lt;B&gt; I &lt;S&gt; code &lt;E&gt; → 6 tokens</blockquote><blockquote>&lt;B&gt; You &lt;S&gt; code &lt;E&gt; → 6 tokens</blockquote><p>Then:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*2q4yO_VMJIDk2jNuoe1iPw.png" /><figcaption>Embedding encoding (word level) example</figcaption></figure><p>Simpler, right? This is because the Embeddings layers will also transform into matrices and add more information with decimal values instead of integer values (0, 1). But we don’t need to worry about that.</p><p>About the decoding process. Well, with the dictionary in hand, just do the reverse process.</p><h4>Sequence to Sequence</h4><p>The most popular approach in this area, Sequence to Sequence, or Seq2Seq for the most intimate, this method consists of using Recurrent Neural Network to work with text. “Work with text” because you can classify, extract information, understand feelings, correct, generate dialogues and among other activities, all from text. Powerful, I know.</p><p>A little bit of history, the highlights in this area initially emerged with the LSTM layers, in which it was studied how to improve the learning. For instance, there is an <a href="https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf">article</a> that presents the process of inverting sentences, as it found a better use in training.</p><p>Anyway, the concept here is the same. Seq2Seq has one Encoder and Decoder model. The encoder will obtain the states of the input that will pass to the decoder, which will process the states of the input and the states of the ground truth in order to give the prediction output. <em>Complicated</em>.</p><p>If you understood above, you noticed that there is more than one input. Therefore, as input for our model, we have: (i) encoder input; (ii) decoder input (ground truth); and for calculate loss function: (iii) ground truth. The detail is that each of these inputs has its own encoding (text → number). Then:</p><blockquote><strong>Encoder input</strong>: <strong>&lt;S&gt;</strong> your_text <strong>&lt;E&gt;</strong> &lt;P&gt; &lt;P&gt;…</blockquote><blockquote><strong>Decoder input</strong>: <strong>&lt;S&gt;</strong> your_text &lt;P&gt; &lt;P&gt;…</blockquote><blockquote><strong>Ground truth</strong>: your_text <strong>&lt;E&gt;</strong> &lt;P&gt; &lt;P&gt;…</blockquote><p>There is more than one approach to encoding the Encoder inputs. However, this one that I presented covers the current models (<em>that use Bidirectional Recurrent Neural Networks </em><strong><em>#</em></strong><em>spoiler</em>).</p><p>In this <a href="https://medium.com/shortcutnlp/how-to-implement-seq2seq-lstm-model-in-keras-shortcutnlp-f55fadd7f812">post</a> and image below, you can see a different approach to encoding the inputs, with no Start/End token in encoder input sentence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/0*B19ipG9npfyT-9OS.png" /><figcaption>Seq2Seq workflow example (with no Start/End token in encoder input sentence)</figcaption></figure><h4>Bahdanau Attention</h4><p>Everything changed with the arrival of the Attention layer proposed by <a href="https://arxiv.org/pdf/1409.0473.pdf">Bahdanau</a> (originally in 2014). Now the models have gone from 60% to 95% accuracy. Just by applying the attention process between the encoder and decoder states.</p><p>And what means <em>Attention</em> process? in general and quickly, Attention is an “<em>alignment</em>” between two vectors/matrices, based on <em>scores</em>. Bahdanau proposed calculating these scores, based on concatenations. This makes processing, memory usage and training time an aggravating factor, even though it provides beautiful results and is now the most used by the most well-known NLP tools.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/328/1*NchyioLJtZGclqYlyHE9tA.png" /><figcaption>Bahdanau Attention Mechanism</figcaption></figure><h4>Luong Attention</h4><p>As a way of improving, <a href="https://arxiv.org/pdf/1508.04025.pdf">Luong</a> (2015) proposed a new approach to calculate Attention scores, now using dot product. In this way, the model was extremely light and fast in training, even delivering a result just below the Bahdanau (at least in my experiments).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/1*jd8PhOk7c9K2Ttirmwu9Dw.png" /><figcaption>Luong Attention Mechanism</figcaption></figure><h4>Transformer</h4><p>Finally, Transformer model, or Multi-Head Attention by Vaswani et al. (2017) and with the best title paper <a href="https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf"><em>Attention is All you Need</em></a><em>.</em></p><p>I really created high expectations for this model and followed step by step in Google’s excellent tutorial (<a href="https://www.tensorflow.org/tutorials/text/transformer">Transformer Model for Language Understanding</a>).</p><p>The concept is really interesting. Also use dot product process, but there are no Recurrent layers, <em>just attention</em>. Yeah, the process here is based only on recursively attention, or self attention.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*h6d-tCVUudO-CXzY4Gattw.png" /><figcaption>Encoder and Decoder Transformer Model</figcaption></figure><p>The bad thing here is the decoding process. Because if recursion is used to decode. <em>My context </em>was to use sentences with 128 tokens (high value for the usual). Then imagine Self Attention decoding in token by token loop, sentence by sentence. If you spend a lot of time so a huge trade-off to me, because it is extremely fast to train and with a very nice result, but slow to decode. ~<em>I need to hit a code in C ++!~</em></p><p>Anyway, today there are studies aimed at solving this problem, improving, of course, the results (<a href="https://arxiv.org/abs/1901.02860">Transformer-XL</a> and <a href="https://arxiv.org/abs/1901.11117">Evolved Transformer</a> for example).</p><p>I know I explained it very quickly and superficially, but I don’t want to go into the code details or each implementation and operation. For this, all three approaches follow the same coding and decoding logic, same flow as Encoder Model and Decoder Model. In addition, you can follow the char or word level, depending on the objective.</p><h3>Datasets</h3><p>This step is complicated for the HTR area. Let me explain, both Machine Translation and Grammatical Error Correction have huge datasets to work with text. I can list <a href="https://www.cl.cam.ac.uk/research/nl/bea2019st/">BEA2019</a>, <a href="https://www.comp.nus.edu.sg/~nlp/conll13st.html">CoNLL13</a>, <a href="https://www.comp.nus.edu.sg/~nlp/conll14st.html">CoNLL14</a> and <a href="https://research.google/pubs/pub41880/">One Billion by Google</a> (the project has support for them :]). In these datasets there is no partitioning methodology, so we can divide it by 10% for validation, 10% for testing and the rest for training. The training will works fine and after a long time, our model will predict the correction of the sentence. Everything OK so far.</p><p>However, I want to work with HTR datasets, to correct them using a small model. Here I list <a href="http://transcriptorium.eu/datasets/bentham-collection/">Bentham</a>, <a href="http://www.fki.inf.unibe.ch/databases/iam-handwriting-database">IAM</a>, <a href="http://www.a2ialab.com/doku.php?id=rimes_database:start">Rimes</a>, <a href="http://www.fki.inf.unibe.ch/databases/iam-historical-document-database/saint-gall-database">Saint Gall</a>, <a href="http://www.fki.inf.unibe.ch/databases/iam-historical-document-database/washington-database">Washington</a>. In addition, the purpose of these datasets is only the recognition of text in image for the digital environment. Here we got some problems.</p><p>The <em>first problem</em> here is the scarcity of data. BEA2019 has about 35 thousand sentences, while in the largest HTR dataset I had access to, it had 11 thousand.</p><p>The <em>second</em> problem, the most serious, is the lack of distribution of tokens within the partitioning of the HTR datasets. And why the most serious? Well, we can’t train a model to learn “I do”, and put the sentence “I nned” to predict. The result will be “I do” (<em>wrong</em>), because the model doesn’t know the existence of the “need” token and then correct it correctly.</p><p>It took me a long time to realize this and the only solution (not the ideal) I found was to generate new sentences from the dataset using word level n-grams (<em>yes!!</em>). The process is somewhat similar, but instead of just using a value for N, we are going to use some values and some restrictions. I called it <em>multigrams</em> to make it easier to understand. So to exemplify:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/1*LdtcVsBT09xsehXlVug_7w.png" /><figcaption><em>Multigram Process example</em></figcaption></figure><p>With the partitioned dataset, I redistributed the multigrams on the training and validation partition (to maintain the 10% ratio) and kept the original test partition. This increased the volume of data considerably.</p><p>Remember, if you use a proper dataset for text, you don’t need this step. Therefore, for the HTR context, the ideal scenario would be a text image dataset for recognition, in which your tokens are in equal proportions in the three partitions (training, validation and testing).</p><p>You can also find more details and some images of these datasets in this previous <a href="https://medium.com/@arthurflor23/handwritten-text-recognition-using-tensorflow-2-0-f4352b7afe16">post</a>. However, in this project only it texts are used.</p><h4>Bentham</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*4HLR1u0BaTazyqmIGm2Xjw.png" /></figure><p><a href="http://transcriptorium.eu/datasets/bentham-collection/">Bentham</a> database is a collection of manuscripts written by English philosopher Jeremy Bentham (1748–1832).</p><h4>IAM</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/291/1*0snpbugEtz-72V28pTVBnQ.png" /></figure><p>The Institut für Informatik und Angewandte Mathematik (<a href="http://www.fki.inf.unibe.ch/databases/iam-handwriting-database">IAM</a>) database contains forms with English manuscripts.</p><h4>Rimes</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/306/1*rN4LNKVDMjI2ShfLbTNL_Q.png" /></figure><p>The Reconnaissance et Indexation de données Manuscrites et de fac similÉS (<a href="http://www.a2ialab.com/doku.php?id=rimes_database:start">Rimes</a>) database is a collection of text written in French.</p><h4>Saint Gall</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*VTVhXYWZXdCWFEXDosZx6Q.png" /></figure><p><a href="http://www.fki.inf.unibe.ch/databases/iam-historical-document-database/saint-gall-database">Saint Gall</a> database brings manuscripts in Latin from the 9th century.</p><h4>Washington</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/351/1*BDgOJEgYIhU6VJTNi60qBQ.png" /></figure><p>Finally, <a href="http://www.fki.inf.unibe.ch/databases/iam-historical-document-database/washington-database">Washington</a> database (the smaller) was created from the George Washington Papers (English) at 18th century.</p><h3>Experiment Setup</h3><p>In this stage, the usual process was used for the statistical approach: creation of the <em>corpus</em> from the dataset; creation of the dictionary/Language Model with the occurrences; find the best <em>N</em> value for that dataset + technique; finally, get the possible correction.</p><p>For the Neural Network approach, the training step is used to adjust the weights; to then predict the possible correction.</p><p>Specifically for this context of Neural Networks, Spelling Correction and HTR, I used the char level (One hot enconding) and kept the original partitioning of the datasets (with the addition of multigrams in training and validation).</p><p>In addition, I didn’t bring the results using Bahdanau Attention and the Transformer model, as they didn’t meet my expectations (training time or decoding, for example), but I kept the code implementation in the project repository as well.</p><p>Thus, I used a combination of the Bahdanau’s Model (who brings encoder step with Bidirectional layer), using Gated Recurrent layers (instead conventional LSTM), using Luong Attention with Layer Normalizaton after (this last I learned from the Transformer model). ~yes, <em>a great mix</em>~</p><p>Finally, I kept the units value at 512 and dropout at 0.2 (the normal for small models), using batches with 128 sentences. ReduceLRonPlateau and Early Stopping were also applied after 15 and 20 epochs without improvement in the value of the loss of validation, respectively.</p><p>In summary, the training was created to finish as quickly as possible. If someone needs to keep training for longer, or increase the model’s capacity, feel free to increase the parameters.</p><h4>Noises</h4><blockquote>In the end, how will the model learn to correct sentences if we only have the ground truth?</blockquote><p>Yeah, this step is really important. We will generate random errors in the data. For the validation partition only once (of course), for training data, we generate each interaction.</p><p>It is a simple function that generates a noisy sentence before encoding. I found this great <a href="https://machinelearnings.co/deep-spelling-9ffef96a24f6#.2c9pu8nlm">post</a> and <a href="https://github.com/MajorTal/DeepSpell">repository</a> from Tal Weiss, where he uses a conventional process of randomly add/remove/transpose/replace some token in sentence.</p><p>In addition, I had to go a little further, because this project has a connection with my HTR project, so I added two more possibilities in <em>remove mode</em>, which were: <em>accentuation</em> and <em>repeated letters</em> in sequence. So, I kept a percentage of 10% to 15% in the generated error rate, how you can see below an example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*naulIUSCihUiRarYUj9zIA.png" /><figcaption>Sample of the noise generation process</figcaption></figure><h4>Metrics</h4><p>Unlike Machine Translation and GER metrics (such as precision, or BLEU), which aim to evaluate the correction a little more deeply and semantically. Here I made another change, according to the area of Optical Character Recognition (OCR). So, I used the metrics Character Error Rate (CER), Word Error Rate (WER) and Sequence Error Rate (SER).</p><p>In addition to the OCR metrics, I also analyzed the time in seconds when executing the correction/prediction on each item.</p><p>I know they aren’t the best metrics for this type of activity (at least I haven’t seen any articles with this evaluation approach), but my goal was to integrate with the HTR system, so I kept the metrics throughout the workflow.</p><h4>Google Colab: Tutorial Mode</h4><p>Finally, all training and predictions were conducted on the <a href="https://colab.research.google.com/">Google Colab</a> platform. The platform offers Linux operating system, with 12GB ram and Nvidia Tesla P100 GPU 16GB memory. ~<em>thank you again, Google</em> ❤❤~</p><h3>Results</h3><p>First to explain, all inputs are the <em>outputs</em> of the first project (<a href="https://medium.com/@arthurflor23/handwritten-text-recognition-using-tensorflow-2-0-f4352b7afe16">HTR</a>). That said, we have 5 datasets and 3 optical model architectures (Bluche, Flor, Puigcerver). So there are 15 types of input for the 5 approaches used in this experiment (4 statistical and 1 Se2Seq). ~<em>a lot of results</em>~</p><p>In general, the results were favorable to Seq2Seq, especially if the input has a low error rate. However, the runtime stayed with Kaldi + SRILM (<em>of course!</em>). But it is interesting to note that SymSpell comes close in speed and result (remember, <em>c++ vs python !!</em>). In addition, Norvig approach had the slowest execution time and Similarity the worst result.</p><h4>Bentham</h4><p>Bentham dataset had very close results. Bluche inputs have almost 13% error, in which the model’s training was between 10% — 15%, so Seq2Seq lost to Kaldi in this architecture (CER value only). However, Seq2Seq reached an incredible 2.91% CER, 6.75% WER and 33.12% SER.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_nnBDA7eeY_9jBdstcCugQ.png" /><figcaption>Measures in the Bentham dataset</figcaption></figure><p>Some examples of corrections:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FhIL-cwMK6RV1TggqUF__A.png" /><figcaption>Sample results in the Bentham dataset</figcaption></figure><h4>IAM</h4><p>The IAM dataset was the most complicated by the tokens in it sentences. Again, Bluche inputs have almost 13% error, then Seq2Seq lost to Kaldi in this architecture (CER value only) and Seq2Seq reached 2.85% CER, 6.41% WER and 32.65% SER.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SIPNasJHU7Ysn1B_yg5scA.png" /><figcaption>Measures in the IAM dataset</figcaption></figure><p>Some examples of corrections:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*55uPbLuFYyfxbYK5dm0mOw.png" /><figcaption>Sample results in the IAM dataset</figcaption></figure><h4>Rimes</h4><p>Rimes dataset starts to become interesting, as the sentences have a certain context and better sharing of tokens. Seq2Seq reached 1.71% CER, 4.88% WER and 23.71% SER. ~<em>fantastic</em>~</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*61D02uppoSLe9PHtUIrCgQ.png" /><figcaption>Measures in the Rimes dataset</figcaption></figure><p>Some examples of corrections:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B45WOK2-Byw9o3RYyaipyQ.png" /><figcaption>Sample results in the Rimes dataset</figcaption></figure><h4>Saint Gall</h4><p>Saint Gall dataset was also easy, because it presents Latin language, because it has the Latin language, it has no punctuation or accentuation signs. The amount of tokens was the smallest. Seq2Seq reach 2.11% CER, 5.03% WER, 24.53% SER.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K8Lo3nNuHKNSHv_0MIescw.png" /><figcaption>Measures in the Saint Gall dataset</figcaption></figure><p>Some examples of corrections:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YUr3HHgpsmzYbEY8XsBFeA.png" /><figcaption>Sample results in the Saint Gall dataset</figcaption></figure><h4>Washington</h4><p>Finally, Washington has the smallest database, but not as complicated as IAM or Bentham. Seq2Seq had 1.29% CER, 2.97% WER, 16.88% SER. Most errors were concentrated in a few sentences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XeXArXtGy8HuEbXN5DDnjA.png" /><figcaption>Measures in the Washington dataset</figcaption></figure><p>Some examples of corrections:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z5GqKKUgegPS0FAp9m50zw.png" /><figcaption>Sample results in the Washington dataset</figcaption></figure><p>Even though I don’t get into much of the discussions here, it is important to highlight the variations in the <em>N</em> values for each statistical approach, in each architecture and on each base, while in a Seq2Seq we don’t have to worry about that.</p><p>Another interesting point is the relationship between the percentage of error of the inputs with that used in training. First, it can cause incompatibility in the correction (seen with the Bluche architecture on the Bentham and IAM bases) and second, with the increase in the amount of noise, the model may go into overfitting, unless we also increase the number of neurons.</p><h3>Conclusion</h3><p>Dear reader, if you have read this far, congratulations! If you jumped this far, well, let’s go ahead anyway. After all the study, implementation and experimentation within the context of Handwritten Text Recognition, I came to some points in my head:</p><ul><li>SymSpell is a great option for quick, practical implementations and good results. ~<em>nothing to complain about here</em>~</li><li>Kaldi + SRILM is a good alternative and a powerful tool, however, I think it quite laborious and complicated (specially for <em>newbies</em> like me). From that point of view, it would be better to invest a little more time and dedication in the performance of Seq2Seq/Transformer model for this task.</li><li>I realized that encoder-decoder models is still at the beginning in the HTR area, even with good results, it isn’t all the potential that it can offer. Maybe we can say that HTR research field haven’t left behind yet the statistical approach, making hard to work with incompatible/restrict datasets, for example. ~<em>I know it’s not easy, but nobody said it would be</em>~</li></ul><p>Therefore, this implementation is a simple example of the power of seq2seq in a simple spelling correction task. If you want to improve performance (datasets with more tokens, or multi-language), you may be able to try more training epochs, or higher number of units, or even try the transformers (<em>they are amazing</em>).</p><p>I wrote this post almost a year ago and we are now able to publish the article “<a href="https://www.mdpi.com/2076-3417/10/21/7711">Towards natural language processing as a spell check for offline handwriting recognition systems</a>”, as a result of my master’s project. The concepts used are the same, but for more complete content and updated results, I also recommend reading the article. Finally, feel free to improve and reference the project ~ <em>Cheers!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a063f428c106" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Growing Experience in Web Development]]></title>
            <link>https://arthurflor23.medium.com/growing-experience-in-web-development-a648f98be417?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/a648f98be417</guid>
            <category><![CDATA[personal-growth]]></category>
            <category><![CDATA[personal-development]]></category>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[experience]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Thu, 02 Apr 2020 23:14:59 GMT</pubDate>
            <atom:updated>2020-04-02T23:41:51.603Z</atom:updated>
            <content:encoded><![CDATA[<p>A quick story about my web projects and their weaknesses ~ <em>yes, weaknesses!</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kUlaUl5OZnfVWUiegZ2biQ.png" /></figure><p><em>And so it is… just like you said it would be… ~ </em>After two years away, I return to a web project as a front-end developer. I created the most complicated logical components of my career and as Damien Rice said ~ <em>No love, no glory… No hero in her sky</em>. In addition, it motivated me to write about my experience in web projects and I will bring specifically only four that I considered so important to my life.</p><p>Unfortunately, will not have code or technical code explanations in this post, this is not the focus here. And as you, my dear reader, must have read in the subtitle, I will focus on the weaknesses of the projects with my current look. In fact, I believe that this is the only way we can see how much we have evolved from one project to another.</p><p>Anyway, let’s do it by steps. The first one is Otatop, my bachelor degree project; the second is Si-dade, my startup project; third is Leader, my first app within a company; and finally, the Labeling, a website with image editor functionality.</p><h4>1. Otatop</h4><p>The year was 2016, and I was finishing my bachelor degree. Until there, we just saw Java Web and particularly I didn’t like. So, I decided to use JavaScript (<em>yeah</em>) with <a href="https://jquery.com/">jQuery</a>, <a href="https://www.jointjs.com/">JointJS</a> and Intel XDK frameworks (<em>ouch</em>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/1*Rbe2quYBFjKLMPwExlIA7g.png" /></figure><p>Anyway, this project aimed to create basic systems based on the Entity-Relationship Model. The ER model would also be modeled from the platform and it would also generate a normalized database.</p><p>For my bachelor degree I used the Database field to study, write the concepts and improve the logic functions in code. However, this project never saw the sunlight.</p><p>Looking at it today, I realize how non-optimized and unstructured it was. To give an idea, I used jQuery to dynamically deal with the DOM and thus be able to create the relationships of the entities within the diagram on the canvas with the JointJS (<em>oh God</em>). Looking at the repository, the <em>node_modules</em> folder is on GitHub (<em>shameful laughs</em>). In the end, I made a project usable and friendlier than other tools of the time.</p><p>Finally, I still remember the days worked on this project and the happiness I had when I was able to implement the relationship logic and the normalization steps through the diagrams. Good times.</p><h4>2. Si-dade</h4><p>The years 2017 and 2018 were purely learning for me. On the one hand, I was trying to set up a Startup with friends, and we were able to enter the city’s incubation program. On the other hand, we needed to develop the new project with recent technologies that are easy to handle. So, in the front-end I chose the <a href="https://ionicframework.com/">Ionic</a> Framework (version 2+).</p><p>Here is an interesting point. I had left jQuery, JavaScript and Intel XDK straight to Ionic with <a href="https://angular.io/">Angular</a> and TypeScript. I really don’t need to say how amazed I was and how much information I had to absorb to develop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*DPCm6aXjFC8Kt89XctvaSg.png" /></figure><p>The project aimed to map points in the city where people went through some kind of danger. In this way it would be possible to plot a map of the city with the “main” places considered to be dangerous and would serve for the public government to make decisions about it.</p><p>We tried hard at that time to make the startup work. But again, it didn’t go forward. Today, I consider that we lost a lot of time in development (<em>maybe because we are learning in parallel? I don’t know</em>), but this delayed the resources (money and motivation) to keep the company alive (<em>lesson learned</em>).</p><p>Anyway, my criticism here goes to the separate development between the mobile application and the administrative panel. Today I know that it is possible to do everything in a single project (single and responsive code), being well-structured and managing sessions of the common and administrative users.</p><h4>3. Leader</h4><p>In my first official job, at the end of Startup and also between 2017 and 2018, I had a mobile project to develop with full responsibility for development decisions and as I was the only programmer, I had to do all the work (basically a full stack).</p><p>In this project, I went first to Ionic on mobile and for the back-end I chose Node.js. Needless to say, I had never used Node (<em>haha</em>), however, I used the Lumber Framework (from <a href="https://github.com/ForestAdmin/lumber">ForestAdmin</a>) to manage the administrative side. This allowed me to develop the project within the very short time I had.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*xYRA8-TLMJySLjXUT1nNSA.png" /></figure><p>The project aimed to manage the itinerary of salespeople’s activities through the mobile application, allowing the generation of reports to supervisors on the administrative side. This helped in decision-making and created better planning.</p><p>This project was a success in the company, but shortly after finishing the project documentation, I found a better job opportunity in the capital and honestly I don’t know if they continued using the project, since I was the only programmer in the company.</p><p>Anyway, my points here goes to the development of the project totally alone (a team was really needed); the development of expendable features; unnecessary complex functions; and the damn stupidity of uploading unnecessary files to the repository (never do that!).</p><h4>4. Labeling</h4><blockquote>I’ll spend more text here, because this project motivated me to write this post and is fresh in my mind.</blockquote><p>A lot has happened since Leader until today. In fact, this project appeared between 2019 and 2020 and I had no intention of participating, as I currently work more in the master’s degree with machine learning. However, I accepted and again I was free to define the technologies used in the front-end, as well as the creation of mockups.</p><p>The project had the proposal to develop only a website and because I had a short time, I chose Angular 9.x (for already knowing) with Nebular framework (UI) in the development of the front-end. What surprised me how much the tools have evolved and how easy it was to develop a fully responsive and modulated project.</p><blockquote>No project icon here, because it’s a private company :]</blockquote><p>The objective was to create a website that would allow users to manage (through profiles) label activities to other users. This labeling would be done from already trained models (from the company) and also through the user himself.</p><p>Here we got a point. Profiles, session and activity management, reports, webservice, JSON objects… everything until then was predictable, but edit the image labels? Soon I imagined the Canvas + jQuery that I used in the first project and I would definitely do it in a more correct way this time.</p><p>Fortunately I did it in a more coherent way through the incredible <a href="http://fabricjs.com/">Fabric.js</a>. Unfortunately, this functionality cost me a hundred times more time than I imagined. To understand the complexity, the image that canvas would load would have a scale factor and a centering adjustment according to the responsiveness of the canvas. If the window size was changed, the image would adjust itself (responsive), but that would include all objects on screen as well (labels). Not only that, but the user could zoom in as well. So it would be a factor of scale, centering and zoom in the image and labels. <em>~160 hours only to make this component… No love, no glory… No hero in her sky…~</em></p><p>Let’s go to the next step: components integration. Imagine that we have the Figure 1. Three components on the screen: the first a list of images; the second, the image editor with the label management; the third, a form created dynamically through webservice parameters (already including dynamic field validations and everything you can imagine).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*v0bHDEzIhjyR4kqsLI2TFQ.png" /><figcaption>Figure 1. Mockup example</figcaption></figure><p>Now, the webservice send me a standard object format (pattern X). The Angular form with validation is another pattern object (pattern Y); and Fabric.js (canvas objects) is another pattern object (pattern W). However, form interactions must update canvas objects and canvas interactions must update form values. Finally, to send back the values to the server, must return to the standard object format…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*msjZDlh8NzE6gmmv.gif" /></figure><p>Saving you from further explanations, the final code of the components was complex, but well modulated through Angular services. Thus, this is (<em>for now</em>) my new work pride (Figure 2). ~<em>dark theme included!~</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vaku3rCxuBXBtTtpJomWvA.png" /><figcaption>Figure 2. Labeling example</figcaption></figure><blockquote>I’m really curious to know what the Arthur three years later will think about this project.</blockquote><p>If you’ve read this far, thank you! This post helped me to put out some web projects I went through and remember a good part of them.</p><p>In addition, this post made me reflect on how much I have changed in recent years. Whether in projects, teams, codes, decisions. In fact, that same web development learning curve, I’m going through in machine learning (check out the other <a href="https://medium.com/@arthurflor23">posts</a>). Today I look at my first projects in the field of AI and I already see their weaknesses.</p><p>Finally, for you who like machine learning, I will publish a new post sometime about the second part of my master’s degree project. #spoiler #NLP</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a648f98be417" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Handwritten Text Recognition using TensorFlow 2.0]]></title>
            <link>https://arthurflor23.medium.com/handwritten-text-recognition-using-tensorflow-2-0-f4352b7afe16?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4352b7afe16</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[ocr]]></category>
            <category><![CDATA[experience]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Sat, 21 Sep 2019 16:49:26 GMT</pubDate>
            <atom:updated>2023-03-02T14:44:43.968Z</atom:updated>
            <content:encoded><![CDATA[<p>An easy-to-use implementation of offline HTR system (line-level)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6if352Mo5SrYfplfxNY8kg.png" /></figure><p>How long my friend. Hope you’re fine. This time I bring to you a new project/post. Now with offline Handwritten Text Recognition (line-level), basic concepts, state-of-art models, my new proposed model, results and conclusions. All this using TensorFlow 2.0, through an easy-to-use code.</p><p><strong>Firstly</strong>, I would like to thank the readers of the other posts and contacts I had. Feel free to contact me, I will help you whenever possible. <strong>Secondly</strong>, even with this title name, this post will have no code and no technical approach (maybe I’ve lied here), but rather a description of the project development and some points that I find interesting.</p><p>For more detail, you can read the project presentation, well as other approaches used with a little more in depth. In addition, for code lovers, you can also find the open source, documentation and even a tutorial:</p><ul><li><a href="https://github.com/arthurflor23/handwritten-text-recognition">https://github.com/arthurflor23/handwritten-text-recognition</a></li><li><a href="https://dl.acm.org/doi/abs/10.1145/3395027.3419603">HTR-Flor++: A Handwritten Text Recognition System Based on a Pipeline of Optical and Language Models</a></li></ul><h3><strong>Introduction</strong></h3><p>For those starting in the Optical Character Recognition (OCR) environment, I find it interesting to bring a brief context.</p><p>According to [1], OCR systems have two categories: <em>online</em>, in which input information is obtained through real-time writing sensors; and <em>offline</em>, in which input information is obtained through static information (images). Within the <em>offline</em> category, there is the recognition of typed and manuscript text (Figure 1).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/679/1*fVcnjlZ4-u-8WBZqHq_QaQ.png" /><figcaption>Fig. 1: Classification of Optical Character Recognition from Sonkusare and Sahu (2016)</figcaption></figure><p>So, Handwritten Text Recognition (HTR) has the purpose of transcribing cursive text to the digital medium (ASCII, Unicode) [2].</p><p>For many years, HTR systems have used the Hidden Markov Models (HMM) for the transcription task [3]–[5], but recently, through Deep Learning, the Convolutional Recurrent Neural Networks (CRNN) approach has been used to overcome some limitations of HMM [6], [7]. To exemplify a CRNN model, I bring the [8] model (Figure 2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*QYH2MLTiU_RlvSYFpUjV_w.png" /><figcaption>Fig. 2: Overview of a CRNN (source: <a href="https://towardsdatascience.com/build-a-handwritten-text-recognition-system-using-tensorflow-2326a3487cd5">Build a Handwritten Text Recognition System using TensorFlow</a>)</figcaption></figure><p>The workflow can be divided into 3 steps. <strong>Step 1</strong>: the input image is fed into the CNN layers to extract features. The output is a feature map. <strong>Step 2</strong>: through the implementation of Long Short-Term Memory (LSTM), the RNN is able to propagate information over longer distances and provide more robust features to training. <strong>Step 3</strong>: with RNN output matrix, the Connectionist Temporal Classification (CTC) [9] calculates loss value and also decodes into the final text.</p><p>For more detail on this process, I recommend reading <a href="https://towardsdatascience.com/build-a-handwritten-text-recognition-system-using-tensorflow-2326a3487cd5"><em>Build a Handwritten Text Recognition System using TensorFlow</em></a> by Harald Scheidl (2018).</p><p>Lastly,<strong> is very important explain that for this post</strong>, step 3 (CTC) is the same for all architectures presented, then the Vanilla Beam Search [10] method is used, since it doesn’t require a dictionary for its application, unlike other known methods such as Token Passing [11] and Word Beam Search [8]. Therefore, the architectures presented in the following sections only act in steps 1 and 2.</p><p>In addition, the <strong>charset</strong> for encoding text is also the same for all datasets. So, the list used consists of 95 printable characters from ASCII table (Figure 3) by default and doesn’t contain accented letters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7vYunynRWejDksRf2SfkrQ.png" /><figcaption>Fig. 3: charset adopted to encoding text</figcaption></figure><p>Now you might be wondering why use the same CTC and charset for the architectures and datasets. Well, for this experiment purposefully, I wanted to bring a more uniform approach between the models and databases. Since the use of specific dictionaries and charsets can bring better results in some scenarios, but not so much in others. For example, a charset with only 60 characters is “easier to hit” (1/60) than another with 125 (1/125). In fact, the approach here becomes more difficult for text recognition by the models.</p><h3>Datasets</h3><p>For the experiment, it was used the free segmentation approach of text lines of <a href="http://transcriptorium.eu/datasets/bentham-collection">Bentham</a>, <a href="http://www.fki.inf.unibe.ch/databases/iam-handwriting-database">IAM</a>, <a href="http://www.a2ialab.com/doku.php?id=rimes_database:start">Rimes</a> and <a href="http://www.fki.inf.unibe.ch/databases/iam-historical-document-%20database/saint-gall-database">Saint Gall</a> datasets.</p><p>The methodology of partitioning (training, validation and test) of the data was kept original for each dataset, except for Rimes, since it doesn’t have the validation partition. For this case, the validation data is a subset of 10% of the training partition.</p><p>Bringing a brief detailing of each dataset, Bentham [12] is a collection of manuscripts written by English philosopher Jeremy Bentham (1748–1832). This historical dataset (Figure 4) is the most complex among the four adopted. It also has a considerable amount of punctuation marks in the texts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9Sk1fevaclk-2VNJumWrcQ.png" /><figcaption>Fig. 4: Sample images from Bentham and data numbers per partition</figcaption></figure><p>The Institut für Informatik und Angewandte Mathematik (IAM) database [13] contains forms with English manuscripts, which can be considered as a simple base, since it has a good quality for text recognition (Figure 5). However, it brings the challenge of having several writers, that is, the cursive style is unrestricted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xsUfUB4g4171IUqSzcwY6Q.png" /><figcaption>Fig. 5: Sample images from IAM and data numbers per partition</figcaption></figure><p>The Reconnaissance et Indexation de données Manuscrites et de fac similÉS (Rimes) database [14] is a collection of text written in French language by several writers (Figure 6). The text recognition is also considered easy because there is a good writing of the texts, however, the French language brings more accented words.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R1cxdOhguyMZQw_QXPB_ig.png" /><figcaption>Fig. 6: Sample images from Rimes and data numbers per partition</figcaption></figure><p>Lastly, the Saint Gall database [15] brings manuscripts in Latin from the 9th century of only one writer (Figure 7). The images obtained are already binarized and normalized and the biggest challenge for this collection is to deal with overfitting, since it has around 1,400 text lines in total and the writing style is very regular.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3xdyl44S72P7YoCbeJorPw.png" /><figcaption>Fig. 7: Sample images from Saint Gall and data numbers per partition</figcaption></figure><p>In short, each explored database brings a certain challenge. I know that there are specific techniques to solve each of these challenges, but the purpose of the study is to bring methods and approaches that are common to the all datasets presented (just to remember).</p><p>Finally, to make an easy input of the data into model, the project was prepared to transform each dataset. The transformation process is simple. It put already preprocessed images, ground truth texts and encoded texts into only one HDF5 file. This possibility a fast loading of the data, easy uploading to Google Drive (if you run there) and decreases the storage space (e.g. dataset with 2.8gb becomes 360mb).</p><h3>Architectures</h3><p>I chose two model approaches from state-of-art as inspiration for mine. However, I won’t go into the smallest details of each one, but I will leave some important points and, of course, references of the works.</p><p>I would also like to comment that the architectures I chose, correspond to approaches that use BLSTM in the recurrent layer (RNN) and that were eventually compared with MDLSTM [16] as an alternative to computational cost and high complexity. If you are interested in the details and approaches used in each study, I recommend the readings:</p><ul><li><a href="http://www.jpuigcerver.net/pubs/jpuigcerver_icdar2017.pdf"><em>Are multidimensional recurrent layers really necessary for handwritten text recognition?</em></a> (J. Puigcerver, 2017)</li><li><a href="http://www.tbluche.com/files/icdar17_gnn.pdf"><em>Gated convolutional recurrent neural net- works for multilingual handwriting recognition</em></a> (T. Bluche and R. Messina, 2017)</li></ul><p>In this way, the first architecture (CNN-BLSTM), introduced by Puigcerver [17], has a high level of recognition rate and a large number of parameters (around 9.6 million). The Figure 8 shows in detail the distribution of parameters and hyperparameters through 5 convolutional layers and 5 BLSTM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/301/1*hopSrn4VvoWXVxmVEnpoAg.png" /><figcaption>Fig. 8: Workflow of the Puigcerver architecture</figcaption></figure><p>The second architecture (Gated-CNN-BLSTM), now introduced by Bluche and Messina [18], has the highlight of the Gated-CNN approach. In summary, this technique aims to extract more relevant features of the image. Instead Puigcerver approach, this model has very few parameters (around 730 thousand), making it more compact and faster. The Figure 9 shows in detail the distribution of parameters and hyperparameter sthrough 8 convolutional layers (3 gated included) and 2 BLSTM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*948_iqinXU5W5TqdifAmsg.png" /><figcaption>Fig. 9: Workflow of the Bluche et al. architecture</figcaption></figure><p>Lastly, the proposed architecture (Gated-CNN-BLSTM) that was inspired by [17] and [18], aiming at: (<em>i</em>) to achieve results compatible with Puigcerver model; and (<em>ii</em>) to keep the small number of parameters (not exceeding one million), such as Bluche et al. model.</p><p>It was a great challenge to combine the two advantages of each architecture into a single one, however, in the developmental, I applied another Gated approach, once presented by [19] in the context of Language Model. Other methods that also brought differences in results (improvements) were the Batch Renormalization [20] and the Parametric Rectified Linear Unit (PReLU) activator [21].</p><p>Thus, the proposed Gated-CNN-BLSTM architecture preserves the low number of parameters (around 820 thousand) and high recognition rate. The Figure 10 shows in detail the distribution of parameters and hyperparameter through 11 convolutional layers (5 gated included) and 2 BLSTM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/401/1*uD6qtugHQtb7uqcpFx3z_w.png" /><figcaption>Fig. 10: Workflow of the Flor architecture</figcaption></figure><p>For training, it was used the RMSProp optimizer [22] with the standard learning rate of each model to incrementally update the parameters using CTC loss gradients.</p><h3>Experiment Setup</h3><p>An important help to this project it was the <a href="https://github.com/ysoullard/CTCModel"><em>CTCModel: a Keras Model for Connectionist Temporal Classification</em></a> [23]<em> </em>repository/paper<em>.</em> In this way, it was possible to abstract the workflow of the HTR system, making easy-to-use code. In short, instead of you worry about the loss value ​​and decoding of outputs (usually in a “manual” way), just call the training and prediction methods as in the Keras, in this case, TensorFlow Keras. Simple like that.</p><p>So, each model was trained to minimize the validation loss value of the CTC function. To improve and normalize images, it was applied two techniques in the preprocessing: (<em>i</em>) <a href="https://github.com/fanyirobin/text-image-binarization">Illumination Compensation</a> [24] to remove shadows and balance brightness and contrast; (<em>ii</em>) <a href="https://github.com/githubharald/DeslantImg">Deslanting</a> [25] to remove the cursive style. A comparison of the image without/with preprocessing is shown in Figure 11.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x5M_QHzEWsD4DM75V7aaBg.png" /><figcaption>Fig. 11: Bentham preprocessing example: original (a), only normalization (b), illumination compensation + deslanting + normalization (c)</figcaption></figure><p>Continuing, the 1024x128 resizing (with padding) was also performed on all input images and Data Augmentation application to increase the amount of the training partition through morphological and displacement transformations (only minor variations).</p><p>It was set for all the minibatch of 16 size and Early Stopping after 20 epochs without improvement in validation loss value. For the best use of each model, within the 20 tolerance epochs, also used ReduceLRonPlateau schedule [26] with a decay factor of 0.2 after 10 epochs without improvement also in validation loss value.</p><p>The metrics used to validate the results, calculated through the Levenshtein Distance [27] between the ground truth and the predictions, are: (<em>i</em>) Character Error Rate (CER), and (<em>ii</em>) Word Error Rate (WER). As expected, the WER results tend to be greater than CER, since it corresponds to the distribution of the error of the characters in words [28].</p><p>Finally, all training and predictions were conducted on the <a href="https://colab.research.google.com/">Google Colaboratory</a> (Colab) platform. By default, the platform offers Linux operating system, with 12GB ram and Nvidia Tesla T4 GPU 16GB memory (thank you so much, Google ❤).</p><h3>Results</h3><p>For all tests, doesn’t apply any kind of post-processing, such as Language Model, but only the CTC decoding in the final text is taken into account (<em>oh yeah</em>). Fortunately, the proposed model achieved good results and there is a discussion to highlight for each dataset.</p><p>Starting with Bentham dataset, the punctuation marks correspond about 20% of the error rate. In this way, disregarding the punctuation marks, the recognition rate outperforming around 5%. The Table 1 shows the results details between the models in these three scenarios: (<em>i</em>) text without any type of normalization (default); (<em>ii</em>) only punctuation marks; and (<em>iii</em>) text without punctuation marks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/1*9iu66wamG2MCOBL5zKOOTQ.png" /><figcaption>Table 1: Line-level results on the Bentham database</figcaption></figure><p>For IAM database, the proposed architecture also achieved better results compared to the others, having an improvement of up to 8%. The details are shown in Table 2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*CKCgz6RJ7ygD40Cb7IETLA.png" /><figcaption>Table 2: Line-level results on the IAM database</figcaption></figure><p>In the Rimes database, punctuation marks don’t influence as much as accent marks used by the French language. In this case, the accent errors are equivalent to about 19% of the error rate. Thus, for both scenarios, the proposed architecture achieved up to 18% improvement. The Table 3 shows the details in both scenarios.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*zD9n5zq9qd_gaK62Chx_Pw.png" /><figcaption>Table 3: Line-level results on the Rimes database</figcaption></figure><p>Finally, the Saint Gall database still suffers from overfitting, as expected, and doesn’t have punctuation and accentuation, referring to the Latin language. In this scenario the results were closer, but also got an improvement of up to 6%. Table 4 shows the details of the results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*US0za9Yf-dkETnezsU_ciw.png" /><figcaption>Table 4: Line-level results on the Saint Gall database</figcaption></figure><p>For this post, I brought brief information about the results. However, if you are interested, there are more details in the GitHub repository (<em>doc</em> folder), or if you prefer, in the project paper.</p><h3><strong>Conclusions</strong></h3><p>In this quick post, I presented my recent project on Handwritten Text Recognition (HTR). Overall, after 3 months of study and development, I saw that many HTR projects tend to be very “manual” in model training (when there is code). Of course, traditional Optical Character Recognition (OCR) systems are simpler and by Keras it’s possible to develop easier, for example, in tutorials with the MNIST dataset. However, for HTR this scenario is not the same.</p><p>The <a href="https://github.com/ysoullard/CTCModel/blob/master/CTCModel.py">CTCModel</a>, now I extended to <a href="https://github.com/arthurflor23/handwritten-text-recognition/blob/master/src/network/model.py">HTRModel</a>, makes development much simpler and more automatic. Anyway, you can check through the <a href="https://github.com/arthurflor23/handwritten-text-recognition/blob/master/src/tutorial.ipynb">tutorial</a> in Jupyter Notebook (you can run through main python file too). I hope at least one of the two codes helps new text recognition projects.</p><p>In addition, until arriving at the proposed architecture, many tests were performed (a lot) and even the order of activation and normalization layers influence the results. One approach I haven’t tested is to change the traditional convolutional layers to more specific ones, such as Depthwise Convolutional [29] or Octave Convolutional [30], for example. ~ <em>I leave this tip ~</em></p><p>Thus, let me know if you liked it, or whether it helped or not. Thanks for read until here, see you!</p><h3>References</h3><p>[1] M. Sonkusare and N. Sahu, “A survey on handwritten character recognition (hcr) techniques for english alphabets”, Advances in Vision Computing: An International Journal, vol. 3, pp. 1–12, 03 2016.</p><p>[2] B. L. D. Bezerra, C. Zanchettin, A. H. Toselli, and G. Pirlo, Handwriting: Recognition, Development and Analysis. Nova Science Pub Inc, jul 2017.</p><p>[3] H. Bunke, M. Roth, and E. G. Schukat-Talamazzini, “Off-line cursive handwriting recognition using hidden markov models”, Pattern Recognition, vol. 28, pp. 1399–1413, 09 1995.</p><p>[4] B. S. Saritha and S. Hemanth, “An efficient hidden markov model for offline handwritten numeral recognition”, 2010.</p><p>[5] A. H. Toselli and E. Vidal, “Handwritten text recognition results on the bentham collection with improved classical n-gram-hmm methods”, in Proceedings of the 3rd International Workshop on Historical Document Imaging and Processing (HIP@ICDAR), 2015.</p><p>[6] A. Graves, S. Fernández, M. Liwicki, H. Bunke, and J. Schmidhuber, “Unconstrained on-line handwriting recognition with recurrent neural networks” in Advances in Neural Information Processing Systems 20, vol. 20, 01 2007.</p><p>[7] P. Voigtlaender, P. Doetsch, and H. Ney, “Handwriting recognition with large multidimensional long short-term memory recurrent neural networks”, in 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), 10 2016, pp. 228–233.</p><p>[8] H. Scheidl, S. Fiel, and R. Sablatnig, “Word beam search: A connectionist temporal classification decoding algorithm”, in 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). Niagara Falls, United States: IEEE Computer Society, 08 2018, pp. 253–258.</p><p>[9] A. Graves, S. Fernández, and F. Gomez, “Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks”, International Conference on Machine Learning, pp. 369–376, 2006.</p><p>[10] K. Hwang and W. Sung, “Character-level incremental speech recognition with recurrent neural networks”, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 3 2016. [Online]. Available: <a href="http://dx.doi.org/10.1109/ICASSP.2016.7472696">http://dx.doi.org/10.1109/ICASSP.2016.7472696</a></p><p>[11] A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber, “A novel connectionist system for unconstrained handwriting recognition”, IEEE transactions on pattern analysis and machine intelligence, vol. 31, pp. 855–68, 06 2009.</p><p>[12] M. Diem, S. Fiel, F. Kleber, R. Sablatnig, J. M. Saavedra, D. Contreras, J. M. Barrios, and L. S. de Oliveira, “Competition on handwritten digitstring recognition in challenging datasets (hdsrc 2014)”, 14th International Conference on Frontiers in Handwriting Recognition (ICFHR), vol. 2014, 12 2014.</p><p>[13] U.-V. Marti and H. Bunke, “The iam-database: An english sentence database for offline handwriting recognition”, International Journal on Document Analysis and Recognition, vol. 5, pp. 39–46, 11 2002.</p><p>[14] E. Grosicki, M. Carre, J.-M. Brodin, and E. Geoffrois, “Rimes evaluation campaign for handwritten mail processing”, in ICFHR 2008 : 11th International Conference on Frontiers in Handwriting Recognition. Montreal, Canada: Concordia University, 8 2008, pp. 1–6. [Online]. Available: <a href="https://hal.archives-ouvertes.fr/hal-01395332">https://hal.archives-ouvertes.fr/hal-01395332</a>.</p><p>[15] A. Fischer, E. Indermühle, H. Bunke, G. Viehhauser, and M. Stolz, “Ground truth creation for handwriting recognition in historical documents”, ACM International Conference Proceeding Series, pp. 3–10, 01 2010.</p><p>[16] B. Moysset and R. O. Messina, “Are 2D-LSTM really dead for offline text recognition?” CoRR, vol. abs/1811.10899, 2018.</p><p>[17] J. Puigcerver, “Are multidimensional recurrent layers really necessary for handwritten text recognition?” 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 67–72, 11 2017.</p><p>[18] T. Bluche and R. Messina, “Gated convolutional recurrent neural net- works for multilingual handwriting recognition”, 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 646–651, 11 2017.</p><p>[19] Y. N. Dauphin, A. Fan, M. Auli, and D. Grangier, “Language modeling with gated convolutional networks”, 2017.</p><p>[20] S. Ioffe, “Batch renormalization: Towards reducing minibatch<br>dependence in batch-normalized models”, CoRR, vol. abs/1702.03275,<br>2017. [Online]. Available: <a href="http://arxiv.org/abs/1702.03275">http://arxiv.org/abs/1702.03275</a></p><p>[21] K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification”, in 2015 IEEE International Conference on Computer Vision (ICCV), 12 2015, pp. 1026–1034.</p><p>[22] T. Tieleman and G. Hinton, “Lecture 6.5–rmsprop: Divide the gradient by a running average of its recent magnitude”, COURSERA: Neural Networks for Machine Learning, 2012.</p><p>[23] Y. Soullard, C. Ruffino, and T. Paquet, “CTCModel: Connectionist Temporal Classification in Keras”, 2018.</p><p>[24] K.-N. Chen, C.-H. Chen, and C.-C. Chang, “Efficient illumination compensation techniques for text images”, Digital Signal Processing, vol. 22, no. 5, pp. 726–733, 2012. [Online]. Available: <a href="http://www.sciencedirect.com/science/article/pii/S1051200412000826">http://www.sciencedirect.com/science/article/pii/S1051200412000826</a></p><p>[25] A. Vinciarelli and J. Luettin, “A new normalization technique for cursive handwritten words”, Pattern Recognition Letters, vol. 22(9), pp. 1043–1050, 07 2001.</p><p>[26] F. Chollet, “Keras: Deep learning library for theano and tensorflow”, 2015.</p><p>[27] ] V. Levenshtein, “Binary codes capable of correcting deletions, insertions and reversals”, Soviet Physics Doklady, vol. 10, p. 707, 1966.</p><p>[28] J. A. Sánchez, V. Romero, A. H. Toselli, M. Villegas, and E. Vidal, “A set of benchmarks for handwritten text recognition on historical documents”, Pattern Recognition, vol. 94, pp. 122–134, 2019.</p><p>[29] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weylan, M. Andreetto, H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications”, 2017. [Online]. Available: <a href="https://arxiv.org/pdf/1704.04861.pdf">https://arxiv.org/pdf/1704.04861.pdf</a></p><p>[30] Y. Chen, H. Fan, B. Xu, Z. Yan, Y. Kalantidis, M. Rohrbach, S. Yan, J. Feng, “Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution”, 2019. [Online]. Available: <a href="https://arxiv.org/pdf/1904.05049.pdf">https://arxiv.org/pdf/1904.05049.pdf</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4352b7afe16" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Experience on Implementing a state of art Deep Learning model using TensorFlow 2.0]]></title>
            <link>https://arthurflor23.medium.com/experience-on-implementing-a-state-of-art-deep-learning-model-using-tensorflow-2-0-9b77ae63adf6?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/9b77ae63adf6</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[experience]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Thu, 15 Aug 2019 12:45:11 GMT</pubDate>
            <atom:updated>2020-09-21T18:13:02.283Z</atom:updated>
            <content:encoded><![CDATA[<h3>Experience on Implementing a State-of-the-Art Model Using TensorFlow 2.0</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TQkw5_VQfKewwwsiKxJfOA.png" /></figure><p>In this post, Carlos Timoteo (my co-author on this post) and I share our experience on implementing a Deep Learning model using <a href="https://www.tensorflow.org/beta/">TensorFlow 2.0</a>. We’re going to describe on some level of detail our experience.</p><h3>Introduction</h3><p>TensorFlow (TF) is an open-source machine learning framework for research and production use cases. The TF project is maintained by several companies in the industry, including Google Alphabet. Being the second largest open-source project on <a href="https://github.com/tensorflow/tensorflow">GitHub</a> by numbers of contributors, providing an easy to read documentation and several tutorials and guides.</p><p>We decided to adopt TensorFlow because of its huge ecosystem to help implement a real solution to a research problem. The TF presents a higher level abstraction to build and train deep learning models by using Keras API. TF.Keras is an implementation of the Keras API optimized within TensorFlow. <a href="http://keras.io/">Keras</a> API is based on three key principles: user-friendly; modular and composable; and easy to extend. More details on <a href="https://www.tensorflow.org/about/">TensorFlow</a> and <a href="https://www.tensorflow.org/guide/keras">TF.Keras</a> can be found online.</p><p>Besides, TensorFlow enables researchers to build and train state-of-the-art models even with scarce resources. It provides you to use Keras Functional API and Model Subclassing API to create complex topologies. If you need easy prototype and fast debug, use eager execution. TensorFlow also supports an ecosystem of powerful add-on libraries and models to experiment with, including <a href="https://www.tensorflow.org/hub">TFHub</a>, <a href="https://medium.com/tensorflow/introducing-ragged-tensors-ac301c31fd38">Ragged Tensors</a>, <a href="https://www.tensorflow.org/probability">TensorFlow Probability</a>, <a href="http://ai.googleblog.com/2017/06/accelerating-deep-learning-research.html">Tensor2Tensor</a> and <a href="https://github.com/google-research/bert">BERT</a>. Here is a <a href="https://www.tensorflow.org/beta/guide/effective_tf2">guide</a> on how you can effectively use TensorFlow 2.0.</p><h4>Arthur’s Journey on Machine Learning</h4><p>Well, before working on TensorFlow, my journey in neural networks started with Keras. At that time, I was in the early days on learning machine learning and others researchers recommended me to use Keras, by stating that it was TensorFlow 1.x on the easy way. So why not? I decided to go with Keras. The curious thing is that even without knowing anything in the area and reading a lot of articles and tutorials, I could implement a nice research application project, check the repo <a href="https://medium.com/@arthurflor23/surface-crack-detection-92fb27bf0f18">here</a>. It was really easy and practical to work with Keras.</p><blockquote>Why start talking about Keras? Wouldn’t it be TensorFlow 2.0?</blockquote><p>And you are right! I just would like to give some context before having first contact with TF 1.x and TF 2.0. I’ve been introduced to TensorFlow 1.x and 2.0 by Carlos Timoteo, when we’ve worked together on January 2019. In that opportunity, we’ve organized a Google <a href="https://events.withgoogle.com/cloud-studyjam/">Cloud Study Jam</a> on Machine Learning in Recife. Well, I ended up choosing TF 2.0 (still beta release) as the framework for my dissertation project, in February 2019, for a few reasons. First, what I found was the same Keras framework, I had zero down-time for adaptation. Second, it was cleaned, improved and maintained by TensorFlow team, keeping the same easy logic to build ML solutions. Last, I’m not an old school researcher, any tool that gives me ease, usability and good performance, I would just adopt it (Figure 1). Wonderful, isn’t it?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wnE-HSBn6D_IC9NK" /><figcaption><strong>Fig. 1</strong>: Fry wanting TensorFlow 2.0</figcaption></figure><p>Another quick note, with TF 2.0 Beta, I had no critical bugs, but of course, you can find a few bugs. If you find any, please file an issue to the maintenance team on the GitHub <a href="https://github.com/tensorflow/tensorflow">repository</a> (this is very important for our <a href="https://www.tensorflow.org/community/">community</a>).</p><h3>Building a model using TF 2.0</h3><p>If you’re used to build models using TF 1.x, dealing with sessions and operations… just forget and embrace the pythonic way of programming. In the new version, we have a much easier to understand environment (especially for beginners). So we now have two high-level APIs, <a href="https://www.tensorflow.org/guide/estimators">Estimator API</a> and Keras. Those are the options to build a model:</p><ol><li>Using <a href="https://www.tensorflow.org/beta/tutorials/estimators/premade_estimators">Estimator API</a>, for when you need an out-of-box model, such as a Linear Classifier or Regressor model. Even though they are generic models, it’s a really good solution for fast prototyping and to establish baseline performance;</li><li>Using Keras <a href="https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/Sequential">Sequential</a> class, in which we sequentially stack layer by layer. Imagine this as a linear stack of layers. The data input flows linearly layer by layer, in which layers infers input shape, except the first. This is the easiest way to build simple models, but offers less flexibility;</li><li>Using Keras <a href="https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/Model">Model</a> class, in which it allows greater flexibility in the construction of layers in a functional way. Here we can connect non-sequential layers, such as a DNN called U-net. This <a href="https://www.kaggle.com/advaitsave/tensorflow-2-nuclei-segmentation-unet">post</a> exemplifies how to code a U-net using TensorFlow 2.0.</li></ol><p>In the last two options, at the end of construction you must compile the model which means configuring learning process parameters like optimizer, loss function and evaluation metrics. Very easy, isn’t it? In addition, TF 2.0 still maintains the `<a href="https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/nn">tf.nn</a>` API if you want to get into lower levels of hardcore style code. In this blog <a href="https://medium.com/google-developer-experts/demystify-the-tensorflow-apis-57d2b0b8b6c0">post</a>, you can find a better discussion on all options.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iIBCRBNthXYZk-6N" /><figcaption><strong>Fig. 2</strong>: TF 2.0 Model Building APIs. `tf.layers` are not supported anymore. (<a href="https://medium.com/google-developer-experts/demystify-the-tensorflow-apis-57d2b0b8b6c0">source</a>)</figcaption></figure><p>In any case, it’s recommended to use `tf.keras` as a development option as it combines great flexibility in creating any kind of model by combining functions. That way, TF 2.0 automatically executes and manages all low-level content.</p><p>Another feature provided by TF 2.0 is the options to dynamically build computational graphs using Eager Execution. You can use Eager Execution to easily test model building and model training. In this <a href="https://www.tensorflow.org/beta/guide/eager">tutorial</a> you can find examples on how to use Eager Execution. Pytorch early adopters have not a reason anymore to convince you to migrate.</p><p>For those who are in love with Transfer Learning, <a href="https://www.tensorflow.org/hub">TensorFlow Hub</a> is a library for publication, discovery, and consumption of reusable pre-trained machine learning models and layers. Here you find a <a href="https://www.tensorflow.org/beta/tutorials/images/hub_with_keras">tutorial</a> on how to use TF Hub with TF 2.0 Keras API to perform transfer learning.</p><p>If you are a researcher and still are not convinced on why you should use TensorFlow for research purposes check <a href="https://www.tensorflow.org/tfrc">TensorFlow Research Cloud </a>(TFRC) and <a href="https://github.com/tensorflow/models">TensorFlow Models</a>. TFRC program enables researchers to apply for access to a cluster of more than 1,000 Cloud TPUs. TensorFlow Model repository contains machine learning models implemented by researchers, including Google Brain and Deepmind researchers.</p><h3>Feed data to training routine</h3><p>There are two ways to train a model built using Keras <a href="https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/Model">Model</a> class. By calling `fit()` method, in which we train for a fixed number of epochs; or by calling `fit_generator()` method, in which we use <a href="https://jeffknupp.com/blog/2013/04/07/improve-your-python-yield-and-generators-explained">python generator</a> or <a href="https://www.tensorflow.org/guide/datasets">Dataset API</a> (`tf.data.Dataset`). The model trains on data yield batch-by-batch, all like Keras! Oh, by the way, TF Keras!</p><p>In Arthur’s project, there was a lot of image data to load, so the `fit_generator()` was used, because that way the memory with all the data was not exceeded. However, what we find fascinating here is the possibility to configure all training through parameters and callback functions. Split training and validation data. Define Learning rate Schedules, Early Stopping, Warms Restarts, Tensorboard, Model Checkpoint and many other options.</p><p>We could also use `<a href="https://medium.com/@arindambaidya168/https-medium-com-arindambaidya168-using-keras-imagedatagenerator-b94a87cdefad">ImageDataGenerator</a>()` a generator class that allows a variation of input data (images), creating a type of real-time training data augmentation with no memory cost. Would be great, but not for now.</p><p>As mentioned before, another approach is to use Dataset API. It’s important to mention that Tensorflow team introduced this API based on the <a href="https://www.tensorflow.org/guide/performance/datasets">performance analysis</a>. It utilizes multithreading in C++ level. So it would be really nice to work with it, because it’s easier to ingest and manipulate data from different sources than using <a href="https://docs.w3cub.com/tensorflow~python/reading_data/">QueueRunner</a>. It also enables you to build complex data feeding pipelines.</p><p>Now, we introduce a limitation that we ended up facing, or would it be a lack of performance? Anyway.. using `ImageDataGenerator()` in a model training, we depend on scikit-image package for image transformations (data augmentation function). The problem found was that the training was taking longer than expected. Therefore, we decided to create a little transform function using OpenCV (following the same logic as TF Keras, but simplified). Well, by implementing it by hand, the training now takes 20 seconds less per epoch. It looks like not too much, but for a 1,000 or 10,000 training epochs with the same amount of data, it’s a difference of several hours already.</p><p>Not convinced by all the benefits, Arthur have decided to suffer a little bit working with TF 1.x, dry-running and analyzing HTR implementations, and it was painful (<em>oh God</em>). HTR models are quite complex because they have four inputs and two outputs (adding the Connectionist Temporal Classification). To train such an optical model using TF 1.x takes too long and requires manual implementation of training routine. In the Figure 3, we show how to implement manually a training routine in which you feed data to model batch-by-batch, update the weights of the neural network, check if the model has learned or not, and implements early stop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xGx9bLv0A-qZwXW6" /><figcaption><strong>Fig. 3</strong>: Example of the HTR train code using TF 1.13</figcaption></figure><p>Using TF 2.0, you can create the same training routine for the same optical model for an HTR system by calling `fit_generator` method informing appropriate callbacks (Figure 4), and of course, extending the Model class from TensorFlow 2.0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/0*iJrtSSxiDWhBWMvX" /><figcaption><strong>Fig. 4</strong>: Example of the HTR train routine using TF 2.0</figcaption></figure><h3>Training on scale using TF 2.0</h3><p><a href="https://www.tensorflow.org/beta/guide/distribute_strategy">TensorFlow Distributed Strategy</a> is an API to distribute synchronously or asynchronously training across multiple GPUs, CPUs or TPUs. Using this API, you can scale existing models and training routines implemented using TF 2.0 Keras or Estimator API with minimal code changes.</p><p>To be honest, having this capability was fantastic! We could perform scalable training using <a href="https://colab.research.google.com/">Google Colab</a>, a platform that provides a free cloud service based on Jupyter Notebooks that provides free GPU and TPU. Thus, by using the `fit_generator()` method and GPU hardware on Colab, we could train a large amount of data without worrying about hardware allocation (like we would do using TF 1.x). On Figure 5, you can check training metrics on <a href="https://www.tensorflow.org/tensorboard">Tensorboard</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/756/1*QbLlmMPQPLF1lrFO4DidWQ.png" /><figcaption><strong>Fig. 5</strong>: Training converging example from Tensorboard</figcaption></figure><p>There, Arthur’s project, CPU was used for data ingestion and preprocessing, such as train/validation/test splitting, and GPU for model training. Thus, it was possible to optimize the use of resources during the training time.</p><p>What about TPU? Worth it? Well, we haven’t used it. TPU is an AI application-specific hardware accelerator developed by Google for machine learning use cases. To give an idea, we can say that the CPU can handle tens of operation per cycle; whether the GPU can handle tens of thousands of operations per cycle. TPUs can handle up a hundred thousand operations per cycle. The Figure 6 brings a perspective when comparing those hardware acceleration technologies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pvaZ4p9mpEgFZk2x" /><figcaption><strong>Fig. 6</strong>: Computing primitives in CPU, GPU and TPU, respectively (<a href="https://iq.opengenus.org/cpu-vs-gpu-vs-tpu">source</a>). The dot is a matrix multiplication operation.</figcaption></figure><p>In addition, you can also use data manipulation and training with GPU or TPU only. Depending on the project requirements, you may need to use <a href="https://www.tensorflow.org/tfx">TFX</a> and <a href="https://cloud.google.com/dataflow/">Google Cloud Dataflow</a>.</p><h3>Check Documentation</h3><p>A final tip for those who will venture into ML frameworks. Check the API <a href="https://www.tensorflow.org/versions/r2.0/api_docs/python">documentation</a> on the website!</p><p>We can find the descriptions, parameters of the methods, tutorials, references to the original technical papers from authors who proposed the methods. Besides, for every parameter there are brief suggestions on their use and in which context it should be applied. It really helps a lot if you don’t know how to apply things.</p><p>For instance, the <a href="https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/layers/BatchNormalization">BatchNormalization</a> class, we can find some interesting information in the docs:</p><ol><li>Short Description: Base class of Batch normalization layer (Ioffe and Szegedy, 2014);</li><li>Research Paper: <a href="https://arxiv.org/abs/1502.03167">Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift</a>;</li><li>Parameter <strong>fused</strong>: if True, use a faster, fused implementation, or raise a ValueError if the fused implementation cannot be used. If None, use the faster implementation if possible. If False, do not used the fused implementation;</li><li>Parameter <strong>renorm</strong>: whether to use <a href="https://arxiv.org/abs/1702.03275">Batch Renormalization</a>. This adds extra variables during training. The inference is the same for either value of this parameter.</li></ol><p>So, have you ever heard about fused and renorm implementation? Did you know that Batch Renormalization is best used for mini-batch? Or only by using the fused parameter (True), training becomes faster, because it uses optimized implementation?</p><h3>Conclusions</h3><p>Even though TensorFlow 2.0 is in Beta, we didn’t see any major problems using it. On the contrary, it made it possible to develop a project in the simplest and most effective way for me. TF 2.0 still have much to offer regarding TPU support, bug fixes, as well as some documentation updates.</p><p>We hope this post can help those who are just starting in this area of study and describe some of our experience with the framework. Certainly, we’ll use TPUs in future projects, but GPUs are already sufficient for most studies.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9b77ae63adf6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Text Segmentation]]></title>
            <link>https://arthurflor23.medium.com/text-segmentation-b32503ef2613?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/b32503ef2613</guid>
            <category><![CDATA[handwriting]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[text]]></category>
            <category><![CDATA[opencv]]></category>
            <category><![CDATA[segmentation]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Fri, 29 Mar 2019 12:49:33 GMT</pubDate>
            <atom:updated>2023-03-02T14:44:22.287Z</atom:updated>
            <content:encoded><![CDATA[<p>Document scanner until word segmentation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*K6Hjzt8uMtTaeIl0cGPd9g.png" /><figcaption>Fig. 1: Sample of text segmentation</figcaption></figure><p>Hey, welcome back! This project was about text segmentation. More specifically, the process of handwritten text segmentation using digital image process.</p><p>At first, the repository was created to be a start point to another project. So, the idea is to start the segmentation process to make something after with results. For code lovers:</p><ul><li><a href="https://github.com/arthurflor23/text-segmentation">https://github.com/arthurflor23/text-segmentation</a></li></ul><p>The initial goal was to study and implement a handwritten text line segmentation. Simple… And here I come, with more complication, study, and implementation. I must say that at the end of the study, I learned that I should not use this implementation in a real-world problem (I’ll get there).</p><p>So, to make a line segmentation, for after do a word segmentation, first, you must think the <strong>binarization technique</strong> and most importantly: <em>how your data </em><strong><em>will be</em></strong>.</p><p>Why did I say that? Because you can get simple data images from <a href="http://www.fki.inf.unibe.ch/databases/iam-handwriting-database">IAM dataset</a>, or get some like <a href="http://transcriptorium.eu/datasets/bentham-collection/">Bentham dataset</a>, for example. So it depends on what you are working on. I like working with more difficult scenarios, because if the project achieves good results, consequently it will bring great results in simpler scenarios.</p><blockquote>I already notice that this project was in the simplest scenarios. This brought me a bit of frustration, but of course, a lot more knowledge.</blockquote><h3>Document Scanner</h3><p>The first step, document scanner, isn’t necessary if you get ready datasets, but think that images come from a smartphone camera or simple scan. Anyway, is important <em>normalize</em> the input images to get only the important area to segmentation.</p><p>This is an automatic process. In reality, the perfect scenario to this would be to create an <em>interface</em> with the user, like the <a href="https://play.google.com/store/apps/details?id=com.intsig.camscanner">CamScanner</a> app does, for example. In this step, we detecting the predominant contour in image and segment using a <em>four-point perspective</em> <em>transformation</em>. Here I leave two very good links:</p><ol><li><a href="https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/">4 Point OpenCV getPerspective Transform</a>;</li><li><a href="https://www.pyimagesearch.com/2014/09/01/build-kick-ass-mobile-document-scanner-just-5-minutes/">How to Build a Kick-Ass Mobile Document Scanner in Just 5 Minutes</a>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/1*F0IvQaFfa9ZrGxI3LsEyog.png" /><figcaption>Fig. 2: Document scanner process from <a href="https://www.pyimagesearch.com/2014/09/01/build-kick-ass-mobile-document-scanner-just-5-minutes/">How to Build a Kick-Ass Mobile Document Scanner in Just 5 Minutes</a></figcaption></figure><h3>Binarization</h3><p>Exist a lot of techniques of <a href="https://felixniklas.com/imageprocessing/binarization">binarization</a>. Among those studied and implemented that I found:</p><ol><li>A Threshold Selection Method from Gray-Level Histograms (<strong>Otsu</strong>);</li><li>An Introduction to Digital Image Processing (<strong>Niblack</strong>);</li><li>Adaptive Document Binarization (<strong>Sauvola</strong>);</li><li>Text Localization, Enhancement, and Binarization in Multimedia Documents (<strong>Wolf</strong>);</li><li>Binarization of Historical Document Images Using the Local Maximum and Minimum (<strong>Su</strong>).</li></ol><p>What is the best? Well, that depends. As the study aims to get the best from different inputs, I’ve put together some slightly different images to perform the tests (about 10), as Fig. 3 shows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*aU74RHuUYr3omgMxltGJhw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*NCmelgPFsHGepHSk7UsFuQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/611/1*6hXT3dq5RhhMSr2m1ZkDTA.png" /><figcaption>Fig. 3: Sample from binarization test</figcaption></figure><p>The point is, somehow, I found the paper “<em>Efficient illumination compensation techniques for text images</em>” about <strong>Illumination Compensation</strong>. This paper presents a new approach to light normalization of the image (Fig. 4).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/1*B3ma4lAsTrzaNCMvyCamFQ.png" /><figcaption>Fig. 4: An illumination compensation test with an image a lot of shade</figcaption></figure><p>With this technique applied, I get the best results from images (Fig. 5), but of course, new processing in python too. To be faster, I changed all preprocessing to c++, and the response was immediate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/1*90o-zNHJfELhMP9hVcpgHQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*03EXfQHOfkoylx9Yh1DVXw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*uun8bdNci4ATIPB6Z_H4Rg.png" /><figcaption>Fig. 5: New binarization test with illumination compensation before</figcaption></figure><p>In the end, I chose the Sauvola method with illumination compensation at all.</p><h3>Line Segmentation</h3><p>Finally, we arrive at the start point of this project! <em>haha</em></p><p>To have a good orientation of what techniques are present in the middle of text segmentation, I recommend the paper “<em>Text-Based Image Segmentation Methodology</em>”. In it, I was able to find the <strong>levels of text segmentation</strong> (line, word, character) and <strong>methodologies</strong> (pixel counting approach, histogram approach, Y histogram projection, text line separation, false line exclusion, line region recovery, smearing approach, stochastic approach, water flow approach), very useful to start.</p><p>I’ve implemented 2 line segmentation methods to see which would be the most appropriate:</p><ol><li>A Statistical approach to line segmentation in handwritten documents;</li><li>A* Path Planning for Line Segmentation of Handwritten Documents.</li></ol><p>Well, both are really good (Fig. 6), but the second is really slow for this project, because depending on the image, the algorithm takes 2 minutes or more, to result in a very similar segmentation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nSnc1UA4QeDL7ph5vlc96A.png" /><figcaption>Fig. 6: “A Statistical approach to line segmentation in handwritten documents” (a), and “<em>A* Path Planning for Line Segmentation of Handwritten Documents</em>” implementation (b). Both with Saint Gall dataset image.</figcaption></figure><p>To improve the lines normalization, also was implement a <strong>deslanting</strong> process from “<em>A New Normalization Technique For Cursive Handwritten Words</em>”. This process removes the cursive writing style and it is used as a preprocessing step for handwritten text recognition (Fig. 7).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/412/0*E_1clfAbK5WH4BEE" /><figcaption>Fig. 7: Sample of deslanting image from <a href="https://github.com/githubharald/DeslantImg">https://github.com/githubharald/DeslantImg</a></figcaption></figure><h3>Word Segmentation</h3><p>In this step, I can highlight the papers:</p><ol><li>Scale-Space Technique for Word Segmentation in Handwritten Documents;</li><li>Word Segmentation Method for Handwritten Documents based on Structured Learning;</li><li>Keyword spotting in historical handwritten documents based on graph matching;</li><li>Toward a Dataset Agnostic Word Segmentation Method.</li></ol><p>For implementation, I really found it interesting to approach with deep learning, like “<a href="https://github.com/gaxler/dataset_agnostic_segmentation"><em>Toward a Dataset Agnostic Word Segmentation Method</em></a>”. They presented a process with three models to do word segmentation (Fig. 8).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GuDYztnHvVNwgeD1ARh8Sg.png" /><figcaption>Fig. 8: A sample document from the ICDAR dataset (a), heatmap generated by the heatmap network (b), and a smooth heatmap generated by the smoother network (c).</figcaption></figure><p>To be honest, at this point, I just wanted a simple method, small and fast to implement. So “<em>Scale-Space Technique for Word Segmentation in Handwritten Documents</em>” gave me this, and of course, the method brought good results to simple images (Fig. 9).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*NBJXPfd-YzFoW1Hfmks5Gg.png" /><figcaption>Fig. 9: Sample word segmentation from lines</figcaption></figure><p>Another approach to the segmentation step that I found, this time with compressed images:</p><ol><li>Extraction of Line-Word-Character Segments Directly from Run-Length Compressed Printed Text-Documents;</li><li>Text line Segmentation in Compressed Representation of Handwritten Document using Tunneling Algorithm;</li><li>Word Segmentation Directly in Run-Length Compressed Handwritten Document Images using Connected Component Analysis.</li></ol><h3>Results</h3><p>It was 10 images to test, but all the code/results you can find in <a href="https://github.com/arthurflor23/text-segmentation">https://github.com/arthurflor23/text-segmentation</a>. For now, here some final results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NcWFfzsw__WGUgg9hoh8bQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GgpTbKTCihQAzh2G-NqKIg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4G30_4_bT0fTji2_kpAiIA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6jflOuFQC7IGukb_kJhKig.png" /></figure><h3>Conclusions</h3><p>It was a great experience for me, and with this project, I will change somethings about my computer vision project (<em>it will be the next post, I think</em>).</p><p>Then, this process of the segmentation is good to simple images of the manuscripts, because the final code is <em>static. </em>If anything comes up different from the input, the line/word segmentation will not go very well. And we don’t have to go very far with “anything different”, such as a book edges, for example.</p><p>Anyway, this factor directly impacts the results of my dissertation research, so I will have to find some way out in the coming months. ~<em>maybe the computer vision project helps, maybe not</em>~</p><h3>References</h3><ol><li>Arivszhagan, M., Srinivasan, H., Srihari, S. (2007). <strong>A Statistical approach to line segmentation in handwritten documents</strong>. The International Society for Optical Engineering;</li><li>Axler, G., Wolf, L. (2018). <strong>Toward a Dataset Agnostic Word Segmentation Method</strong>. 2018 25th IEEE International Conference on Image Processing;</li><li>Chen, K.-N., Chen, C.-H., Chang, C.-C. (2012). <strong>Efficient illumination compensation techniques for text images</strong>. Digital Signal Processing, v. 22, p. 726–733;</li><li>Javed, M., Nagabhushan, P., Chaudhuri, B. (2013). <strong>Extraction of Line-Word-Character Segments Directly from Run-Length Compressed Printed Text-Documents</strong>. 2013 Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing, and Graphics;</li><li>Lazzara, G., Géraud, T. (2014). <strong>Efficient Multiscale Sauvola’s Binarization</strong>. 2014 International Journal on Document Analysis and Recognition manuscript;</li><li>Manmatha, R., Srimal, N. (1999). <strong>Scale-Space Technique for Word Segmentation in Handwritten Documents</strong>. SCALE-SPACE ’99 Proceedings of the Second International Conference on Scale-Space Theories in Computer Vision, p. 22–33;</li><li>Mehul, G., Ankita, P., Namrata, D., Rahul, G., Sheth, S. (2014). <strong>Text-Based Image Segmentation Methodology</strong>. 2nd International Conference on Innovations in Automation and Mechatronics Engineering;</li><li>Niblack, W. (1986). <strong>An Introduction to Digital Image Processing</strong>. Strandberg Publishing Company Birkeroed, Denmark;</li><li>Ntirogiannis, K., Gatos, B., Pratikakis, I. (2014). <strong>A Combined Approach for the Binarization of Handwritten Document Images</strong>. Pattern Recognition Letters 35;</li><li>Otsu, N. (1979). <strong>A Threshold Selection Method from Gray-Level Histograms</strong>. IEEE Transactions on Systems, Man and Cybernetics 9, 62–66;</li><li>Rosebrock, A. (2014). <strong>4 Point OpenCV getPerspective Transform Example</strong>. Retrieved February 27, 2019, from <a href="https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/">https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example</a>;</li><li>Rosebrock, A. (2014). <strong>How to Build a Kick-Ass Mobile Document Scanner in Just 5 Minutes</strong>. Retrieved February 27, 2019, from <a href="https://www.pyimagesearch.com/2014/09/01/build-kick-ass-mobile-document-scanner-just-5-minutes/">https://www.pyimagesearch.com/2014/09/01/build-kick-ass-mobile-document-scanner-just-5-minutes</a>;</li><li>Ryu, J., Koo, H. Il, Cho, N. Ik. (2015). <strong>Word Segmentation Method for Handwritten Documents based on Structured Learning</strong>. IEEE Signal Processing Letters, v. 22;</li><li>Sauvola, J., Seppanen, T., Haapakoski, S., Pietikainen, M. (1997). <strong>Adaptive Document Binarization</strong>. IEEE Computer Society Washington;</li><li>Stauffer, M., Fischer, A., Riesen, K. (2018). <strong>Keyword Spotting in Historical Handwritten Documents Based on Graph Matching</strong>. Pattern Recognition 81, p. 240–253;</li><li>Su, B., Lu, S., Tan, C. L. (2010). <strong>Binarization of Historical Document Images Using the Local Maximum and Minimum</strong>. Document Analysis Systems;</li><li>Surinta, O., Holtkamp, M., Karabaa, F., Oosten, J.-P. van, Schomaker, L., Wiering, M. (2014). <strong>A* Path Planning for Line Segmentation of Handwritten Documents</strong>. 14th International Conference on Frontiers in Handwriting Recognition;</li><li>Vedere, A. R., Nagabhushan, P. (2018). <strong>Text line Segmentation in Compressed Representation of Handwritten Document using Tunneling Algorithm</strong>. International Journal of Intelligent Systems and Applications in Engineering, p. 251–261;</li><li>Vedere, A. R., Nagabhushan, P., Javed, M. (2018). <strong>Word Segmentation Directly in Run-Length Compressed Handwritten Document Images using Connected Component Analysis</strong>. IET Image Processing;</li><li>Vinciarelli, A., Luettin, J. (2001). <strong>A New Normalization Technique for Cursive Handwritten Words</strong>. Pattern Recognition Letters 22;</li><li>Wolf, C., Jolion, J.-M., Chassaing, F. (2002). <strong>Text Localization, Enhancement, and Binarization in Multimedia Documents</strong>. International Conference on Pattern Recognition (ICPR), v. 4, p. 1037–1040.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b32503ef2613" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Surface Crack Detection]]></title>
            <link>https://arthurflor23.medium.com/surface-crack-detection-92fb27bf0f18?source=rss-13120c388894------2</link>
            <guid isPermaLink="false">https://medium.com/p/92fb27bf0f18</guid>
            <category><![CDATA[segmentation]]></category>
            <category><![CDATA[unet]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[detection]]></category>
            <dc:creator><![CDATA[Arthur F. S. Neto]]></dc:creator>
            <pubDate>Mon, 25 Mar 2019 13:40:48 GMT</pubDate>
            <atom:updated>2023-03-02T14:44:06.678Z</atom:updated>
            <content:encoded><![CDATA[<p>An experiment with deep learning to segmentation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*0MAeRN5Ut9nWab2sSLF2pw.png" /><figcaption>Fig. 1: A simple segmentation from deep learning</figcaption></figure><p>Hey! How are you? I hope that well, dear reader. So, let’s go to another project?</p><p>I think this is the most interesting project I’ve ever done until now. Maybe because with it I learned <strong>python</strong> and deep learning. Well, this is the <strong>first</strong> project from the master’s degree with Deep Learning.</p><p>The goal was detection (somehow) surface cracks in tiles, ceramics, anything like that. The class received <em>the</em> <em>dataset</em> to this, and well… the dataset was s*cks (we’ll get there). For code and paper lovers:</p><ul><li><a href="https://github.com/arthurflor23/surface-crack-detection">https://github.com/arthurflor23/surface-crack-detection</a></li><li><a href="https://www.scielo.br/scielo.php?script=sci_arttext&amp;pid=S1678-86212021000100139&amp;nrm=iso">Processamento digital de imagens para detecção automática de fissuras em revestimentos cerâmicos de edifícios</a> (Digital image processing for automatic detection of cracks in buildings coatings)</li></ul><h3>Related Studies</h3><p>First, let’s go to some related studies to look for similar projects and to understand which approaches are most used.</p><p>The paper “<em>Multi-resolution approach for fine structure extraction: Application and validation on road images</em>” give me this:</p><blockquote><a href="https://en.wikipedia.org/wiki/Thresholding_(image_processing)">Threshold</a> methods are popular and simple but inefficient (the amount of false positives and of false negatives is important). Methods based on <a href="https://en.wikipedia.org/wiki/Mathematical_morphology">Morphology</a> reduce the number of false positives but they are strongly dependent on the choice of the parameters. <a href="https://en.wikipedia.org/wiki/Artificial_neural_network">Neural Network</a> techniques are efficient but they need a learning step that is quite expensive and non-automatic. Finally, <a href="https://en.wikipedia.org/wiki/Multi-scale_approaches">Multi-Scale</a> methods seem to be the most efficient: unlike other methods, they lead to less false positives, but more false negatives.</blockquote><p>Ok, now I had some orientation of the existing paths. The next paper “<em>Concrete surface crack detection with the improved pre-extraction</em>” presents a new algorithm to cracks detection (CrackHHP). Fig. 2 shows the result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*-y2sf1IsVFlz0cGso9xYTg.png" /><figcaption>Fig. 2: Percolation results of the high-efficiency algorithm</figcaption></figure><p>It’s really amazing, but the problem project was more specific, more dynamic, harder to detect. So, the next papers worked with Neural Network to detection and segmentation. The paper “<em>Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks</em>” presented a CNN architecture which is the original configuration for concrete crack detection (Fig. 3). This new approach gives some really good results (Fig. 4).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LJYPyANUhX5jPlThnF0wpA.png" /><figcaption>Fig. 3: Overall architecture: L#: layers corresponding to operations (L1, L3, L5, and L7: convolution layers; L2 and L4: pooling layers; L6: ReLU layer; L8: softmax layer); C#: convolution; P#: pooling; BN: batch normalization</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bq0KPEJCWcEnYZ7a3Nt0VA.png" /><figcaption>Fig. 4: Result of image scanning using a trained CNN from Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks</figcaption></figure><p>The paper “<em>Concrete Cracks Detection Based on Deep Learning Image Classification</em>” again using deep learning to concrete crack detection:</p><blockquote>The basis for CNN development relies on transfer‐learning, i.e., we build upon a pre‐trained open‐source model <a href="https://neurohive.io/en/popular-networks/vgg16/">VGG16</a> and use an experimental study to evaluate the influence of training parameters such as learning rate, number of neurons (or nodes) in the last fully connected layer, and training dataset size on the accuracy of the classification model.</blockquote><p>But the paper that really got me was the “<em>Semantic Metric 3D Reconstruction for Concrete Inspection</em>” from <a href="https://ericlyang.github.io/project/deepinspection/">Robotic Inspection</a> project (Fig. 5).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*Z5sUrru1VZZkcT0eobtFYA.png" /><figcaption>Fig. 5: Image from Robotic Inspection</figcaption></figure><p>They introduced the <a href="https://github.com/ccny-ros-pkg/inspectionNet_Segmentation">InspectionNet</a> (Fig. 6) architecture to crack and spalling detection:</p><blockquote>The InspectionNet is motivated by HED and U-net, and these two deep neural networks are end-to-end fully pixel- level segmentation for edge segmentation.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JD3-Q0MqbdbgSOKB4kMcwQ.png" /><figcaption>Fig. 6: The InspectionNet model, the model has a total of 37 layers which is motivated by U-net and HED net</figcaption></figure><p>In this paper, I learned about U-net and most important, <em>they have created the dataset to train</em>, so I would too. Of course, their dataset already has more than 500 images, well defined. Mine would be a small dataset of 100 images, <em>unfortunately</em>.</p><h3>Project Approach</h3><p>I really complicated a simple code. All code has been modularized, so I can change the dataset, change the preprocessing of the image, change de neural network architecture.. all by parameters, and of course, the files with the new code and datasets.</p><p>So, if we want biomedical segmentation, we can. If we want a road crack segmentation, we can. Concrete crack segmentation, we can. Surface crack segmentation, we can.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/345/1*k1WA6CdaBee8zHdc7SiNXA.jpeg" /><figcaption>Fig. 9: Yes, we can!</figcaption></figure><p>Obviously, it was very complex with many possibilities (the results are in GitHub), but a lot of unnecessary complexity for a project. And I only came to learn the concept of <a href="https://martinfowler.com/bliki/Yagni.html">Yagni</a> later (I recommend, dear developer).</p><h3>The dataset</h3><p>The dataset contained 67 very different images (Fig. 10). Among them, we could group in groups of similarity, like angulation, distance or even by type of material.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xoc-oRdlyMyt2XKmg_UF4w.png" /><figcaption>Fig. 10: Sample of the initial images</figcaption></figure><p><strong>No pattern</strong>, <strong>no labels</strong> and worse, <strong>no dataset</strong> with this style of image over the Internet. With this, the approach was to find more images (to the training set) and create the labels by hand.</p><p>It was really hard, but I found 100 images and I took the most varied images <em>on purpose</em>. The labels were created on <a href="https://www.gimp.org/">Gimp</a> (Fig. 11).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*g9uwf-dseVBVKIOr8G8LHw.png" /><figcaption>Fig. 11: Sample of the training dataset and labels</figcaption></figure><p>The <em>U-net</em> would be able to segment with only 100 images (the biomedical segmentation was the proof), but for these images, the dataset would have to be bigger. So I made a simple date augmentation to 1,000 images.</p><h3>Data preprocessing</h3><p>For data preprocessing, well, a simple binarization wasn&#39;t been enough to all images (Fig. 12 a). Then, a series of techniques were applied for all the images could generate a binarization that showed the crack (Fig. 12 b):</p><ol><li>Light equalization;</li><li>Gaussian filter (with kernel 3x3);</li><li>Light and contrast adjust;</li><li>Inversion;</li><li>Erode and dilate functions (with kernel 5x5);</li><li>Finally, Otsu thresholding.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*Av3DzALM-wmW8NviHoGBZg.png" /><figcaption>Fig. 12: Simple binarization (a) and series of techniques to the binarization (b)</figcaption></figure><h3>U-net</h3><p>The paper “<em>U-Net: Convolutional Networks for Biomedical Image Segmentation</em>” give me all that I want. The architecture (Fig. 7) was created to be small and fast (compared to other architectures). At first, it was created for biomedical segmentation (Fig. 8), but I could try, right?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*5-YCn3wRM8DjGWwPu8MBGg.png" /><figcaption>Fig. 7: U-net architecture (example for 32x32 pixels in the lowest resolution). Each blue box corresponds to a multi-channel feature map. The number of channels is denoted on top of the box. The x-y-size is provided at the lower left edge of the box. White boxes represent copied feature maps. The arrows denote the different operations</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/657/1*Zuo4lhLNhMLgjASg-GPFzQ.png" /><figcaption>Fig. 8: Overlap-tile strategy for seamless segmentation of arbitrary large images (here segmentation of neuronal structures in EM stacks). Prediction of the segmentation in the yellow area requires image data within the blue area as input. Missing input data is extrapolated by mirroring</figcaption></figure><p>No surprise, but the papers that I found it was to crack detection like roads and bridges, nothing so specific and small to detect as surface cracks in tiles. Did you see this? But, now, I have the knowledge about U-net and the satisfaction that other people have created their own dataset and I would not be the only one. <em>It’s a normal thing in this area (which I had no idea)</em>.</p><p>For the training process, I really made a lot of mistakes, so I stopped and started from scratch. I think this process took 2 weeks (<em>haha</em>), but remember, I was learning everything at the time and all this I took good advantage to learn other things, like <a href="https://medium.com/greyatom/what-is-underfitting-and-overfitting-in-machine-learning-and-how-to-deal-with-it-6803a989c76">overfitting and underfitting</a>, for example.</p><p>Finally, the final training process was realized in a notebook with i7, 16gb ram and a GPU Nvidia 940M with 4gb for about 10 hours (<em>thank you, Lord</em>).</p><p>But wait, 10 hours because I had 10,000 <em>diverse</em> images (with data augmentation). <strong>Biomedical</strong> or <strong>concrete crack</strong> segmentation, for example, required only 100 images and 30 minutes training (Fig. 13).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qwqdCTC_VZcm2iSh2ZzGEw.png" /><figcaption>Fig. 13: original image grayscale (a), preprocessed image (b), deep learning response overlapped on the original image (c)</figcaption></figure><h3>Results</h3><p>In the end, some of the final results. The rest of the images are in my repository (<a href="https://github.com/arthurflor23/surface-crack-detection">https://github.com/arthurflor23/surface-crack-detection</a>).</p><p>I will follow the structure of Fig. 13 (original, preprocessed, overlapped), so:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r_9JLn582H5wVPjMwwKxnA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j3SEVJJMJ8H6yMRIHiIZSA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2TffnYCtgju_tLCUy230UA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vgPLIIkbO34rti0huzn_Nw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xWaS26JYB_M-wX4Ezc8Sqw.png" /><figcaption>Fig. 14: Sample of the results, original images (left), preprocessed images (center), deep learning response overlapped on the original image (right)</figcaption></figure><h3>Conclusions</h3><p>Don’t think that all the results were good, because they are 67 images in total. Then, this project gave me a lot of knowledge. I never used Python, for real. Ok, maybe the code isn’t the best, whatever.</p><p>To finish, I think the dataset was the <em>problem</em> here:</p><ol><li>Few images available;</li><li>Poor images quality;</li><li>Non-standardized images;</li><li>A <em>big</em> and static preprocessing step was necessary.</li></ol><p>BUT, the project can open doors, because it’s possible to work across multiple targeting scenarios. Besides, it proved very useful and versatile even with all the odds, don’t you think?</p><h3>References</h3><ol><li>Cha, Y. , Choi, W. and Büyüköztürk, O. (2017), Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks. Computer‐Aided Civil and Infrastructure Engineering, 32: 361–378. doi:10.1111/mice.12263</li><li>Liang Yang, Bing Li, Wei Li, Biao Jiang, Jizhong Xiao (2018) Semantic Metric 3D Reconstruction for Concrete Inspection. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.</li><li>Nicolas Coudray, Argyro Karathanou, Sylvie Chambon (2010), Multi-resolution approach for fine structure extraction: Application and validation on road images. International Joint Conference on Computer Vision Theory and Applications, VISAPP.</li><li>Qu Z, Ju F-R, Guo Y, Bai L, Chen K (2018), Concrete surface crack detection with the improved pre-extraction and the second percolation processing methods. PLoS ONE 13(7). https://doi.org/10.1371/journal.pone.0201109</li><li>Ronneberger, O., Fischer, P., &amp; Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. <em>MICCAI</em></li><li>Silva, W.R.L.; Lucena, D.S. (2018), Concrete Cracks Detection Based on Deep Learning Image Classification. Proceedings, 2, 489. doi:10.3390/ICEM18‐05387</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92fb27bf0f18" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>