<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Servin Osmanov on Medium]]></title>
        <description><![CDATA[Stories by Servin Osmanov on Medium]]></description>
        <link>https://medium.com/@servinosmanov?source=rss-45c456062f69------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*a6mi0YhNs8Y-xOwV.</url>
            <title>Stories by Servin Osmanov on Medium</title>
            <link>https://medium.com/@servinosmanov?source=rss-45c456062f69------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 01:04:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@servinosmanov/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Paid $53 for 15% of a Training Run. Here Is Everything Else We Got Wrong.]]></title>
            <link>https://medium.com/@servinosmanov/i-paid-53-for-15-of-a-training-run-here-is-everything-else-we-got-wrong-13cabc90c4a6?source=rss-45c456062f69------2</link>
            <guid isPermaLink="false">https://medium.com/p/13cabc90c4a6</guid>
            <category><![CDATA[mls]]></category>
            <category><![CDATA[crimean-tatars]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mistakes]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Servin Osmanov]]></dc:creator>
            <pubDate>Thu, 19 Nov 2026 17:50:51 GMT</pubDate>
            <atom:updated>2026-11-19T17:52:46.803Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y-A7CUH6AEMGKZGShuS1gw.jpeg" /><figcaption>Time is money</figcaption></figure><p>Every real project has mistakes in it. Usually they get one sentence somewhere between the results, if that. I wanted to do the opposite for once and put all of ours in one place: the ones I’ve already told you about in this series, and the ones I’ve kept quiet until now.</p><p>In the <a href="https://medium.com/@servinosmanov/every-third-word-was-wrong-heres-how-my-language-got-working-ears-07172763bc0b">second article of this series</a> I promised to tell the story of how I rented other people’s graphics cards in the cloud and burned money on them without really doing the maths. In the <a href="https://medium.com/@servinosmanov/my-uncle-wrote-a-book-a-machine-read-it-in-my-voice-e955639f93ab">third</a> I said the promise still stood. This is me keeping it.</p><h3>Why rent a computer at all</h3><p>A quick recap for anyone new. I’m building speech and language technology for Crimean Tatar, my native language, which is on the endangered list. In the <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">first article</a> I explained why, and in the <a href="https://medium.com/@servinosmanov/our-ai-started-answering-in-turkish-the-data-told-me-why-c5122511e934">fourth</a> where the data comes from.</p><p>The speech recogniser and the synthetic voice train at home, on one graphics card; the whole training run in the second article took an hour and a half. But we also need a language model, one that can write and answer in Crimean Tatar, and that is a different animal. It needs a card with roughly six times the memory of mine. Buying one would make no sense for a project like ours, so you rent it by the hour, like a car. And like a rented car, the meter runs whether you’re driving or sitting in a car park trying to work out why the engine won’t start.</p><h3>The short version</h3><p>Over about a month and a half we spent a bit over $300 on rented cards. That wasn’t one big bill but a string of attempts. We trained, looked at what came out, changed the setup and ran again, and each time the run came out cheaper and more reliable, until we had a recipe that works.</p><p>I’ll be honest about how that felt. I was angry that we spent so much money. But I’m glad we ended up with the right “recipe” for future training runs: now we know what a run costs and what we shouldn’t be paying for. The chart below tells the whole story in one picture, and the rest of this article is the long version.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y-lsnOqVGsaZ1uqqQ2F1Wg.png" /><figcaption>What one training run cost, from the first attempt to the latest cycles</figcaption></figure><h3>$53 for 15% of the job</h3><p>The first rented card was an H200, about $4.60 an hour. The plan said the job would take nine to eleven hours and cost $33 to $40. After eleven and a half hours and $53 I stopped it. It had done a small part of the work, and the rest would have taken almost two more days, because the card was working at a few percent of what it can do. When we dug into it, we found three reasons, and each would have been enough on its own.</p><p>The biggest one was that we were paying for empty space. Our training conversations are short, a handful of words each. But the model doesn’t read them one at a time. It reads them in batches where every example has to be the same length, so each short conversation got topped up with blank filler to match. Imagine hiring a lorry by the hour and sending it across town with one shoebox per trip, the rest of the back full of air. That was us. Almost all the time the card was diligently processing filler, and we were paying for it. That is why the estimate was off by a factor of twenty.</p><p>The second reason was saving. Training stops now and then to save its progress, so that if something breaks you don’t start from scratch, which is sensible. But we were saving very large snapshots, very often, onto a slow disk, and every time it saved, the card simply stood still and waited. And the third reason was almost embarrassingly simple: the next batch of data was being prepared more slowly than the card could eat it, so the card waited again. A powerful machine with a slow kitchen.</p><p>The part that stung most was that before renting anything, we had tested everything at home, and everything worked. But that test only proved the program was correct. Whether it would be affordable, we never checked, because at home time is free. “It works” and “it pays for itself” turned out to be two different tests, and we had only run the first one.</p><p>Not everything was lost, though. The first half of the training finished completely, and we kept what it produced.</p><h3>Five machines in one day</h3><p>The empty space was the biggest problem, so we fixed it first. Instead of padding each short conversation, we started packing them tightly, one after another, so that every batch is full of real text and the lorry leaves with a full load. The maths said the same work would now cost about twenty times less.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6W18BGf7O3whtQW7uP_HPw.png" /><figcaption>The price of padding: the same training stage with and without filler</figcaption></figure><p>Feeling much better, I started the second attempt, and over the next day we had to rent five machines in a row. Each one died before it finished, and we paid for every one of them.</p><p>What was strange was that the training itself never crashed. Not once. Everything around it crashed: the scripts that set things up and the programs that keep an eye on the training. One machine fell over because of a check that needed more memory than the card had; that check had never had time to run during the first attempt, so nobody knew it was a problem. Another machine was switched off by our own guard program. It found an old note left over from the previous attempt, a file that said “this run failed”, believed it, and shut down a perfectly healthy training four minutes before the end of a stage.</p><p>That’s when I realised nobody had ever tested these pieces together. So we added a rehearsal. Before every rental we now run the whole chain at home, with the same steps, the same checks and the same guards, only with a tiny model and for free. One rehearsal later caught six bugs, and in the cloud we would have paid for every one of them.</p><p>But the most expensive thing that day wasn’t any single bug. It was how we reacted to them. Whenever anything went wrong, we did the same thing: switched the machine off, fixed the problem, and started again on a fresh one. A fresh machine has to install everything again and download tens of gigabytes again, all while the meter runs. And when a machine is gone, the notes about what happened on it are gone too, so afterwards you’re left guessing. The fifth machine that day worked for more than four hours, crashed, and vanished before we could copy its notes off. I still don’t know why it died.</p><p>That’s where our main rule came from: don’t switch it off, fix it in place. Keep the machine, keep its notes, repair the problem and restart right there. It’s like a flat tyre on a bike ride. You patch it at the side of the road; you don’t abandon the bike and walk home to buy a new one. The next training went through from start to finish on the first attempt and cost about $41 for the whole job, when the first time we’d paid $53 for 15% of it. On the run after that, a short test on the machine itself found three problems. We fixed them on the spot, and the whole thing cost less than $13.</p><h3>The guard that stayed quiet, and the guard that panicked</h3><p>I keep mentioning a guard, so let me explain. It’s a small program we wrote that watches the training, saves the result and switches the machine off when the work is done, so we don’t pay for an idle computer. It’s the thing that’s supposed to stand between us and the running meter, and it made mistakes in both directions.</p><p>Once, the training finished and the guard didn’t notice. It had broken earlier that day, quietly, without telling anyone, and the machine sat there with the finished result for more than half an hour, meter running, until I went and looked myself. Since then we give the guard a fire drill before every run: we trigger an alarm on purpose and check that it reaches us. And near the end of a run, I check the machine by hand too.</p><p>Another time the guard went the other way and was too strict. It saw the card working, then resting, then working, then resting, and didn’t know the card was resting because it was saving the result. It decided the training had frozen and switched the machine off 28 seconds before the end. The result survived, but some of the files that go with it came out empty, and we had to rent another machine just to rescue everything. Now only a real stop counts as frozen, and a pause to save is allowed.</p><p>The guard also doesn’t live on the rented machine any more. It lives at home, so if the rented machine dies, the guard doesn’t die with it and is still there to collect what’s left.</p><h3>When the money runs low</h3><p>Money had a similar story. The first rule was simple: hit the limit, switch everything off. Then, right at the start of one training, the guard did a quick calculation, decided the money might not be enough, and almost stopped a completely healthy job. We caught it in time, but it showed us that a hard money limit is dangerous. It can kill a training run that is nearly finished, and then everything you’ve already paid for is lost. So now, when the system gets close to its limit, it saves everything first and only then shuts down.</p><h3>Prices that weren’t real</h3><p>Another mistake was a simpler one: believing the price list. One cloud showed the card we needed at almost half the price, but it wasn’t actually available anywhere. Another offered an affordable card but paired it with so little memory and disk space that you couldn’t train on it. In the end we found a good option with the chip maker itself, through its developer programme, an MI300X at a fair hourly rate. Checking all those options cost us less than a cent, and since then, before we believe a price, we check that we can actually get the card.</p><h3>Mistakes in measuring</h3><p>These cost almost nothing in money, but in a way they’re more dangerous. A wasted hour shows up on the bill. A wrong number looks exactly like a right one. Some of these I’ve told before, in the <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">first article</a>, the <a href="https://medium.com/@servinosmanov/every-third-word-was-wrong-heres-how-my-language-got-working-ears-07172763bc0b">second</a>, the <a href="https://medium.com/@servinosmanov/my-uncle-wrote-a-book-a-machine-read-it-in-my-voice-e955639f93ab">third</a> and the <a href="https://medium.com/@servinosmanov/our-ai-started-answering-in-turkish-the-data-told-me-why-c5122511e934">fourth</a>, so here I’ll add three I haven’t.</p><p>Crimean Tatar is written in both Latin and Cyrillic letters. One day our recogniser, which writes in Latin, was checked against a book printed in Cyrillic, and the comparison came out at almost 100% wrong. For a moment it looked as if the whole dataset was rubbish. The real reason was almost funny: the comparison was counting every letter as a mistake because the alphabets didn’t match. Once both texts were in the same alphabet, the numbers came back to normal.</p><p>Another time the synthetic voice twice in a row jumped to a strange high note, and I started reworking the data to fix it. Then we listened not to two attempts but to 125, and there were no jumps at all. I had simply happened to hear the same thing twice, and since then the rule is that once or twice doesn’t count.</p><p>The third mistake was the most unpleasant. More than half of the texts we used to test the model also appeared in its training material. It’s like giving a student an exam made of the same problems they solved in class: the grade comes out higher than what they actually know. So the scores looked better than they were, and we built a new test from writings that appear nowhere in the training data.</p><h3>What we took away</h3><p>Put all of this together and the picture looks like this. At first almost every run went wrong somewhere, and now failures are rare and cheap.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DALwQfG-mfLtKwTUcduhDQ.png" /><figcaption>How often a run went wrong, by date</figcaption></figure><p>The guard no longer stays quiet or panics: it goes through a fire drill before every run, lives outside the rented machine, and can tell a frozen training from a pause. Near its limit the system doesn’t kill live work because of a number, it saves everything first. Most of the money went in the days when every error meant switching the machine off and rebuilding from nothing, and now that we fix mistakes on the spot, a failure costs a couple of dollars rather than a whole run. The cost is worked out before a run and checked against the first minutes of real work, and everything goes through a free rehearsal at home first. A training cycle now costs about $10 to $12.</p><p>A cheap cycle isn’t automatically a good one, of course. One version trained on new data turned out no better than the previous one in a <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">blind comparison</a> (I described how it works in the first article), so it never went into use. But that kind of failure now costs $10 instead of $53, and it leaves a full report behind on what went wrong.</p><h3>Where we are now</h3><p>We’re not giving up on training in the cloud. Right now we’re collecting data, and when we train again it will be on rented machines as before, only each run is now many times cheaper and doesn’t break out of nowhere. And the work is paying off: the latest version of our speech recogniser came out better than the one before it. It is more accurate against our reference transcripts, and it won more often in the same blind comparison, where you listen without knowing which version is which.</p><p>What holds us back now isn’t hardware or money. It’s live, conversational speech that a person has listened to and written down. We’ve learned to count graphics cards. What our models are still missing is living language.</p><p><strong>Want to help?</strong> What this project needs most is native speakers. Listen to a short recording and tell us whether the machine wrote it down correctly. Read a page of text and mark which word is Turkish and which is ours. Write a dialogue with the real colour of the language in it. Record a few minutes of reading in your own voice. You don’t need to know anything about programming, only to know your language well. Write to <a href="/service/mailto:ai@ay.mw">ai@ay.mw</a> or on Telegram: <a href="https://t.me/ana_yurt_dev">t.me/ana_yurt_dev</a>.</p><p>And if you’d like to support the work financially, you can do it here: <a href="https://ko-fi.com/anayurt">ko-fi.com/anayurt</a>. After everything in this article, each dollar goes a lot further than it used to, and that support turns into new recordings of living speech, which need to be made now.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13cabc90c4a6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Our AI Started Answering in Turkish. The Data Told Me Why.]]></title>
            <link>https://medium.com/@servinosmanov/our-ai-started-answering-in-turkish-the-data-told-me-why-c5122511e934?source=rss-45c456062f69------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5122511e934</guid>
            <category><![CDATA[mls]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[language-learning]]></category>
            <category><![CDATA[pytho]]></category>
            <dc:creator><![CDATA[Servin Osmanov]]></dc:creator>
            <pubDate>Fri, 30 Oct 2026 00:19:19 GMT</pubDate>
            <atom:updated>2026-10-30T00:19:19.967Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cEr2EAxapxEPolQqXiQ44A.png" /></figure><h3>Our AI Started Answering in Turkish. The Data Told Me Why.</h3><p>One day our Crimean Tatar language model answered me with “Ayt, men dinlemeye hazırım.” It means “Go on, I’m ready to listen,” and almost every word in it is right. The shape is wrong. In Crimean Tatar it’s “Ayt, men diñlemege azırım.” The model had spoken Turkish with our vocabulary, the way a guest might who learned the language from the wrong neighbour.</p><p>That sentence turned out to be a thread. Pull it, and you end up looking at where every piece of our data came from.</p><p>In the <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">first article of this series</a> I wrote that Crimean Tatar had no dataset good enough for serious work, and that I started by cleaning up an existing speech set. In the <a href="https://medium.com/@servinosmanov/every-third-word-was-wrong-heres-how-my-language-got-working-ears-07172763bc0b">second</a> a machine learned to hear our speech. In the <a href="https://medium.com/@servinosmanov/my-uncle-wrote-a-book-a-machine-read-it-in-my-voice-e955639f93ab">third</a> a synthetic voice read a whole novel. People kept asking me the obvious question: if there was no data, what did all of this learn from?</p><p>This is that story. One thing I won’t do here: I don’t name the archives, publications, channels or people who shared material with us, and I don’t talk about our arrangements with them. What I can tell you is what kinds of material there were, what was wrong with them, and how many surprises each one hid.</p><h3>A shelf full of things that weren’t datasets</h3><p>Listed honestly, we had more than I expected. Printed books and magazines from many decades: novels, stories, poetry, schoolbooks, newspapers. Broadcast recordings: news, programmes, studio conversations, interviews. Audiobooks read from cover to cover by professional narrators. Dictionaries, without which you can’t even check whether a word is spelled right. My own recordings. And what teachers bring: corrections, dialogues, recordings of their voices.</p><p>Almost none of it was a dataset. A dataset, for speech, means recordings paired with the exact text spoken in them; for a language model, it means clean text you can trust. A book is a file on a shelf. A broadcast is hours of sound without a single line of text. An audiobook has its sound and its printed text living separate lives: you have to find the text, check it’s the same edition, and line the two up word by word. Sometimes there’s no digital text of the book at all.</p><p>So the work was turning all of that into pairs of sound and text, and into text that deserved trust. Every kind of material brought its own surprise.</p><h3>The scans that were already text</h3><p>Books came first. The plan looked obvious. Most older Crimean Tatar books exist only as scans, so they would have to go through recognition, which turns a picture of a page into text. That’s slow, and on our language it makes mistakes, especially on letters Russian doesn’t have.</p><p>Before starting, I checked what was actually on the shelf, and the question turned out to be aimed slightly wrong. About four fifths of the Crimean Tatar pages already carried a text layer: underneath the picture of the page sat real text you could simply take. Recognition was only needed for the remaining fifth.</p><p>But “there is text” isn’t the same as “the text is good.” The first look showed three different kinds of damage, and none of them was our fault. Someone else’s software had made those text layers long ago, often an old recogniser, and the text got spoiled back then, or while being copied from file to file.</p><p>The first kind was a soft sign standing where a hard one belongs. In Cyrillic, our deep <em>q</em> and <em>ğ</em> are written къ and гъ, and programs trained on Russian kept turning them into кь and гь. Here’s the nice part: кь and гь never occur in Crimean Tatar Cyrillic at all, so the damage can be undone mechanically. I didn’t take my own word for it, though. I first ran the repair over text I knew was healthy. It had to change nothing there, and it didn’t change a single letter. Only then did it go over the whole archive, where it fixed tens of thousands of pages.</p><p>The second was the wrong code page. Latin pages read like tatarlýq and qorquya tüþüp: the text underneath was right, but its letters had been read in the wrong encoding, so ı turned into ý and ş into þ. That’s reversible too: tatarlıq, qorquya tüşüp.</p><p>The third was the same book, several times. The same works sat in different editions, different collections, under different file names. In the <a href="https://medium.com/@servinosmanov/every-third-word-was-wrong-heres-how-my-language-got-working-ears-07172763bc0b">second article</a> I told how four audiobooks turned out to be stored twice under completely unrelated names. Text did the same thing. File names barely caught any copies. Comparing the content itself, page against page, found about 7% repeats. Without that, the model would have learned them twice.</p><p>How do you tell a good page from a bad one when nobody has proofread it? Our <a href="https://ana-yurt.com/lugat">dictionary</a> (<a href="https://apps.apple.com/us/app/q%C4%B1r%C4%B1mtatar-lu%C4%9Fat%C4%B1/id1457493656">iOS</a>, <a href="https://play.google.com/store/apps/details?id=com.anaurt.lugat">Android</a>), where this whole journey began, came to the rescue. For each page I counted the share of words that start with a real dictionary word. Start with, not match: our language glues ending after ending onto a root, so an exact match tells you almost nothing. Russian books, which I ran as a control, didn’t get a single page through. Turkish books partly did. Remember that; it comes back.</p><p>The result surprised me. The amount of text we could train on grew roughly tenfold, and most of that growth needed no recognition at all. The expensive part wasn’t reading scans. It was deciding whether to trust the text that was already there.</p><h3>The text that pretended to be ours</h3><p>Which brings me back to the Turkish answer.</p><p>When I started measuring, Turkish shapes in the model’s answers had become several times more common than in the previous version. The cause was in the data. Among the text we’d gathered were periodicals published outside Crimea. They carry real Crimean Tatar pages, and next to them whole pages in Turkish, sometimes in the same issue. To someone who knows neither language, they look alike. To a program, too.</p><p>This is where I learned the most important thing in this whole article. An ordinary language filter asks, “Is this Crimean Tatar or not?” and looks at small function words and endings. For Turkish the answer is almost always “looks like it”: the two languages share too many roots and too many little words. My own paragraph-level filter, run across the corpus, found over ten thousand pieces worth removing: Russian, Ukrainian, English. It found signs of Turkish in just 40 of them. It simply couldn’t see Turkish.</p><p>So I built my own detectors that look for things Crimean Tatar never does. The Turkish verb ending <em>-yor</em>. <em>değil</em> where we write <em>degil</em>. <em>iki</em> where we say <em>eki</em>. The ending <em>-meye</em> where ours is <em>-mege</em>, exactly the mistake in “dinlemeye.” Each detector was tested both ways: on clean Crimean Tatar it must stay silent, on real Turkish it must fire. Later a second one appeared, for other Turkic languages, and it found a few dozen pieces in the corpus written entirely in Kazakh, Tatar or Uzbek.</p><p>Several thousand fragments went into quarantine. Not into the bin. Each one is kept as it was, marked with where it came from, so any decision can be revisited.</p><p>There was a tempting idea: don’t throw “Turkish spelling” away, correct it automatically, k to q, n to ñ. I tried it and put it aside. The program fixed the form without understanding the meaning. It turned <em>balaların</em> (“of the children”) into <em>balalarıñ</em>, which means “your children.” The right form is <em>balalarnıñ</em>. Every one of those corrections looked like a real Crimean Tatar word, and that’s exactly what made them dangerous. Automation that makes believable mistakes is worse than automation that does nothing.</p><p>And then the other side of it. Our dictionaries mark some words as dialect, like the south-coast <em>ben</em> instead of <em>men</em> for “I.” My Turkish checks naturally picked on them. I went through the whole list and decided: these are dialects, different ways our own language is spoken, and they stay. Yes, some of them look Turkish. They’re still ours, and I won’t scrub them out of the corpus. The difference is handled elsewhere: dialect forms stay in the training text, and in the model’s answers they’re counted separately. I want the model to answer in the literary language, but it can learn from everything the language lives in.</p><h3>Sound without text</h3><p>Sound was harder, because most recordings come with no text at all.</p><p>There was an audiobook several hours long: a good narrator, a clean recording. I never found the printed text of that edition in any digital form. So the trick from the <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">first article</a>, lining a known text up against the sound, couldn’t work here.</p><p>Instead, our recogniser transcribes the recording, and then a second one, built completely differently and trained separately, listens to the same sound. Only the pieces where the two transcripts agree almost entirely go into the corpus. Two models that make different mistakes rarely make the same one. When our recogniser got better and the cutting got more precise, the same book gave 2.6 times more usable sound, and the spots where the old model got stuck repeating one phrase fifty times disappeared.</p><p>I’ll be honest, though: agreement isn’t proof. Two wrong models can, in principle, agree, and you can’t check them against the book, because there’s no book. You can only listen. That’s why we started checking every doubtful piece by ear.</p><p>Then came the broadcasts, more than a thousand hours of them. Same principle: the machine transcribes, and only what it’s sure of, and what the second model agrees with, goes into training. A strict filter kept about a quarter.</p><p>And this is where a very instructive trap caught me. I looked at what had passed, and more than 80% of it was news. Newsreaders speak clearly, from a script, and the recogniser hears them well already. Conversation, where people interrupt each other, lose their thread and talk like people do, got through only in small part: about one hour in nine, against one in three for news. A filter built on confidence keeps what the model can already do and throws away exactly what it needs to learn. In the next round I capped the share of news and let conversation through with a gentler threshold.</p><p>That left a question: how do you know whether things got better? The test texts for real speech had been made by the machine itself, the old version of the recogniser. So any new model that disagreed with the old one automatically looked worse. I took the pieces where the two disagreed and listened to them myself, blind, without knowing whose transcript was whose. It took me many hours. It turned out the test texts themselves were wrong in about 15% of their words, and the new model was actually right noticeably more often than the old one.</p><p>Today the recogniser running on our server is the one trained on real speech, and our partners use it a lot. On short real recordings it makes about a third fewer mistakes than before. On long conversations the old and new are level, around 30% of words wrong, and that’s the honest state of things. The next round, with several times more hours of conversation, came out level too, and I couldn’t find a winner even by ear. So “just add more hours” doesn’t work here. That’s a result as well.</p><h3>The people</h3><p>Everything above rests on people.</p><p>Finding them turned out to be one of the hardest parts. It’s hard to find people who can help, and not because they don’t want to. They don’t have the time. Many agreed; only a few actually work with us, and that has slowed things down a lot. So we keep looking. We’ve posted calls on social media, and people who care have answered.</p><p>My first rule is simple: the voices and recordings you hear in our examples and in our synthetic voices are used with their owners’ consent. If someone changes their mind, we take their voice out. No arguments.</p><p>Crimean Tatar teachers work with us through a dedicated portal. They write dialogues the model learns to talk from. They check lines of the corpus and mark where a text isn’t Crimean Tatar. They go through dictionary notes and decide which word is Turkish and which is ours. They correct drafts the machine wrote. And they record their voices. Those recordings have never once gone into training: they became the exam for the recogniser. A teacher on an ordinary microphone, not a studio one, is the closest thing we have to how real people will talk to the machine. So we keep them for testing and never train on them.</p><p>There are also partners who record oral histories: people telling the stories of their lives in their own language, often in a local dialect. They sent us an interview with subtitles they had checked by hand. It became one of our hardest exams: real conversation, dialect and pauses, with text written by a person, not a machine. The model doesn’t learn from it either. Recordings like that are too valuable to spend on training, and they’re the only place you can see how the machine copes with a living conversation.</p><p>And my own voice. The machine read the second book in our library in it, the one I wrote about in the <a href="https://medium.com/@servinosmanov/my-uncle-wrote-a-book-a-machine-read-it-in-my-voice-e955639f93ab">third article</a>. And my ears: I listen to the doubtful pieces myself when the automatic check can’t decide who’s right.</p><h3>When we tried to make our own data</h3><p>When you have little data, it’s tempting to make some. We had two tools for it: speech synthesis, which can read any text, and large language models, which can write.</p><p>First, synthetic speech for training the recogniser. The idea is simple: a synthetic recording arrives with perfect text, because we chose the text before the machine read it. I mixed in synthetic speech in several voices until it made up half of all training clips. Each synthetic clip was first run through the recogniser, and the ones that didn’t match their text were dropped by a threshold I set in advance. About 5% fell out.</p><p>Before the run I wrote down my prediction: a little synthetic speech might help, half of it would hurt. It went the other way. On an exam of two audiobooks the model had never heard, word errors went down from 17.0% to 15.7%, and the gain held in noise, in echo and over a telephone line.</p><p>But there’s a second half to that truth. That exam is audiobooks, clean reading in a quiet room. On recordings closer to real life, the model with synthetic speech looked worse than the one trained only on real speech. I can’t call that check final: its test texts were also made by the old model, and each version was trained only once. Still, that model didn’t go into service. The one on our server was trained on real speech, and on real speech it’s better.</p><p>Then, machine-written text for the language model. My rule here is that it doesn’t matter who wrote a text, a person or a machine. What matters is that it’s clean, correct Crimean Tatar. Generate as much as you like, but every batch goes through the dictionary, the Turkish checks, the transliteration check, and some of it through my eyes.</p><p>I asked several large language models to write dialogues in Crimean Tatar and read them blind, without knowing which was whose. Out of 26, not one was usable as it stood. The best could be saved with edits; the rest were mostly Turkish with some Uzbek and Kazakh mixed in. And the worst part: the automatic checks could barely tell the models apart. One that I rejected completely got the same score as the others, because my detectors back then could see Turkish but not Kazakh or Uzbek. That’s the moment the second detector came from.</p><p>The second attempt was stricter: longer dialogues, written to a detailed brief, and every one of them passed every automatic check without a single flag. I read six of them blind. Not one was even worth correcting. Two I rewrote myself; the other four were hopeless, “lots of words nobody would understand, and the wrong forms.” The automation said clean. The ear said: that’s not our language.</p><p>That’s when I understood synthetic data wasn’t going to work for us here, and I was upset. Synthetic data hadn’t solved the problem, and we still can’t find enough teachers.</p><p>My conclusion is the same for sound and for text. Synthetic data is good, but it doesn’t solve the main problem. We need more people who will write dialogues with the colours of the language, with its character and its soul.</p><h3>Why it all comes down to native speakers</h3><p>Put together, the picture looks like this. The texts turned up where nobody looked for them: in the layers under the scans, which needed repairing and checking, not reading. The sound turned up in broadcasts and audiobooks, but its text had to be dug out by two machines checking each other. Synthetic data added a little, and only where a person kept an eye on it.</p><p>And the most important decisions in this whole story were made by a native speaker’s ear. It told Turkish apart from dialect. It heard that the test texts themselves were wrong in one word out of every six or seven. And it said “that’s not our language” about dialogues the instruments had called clean. No filter did any of that.</p><p>Audio examples for this article: <a href="https://ai.ana-yurt.dev/en/blog/2026-10-otkuda-dannye/">https://ai.ana-yurt.dev/en/blog/2026-10-otkuda-dannye/</a>.</p><p><strong>Want to help?</strong> What this project needs most is native speakers. Listen to a short recording and tell us whether the machine wrote it down right. Read a page of text and mark which word is Turkish and which is ours, from a dialect. Record a few minutes of reading in your own voice. You don’t need to program for any of it; you need to know your language well. In this story the most important decisions were made by a native speaker’s ear. Write to <a href="/service/mailto:ai@ay.mw">ai@ay.mw</a> or on Telegram: <a href="https://t.me/ana_yurt_dev">t.me/ana_yurt_dev</a>.</p><p>And if you’d like to support the work itself, you can do that here: <a href="https://ko-fi.com/anayurt">ko-fi.com/anayurt</a>. That support becomes new recordings of living speech, and recording it matters now, while there are still people to learn from.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5122511e934" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Uncle Wrote a Book. A Machine Read It in My Voice.]]></title>
            <link>https://medium.com/@servinosmanov/my-uncle-wrote-a-book-a-machine-read-it-in-my-voice-e955639f93ab?source=rss-45c456062f69------2</link>
            <guid isPermaLink="false">https://medium.com/p/e955639f93ab</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[rsa]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[language-learning]]></category>
            <category><![CDATA[text-to-speech]]></category>
            <dc:creator><![CDATA[Servin Osmanov]]></dc:creator>
            <pubDate>Fri, 09 Oct 2026 17:57:37 GMT</pubDate>
            <atom:updated>2026-10-09T17:57:37.100Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5LmTYmxJaJdf91huWTfx3g.png" /><figcaption>listening to Audio books in Crimeantatar language is real</figcaption></figure><p>There’s a book in our online library called <em>Küreş meydanını ot basmaz</em>, which comes out roughly as <em>Grass Won’t Grow Over the Wrestling Ground</em>. Its author, Yunus Qandım, is my uncle. A machine has read all twenty-eight chapters of it aloud in Crimean Tatar, six hours and twenty-nine minutes of sound, and it read them in my voice.</p><p>Usually, when you hear your own voice on a recording, it sounds foreign and a bit strange. This time it was different. When I heard the first sentence, my happiness knew no bounds: the quality turned out to be very good.</p><p>That was the second book the machine read. The first was a novel, and that’s the one this article is mostly about, because it showed me everything a short demo hides. I’ll come back to my uncle’s book once it’s clear what it took to get there.</p><p>In the <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">first article of this series</a> I gave that novel a few lines: a synthesised voice had read Şamil Alâdin’s <em>Merdiven</em> (the word means “ladder” or “staircase”) from start to finish, sixteen chapters, 1 hour 58 minutes of sound. At the end of the <a href="https://medium.com/@servinosmanov/every-third-word-was-wrong-heres-how-my-language-got-working-ears-07172763bc0b">second article</a> I promised to tell the story behind those lines. Here it is.</p><p>Speech synthesis, TTS for short, is the computer reading text out loud. In this project it’s the voice. Without it there are no audiobooks, no reader app that speaks the text to you, and an assistant could only type its answers in silence.</p><h3>Fourteen sentences can’t tell you much</h3><p>Making synthesis sound good on a handful of sentences is easy. You pick the ones that came out well, put them on a page, and everything sounds great. I had my own version of that: fourteen test sentences, packed on purpose with the sounds that are hardest for our language, the deep <em>q</em>, <em>ğ</em>, <em>ñ</em> and <em>dj</em>. I used them to compare one version of the model with another.</p><p>But fourteen sentences say nothing about how a voice behaves for ten minutes straight. Whether the timbre drifts by the end of a chapter. What the machine does with dialogue, with names and numbers, with lines one or two words long. How it breathes. You can’t hear any of that in a fifteen-second clip. In a book you hear all of it.</p><p>So the next step was one whole chapter: the first chapter of <em>Merdiven</em>, 1,608 words, about twelve minutes. The goal wasn’t to admire it. The goal was to find what breaks.</p><p>Why <em>Merdiven</em>? I love Şamil Alâdin very much, and I wanted the first book a machine read to be one of his. The novel also suited the job: it’s even, narrative prose. Humorous stories or children’s books, where the voice has to play, the model wasn’t ready to read yet.</p><p>Before listening to anything, I made sure the text had never been in the material the model learned from: not a single match of six words in a row against nearly 1.4 million words. Otherwise the machine wouldn’t be reading, it would be remembering. Later the whole book went through the same check, also with zero matches.</p><p>The first number was a lesson in itself. One way to grade synthesis is to read it back with a speech recogniser for our language and count the characters that differ from the text. On the test sentences the machine got about 14% of characters wrong; on the chapter’s ordinary prose, under 2%. The model hadn’t improved overnight. The test sentences were built to be nasty, and ordinary text isn’t. Since then I don’t pass off the test-set number as how the model really reads.</p><p>The overall verdict on the chapter surprised me. The voice held. The timbre barely changed from the first minute to the last, and the pace stayed even. The faults weren’t in the model. They were in the text and audio handling around it: how the text gets cut into sentences, how short sentences get glued together, how breaths get trimmed. I’d written that part myself, which made it good news. It could be fixed without retraining anything.</p><h3>How the voice works, in two paragraphs</h3><p>The details were in the <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">first article</a>, so briefly. Ready-made models for neighbouring languages read our deep <em>q</em> as a plain <em>k</em>, and qara (black) comes out as <em>kara</em>. So I picked a base that already knew the sound: a multilingual model that had heard plenty of Kazakh, Uyghur and Bashkir. It had no Crimean Tatar in it, and a writing trick covers that: our къ, гъ, нъ are rewritten into the Bashkir letters ҡ, ғ, ң, which the model knows, and from then on it reads our text as if it were Bashkir.</p><p>The voice itself comes separately. The model I fine-tuned learns the language, not the timbre. It takes the timbre from a short sample, a few seconds of someone’s recording along with its exact text. <em>Merdiven</em> is read in Sevil’s voice, and every sentence of the book was voiced from the same seven-second sample.</p><p>The writing trick had a fourth rule, and it deserves its own section.</p><h3>One letter you can only judge on a long text</h3><p>In the <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">first article</a> I told the story of a word that came out wrong. Our word <em>nice</em> (no relation to the English one: it means “how many” and sounds roughly like <em>nee-djeh</em>) was being read as <em>nizhe</em>, which happens to be a Russian word meaning “lower”. The same rewriting rule was quietly turning our <em>dj</em> into <em>zh</em>, so the model was literally being handed a Russian word. What happened next is the part worth telling.</p><p>To fix it, I compared five ways of writing that sound. On the fourteen test sentences the two best were indistinguishable; both fixed the word. So I re-voiced the whole chapter, 195 sentences, with each of them. One of the two, the Tatar letter җ, fixed <em>nice</em> and damaged everything else: across the chapter it produced 27% more character errors. Plain <em>dj</em> fixed the word and broke nothing.</p><p>On fourteen sentences the two looked equally good. A full chapter showed at once that one of them did harm. That kind of discovery is exactly why I wanted the machine to read a whole book.</p><h3>What broke in the first chapter</h3><p><strong>Gluing short sentences.</strong> At first I glued short sentences onto their neighbours. The logic seemed solid: the model had learned from recordings no shorter than three seconds, so a two-word line was better not handed to it alone. I never checked that logic. I should have.</p><p>On the glued pieces the model started dropping whole sentences:</p><p><strong>Asked to read:</strong> <em>Men yañılğanım. O, oca degil eken.</em></p><p><strong>What it read:</strong> <em>O, oca degil eken.</em></p><p>“I was wrong. He isn’t a teacher after all” turned into just “He isn’t a teacher after all.” The glued pieces made up a quarter of the chapter’s text and produced 43% of its errors.</p><p>So I finally tested the assumption and voiced the chapter with no gluing at all. Short sentences really do come out worse, about two and a half times as many errors, though in absolute terms that’s about two percentage points, and errors keep falling only up to roughly seventy characters. But that penalty is smaller than the price of gluing, which was losing whole pieces of text. I switched gluing off.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ECZ0Z2rdHAYRGLAnxeTzw.png" /><figcaption><em>Chapter one voiced without gluing. Everything under 43 characters used to be glued to its neighbours, yet errors keep falling until about seventy.</em></figcaption></figure><p><strong>The trim that ate the first word.</strong> Synthesis sometimes opens a phrase with an intake of breath, like a live narrator, so I wrote a trim that cut everything before the first voiced sound. But plenty of words begin with a voiceless consonant: <em>Çañ qaqıldı</em> turned into <em>Qaqıldı</em>, <em>Tışta, yeşergen…</em> into <em>yeşergen…</em>. In 111 sentences out of 147 the trim cut into the first word, and the chapter read better with no trim at all than with my “fix”. The new trim goes by loudness instead. A consonant is quiet, but it’s still much louder than the silence in front of it. With the new trim the chapter reads better than with none.</p><p><strong>Breaths that weren’t there.</strong> My first check said 89% of sentences began with a breath, which scared me. It turned out the trim had been grading itself, counting those same voiceless consonants as breath. An honest count found a breath at the start of one sentence in five. Most breaths were actually between words, where the trim never looked, in 76 sentences out of 195. Those I learned to turn down to the level of the background without moving a single timestamp.</p><p><strong>Pace.</strong> The machine reads faster than a live narrator, 17.4 characters per second against 14.5, so the chapter I expected to run fifteen minutes took twelve, which sounds a little hurried for an audiobook. The pauses I didn’t invent. I measured them on ten minutes of a live narrator’s reading: the median gap between sentences was 288 milliseconds. So the machine now gets 300 milliseconds between sentences, a bit more when the speaker changes and a bit more again between paragraphs.</p><p>After all the fixes, the chapter’s character error rate was 1.84% instead of 2.56% in the first version. Same model. All that changed was what I fed it and what I did with the sound afterwards.</p><h3>Sixteen chapters, each on its own</h3><p>Then came the other fifteen chapters, and the whole book found what one chapter couldn’t. The three asterisks that separate scenes were going to the model as text, and it dutifully “read” them. The novel’s last line, the author’s date “1940 s.”, was taken for ordinary text, so the novel ended with the machine reading the year out loud. And five sentences were torn in half wherever a paragraph broke off on an ellipsis.</p><p>In chapter seven a safeguard I’d built into the assembly refused to build the chapter: the words of one sentence, glued back together, didn’t give back the original text. The cause was a double space in the book’s file. You can’t hear it or see it on screen, but a reader that shows the text word by word would have silently lost it. After that I also changed the order of work: if a safeguard trips in one chapter, the other chapters carry on. A book that stops at chapter seven in the middle of the night is a lost night.</p><p>I checked every chapter separately, and one looked bad: chapter four, at 8.3% character errors, four times the book’s median. I went and listened, and the chapter was fine. The whole error came from one two-word line, “– dep tüşündi.”, on which the checking recogniser got stuck and wrote it down twenty-five times in a row. Without that line the chapter sits at about 2%, like all the others. The same line tripped up the check in chapter two as well; it was just less visible there. The machine doing the reading wasn’t wrong. The one doing the checking was.</p><p>That’s also why a single average for the whole book would be the wrong number. One line like that spoils the average and tells you nothing about the reading. Chapter by chapter the picture is even: median 2.15%, best chapter 1.5%, worst 2.7%. Quality doesn’t slide toward the end of the book or on long chapters. The longest one, chapter two, runs nineteen and a half minutes and comes in at 2.5%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kmx1pEuGgQXj1C7RZL9PtQ.png" /><figcaption><em>The hatched tops are what the checker added by failing on short lines, not the reading itself. Nearly all of it is one line, “– dep tüşündi.”, which sent the recogniser into a loop in chapter four and again in chapter two.</em></figcaption></figure><p>As I wrote in the <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">first article</a>, two hours of audio take about three hours of work on a home computer. Less than half of that is the synthesis itself; more goes into tying every word to its moment in time and checking everything. It’s worth it.</p><h3>A reader that follows the voice</h3><p>I didn’t want the book only as a sound file. I wanted a reader: the text on the screen, the voice reading, the current word lit up, and a click on any word jumps the reading there.</p><p>For that you need to know when every word is spoken, and the trick that started the whole project helped again: lining sound up against text you already know. This time the known text is simply what the machine was given to read. Each sentence is aligned against its own text and then placed on the chapter’s timeline. In all sixteen chapters, 100% of the words have a time; only dashes and marks that aren’t pronounced are left without one.</p><p>In the <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">first article</a> I told the story of word timings that had drifted 44 seconds by the end of a chapter. Now the build refuses to run if the end of the last sentence and the length of the audio file disagree by more than half a second. And because the first version of the reader was built on the old narration, the one that still said <em>nizhe</em>, every chapter now stores a fingerprint of its audio file: a stale narration shows up in the data, not only to the ear.</p><p>The reader lives on ana-yurt.com, one page for both reading and listening. You can open <em>Merdiven</em> and listen to all of it: <br><a href="https://ana-yurt.com/kutuphane/3444/read">ana-yurt.com/kutuphane/3444/read</a>.</p><h3>Whose voice it is</h3><p>If the timbre comes from a sample, then choosing a voice means choosing a few seconds of recording. And the sample turned out to matter more than I’d thought.</p><p>I first tried my own voice on single sentences and heard long, misplaced pauses in the middle of them. I blamed the model. It was the sample: 0.77 seconds of silence before the words and 1.77 seconds after. The model takes the pace from a sample as well as the timbre, how much time to spend on each letter, and it was spreading that extra time through the phrase as pauses. I trimmed the silence off the edges, and the median total of pauses inside a sentence dropped from 930 to 85 milliseconds. In a blind comparison, listening without knowing which version was which, the trimmed one couldn’t be told apart from the old: 7 to 6. The defect went away, and the voice stayed.</p><p>Then my voice sounded dull to me. That was the microphone: a lapel mic on the chest sits off to the side of the mouth and loses exactly the frequencies that were missing. No filter paints that back in, because what the microphone never recorded isn’t in the recording. I had to record the samples again.</p><p>I also tried to help the male voice through training, by adding more male speech to the material. Blind, the new version lost to the old one 3 to 11, and one of the female voices drifted lower in pitch, from 208 to 199 hertz. I only measured pitch after my ears had spoken, and both pointed at the same voice. I didn’t keep that version.</p><p>Once the pauses and the microphone were sorted out, I decided to try my own voice on a whole book. That’s how the second book in our library, the one I started with, came to be read in my voice: Yunus Qandım’s <em>Küreş meydanını ot basmaz</em>, 28 chapters, 6 hours 29 minutes of sound. You can listen to it too: <a href="https://ana-yurt.com/kutuphane/3516/read">ana-yurt.com/kutuphane/3516/read</a>. It was read by the current version of the model; <em>Merdiven</em> was read by an earlier one.</p><p>I didn’t pick that book by chance either. Its author is my uncle, and I really wanted his book to be heard in my own voice.</p><h3>What I haven’t solved yet</h3><p><strong>Stress.</strong> This story went round in a circle, and it matters to me more than the others. I noticed that the machine sometimes read the name Riza as <em>Rıza</em>, with the dotless ı, a vowel made further back in the mouth. A check confirmed it: in about half of the sentences, and the same with the name Aziz. A fix turned up too: write the <em>i</em> as <em>iy</em> on the input. The back vowel disappeared completely, and the errors in the name itself went down. The instruments were happy.</p><p>I listened blind and didn’t accept it. In my note on the blind page I wrote: “if I hear the I, the stress is wrong.” The fix brought back the right vowel and dragged the stress onto the first syllable, while ours is on the last. Both versions were wrong, each in its own way.</p><p>I wasn’t too upset. These are first models, and I allow for mistakes like that in them. I decided to come back to this problem later.</p><p>Meanwhile I started measuring stress itself. In our language the last syllable of a word is usually drawn out a little, and live narrators stretch some words a lot and others hardly at all. The model gets the average right but reproduces only about six tenths of that range: too little where narrators stretch most, too much where they stay flat. The original model, before my fine-tuning, is just as flat. My current suspect is that the model sets the length of the whole phrase in advance, so a longer last syllable would have to borrow from its neighbours. That’s still a hypothesis, and testing it means working on the model itself, not on the text it’s given.</p><p><strong>Breaths.</strong> Cleaning up breaths has never once made anything worse, but short ones, about a tenth of a second long, partly survive. My own voice has the most: in one measurement breaths took up 15% of the sound, against about 1% for Sevil. Trimming the silence in the sample cut that by more than half, but not to zero.</p><p><strong>Pauses on emotional lines.</strong> It seemed to me the machine paused longer where the text got emotional. On sentences matched for length and number of commas I found no link: emotional lines in prose just tend to be longer. But I could only measure how long the pauses were, not where they fall, and a pause in the wrong place in an exclamation grates far more than the same pause at a comma. For me that question is still open.</p><p><strong>Dialogue.</strong> The model sees the dash before a line of dialogue, but it reads a change of speaker as a new sentence, not as a new voice.</p><p><strong>Numbers.</strong> I turn them into words in advance, but recognition can’t check how they were read: it writes what it hears as digits, and “1924” turns into “19124”. For now only the ear checks numbers.</p><h3>What the novel taught me</h3><p>The voice turned out to be the most reliable part of the whole chain. Nearly everything that broke was not in the model but in the handling of text and sound around it: the gluing, the trim, the letters, the timings, the double space. Some of it the instruments found, and the ear never would have. Some of it the ear found, and the instruments couldn’t even score it: <em>nice</em>, the stress in Riza, the breaths between words. And the whole book found what neither fourteen sentences nor one chapter would have.</p><p>When I’d listened to the book all the way to the end, I felt proud: the work hadn’t been in vain, and it gave an excellent result. But that’s no reason to relax. You can still hear places in the book that need more work, which means we still have a lot of work ahead.</p><p>This is the third article in the series. Next I’ll go into how I chose a base for the voice that already knows how to say our <em>q</em>, and why neighbouring Turkish won’t do for that. After that, as promised, the one about cloud graphics cards.</p><p>Audio samples and the rest of the project: <a href="https://ai.ana-yurt.dev/">https://ai.ana-yurt.dev/</a>.</p><p><strong>Want to help?</strong> What this project needs most is native speakers. Listen to a chapter and mark where the stress falls on the wrong syllable, where a name is read wrong, where a pause breaks a phrase in two. You don’t need to program for any of that; you need to hear your own language well. In this book the most important finds were made by ear. Write to me: <a href="/service/mailto:support@ana-yurt.com">support@ana-yurt.com</a>.</p><p>And if you’d like to support the work itself, you can do that here: <br><a href="https://ko-fi.com/anayurt">ko-fi.com/anayurt</a>. That support becomes new recordings of living speech — and recording it matters now, while the people who carry the language are still here to be recorded.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e955639f93ab" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Every Third Word Was Wrong. Here’s How My Language Got Working Ears.]]></title>
            <link>https://medium.com/@servinosmanov/every-third-word-was-wrong-heres-how-my-language-got-working-ears-07172763bc0b?source=rss-45c456062f69------2</link>
            <guid isPermaLink="false">https://medium.com/p/07172763bc0b</guid>
            <category><![CDATA[mls]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[tts]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[rsa]]></category>
            <dc:creator><![CDATA[Servin Osmanov]]></dc:creator>
            <pubDate>Wed, 23 Sep 2026 19:43:51 GMT</pubDate>
            <atom:updated>2026-09-23T21:31:03.986Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SsIjjEfO-8g-EV2Dw9yFYg.png" /></figure><p>Last time I wrote about <a href="https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4">why I decided to walk the whole road</a> — from a dictionary to a voice assistant you can actually talk to in Crimean Tatar — and why speech turned out to be the part that couldn’t wait. I promised to come back and tell you how the machine was taught to hear. This is that story, and it starts with a humbling number.</p><p>Speech recognition — the machine listening and writing down what it hears, ASR for short — is the ears of this project, and everything hangs off them. Without ears, old recordings stay as sound nobody can search, an assistant doesn’t know what you said to it, and there are no sound-and-text pairs to train a voice on. Almost anything else can be postponed. Not the ears.</p><p><strong>The exam I gave to a model everyone trusted</strong></p><p>I already had a model that could recognise Crimean Tatar. Or rather, that was understood to: a large Whisper recogniser, fine-tuned on our language. We’d been using it carefully, on clean recordings, where it looked respectable. I had read its transcripts and nodded along. What I had never done was sit it down for a proper exam with a number at the end.</p><p>So the first thing I did in this stage was not to improve anything. I tested it.</p><p>Recognition quality is measured as the share of wrong words — how many words you’d have to insert, delete or replace to arrive at the correct text. The answer came back: “<strong><em>34.6% word errors”</em></strong>, and 11.9% by characters.</p><p>Thirty-four and a half percent is every third word missed. That isn’t “a few rough edges.” A transcript with every third word wrong is no good for searching an archive, no good for training a voice, and nowhere near good enough to talk to; a human reads it as cipher. The only other openly available model for our language did worse on the same exam. So the starting point was clear: the best ears Crimean Tatar had were wrong about every third word.</p><p><strong>Exam first, lessons afterwards</strong></p><p>Why I started with a test rather than with training is probably the most useful thing I learned in this stage. You have to test a model on material it has never heard. People usually split data at random — nine clips into training, every tenth into testing. For a language like ours that’s meaningless: neighbouring clips come from the same recording, the same reader, the same day and microphone, and the slicing tool regularly cuts one sentence across two adjacent clips, so half a sentence goes into training and the other half into the exam. What the model then demonstrates is that it recognises a voice and a room, not a language.</p><p>So I set aside “<strong><em>two entire books”</em></strong>, with two readers the model had never encountered: 893 clips, 1 hour 52 minutes. That takes away the voice, the microphone, the era and the vocabulary of any particular text in one move.</p><p>This sounds like an obvious precaution. It turned out not to be enough.</p><p><strong>The four books that existed twice</strong></p><p>My material had accumulated over years, organised in different ways at different times. Before launching anything, I checked something banal: was the exam material really absent from the training pile?</p><p>By filename, yes. Zero matches, not one overlap; the two sets of names looked like they came from different universes.</p><p>So I stopped comparing names and compared the texts themselves, looking for training clips that contained the same six words in a row as some exam clip.</p><p>Four audiobooks were sitting in my materials twice. The same recordings, sliced by two different tools at two different times, filed under names with nothing in common. The share of each book’s clips that had a twin in the training set:</p><p>- “<strong><em>the book I’d picked to test on”</em></strong> — <em>96.9%</em> (651 of 672 clips)<br>- “<strong><em>the second exam book”</em></strong> — 58.8%<br>- “<strong><em>a third book”</em></strong> — 82.1%<br>- “<strong><em>a fourth book”</em></strong> — 69.8%</p><p>That top line is the book I was about to grade my model with.</p><p>There wasn’t fear in that moment, or anger. Just a plain realisation that none of my earlier measurements could be trusted, and one consoling thought on top of it — thank God this surfaced before the training run and not after. If I hadn’t noticed, everything downstream would have gone smoothly. The model would have trained on recordings it would later be examined on, different slicing, same sentences, same voices, and produced a beautiful number. I’d have been delighted, written it up, maybe published the model. And the number would have meant exactly nothing. It’s an exam where the questions were handed out in advance.</p><p>The nastiest thing about this trap is that it has no symptoms. Everything works, nothing crashes, and the result is better than you hoped — which is precisely the part that should worry you.</p><p>The conclusion I wrote down afterwards: “<strong><em>in a low-resource language, re-sliced copies of the same recordings are not the exception, they’re the norm.</em></strong>” There’s little material, so people come back to it with different tools, and each pass produces a fresh set of files wrapping old content. Hunt for those copies by text, never by name. After the cleanup the training set was re-verified from scratch: not one exam clip, not one match on six consecutive words. Not “I’m confident I cleaned it” — measured, and written down.</p><figure><img alt="Chart: share of each book’s clips that had a twin in the training set" src="https://cdn-images-1.medium.com/max/1024/1*Yyh1SI6KuTwMF5Cpz49ABQ.png" /></figure><p>*Four books that sat in my materials twice. By filename they didn’t overlap at all; the overlap only showed up in the text.*</p><p><strong>Ninety minutes of training</strong></p><p>And now the part everyone assumes is the main event, which turned out to be the fastest and dullest stretch of the stage.</p><p>You don’t have to retrain a model end to end. You can leave the main model untouched and bolt a small add-on next to it — a few extra layers that nudge its behaviour. Only the add-on learns. Mine came to “<strong><em>2% of the model’s parameters”</em></strong>: thirty-one million trainable against one and a half billion frozen. The training material was fifteen and a half hours of speech, what survived the cleaning: three passes, 705 steps.</p><p><strong><em>“An hour and a half on one home graphics card.”</em></strong> Peak video memory 10.4 gigabytes, so it fits in an ordinary gaming card rather than a server rack. The finished add-on weighs “126 megabytes”. You could send it to someone in a messenger.</p><p>The exam result: “<strong><em>20.1% word errors”</em></strong> instead of 34.6%, and 9.4% by characters instead of 11.9%. Forty-two percent fewer word errors.</p><p>I’ll be honest — there was no elation when the training finished. A number is a number; it got better, which was the plan. The pride arrived later, when I pushed material through the model that it had no business handling at all: films, podcasts, noisy recordings. The transcripts came back reading almost perfectly. That was the moment it stopped being a row in a table and became working ears.</p><p>Both exam books improved, and the one with the other reader improved more — so the model hadn’t memorised one person’s delivery. One detail from the run is worth more than the headline number: “<strong><em>at the first checkpoint the model was worse than before training.”</em></strong> By step 100 the add-on has half-learned its new behaviour, already interferes with the old one, and gives nothing back yet. Had I looked there and concluded the idea was a bust, I’d have shut it down and been wrong.</p><p><strong>Then the errors kept falling, with no training at all</strong></p><p>Here’s the interesting half of the story. A finished model has not only what it knows, but “<strong><em>how it picks its words when it answers”</em></strong>. By default it takes the simplest route: at each step it grabs whatever word looks most likely right now, and never looks back. Fast — and occasionally catastrophic, because once it turns down the wrong street it can’t correct itself and starts walking in circles. That’s exactly what was happening in the worst exam clips. Not “couldn’t make it out” — stuck in a loop:</p><blockquote>Qıznıñ qıznıñ qıznıñ qıznıñ …<br>– dedi Akimoviç, — dedi Akimoviç, — dedi Akimoviç, …</blockquote><p>The alternative is to make the model hold several candidate phrasings in mind at once and choose the best whole phrase rather than the best next word. It costs time and requires not one gram of retraining. I tried 24 different settings — not on the exam material, but on a small set held back specifically for choosing (more on that below). The winner went to the exam once.</p><p><strong><em>“17.0% word errors and 7.0% by characters.”</em></strong> Not a single weight changed. All that changed was how carefully the model assembles its answer, and it cost 3.1 times longer per transcription.</p><figure><img alt="Chart: three stages of recognition — word and character errors" src="https://cdn-images-1.medium.com/max/1024/1*Bs6iulRpC_NawTxxjwd2qQ.png" /></figure><p>*Left to right: the model I started from; after ninety minutes of training; after tuning how the model reads out its answer.*</p><p><strong>Where that win actually came from</strong></p><p>It would have been easy to stop here and write “decoding tuning cuts errors by 15%.” True, and a half-truth. Across the exam, “<strong><em>409 wrong words”</em></strong> were recovered. Of those, “<strong><em>160 came from three clips out of 893</em></strong>”— 0.34% of the material delivering 39% of the entire gain. They were the looping clips: against 41 words of correct text, the model had produced 175 words of error. The remaining 890 clips improved by a modest 10%, evenly, no fireworks.</p><p>Both halves are real, and neither alone would justify running three times slower. But they mean different things. The broad 10% is “the model got tidier.” The three clips are “it stopped occasionally losing its mind.” For an archive the second matters far more: a clip where a sentence has been replaced by five hundred bytes of one repeated word isn’t merely bad, it poisons everything you later train on it.</p><p>After the tuning, there were no such clips at all. Zero out of 893.</p><p><strong>The fix that should have helped and made things worse</strong></p><p>Among the settings there’s a blunt one: forbid the model from repeating itself. It looks like precisely the right weapon against looping.</p><p>I tested it, and it was harmful. Of 255 development clips it “<strong><em>broke 17 and fixed 1”</em></strong> — and twelve of the seventeen had been practically flawless beforehand. You can see why in the transcripts. The ban doesn’t stop the model repeating; it forces it to repeat “<strong><em>incorrectly”</em></strong>:</p><blockquote>in the book — Qartbaba, qartbaba, — tez-tez çapıp kelgen kiçkene Aziz<br>no ban — Qartbaba! Qartbaba! — tez-tez çapıp kelgen kiçkene Aziz<br>ban on — Qartbaba! Qartbabaa! — tez-tez çapıp kelgen kiçkene Aziz<br>ban harder — Qartbaba! Qartbava! — tez-tez çapıp kelgen kiçkene Aziz</blockquote><p>A child is running and calling out: “Qartbaba! Qartbaba!” — Grandfather! Grandfather! The model heard that perfectly. The no-repeat rule wouldn’t let it write down what it had heard, so it mangled the second word rather than let it match the first. The same happened to `bağışla… bağışla…` — “forgive me… forgive me…” — whose second instance became the non-existent `Bağuşla`.</p><p>Calling someone twice, asking twice: that’s ordinary Crimean Tatar, that’s how people talk. A ban can’t tell a living repetition from a machine stuck in a groove, and our language is full of living repetitions.</p><p>Of everything in this stage, this is the story I like best, because it isn’t really about engineering. You cannot fix a language with prohibitions. Any rule of the form “that doesn’t happen” eventually collides with how people actually speak — and it breaks exactly that spot while leaving everything around it intact, so that you don’t notice.</p><p><strong>How not to lie to yourself</strong></p><p>I’ve mentioned a “separate small set,” and that the exam was run only a couple of times. Every number above rests on that, so here is how it works. There aren’t two sets. There are three.</p><p><strong><em>“The training set”</em></strong> — what the model learns from.</p><p><strong><em>“The development set”</em></strong> — 255 clips, half an hour of audio, two short works pulled whole out of the training material. Its job is to “<strong><em>choose”</em></strong>: which checkpoint is better, which of the 24 settings is better. I ran dozens of measurements over it, as often as I liked.</p><p><strong><em>“The exam set”</em></strong> — those two books. I ran the model over it exactly “<strong><em>twice in the entire stage”</em></strong>: once on the winning checkpoint, once on the winning decoding setting. The rule for picking a winner was written down before I looked at any results.</p><p>Why so strict? Because if you run the exam over and over and keep the best result each time, what you select isn’t the best model but the luckiest one. The number comes out beautiful and never reproduces again.</p><p>The development set was checked for contamination too — zero shared clips, zero six-word matches. After the duplicates, I no longer file that under paranoia. It also turned out to be “<strong><em>much easier than the exam”</em></strong>: the same model got 17% of words wrong on it, against 34.6% on the exam, because its pieces come from the same circle of material as the training data while the exam books are two complete strangers reading. So it’s fine for saying “this setting beats that one,” and useless for saying “the model errs this much.”</p><p>That bit me once. Whisper has a built-in watchdog against looping — exactly the right instrument, nearly free. But the development set held “<strong><em>not a single”</em></strong> loop, so there was nothing to test it on, and by my own rules I couldn’t select it. So I have no number for it, and I won’t invent one.</p><p><strong>Seventeen percent in the studio. What about real life?</strong></p><p>Everything above was measured on one kind of sound: one person, one microphone, a quiet room, reading a book. An honest number for its own world, and nearly silent about the world a voice assistant lives in.</p><p>So the next step was to take those same 893 exam clips and ruin them seventeen ways — noise, music, other people’s voices, room echo, a telephone line, faster and slower speech. The text is known, so the error can be measured honestly in each of those worlds.</p><p>Telephone bandwidth and bad audio compression turned out to be “<strong><em>free”</em></strong>; down a phone line the model works a hair better than in the studio. Speaking rate barely matters either — 15% faster or slower costs no more than 6%.</p><p>But “<strong><em>background voices are the real enemy, by a wide margin.”</em></strong> A murmur of other people’s speech costs about twice what steady noise of the same loudness costs, and four times what music costs. When the competing speech is as loud as the speech you want, errors hit “<strong><em>69%”</em></strong> instead of 17 — the model simply stops working. The runner-up is echo: in a reverberant space like a corridor, errors grow two and a half times.</p><p>Which points at something concrete for the assistant to come: the main investment in reliability isn’t “more data,” it’s pulling one voice out of several, and a microphone placed properly.</p><p>One last check pleased me: an open set of Crimean Tatar audiobooks I had never trained on scored practically the same as my own exam. The model hadn’t memorised my readers. It hears the language.</p><p>So 17% isn’t “done.” It’s a reference point. The work with real noise starts here, and it’s already under way.</p><p>— -</p><p>This is the second article in the series. Next: how a synthesised voice read an entire book — sixteen chapters, nearly two hours of audio — and what had to be verified in every chapter separately. After that, the one where I rented someone else’s graphics cards in the cloud and set money on fire computing almost nothing.</p><p>Audio samples and the rest of the project — [https://ai.ana-yurt.dev/](<a href="https://ai.ana-yurt.dev/ru/blog/2026-09-mashina-nauchilas-slyshat/">https://ai.ana-yurt.dev/en/blog/2026-09-mashina-nauchilas-slyshat/</a>).</p><p><strong><em>“Want to help?”</em></strong> What this project needs most is native speakers. Listen to a clip and tell me whether the machine wrote it down correctly; read a transcript and correct it; point out where a word doesn’t sound like ours. You don’t need to program for any of that — you need to hear your own language well. Ears exactly like that found things in this stage that no automated check caught. Write to me: [<a href="/service/mailto:support@ana-yurt.com">support@ana-yurt.com</a>](<a href="/service/mailto:support@ana-yurt.com">support@ana-yurt.com</a>).</p><p>And if you’d like to support the work itself, you can do that here: [ko-fi.com/anayurt](<a href="https://ko-fi.com/anayurt">https://ko-fi.com/anayurt</a>). That support becomes new recordings of living speech — and recording it matters now, while the people who carry the language are still here to be recorded.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=07172763bc0b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Language Is Listed as Endangered. So I Built It a Voice.]]></title>
            <link>https://medium.com/@servinosmanov/my-language-is-listed-as-endangered-so-i-built-it-a-voice-8f855ae40ad4?source=rss-45c456062f69------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f855ae40ad4</guid>
            <category><![CDATA[gemma-4]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[mls]]></category>
            <dc:creator><![CDATA[Servin Osmanov]]></dc:creator>
            <pubDate>Thu, 10 Sep 2026 00:13:26 GMT</pubDate>
            <atom:updated>2026-09-10T00:40:46.104Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6eq-Ae5teMmifqUSaAbIkA.png" /><figcaption>My Language Is Listed as Endangered. So I Built It a Voice.</figcaption></figure><p>The word was `<em>qara</em>`. It means black. It is about as ordinary as a word gets.</p><p>I was listening to an off-the-shelf speech model read it back to me, and it said <strong>kara</strong>. Softly, politely, in Turkish. In Crimean Tatar those are two different sounds carrying two different meanings, and the wrong one had just landed in the middle of a word I have heard since I was a child.</p><p>That is a small thing. It is also not a small thing at all, and it is roughly where this story starts.</p><h4>Almost nothing</h4><p>Crimean Tatar is my language. It is also on the lists of endangered languages, and if you try to use it the way you’d use English or Turkish on a phone, you find out fast what “endangered” means in practice.</p><p>No decent machine translation. No speech recognition. No synthesis, so no way to have a text read to you. No keyboard that suggests the next word. None of the invisible scaffolding that speakers of large languages stand on without ever noticing it is there.</p><p>Not “a little of it.” Almost none of it.</p><p>At some point that stopped being an observation and turned into an itch I couldn’t put down. A language with no tools is only really available to people who grew up inside it, who heard it at home. Everyone else is locked out, including a lot of people who very much want in.</p><p>So I made a plan that, written down, looked presumptuous even to me: walk the whole path. From an ordinary dictionary to a voice assistant you can actually talk to in Crimean Tatar. Not one model, not one tool. The entire chain, one step at a time.</p><p>It didn’t come out of nowhere. Years earlier I’d built a dictionary database, then a phrasebook. When AI tooling started getting genuinely usable, I reached for what was already in my hands: I cleaned up a Crimean Tatar speech dataset and took a first shot at decent synthesis. Both went out in the open — the <a href="https://huggingface.co/datasets/servinosmanov/tts-crh-sevil-fixed">dataset</a> and the <a href="https://huggingface.co/servinosmanov/xtts-crh-sevil-v1">model</a> are public. That first attempt was not great, but holding it made the bigger idea feel possible instead of ridiculous.</p><h4>Why voice, and why now</h4><p>The dictionary and the alphabet converter came first, because they were the most paper-like. Crimean Tatar is written in two alphabets, Cyrillic and Latin, so every text quietly exists in two incompatible versions; a converter isn’t a nice-to-have, it’s plumbing. You can’t teach a machine to pronounce a word if you can’t check how the word is spelled.</p><p>Speech came next, and it turned out to be the hardest stage by a wide margin. It’s what this article is really about.</p><p>Why speech before the translator, before the assistant that’s supposed to be the destination? Two reasons, and only one of them is technical.</p><p>The first is time. A book will sit on a shelf for another twenty years and nothing will happen to it. The people whose living speech can still be recorded are not getting younger. That is the one part of the job that cannot be postponed, and it doesn’t care about my roadmap.</p><p>The second I think of as a flywheel. Speech recognition — the machine listening and writing down what it hears, usually shortened to ASR — turns old recordings into text. Text plus the audio it belongs to is exactly the raw material for training synthesis, the machine reading text out loud, shortened to TTS. Synthesis makes audiobooks, people listen, people speak more, recognition gets better. The circle closes and starts turning on its own. For a language with no stockpile of material, that’s the only way out of poverty under its own power, instead of waiting to be noticed by someone with a research budget. That wait can end in nothing.</p><p>An assistant needs both halves anyway: to talk to it, it has to hear you and answer out loud. Speech isn’t a detour from the route, it’s the middle of it, and useful things fall out along the way — audiobooks for a language that has almost none, a pronunciation trainer, archive recordings turned into searchable text.</p><p>Nobody tried to talk me out of any of it, by the way. The only doubts in the room were mine, and they were technical.</p><h4>One sound</h4><p>Back to `<em>qara</em>`.</p><p>In Crimean Tatar, /q/ and /k/ are separate sounds with separate meanings, and /q/ sits inside words we say every day: `<em>yoq</em>`, `<em>qız</em>`, `<em>vaqıt</em>`. If your model flattens it into /k/, it isn’t speaking with an accent. It’s speaking a slightly different language.</p><p>So I went looking at the neighbours, because “just take a Turkish model, the languages are close” is the advice everyone gives:</p><ul><li><strong>Turkish</strong> — the sound doesn’t exist at all, there’s only one k, and qara comes back as <em>kara</em>.</li><li><strong>Azerbaijani</strong> — the letter q reads as /g/ and weakens further, down to a breath.</li><li><strong>Kazakh, Uyghur, Bashkir</strong> — the sound exists, and it has its own letter: ҡ, қ.</li></ul><p>I verified the Turkish case twice on purpose, because I didn’t want to argue with myself about what I thought I’d heard. The most obvious path breaks pronunciation in the most frequent words in the language. That was when it became clear you cannot simply switch on somebody else’s finished model.</p><h4>What a dataset is, and why we didn’t have one</h4><p>To make a computer speak a language you need recordings of speech and the exact text that goes with them. Many hours of it, cleaned, aligned, cut into matching pieces. That collection is called a dataset. It’s the fuel; everything else is engine.</p><p>For English there are thousands of them. For Crimean Tatar there was not one usable for serious work — not because nobody cared, but because we’re a long way down the queue at the big labs, and we may never come up.</p><p>Which turned the question into a personal one: can one person do this? No lab budget, no server farm. One computer, one good graphics card, evenings and nights.</p><p>The answer, I’ll spoil now, is yes. In months, not years. The interesting part is where those months go, because it isn’t where anyone assumes.</p><h4>The first good decision</h4><p>The standard way to build a speech dataset is to take audio and have a machine transcribe it. That was closed to us: there was no good Crimean Tatar recognition, and bad recognition produces a bad corpus, which produces more bad recognition. A circle turning the wrong way.</p><p>So I flipped it. Audiobooks already have exact text, printed in the book. There’s nothing to recognise. You line the known text up against the sound and cut on word boundaries. That inversion was the most important decision in the project: it walks straight around the thing the language didn’t have.</p><p>Then came the part nobody puts in their write-up. One alignment run over seven and a half hours of recording worked honestly for the first two hours and then silently ruined three and a half hours of audio. No error on screen, no warning. Smooth, confident, completely wrong output. Out of seven hours, a little under two survived.</p><p>Those failures are the worst kind. A crash is a gift, you see the problem instantly. Plausible garbage costs you a week.</p><p>The fix wasn’t in the code, it was in the approach. Instead of three big blocks I split the material into forty chapters and located each one separately, matching short character sequences rather than whole words. Word-level matching found nothing: on a pair I knew for a fact was a match, similarity came back under one percent. Character-level, everything snapped into place — thirty-eight chapters out of forty, and five clean hours out of the same material instead of under two.</p><h4>You can’t add data to fix a foundation</h4><p>Before changing anything, I wanted to know why the model said <strong>kara</strong>. Maybe it had never seen the sound? No. In my set, the letter q showed up in 88% of recordings, more often than k. The model had heard it pronounced correctly thousands of times.</p><p>The problem was underneath. In the base model, before any training on my language, q was already wired to /k/ through English, Spanish, French, and my fine-tuning wasn’t overriding that. Which leads somewhere uncomfortable: it doesn’t matter how much data you add. You’re not short on data, you’re standing on the wrong floor.</p><p>So I went looking for a base by exactly one criterion: that it already knew how to say our sound. That points at the Turkic languages where /q/ exists and has its own letter — Kazakh, Uyghur, Bashkir. I found a multilingual model whose training included fifteen hundred hours of Kazakh, four hundred of Uyghur and two hundred and fifty of Bashkir, against a hundred and twenty-five of Turkish. Precisely the opposite of what intuition recommends.</p><p>It still had no Crimean Tatar in it. I got around that with a writing trick: our къ, гъ, нъ get rewritten into the single Bashkir letters ҡ, ғ, ң the model already knows, and then you talk to it as if you were speaking Bashkir. You could hear it working before any fine-tuning at all — the new base pronounced our /q/ correctly 88–100% of the time. I wasn’t teaching the model a new sound. I had stopped getting in its way.</p><h4>Two and a half times the data, and the number didn’t move</h4><p>My favourite negative result of the whole project.</p><p>The corpus grew: six hours, eleven and a half, fifteen. The machine scores quality by character error rate — feed the synthesised speech back through recognition and count how many characters come out wrong. Here’s what happened:</p><ul><li><strong>base, no Crimean Tatar at all</strong> — 0.173–0.198, accented but readable</li><li><strong>5.9 hours</strong> — 0.136</li><li><strong>11.5 hours</strong> — 0.138, noticeably livelier intonation</li><li><strong>15.3 hours</strong> — 0.141, better again</li></ul><p>Two and a half times the data, and the number didn’t improve. Formally it got very slightly worse. The spread between repeat measurements of the same voice was wider than the entire difference between rows, so the honest reading is that all three runs are indistinguishable by this metric.</p><p>And by ear, every next voice was better than the one before. I compared them blind, not knowing which was which, and the preference was stable.</p><figure><img alt="Chart: corpus size against machine-measured quality" src="https://cdn-images-1.medium.com/max/1024/1*rKIkEcA7XR-dHoDtwIn55g.png" /></figure><p><strong>The corpus grew two and a half times over. The machine metric sat still — while blind listening picked the newer voice every time.</strong></p><p>The explanation matters more than the result. Intelligibility, how clearly the words come across, saturates early, somewhere around six hours. Intonation, liveliness, the breathing of a phrase keep improving well past that. The metric stops right where the remaining quality begins. Anyone who looked at that number and stopped collecting data would have been telling the truth and making the wrong call.</p><p>Underneath it is a rule that cost me several crossed-out “wins”: one measurement means nothing. The same voice showed one number on the first run and a visibly different one averaged over three. Measure three times.</p><h4>A bug no ear would ever catch</h4><p>I only found this one because I was clicking on individual sentences. Listening to a narrated chapter, I clicked sentence 195 and heard the wrong thing. Further into the text the mismatch grew, and it grew evenly:</p><ul><li><strong>sentence 1</strong>–0 s</li><li><strong>sentence 49</strong> — +10.7 s</li><li><strong>sentence 98</strong> — +21.8 s</li><li><strong>sentence 147</strong> — +33.0 s</li><li><strong>sentence 195</strong> — +43.9 s</li></ul><p>My first thought was right in shape and wrong in substance: the model must be falling apart over long text. It wasn’t. The voice held steady from the first minute to the last, and the tempo didn’t move. The audio was fine.</p><p>Two parts of the program were computing time through the chapter independently, and one of them counted the pauses between phrases twice. Forty-three seconds is exactly the sum of those doubled pauses across every seam in the chapter. The sound was correct. The map to the sound was not.</p><p>Listening straight through, this bug is completely inaudible. The file is right. It only shows itself if you play one sentence on its own.</p><h4>A word that was pronounced perfectly</h4><p>The most human story in the project, and the investigation took about a minute.</p><p>I heard a soft “<strong>j”</strong> in a chapter where the book has a “<strong>dj”</strong>, went to dig into it, and there was nothing to dig. The letter-rewriting rule, the same one with ҡ and ғ, was also collapsing “<strong>dj”</strong> into “<strong>j”</strong>. The model had been handed the wrong string, and it pronounced that string flawlessly.</p><p>The rule had glued together a distinction the language actually makes: “<strong>dj”</strong> lives in our own words, “<strong>j”</strong> mostly arrives with loanwords. Once they’re merged, nothing downstream could possibly have said them differently.</p><p>There were fifty-five such words in that chapter. I noticed one, the only one that happened to land on another real word; the other fifty-four just sounded slightly off, and the ear walked past them. Fixing it barely moved the numbers, and it was still right. This is our language, not a statistic.</p><h4>Where we are today</h4><p><strong>Recognition</strong>. The machine listens to Crimean Tatar speech and writes it down. Quality is the share of words it gets wrong, and separately, characters:</p><ul><li><strong>the model I started from</strong> — 34.6% word errors, 11.9% character errors</li><li><strong>+ fine-tuning (an hour and a half on one graphics card)</strong> — 20.1% word errors, 9.4% character errors</li><li><strong>+ tuning how the model reads out its answer</strong> — <strong>17.0%</strong> word errors, <strong>7.0%</strong> character errors</li></ul><p>The second row is training. The third involves no training at all: I didn’t touch a single weight, I just asked the model to pick its words more carefully. Errors ended up at half of where they started, and everything I added on top weighs 126 megabytes.</p><figure><img alt="Chart: recognition errors before and after" src="https://cdn-images-1.medium.com/max/1024/1*pGdfAJneAnn1jn_YEPU6gA.png" /></figure><p><strong>Left to right: the original model, after fine-tuning, after tuning the decoding.</strong></p><p>The hard part wasn’t the training, it was making the evaluation honest. Four of my audiobooks turned out to be sitting in my materials twice, cut up differently, under completely unrelated names — by filename they didn’t overlap at all. For one book I meant to test on, 96.9% of the clips had a twin in the training set. Had I missed that, the model would have sat an exam on questions it had already seen, and I’d have gotten a beautiful number that meant nothing.</p><p><strong>Voice</strong>. A synthesised voice read an entire book: Şamil Alâdin’s novel «Merdiven». Not an excerpt, not one chapter for a demo. The whole thing:</p><p><strong>chapters</strong> — 16</p><p><strong>words</strong> — 15,444</p><p><strong>running time</strong> — <strong>1 hour 58 minutes</strong></p><p><strong>machine time</strong> — 177 minutes, about one and a half times the length of the audio</p><p><strong>word timings verified</strong> — 100% of words, all sixteen chapters</p><p><strong>per-chapter error</strong> — median 0.022, worst chapter 0.027, best 0.015</p><p>Every chapter was checked on its own, rather than judging a book by one lucky chapter, and there’s no decay toward the end. I also proved by measurement, not argument, that this book’s text never got into training: not a single matching word sequence against nearly a million and a half words of corpus. Otherwise this wouldn’t be reading, it would be recitation.</p><p>And the number I think matters most here: <strong><em>two hours of finished audio costs about three hours of machine time</em></strong>, everything included — synthesis, word-level alignment, the checks, a full reverse proofread of every chapter. An audiobook in our language now costs one night on one home computer. Not a studio with hired narrators, and not a year of waiting.</p><h4>The clip I sent to everyone</h4><p>The first time I heard genuinely good narration in Crimean Tatar, I thought I’d lose my mind from happiness. I wanted to stop strangers on the street about it.</p><p>What I actually did was record a clip and send it to friends and family, the ones who knew what I’d been doing in there all those nights. Some of them had never heard our language come out of a machine before, because there was never a machine that could do it.</p><p>That is the whole reason for the rest of it.</p><h4>Where the months really went</h4><p>The most honest thing I can tell you: training models takes hours. An hour and a half here, a few hours there. It is nowhere near the main expense.</p><p>The months went elsewhere. Into collecting and aligning data, and repairing what broke without saying so. Into proving the evaluation was clean and the model hadn’t seen its own exam. Into re-measuring. And into listening with ears, blind, one sentence at a time.</p><p>Near the end I noticed something that surprised me. All the errors found over the project split roughly in half between instruments and a human being with headphones on. The instruments caught that minute-long timing drift; no ear would ever have caught it, because straight through it sounds correct. The human caught breaths landing in the wrong places, bad annotations, and that one collapsed consonant.</p><p>For a language like mine, the cheapest and strongest tool available is one native speaker, fifteen minutes, and a pair of headphones. It outplays the automation. Which leads somewhere genuinely hopeful: you don’t have to be a programmer to take part in work like this. You have to hear your own language well.</p><p>— -</p><p>This is the first article in a series. Next: how the machine was taught to hear Crimean Tatar, how a book got narrated chapter by chapter, how to run blind comparisons so you don’t fool yourself, and how I set money on fire renting someone else’s GPUs and computed almost nothing on them.</p><p>The assistant is still a long way off. But the stretch of road that looked least passable has been crossed, and it turned out to be passable.</p><p>Voice samples and the rest of the project — <a href="https://ai.ana-yurt.dev/blog/2026-09-kak-rodilas-ideya/](https://ai.ana-yurt.dev/blog/2026-09-kak-rodilas-ideya/).">https://ai.ana-yurt.dev/blog/2026-09-kak-rodilas-ideya/</a><a href="https://ai.ana-yurt.dev/blog/2026-09-kak-rodilas-ideya/](https://ai.ana-yurt.dev/blog/2026-09-kak-rodilas-ideya/).">.</a></p><p>If you speak Crimean Tatar and want to help, the project needs your ears far more than it needs code: listening, reading, telling me where the machine gets it wrong. Write to me at <a href="/service/mailto:support@ana-yurt.com">support@ana-yurt.com</a>. And if you’d like to support the work itself, you can do that at <a href="https://ko-fi.com/anayurt">ko-fi.com/anayurt</a> — every bit becomes hours of recordings that live on as data.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f855ae40ad4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Journey Improving a TTS Model for the Crimean Tatar Language]]></title>
            <link>https://medium.com/@servinosmanov/my-journey-improving-a-tts-model-for-the-crimean-tatar-language-66ff85e90a43?source=rss-45c456062f69------2</link>
            <guid isPermaLink="false">https://medium.com/p/66ff85e90a43</guid>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[language-learning]]></category>
            <category><![CDATA[ai-tools]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Servin Osmanov]]></dc:creator>
            <pubDate>Fri, 07 Nov 2025 19:47:21 GMT</pubDate>
            <atom:updated>2025-11-11T15:52:04.353Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UAyu7pihZYV18KejUleJPg.png" /><figcaption>TTS Model for the Crimean Tatar Language</figcaption></figure><h3>From Data Loss to a Fully Working Voice</h3><p>When you work with machine learning, success often hides behind hours of frustration, countless errors, and broken pipelines. This project — improving the <strong>Crimean Tatar TTS (Text-to-Speech)</strong> model — was exactly that kind of journey. What started as a small experiment to fine-tune an existing model turned into a full-scale debugging adventure that taught me more about data integrity, audio processing, and patience than any tutorial could.</p><p>This article continues my earlier piece, <a href="https://medium.com/@servinosmanov/why-language-tech-matters-developing-ai-tools-for-small-languages-fe130073a987"><strong>Why Language Tech Matters: Developing AI Tools for Small Languages</strong></a>, where I discussed the importance of giving smaller languages a voice in the AI era.</p><p>The Starting Point: A Model That Worked — but Only Partially</p><p>My goal was simple: improve the voice model “<strong>Sevil</strong>” for the Crimean Tatar language. I had already worked with similar voices — “<strong>Arslan</strong>” and “<strong>Abibullah</strong>” — using Hugging Face datasets like speech-uk/tts-crh-arslan and speech-uk/tts-crh-abibullah.</p><p>The first training attempts went well — <strong>loss around 0.283</strong>, results acceptable. But something didn’t add up. The dataset had <strong>1,566 audio files</strong>, yet the training logs showed only <strong>415 were being used</strong> — about <strong>26.5%</strong> of the total.<br>That meant <strong>almost three-quarters of my data was silently ignored</strong>.</p><p>At first, I thought it was a fluke. Then I realized it was a <strong>systemic problem</strong> in the Hugging Face datasets API when loading compressed audio from Parquet files.</p><h3>Diagnosing the Problem</h3><p>When I loaded the dataset through datasets.load_dataset(), most files threw errors:</p><p>Error: “Error while decoding audio”<br>Error: “Audio file appears to be empty”</p><p>That didn’t make sense — the audio bytes were clearly present in the Parquet files.<br>After checking manually with pandas.read_parquet(), I confirmed the data was there. The problem wasn’t in the files — it was in how the decoder handled them.</p><p>That’s when I realized: the <strong>datasets API couldn’t decode raw audio bytes correctly</strong>. The data was fine, but the pipeline was broken.</p><h3>Turning Bytes into Sound</h3><p>At this point, I tried everything.<br>I extracted the bytes manually, saved them as .raw files, and tried to load them with librosa and soundfile. Nothing worked.<br>Without proper <strong>metadata</strong> (sample rate, channels, encoding), the files were unreadable.</p><p>Eventually, I discovered the solution: <strong>FFmpeg</strong>.</p><p>By using known dataset parameters — <br>sample rate: 16000 Hz, channels: mono, format: PCM 16-bit little-endian — <br>I could convert all .raw files into clean .wav audio.</p><p>ffmpeg -f s16le -ar 16000 -ac 1 \<br> -i sevil_0000.raw \<br> sevil_0000.wav</p><p>And just like that — <strong>1,566 files successfully converted</strong>.<br>No corruption. No decoding errors. 100% validation success.</p><p>It was the moment of breakthrough — the kind that makes you sit back, smile, and realize you’ve just solved a problem that haunted you for two days straight.</p><h3>Training the Model — Again</h3><p>With clean audio finally ready, I retrained the Sevil model from scratch, this time using <strong>all 1,566 recordings</strong>.</p><p>Training setup (based on my previous configs for “Arslan”):</p><p>num_train_epochs = 500<br>batch_size = 4<br>learning_rate = 1e-4<br>fp16 = True<br>warmup_steps = 2000<br>save_steps = 2000</p><p>The progress was promising: — Loss dropped from <strong>1.14 → 0.80 → 0.50 → 0.27</strong> — The voice quality improved with every epoch<br>- And then… it crashed at <strong>78%</strong> completion.</p><p>The culprit? A familiar one for SpeechT5 users — <br>RuntimeError: torch.cat(): expected a non-empty list of Tensors<br>It turned out to be a bug in <strong>guided_attention_loss</strong>, a component that sometimes fails with uneven sequence lengths.</p><h3>Fixing the Crash</h3><p>Instead of starting over, I resumed training from the last checkpoint (step 16,000) and simply <strong>disabled guided attention</strong>:</p><p>model.config.use_guided_attention_loss = False</p><p>That one line saved the project.<br>Training resumed, completed <strong>98% of the full cycle</strong>, and stabilized with a final <strong>loss of 0.267</strong> — a small numerical improvement, but a big qualitative one.<br>The model became more consistent and robust across new data.</p><h3>Comparing Results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Qt83KSr9p_o6UQ6p0qw7Yw.png" /></figure><p>The difference wasn’t just in metrics — it was in <strong>confidence</strong>.<br>Sevil v2 generalized better, produced smoother intonation, and maintained pronunciation consistency even on unseen words.</p><h3>Lessons Learned</h3><ol><li><strong>Never trust good metrics without checking data coverage.</strong><br>My “good” baseline was trained on just 26% of the data.</li><li><strong>FFmpeg is a lifesaver.</strong><br>It solved what specialized libraries couldn’t.</li><li><strong>Validate every single file.</strong><br>Automation saved hours of manual checking.</li><li><strong>Guided attention loss is optional.</strong><br>Sometimes stability matters more than theoretical accuracy.</li><li><strong>Document everything.</strong><br>By keeping track of every attempt — successful or not — I could understand the full story, not just the happy ending.</li></ol><h3>Why This Project Matters</h3><p>For me, this wasn’t just about fixing one dataset. It was about enabling a <strong>low-resource language</strong> — Crimean Tatar — to have a better voice in the digital world.<br>Improving the Sevil model means clearer pronunciation, smoother prosody, and better accessibility for learners and native speakers alike.</p><p>And for anyone working with custom TTS datasets:<br>Check your files, validate your data, and don’t give up when your model crashes at 78%.<br>That crash might be the best teacher you’ll ever have.</p><p><strong>Author:</strong> <em>Servin Osmanov</em><br>Lead Fullstack Python / ReactJS Engineer</p><p>LinkedIn: <a href="https://www.linkedin.com/in/servin-osmanov/">https://www.linkedin.com/in/servin-osmanov/</a><br>AI researcher and TTS developer for low-resource languages<br>Project: <em>servinosmanov/tts-crh-sevil-fixed </em>on Hugging Face</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=66ff85e90a43" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Judging at CBIT Hacktoberfest 2025: What I Learned from 24 Hours of Open Source Creativity]]></title>
            <link>https://medium.com/@servinosmanov/judging-at-cbit-hacktoberfest-2025-what-i-learned-from-24-hours-of-open-source-creativity-e52d673b534f?source=rss-45c456062f69------2</link>
            <guid isPermaLink="false">https://medium.com/p/e52d673b534f</guid>
            <category><![CDATA[website]]></category>
            <category><![CDATA[hackathons]]></category>
            <category><![CDATA[projects]]></category>
            <category><![CDATA[judgement]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Servin Osmanov]]></dc:creator>
            <pubDate>Thu, 30 Oct 2025 18:33:06 GMT</pubDate>
            <atom:updated>2025-10-30T18:42:47.680Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NxHdAFC1dlwYMkdsGXCozQ.png" /></figure><p>When I first started attending hackathons, I was always the one coding, building, and pitching. This year, for the first time, I got to experience the other side — as a <strong>judge</strong> at the <a href="https://cbit-hacktoberfest25.devpost.com/?ref_feature=challenge&amp;ref_medium=discover#prizes">CBIT Hacktoberfest Hackathon 2025</a>.</p><p>It was an incredible 24-hour online event organized by the <strong>CBIT Open Source Community,</strong> celebrating open-source culture and collaboration as part of the global <strong>Hacktoberfest</strong>. The event gathered hundreds of students from universities across the world — developers, designers, and dreamers ready to learn, code, and share.</p><h4>What the Event Was About</h4><p>Hacktoberfest is all about <strong>celebrating open source</strong>, and this hackathon perfectly captured that spirit. The CBIT edition — now in its 8th year — encouraged participants to collaborate on innovative solutions using modern technologies while contributing to the open-source ecosystem.</p><p>Teams of <strong>3–5 members</strong> worked virtually through <strong>Discord</strong>, tackling real-world challenges in just 24 hours. Despite the distance and time zones, the sense of connection and creativity was palpable.</p><h4>The Judging Framework</h4><p>Every project was evaluated based on a clear, balanced set of criteria (total: 50 points):</p><p>Each project was evaluated out of 50 points across five main criteria:</p><ol><li><strong>Innovation &amp; Creativity (10 points)</strong> — Original ideas and new approaches.</li><li><strong>Collaboration (5 points)</strong> — Teamwork, communication, and shared contribution.</li><li><strong>Implementation (20 points)</strong> — Technical soundness, scalability, and efficiency.</li><li><strong>Design (5 points)</strong> — Usability and user experience.</li><li><strong>Presentation (10 points)</strong> — Clarity, storytelling, and delivery.</li></ol><p>As a judge, I was looking for that spark — projects that combined <strong>solid implementation with a clear purpose</strong>.</p><h4>What I Saw and Learned</h4><p>Some projects were technically ambitious, pushing the boundaries of what’s possible in 24 hours. Others were beautifully simple, focusing on accessibility or social impact. A few used <strong>AI and automation</strong> in creative ways that genuinely surprised me.</p><p>But beyond the technology, what stood out was the t<strong>eamwork.</strong> Many participants were strangers before the event — yet they managed to code, design, and present together like long-time collaborators. That’s the magic of hackathons.</p><h4>The Judging Experience</h4><p>Our panel included engineers and leaders from <strong>Electronic Arts, JP Morgan Chase, Oracle, Deliveroo, ServiceNow, EY,</strong> and more. Being part of such a diverse group gave every discussion depth. Each judge viewed “innovation” slightly differently — and that variety made our evaluations richer.</p><p>I appreciated the effort teams put into their presentations. Even short demos told full stories — from the problem statement to the final prototype. That’s where creativity met clarity.</p><h4>Key Takeaways</h4><p>1. <strong>Innovation isn’t always about technology.</strong> Sometimes it’s about empathy — understanding who you’re helping and why. <br>2. <strong>Good presentation matters.</strong> The best projects told their stories with confidence and focus. <br>3. <strong>Community is everything.</strong> Open-source hackathons like this show how technology connects us beyond borders.</p><p>As a developer, I’ve always believed in continuous learning. Judging this event reaffirmed that growth happens in many forms — whether you’re writing code or evaluating it.</p><h4>Why Hackathons Matter More Than Ever</h4><p>In a world where remote collaboration is the new normal, hackathons remain one of the most human ways to innovate. They combine competition, creativity, and community into one shared experience. For students and professionals alike, they’re the perfect place to experiment and grow.</p><p>I’m grateful to have played a small role in this journey — to witness ideas take shape, to learn from participants, and to see firsthand how open source continues to inspire a new generation of builders.</p><p><strong>Author:</strong><br><strong><em>Servin Osmanov</em></strong><br><em>Lead Software Engineer @ Anvaya Solutions Inc. </em><br>Judge at <a href="https://cbit-hacktoberfest25.devpost.com/?ref_feature=challenge&amp;ref_medium=discover#prizes">CBIT Hacktoberfest Hackathon 2025</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e52d673b534f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Language Tech Matters: Developing AI Tools for Small Languages]]></title>
            <link>https://medium.com/@servinosmanov/why-language-tech-matters-developing-ai-tools-for-small-languages-fe130073a987?source=rss-45c456062f69------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe130073a987</guid>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[language]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Servin Osmanov]]></dc:creator>
            <pubDate>Tue, 28 Oct 2025 04:34:52 GMT</pubDate>
            <atom:updated>2025-10-28T04:43:05.256Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ShjFGSminE68z94h3Er38w.png" /><figcaption>Why Language Tech Matters</figcaption></figure><p>In a world where artificial intelligence is transforming how we communicate, the survival of small languages depends not just on cultural passion — but on technology. <br>While English, Chinese, or Spanish dominate the digital space, thousands of smaller languages remain digitally invisible. Without online presence, data, or digital tools, these languages risk extinction in the 21st century.</p><p>As a software engineer and cultural advocate, I’ve spent years developing digital tools for the <strong>Crimean Tatar</strong> language — a Turkic minority language spoken by less than 300,000 people worldwide. Through this work, I’ve learned that the intersection of <strong>AI and linguistics</strong> isn’t just a research topic; it’s a lifeline for cultural identity.</p><h3>The Challenge: The Technology Gap for Small Languages</h3><p>Mainstream AI models — from GPT-based chatbots to translation engines — are trained primarily on high-resource languages. This creates a serious imbalance: while global communication becomes easier for some, others are left behind.</p><p>For small linguistic communities, the lack of:<br>- digital corpora,<br>- standardized spelling systems,<br>- and high-quality training datasets</p><p>makes it nearly impossible to integrate their languages into modern tools like voice assistants, translation services, or educational apps.</p><p>When a language isn’t “machine-readable,” it risks becoming irrelevant in the digital world — even if it’s still spoken at home.</p><h3>Our Journey: Building Tools for the Crimean Tatar Language</h3><p>In 2018, our team launched a series of educational and linguistic tools under the <strong>Qırımtatar Lugatı</strong> (Crimean Tatar Dictionary) project: <br>- <a href="https://play.google.com/store/apps/details?id=com.anaurt.lugat">Android app on Google Play </a><br>- <a href="https://apps.apple.com/us/app/q%C4%B1r%C4%B1mtatar-lu%C4%9Fat%C4%B1/id1457493656">iOS app on the App Store</a></p><p>Our mission was to give people an accessible and modern way to learn, use, and preserve the Crimean Tatar language.</p><p>The apps quickly grew into more than just dictionaries — they became <strong>living platforms</strong> connecting speakers, learners, and educators. Yet, users wanted more: context-based search, on-the-fly translation, and natural voice pronunciation. <br>This inspired us to integrate <strong>artificial intelligence</strong> to make our tools more flexible and intelligent.</p><h4>The Next Step: Bringing AI to Minority Languages</h4><p>We are now integrating <strong>AI and machine learning</strong> into the Crimean Tatar dictionary project to expand its capabilities:</p><p>- 🧠 <strong>AI-assisted translation:</strong> dynamic, real-time translation between Crimean Tatar, Turkish, English, and Ukrainian. <br>- 🔊 <strong>Speech recognition and synthesis:</strong> allowing users to hear natural pronunciation and practice correct intonation. <br>- 📖 <strong>Adaptive learning:</strong> using AI to personalize vocabulary lessons based on user behavior and progress. <br>- 🪶 <strong>Data-driven NLP foundation:</strong> building scalable open datasets that will support future chatbots, voice assistants, and translation systems.</p><p>To achieve this, we’ve adopted and fine-tuned <strong>open-source speech synthesis models</strong> such as:<br>- <a href="https://huggingface.co/facebook/mms-tts-crh"><strong>facebook/mms-tts-crh</strong></a> — part of Meta’s <em>Massively Multilingual Speech</em> project, providing a solid baseline for Crimean Tatar TTS. <br>- <a href="https://huggingface.co/spaces/robinhad/qirimtatar-tts"><strong>robinhad/qirimtatar-tts</strong></a> — a community-driven model that helps us generate high-quality speech for educational and cultural content.</p><p>By leveraging these models, we’re building an <strong>AI-powered TTS system</strong> that brings Crimean Tatar audio resources to learners, teachers, and content creators for the first time.</p><h4><strong>Why This Matters</strong></h4><p>Language is more than communication — it’s collective memory. <br>When a language disappears, so does a unique worldview and cultural identity.</p><p>AI gives us a chance to reverse that process. With the right tools, small languages can become visible online, connect their communities, and survive in the digital era.</p><p>Projects like <strong>Qırımtatar Lugatı</strong> show that you don’t need a giant corporation to make an impact. <br>A small, passionate team with the right technical vision can give a digital voice — literally — to those who’ve been silent for too long.</p><h4><strong>How Developers and Researchers Can Help</strong></h4><p>If you’re a developer, linguist, or AI researcher, here are a few ways to make a difference:<br>- Contribute to <strong>open-source datasets</strong> for small or low-resource languages. <br>- Support <strong>multilingual NLP and TTS</strong> initiatives on Hugging Face or GitHub. <br>- Collaborate with communities and educators who are digitizing endangered languages. <br>- Help localize educational and cultural apps into underrepresented languages.</p><p>Every dataset, every model, and every contribution brings us closer to a more linguistically inclusive AI ecosystem.</p><h4>Looking Ahead</h4><p>Our roadmap for 2025 includes:<br>- Full integration of <strong>AI-powered translation</strong> within the Qırımtatar Lugatı apps. <br>- Implementation of <strong>voice features</strong> using our fine-tuned TTS models. <br>- Launching an <strong>open API</strong> for developers who want to build tools for Crimean Tatar or similar minority languages.</p><p>Technology alone won’t preserve culture — but it can <strong>amplify</strong> it. <br>For small languages like Crimean Tatar, <strong>AI isn’t just a tool — it’s hope.</strong></p><h4>About the Author</h4><p><strong>Servin Osmanov</strong> is a Senior Full-Stack Engineer and founder of the <a href="https://qirim.online/"><strong>Qırım.Online</strong></a> and <a href="https://ana-yurt.com/"><strong>Ana-Yurt.Com</strong></a> project — an initiative focused on preserving Crimean Tatar culture and language through modern technology. <br>LinkedIn: <a href="https://www.linkedin.com/in/servin-osmanov/">linkedin.com/in/servin-osmanov</a> <br>GitHub: <a href="https://github.com/MrSerWin">github.com/MrSerWin</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe130073a987" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>