<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aleksey Tikhonov on Medium]]></title>
        <description><![CDATA[Stories by Aleksey Tikhonov on Medium]]></description>
        <link>https://medium.com/@altsoph?source=rss-a8bc7a32abd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*CTHVQFXBBCprdvAD97vuuQ.jpeg</url>
            <title>Stories by Aleksey Tikhonov on Medium</title>
            <link>https://medium.com/@altsoph?source=rss-a8bc7a32abd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 07:08:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@altsoph/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[WTF is Voice Steering?]]></title>
            <link>https://medium.com/altsoph/wtf-is-voice-steering-f144d2eb4d0f?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/f144d2eb4d0f</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[evaluation]]></category>
            <category><![CDATA[tts]]></category>
            <category><![CDATA[voices]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Fri, 25 Sep 2026 17:03:05 GMT</pubDate>
            <atom:updated>2026-09-25T17:03:05.819Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zPSdk17ZvC-dQNs-.png" /></figure><p>We trained our new TTS model to follow free-form voice instructions, then needed a way to measure how well it followed them. I decided to share some <a href="https://altsoph.substack.com/p/wtf-is-voice-steering">interesting details of that project in my personal blog</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f144d2eb4d0f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/wtf-is-voice-steering-f144d2eb4d0f">WTF is Voice Steering?</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Auto-cartoons for 2026 Ig Nobel Prize winners]]></title>
            <link>https://medium.com/altsoph/auto-cartoons-for-2026-ig-nobel-prize-winners-e9a968cb9f69?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9a968cb9f69</guid>
            <category><![CDATA[cartoon]]></category>
            <category><![CDATA[humor]]></category>
            <category><![CDATA[generative-art]]></category>
            <category><![CDATA[science]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Sat, 05 Sep 2026 15:12:10 GMT</pubDate>
            <atom:updated>2026-09-05T15:12:49.006Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IbBdY5fCWnNEOnIhyPl5QQ.png" /></figure><p>To test the experimental e2e scientifical cartoon generation pipeline we developed with <a href="https://x.com/SaveTheRbtz">Alexey RBTZ Ivanov</a>, I used it to generate <a href="https://x.com/SaveTheRbtz">illustrations for the latest Ig Nobel 2026 Prize winners</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9a968cb9f69" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/auto-cartoons-for-2026-ig-nobel-prize-winners-e9a968cb9f69">Auto-cartoons for 2026 Ig Nobel Prize winners</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Artificial tongue twisters for TTS evaluation]]></title>
            <link>https://medium.com/altsoph/artificial-tongue-twisters-for-tts-evaluation-226d5387eabd?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/226d5387eabd</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[speech]]></category>
            <category><![CDATA[evolution]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Thu, 06 Aug 2026 16:46:38 GMT</pubDate>
            <atom:updated>2026-08-06T16:47:21.373Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6thlk80HQzGZ8Bki.png" /></figure><p><a href="https://altsoph.substack.com/p/artificial-tongue-twisters-for-tts">How I hacked artificial tongue twisters evolution for fun and profit^W^W^W^W to test our TTS models</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=226d5387eabd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/artificial-tongue-twisters-for-tts-evaluation-226d5387eabd">Artificial tongue twisters for TTS evaluation</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What’s Wrong with TTS Evaluation]]></title>
            <link>https://medium.com/altsoph/whats-wrong-with-tts-evaluation-b0e351431ee7?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/b0e351431ee7</guid>
            <category><![CDATA[metrics]]></category>
            <category><![CDATA[evaluation]]></category>
            <category><![CDATA[speech-recognition]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[text-to-speech]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Fri, 15 May 2026 15:40:08 GMT</pubDate>
            <atom:updated>2026-05-15T15:40:52.865Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*s5MBVGpkKgnnjimu.png" /></figure><p>Besides other things, I am the Head of Evaluations at Inworld AI, where we also build TTS models. Our previous TTS model is still top 1 on the AA TTS leaderboard. Last week we shipped a new one, Realtime TTS-2.</p><p>To make any of that actually move, my team spent the last half year building a proper TTS evaluation system internally. Somewhere in the middle of that work I realized I had accumulated a mildly unhealthy amount of opinions about TTS eval. So here is my post about that:</p><p><a href="https://altsoph.substack.com/p/whats-wrong-with-tts-evaluation"><strong>https://altsoph.substack.com/p/whats-wrong-with-tts-evaluation</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b0e351431ee7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/whats-wrong-with-tts-evaluation-b0e351431ee7">What’s Wrong with TTS Evaluation</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Quantum Persona and Test-time Mode Collapse]]></title>
            <link>https://medium.com/altsoph/quantum-persona-and-test-time-mode-collapse-c638de4331c3?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/c638de4331c3</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[safety]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Fri, 25 Jul 2025 18:15:03 GMT</pubDate>
            <atom:updated>2025-07-25T18:15:03.526Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G7TNHzCcxAICWb3sUwYYHQ.png" /></figure><p><a href="https://altsoph.substack.com/p/quantum-persona-and-test-time-mode">This time I want to explore a phenomenon I’ve been investigating: what happens when a model trained on contradictory information needs to give a single, coherent answer. The experiments demonstrate something I call “test-time mode collapse” — how models quickly lock into one consistent persona, and how we can influence which persona that is.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c638de4331c3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/quantum-persona-and-test-time-mode-collapse-c638de4331c3">Quantum Persona and Test-time Mode Collapse</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What’s Wrong with Chat-Templates Format for LLM]]></title>
            <link>https://medium.com/altsoph/whats-wrong-with-chat-templates-format-for-llm-36c870dbf6da?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/36c870dbf6da</guid>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[chatbots]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Fri, 18 Jul 2025 14:55:39 GMT</pubDate>
            <atom:updated>2025-07-18T14:55:39.545Z</atom:updated>
            <content:encoded><![CDATA[<p>I’d like to discuss the current situation with LLM prompting standards; how we got into this mess; and how to live with it now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vaeXsSV-waVVvYZ1.png" /></figure><p><a href="https://altsoph.substack.com/p/whats-wrong-with-chat-templates-format">Keep reading</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36c870dbf6da" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/whats-wrong-with-chat-templates-format-for-llm-36c870dbf6da">What’s Wrong with Chat-Templates Format for LLM</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[BIG BANG OF AGENT RULES]]></title>
            <link>https://medium.com/altsoph/big-bang-of-agent-rules-2b73e04044cd?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b73e04044cd</guid>
            <category><![CDATA[vibe-coding]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[cursor]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[agents]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Mon, 30 Jun 2025 21:14:51 GMT</pubDate>
            <atom:updated>2025-06-30T21:14:51.298Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>TLDR</strong>: Last weekend, I spent several hours digging through publicly available cursor rules files and analyzing existing usage patterns. Despite the presence of a lot of garbage and auto-generated content, I found several distinct strategies that people use to shape AI agent behavior.</h4><blockquote>What Are Agent Rules, Anyway?</blockquote><blockquote>For those unfamiliar, these rules are configuration files that shape how AI coding agents behave in your project. Think of them as prompting for developers — you write instructions about your preferred coding style, project structure, or workflow, and the agent tries to follow them. And, well, yeah, it’s basically just prompting under the hood, which means not all of them are useful.</blockquote><h3>Intro</h3><p>Last weekend, it was too hot outside, so I finally managed to investigate something that had been bothering me for a while: how people actually use cursor rules in practice. You’ve probably seen these `.cursorrules` files scattered across GitHub repos, but are those really useful or they are just a cargo cult? I’ve read many recommendations and descriptions of different practices; I wrote such rules myself, but I wonder if there are systematic practices to use them. It’s hard to say without proper analysis.</p><p>You can probably guess what happened next — what started as a quick investigation turned into several hours of repository digging and analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*usyvQERZwZfpIwix.png" /><figcaption>…this cartoon was e2e generated by my <a href="https://altsoph.substack.com/p/comic-cartoon-generation">jokes+cartoons generation pipeline</a>…</figcaption></figure><p><a href="https://altsoph.substack.com/i/167197786/intro"><strong>…CONTINUE READING…</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b73e04044cd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/big-bang-of-agent-rules-2b73e04044cd">BIG BANG OF AGENT RULES</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[COMIC CARTOON GENERATION]]></title>
            <link>https://medium.com/altsoph/comic-cartoon-generation-a502683fc5ac?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/a502683fc5ac</guid>
            <category><![CDATA[generative-art]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[humor]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Tue, 01 Apr 2025 15:40:04 GMT</pubDate>
            <atom:updated>2025-04-01T15:40:26.204Z</atom:updated>
            <content:encoded><![CDATA[<h4>Since April Fool’s Day is today, let me share some of my results on automated comic cartoon generation.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dx-QGpXEgNKgD1kCoyoiiQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mfBKNYWNZb_cfrvAigilIA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fFcYw69ZKArqkPgXDsrc4A.png" /></figure><p>Last time, I <a href="https://medium.com/altsoph/humor-arena-7ef24ddcba63">shared</a> how we, with our <a href="https://x.com/framrus">Pavel Shtykovskiy</a>, wrote a <a href="https://arxiv.org/abs/2405.07280">paper on Humor Mechanics, published on ICCC-2024</a>, and how we, with <a href="https://x.com/SaveTheRbtz">Alexey Ivanov</a>, launched <a href="https://humor.ph34r.me/">HUMOR-ARENA</a> to collect human labels and improve automated humor generation and ranking. After that, I decided to check if AI can be used to generate and filter proper cartoons (based on my one-liners previously generated with AI).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*95jVjk8vb3a209ET6JANPg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ST3DTKF2cU5qq0btd6HEsA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vLM5fNFT6T7ru5EivFTwLA.png" /></figure><p>First, not each one-liner (even a good one) can be used as a base for a cartoon. Some jokes are too abstract, some wordplay can not be adequately visualized. That means we need an automated way to understand if the given joke is a good starting point. I’ve collected some examples of good and bad ones and asked a reasoning model (o1) to generate an instruction, a guide. It gave me specific rules, including checks for Visual Clarity, Concrete Elements, Scene Foundation, and so on. So, I took our top generated jokes (from Humor-Arena rating) and filtered them with claude-3.5-sonnet + o3-mini, both armed with that visual instruction. If any of these models thinks the joke is bad for visualizing, we reject it. That leaves us with 25% of jokes from the top.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*icnpI-U5GEPXJWETGoLwxg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3yUwLup_5DCcz4QHBanesA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*69l0kM0q41DGrMgFYbqEDg.png" /></figure><p>Next, we need to generate the cartoons. (Note: this was done before recent releases of new image-gen LLMs, so now it will be even easier)<br>For a generation, I used a pair of o3-mini + DALLE-3 models; the trick is to provide enough details to develop a recognizable style and make a funny cartoon. Since I aimed to match some specific visual style, resembling classics like New Yorker’s or Floyd Gottfredson’s, I took a bunch of examples and reverse-engineered a generalized visual style description.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qmpqvnNkSJpLBixOk_2FTQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p6G4gGfubiuxHskQlCaXGw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_b_H3gYyTEZFshiXCZoW6g.png" /></figure><p>As for funny image creation, vanilla o3-mini wasn’t creative enough to come up with interesting details without hints, so, again, I used a superior model (o1) to generate meta-instructions, a guide on how to create an interesting cartoon based on a given one-liner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FITzGGtc2I4TIXJL3fJiZA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6WSAwV6yt4bBtFCAQDcOew.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BbRJzNNXMA-v1ywITRWwWA.png" /></figure><p>That gives me a lot of cartoons, some of them (I’d say 20–30% I found pretty good, personally). Using cursor, I briefly sketched a script to resize the image and add the text of original joke at the bottom, using comic-sans, you know.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kmAhE7y5ZAuUTFW1qRvPyA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NnVdNOhUJaoCkP5qHYcC0w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3faaTCORVqNwqaREB3Fdxg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a502683fc5ac" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/comic-cartoon-generation-a502683fc5ac">COMIC CARTOON GENERATION</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[HUMOR ARENA]]></title>
            <link>https://medium.com/altsoph/humor-arena-7ef24ddcba63?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/7ef24ddcba63</guid>
            <category><![CDATA[arena]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[humor]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Sun, 08 Dec 2024 12:55:47 GMT</pubDate>
            <atom:updated>2024-12-08T12:55:47.832Z</atom:updated>
            <content:encoded><![CDATA[<h4>TLDR: We developed a novel approach to humor generation that gives human-level results on blind tests. To facilitate further progress in humor generation and understanding, we made <a href="https://humor.ph34r.me/"><strong>HUMOR ARENA</strong></a>, a site where you can participate in side-by-side labeling of various generated one-liners, see the ranking of models, and read the automatic top of generated jokes.</h4><h3>Quick quiz: of these 7 one-liners, only 3 are human-written. Can you guess which ones?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qtsk5cfWOFwkvwa6V6Utdg.png" /></figure><p>Anyone who has asked an LLM to make a joke knows how bad the results usually are. Usually, it responds with one of the memorized standard dad jokes (<em>Why did the scarecrow win an award?.. Why did the tomato turn red?.. Why don’t scientists trust atoms?..</em>). <a href="https://arxiv.org/abs/2306.04563">(Jentzsch and Kersting 2023)</a> show that over 90% of the 1000+ jokes generated by ChatGPT were the same 25 memorized jokes.</p><p>Does this mean that modern models are basically incapable of generating a good joke, or is the problem that we are not explaining the task well enough? A good joke should be original, but in the training data, models can only see old jokes, and the better the joke, the more copies of it are likely to be encountered. What if we isolate this signal and only consider original, new, unique jokes? How can we decide which joke is funnier?</p><p>A successful joke is often based on some pattern, such as a broken expectations, pun, or a play on words. <a href="https://journals.sagepub.com/doi/abs/10.1177/1088868320961909">(Warren, Barsky, and McGraw 2021)</a> refer to more than 20 distinct humor theories attempting to explain humor appreciation. Indeed, the success of a joke also depends on context and audience. If two well-known popular standup comedians switch texts, it is likely that the audiences of both of them will be disappointed. There are <a href="https://www.taylorfrancis.com/chapters/edit/10.4324/9781315827483-2/assessment-appreciation-humor-studies-3-wd-humor-test-willibald-ruch">psychological studies</a> showing the polarization of audiences according to different types of perceived humor (Thanks to <a href="https://scholar.google.com/citations?user=ch2vRdcAAAAJ&amp;hl=en">Pavel Braslavski</a> for discussions and advice on this topic). Long story short, there are <a href="https://ceur-ws.org/Vol-3740/">workshops</a>, <a href="// https://labrc.co.uk/2024/10/19/humour/">conferences</a>, valious research on these questions, but no one knows exactly how it works.</p><p>Earlier this year, my colleague from Inworld.AI, <a href="https://x.com/framrus">Pavel Shtykovskiy</a>, and I decided to apply a data-driven approach to this problem. Taking a dataset of one-liners labeled by a large number of people with pairwise ratings (which of two jokes is funnier), we tried to reconstruct the set of rules behind the determination of the better joke, the so-called <em>humor policy</em>.<br>We then <em>introduced a multi-step reasoning scheme</em> with generation and consequent refinement of associations to generate novel one-liners on a given topic.</p><p>As a result, our generated jokes on blind labeling by humans were significantly funnier on blind labeling by humans than a baseline set of good human jokes — for comparisons, we used a subset of the dataset collected by <a href="https://arxiv.org/abs/1909.00252">(Weller and Seppi 2019)</a> from Reddit jokes, and filtered based on user’s upvotes.</p><p>The results were pretty good to our taste, so we published all the details in our <a href="https://computationalcreativity.net/iccc24/papers/ICCC24_paper_128.pdf">Humor Mechanics</a> paper at The International Conference on Computational Creativity (ICCC) 2024.</p><p>After reading our paper, an old friend of mine, <a href="https://x.com/SaveTheRbtz">Alexey Ivanov</a> from OpenAI, suggested we should create a platform where people can compare jokes generated by different models and thus form a ranking of models by their ability to make people laugh. After spending a few weekends, Alexey and I put together a prototype — <a href="https://humor.ph34r.me/"><strong>Humor Arena</strong></a>. To aggregate pairwise scores into a single rating we used the new <a href="https://github.com/dustalov/evalica">evalica library</a> from Dmitry Ustalov (thanks Dmitry!).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*pwSbCK1gyrSgRhChRMmGQA.png" /></figure><p>To make the result more interesting, we also invented a way to automatically rank jokes based on the current partial pairwise labels, thus creating a beta version of the <a href="https://humor.ph34r.me/top-jokes"><strong>automatic top of jokes</strong></a>. The current top seems to be prone to dark humor and self-criticism. We’ll see when more pairwise scores are accumulated and the rankings are recalculated.</p><p><em>And yes, about the 7 jokes at the beginning of the post — 1, 3, 5, and 6 are model-generated, as are all the others — there were no human jokes among those jokes, sorry.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7ef24ddcba63" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/humor-arena-7ef24ddcba63">HUMOR ARENA</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GALLERY OF UNSEEN]]></title>
            <link>https://medium.com/altsoph/gallery-of-unseen-c690e5146a35?source=rss-a8bc7a32abd------2</link>
            <guid isPermaLink="false">https://medium.com/p/c690e5146a35</guid>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[generative]]></category>
            <category><![CDATA[nanogenmo]]></category>
            <category><![CDATA[evolutionary-algorithms]]></category>
            <dc:creator><![CDATA[Aleksey Tikhonov]]></dc:creator>
            <pubDate>Sat, 30 Nov 2024 17:42:27 GMT</pubDate>
            <atom:updated>2024-11-30T17:42:27.069Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QFhoE9rxz60yXOWmRsmqPw.png" /></figure><h4>TLDR:<strong> It started as a bunch of strange experiments and ideas on unsupervised generation and converged into another NaNoGenMo project. This post contains a write-up on the nuances and technical details of this work.</strong></h4><h3>Stage 1: Evolutionary search for aesthetics</h3><p>First, I was playing with the evolutionary search of the optimal prompt for the <a href="https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0">SDXL model</a> to maximize the aesthetic scores of results measured by Google’s pretrained <a href="https://github.com/idealo/image-quality-assessment">NIMA model</a>. As a starting set of prompt parts, I used something I did before in my <a href="https://altsoph.substack.com/p/freaking-architecture">Freaking Architecture project</a>, so initially, images were biased toward architecture and sculpture. The evolutionary search is not the fastest thing I know, so I wanted to speed up the process and used <a href="https://huggingface.co/docs/diffusers/en/api/schedulers/lcm">LCMScheduler</a> with <a href="https://huggingface.co/latent-consistency/lcm-lora-sdxl">lcm-lora-sdxl</a> to lower the number of inference steps down to 4.</p><p>The genetic part was pretty straightforward: As a genome of an individual, I took a float vector of length 100, with probabilities to select one or another piece of prompt; the pieces were like these:</p><pre>&quot;fluid and dynamic forms&quot;, <br>&quot;golden silver elements&quot;, <br>&quot;googie motifs&quot;, <br>&quot;gray stone&quot;, <br>&quot;hexagonal pattern&quot;, <br>&quot;houses and roads&quot;, <br>&quot;in a ravaged library&quot;, <br>&quot;in a square&quot;, <br>&quot;limestone&quot;,<br>...</pre><p>To evaluate individuals, I randomly sampled up to 15 pieces of the prompt with the corresponding probabilities, merged them together, sampled an image from SDXL, and scored it with NIMA. Since both prompt generation and image generation are stochastic, I repeated both steps up to 10 times and averaged scores. The size of a generation pool was 20; I used generic random mutations and standard cross-over.</p><p>The both average and best scores slowly crawled up with the time:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*mkUiJ_wi3KgpvzmxysapLA.png" /></figure><p>Manual debug showed no signs of degeneration as well:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_QjMewQeTviDABKIwkxwyg.png" /></figure><p>So I ran it in my colab pro account and left it running overnight. In the morning I got 50K+ of completely insane images; it was impossible to even check them all out.</p><p>I decided to focus on those with aesthetic scores greater than 6.0 (a pretty hard baseline), but still, there were 22K+ of them. I had to invent some way to find the most interesting images automatically.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*02TWhii6eMJ7EjJSAZS_bA.png" /></figure><h3>Stage 2: Visual style clustering</h3><p>To continue experiments, I’ve decided to try <a href="https://marimo.io/">marimo</a>— it’s some fresh jupyter analog, and I wanted to give it a try (<em>overall: so far, it looks interesting but a bit unpolished; I should try it again in a half of a year maybe</em>).</p><p>Digging through the pile of images, I’ve noticed there are several distinctly different visual styles standing out — like photos, sketches, paintings, and so on. So, I decided to group pictures by visual style somehow, for starters.</p><p>The general plan was to embed images into vectors, then lower the dimensionality of latent space, then cluster these low-dimensional vectors, and, finally, explore the clusters.</p><p>I tried the <a href="https://github.com/facebookresearch/dino">DINO (v1) model</a> embeddings first, but the resulting clusters were visually too internally diverse, so I didn’t see any clear corresponding style:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fU12KrVf7GH-s4mGcNyKLQ.png" /></figure><p>After a short research, I switched to the generic <a href="https://pytorch.org/vision/main/models/generated/torchvision.models.vgg16.html">VGG16 model</a> and took only the 16th layer weights as a style embedding. That worked much better. After embedding, I made a <a href="https://umap-learn.readthedocs.io/en/latest/">UMAP projection</a> of embeddings into 2d space and ran <a href="https://en.wikipedia.org/wiki/DBSCAN">DBSCAN</a> over it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/544/1*B9GN-WBQSaiku3zSDceh0g.png" /></figure><p>The top 3 style-based clusters had more than 1k images each, and the top 10 had more than 150 images. Visually, these clusters were pretty consistent:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dU7aoDC6XAZ-OpurfE8HWg.png" /></figure><p>For the rest of my experiments, I took the images from the top 3 clusters — one was all about some huge empty gray rooms; another had drawings, collages, and sketches; the third was something like church interiors and strange semi-organic rooms.</p><p>Overall, almost all of these images were pretty good. However, there were still too many very similar ones among them — multiple images of very similar objects; perhaps it was the result of multiple (10x) runs of aesthetic scoring for each individual, so each prompt was used to generate multiple images.</p><p>Anyway, I decided to make some deduplication.</p><h3>Stage 3: Semantic deduplication</h3><p>To do the deduplication, I embedded these selected images again, this time with the CLIP, since it should capture semantics better. Again, I applied UMAP + DBSCAN to get clusters across this subset of images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ETVKP_7IkXrcB0XxAbyGOQ.png" /></figure><p>Thus, I decided to take only one image from each semantic cluster, specifically the one with the maximum aesthetic score, and ended with approximately automatically selected 200 images — they had</p><ul><li>high aesthetic scores (scored by model),</li><li>more or less the same style,</li><li>and were diverse enough.</li></ul><p>They were also generally more or less about art, sculpture, and architecture, so I decided to convert them into something like a guidebook.</p><h3>Stage 4: Essay generation</h3><p>To complete the guidebook, I needed some textual descriptions of my selected images. Indeed, I could just ask some VLM to write these descriptions, but usually, the results of such approaches are weird — such texts are usually full of clichés and general formulations, and when there are many of them, it is instantly obvious that they are very similar in structure.</p><p>To address these issues, I used an approach similar to what I did in our recent <a href="https://github.com/altsoph/humor-mechanics">humor generation project</a> before — I used association generation and a multi-step brainstorming framework. I also generated a short set of hints and recommendations for catalog writers (10 hints).</p><p>Finally, I requested OpenAI GPT-4o to generate a short essay about each image, providing it with the description, associations, and a random subset of the writer’s hints.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eIBSQKa3rWcfOB8Ux4IpOQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LW2BkZczqwDwWj1O6stQ_g.png" /></figure><h3>Stage 5: Final polishing and sharing</h3><p>After the final cleanup, I finished with 150 images and approximately 52K words of text. To make the layout look better, I searched for some automatic framework for publishers. Eventually, I used <a href="https://weasyprint.org/">WeasyPrint</a> — this easy-to-use python library allows to convert generated HTML to PDF and control page layout details with custom CSS instructions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L2QJwFErknEzR1FymJj4zg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tqp7sxJ38Ts3Ff0BGODRBQ.png" /></figure><p>Finally, I made a cover — this is the only part I’ve done manually (still using one of the generated images):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jQMxfuXKyeJTpqUHOjBe8w.png" /></figure><p>The resulting <a href="https://github.com/altsoph/unseen_gallery/blob/main/unseenguide.pdf">PDF</a> and <a href="https://github.com/altsoph/unseen_gallery/tree/main/scripts">code</a> are available on my github. I’ve also made a <a href="https://github.com/NaNoGenMo/2024/issues/23">NaNoGenMo submission</a> to share it with the community; there, I promised to provide more technical details, so that’s the reason I wrote this post.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c690e5146a35" width="1" height="1" alt=""><hr><p><a href="https://medium.com/altsoph/gallery-of-unseen-c690e5146a35">GALLERY OF UNSEEN</a> was originally published in <a href="https://medium.com/altsoph">Altsoph’s blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>