<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Simón Paredes on Medium]]></title>
        <description><![CDATA[Stories by Simón Paredes on Medium]]></description>
        <link>https://medium.com/@paredess?source=rss-2b18950f71ad------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*lFOSAXCvN7jmQb3YEeoKzQ.jpeg</url>
            <title>Stories by Simón Paredes on Medium</title>
            <link>https://medium.com/@paredess?source=rss-2b18950f71ad------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 10:32:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@paredess/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[LLM Prompting is an Art]]></title>
            <link>https://medium.com/@paredess/llm-prompting-is-an-art-404360005560?source=rss-2b18950f71ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/404360005560</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[experience]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[prompt]]></category>
            <dc:creator><![CDATA[Simón Paredes]]></dc:creator>
            <pubDate>Wed, 23 Oct 2024 14:28:14 GMT</pubDate>
            <atom:updated>2024-10-23T14:28:14.789Z</atom:updated>
            <content:encoded><![CDATA[<p>I’m a software engineer, so I approached the task of writing my first prompts using a strict format. The results were not exactly what I wanted. After a year of working in prompt engineering, I can say it still feels more of a craft/art than an engineering problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ucZ7YqofqU6x5FmqB4F-eg.jpeg" /></figure><h3>Working with Llama 3</h3><p>Llama 3 models are great, but getting all their potential out can be quite frustrating. Here I will share my experience and learnings with them.</p><h4>Positive vs Negative Prompts</h4><p><em>I have come to the conclusion that using positive prompting is far more efficient than negative prompting to achieve your result.</em></p><p>First focus all your energy in writing a prompt that tells what you want, and then use negative prompting to make minor corrections.</p><p>This is a good example:</p><pre># Context<br>Your are an agent trying to help a customer to solve a problem.<br>You are talking to them over a phonecall, so be patience if they don&#39;t<br>understand what you said. Be ready to repeat what you just said, but in a<br>sligthly different way for them to understand you better.<br><br># Task<br>Please, identify the problem they have. Many times they will not fully tell<br>their problem or even being able to explain it in the right way. Make follow<br>up questions to clarify.<br><br>....<br><br># Format<br>write your response following this markdown format and nothing else:<br>```markdown<br># Response<br>agent: &lt;write here your response to the customer, it will be processed over<br>a TTS system for the customer to hear you, so only use regular worlds&gt;<br><br># EOF<br>just a marker<br>```<br></pre><p>In the example, you can see how most of it is positive, but there is only one negative part:</p><ul><li>“… <strong>and nothing else:</strong>” I think this is clear enough. We want to avoid the LLM to be “creative” or add extra tokens at the end.</li></ul><p>This is a bad example:</p><pre># Context<br>Your are an agent trying to help a customer to solve a problem.<br>Do not try to talk too long.<br>Avoid repeating the same thing twice.<br><br><br># Task<br>Please, identify the problem they have. <br>Do not assume you understood their problem. <br>The customer doesn&#39;t know what they want. Get the information from them.<br><br>....<br><br># Format<br>Write your response in Markdown, not in JSON or YAML. Make sure you avoid<br>adding wrong character and do not try to add more information than what is <br>being asked.<br>```markdown<br># Response<br>agent: &lt;write here your response to the customer, do not use **, [], UTF8 characters,<br>emoji or any other weird characters&gt;<br>```<br><br>Don&#39;t forge to add this section at the end:<br>```<br># EOF<br>just a marker<br>```<br></pre><h4>Natural language vs Computer language</h4><p><em>LLMs seems to be a lot better at understanding natural language than a computer like instruction.</em></p><p>When I started writing prompts, I wanted them to be as short and directed as possible. So I was using YAML format and providing short, and very directed instructions. This seems to work at first, but it goes down quickly as complexity grows.</p><p>This is an example of those early days prompts:</p><pre>Context: helper agent<br>Task: solve problems<br>How: [make questions,repeat what was told,follow ups]<br>...<br>Output:<br>  Format:<br>    Response: this is your response</pre><p>You can see how it is trying to be as efficient as possible and written in machine format. This seemed to work, but eventually falls apart when the amount of extra information required to solve the task just grows.</p><p>So write your prompts in natural language and do not be afraid of using too many tokens. First arrive to a working solution, then optimize the prompt.</p><h4>Format and Resiliency</h4><p><em>Llama3 works really well with Markdown and YAML</em></p><p>As shown in the first example, my preferred style is to use Markdown and YAML for input and output. This couple has proven to produce the most consistent results in my experience.</p><p>Llama3 is also capable of generating JSON, specially 3.1 on-wards, but the failure rate was in the 30% or more in my case. I’m targeting 90% correct (acceptable) answers.</p><p>Accepting partially correct answers is crucial to reduce cost and roundtrips, specially with smaller models (8B and lower). The response should be preprocessed to remove things that can break a regular parser, or if your format is simple enough, just use regexp or string matching to extract what you need.</p><h4>Order Matters</h4><p><em>LLMs are forward generators, every new token is generated using all the previous ones.</em></p><p>This is really important to keep in mind when you are authoring your prompt. If you want to generate a section that is dependent on another one, then first generate the dependency. Example:</p><pre># Format<br>write your response following this markdown format and nothing else:<br>```markdown<br># Customer Problem Bullets<br>&lt;write here what you know about the customer problem in a bullet list&gt;<br><br># Customer Problem Summary<br>&lt;write a phrase that describes the customer problem, please use the <br>&#39;Customer Problem Bullets&#39; section&gt;<br><br># Customer Problem Clarification<br>&lt;write here a bullet list of questions you would like to ask to better understand<br>the customer problem. It is ok to have an empty list if you have no more questions&gt;<br><br># Response Analysis<br>&lt;write here your analysis of what is the best response to the customer. It can be<br>a question from &#39;Customer Problem Clarification&#39; section or just ackwnoledge <br>you are fully clear and ask the customer to confirm your understanding by reading <br>the section &#39;Customer Problem Summary&#39;&gt;<br><br># Response<br>agent: &lt;write here your response to the customer using the &#39;Response Analysis&#39;<br>section. It will be processed over a TTS system for the customer to hear you,<br> so only use regular words&gt;<br><br># EOF<br>just a marker<br>```</pre><p>Using the wrong order, will not help the LLM to properly process its internal Context and response quality will suffer. You may think you need a larger, more powerful model, but actually the whole problem may be about proper order.</p><h4>Temperature and TopP</h4><p><em>Specially on smaller models, these parameters can help you achieve your expected results.</em></p><p>I will give you a quick and simplified overview of how they work together, but a more in depth explanation is totally recommended for you to read.</p><p>LLMs works by producing a list of possible next tokens. Each of them with a specific probability. It will then apply a series of filters and finally select one randomly.</p><p>Temperature works by adjusting the probability of each token:</p><ul><li>= 1: means no change to the probabilities the model has learn.</li><li>&lt; 1: means exacerbate the probabilities difference, so higher probabilities are made higher and the opposite for lower.</li><li>&gt; 1: is the opposite to “&lt; 1”, so it will decrease the probabilities differences. Higher probabilities decrease and lower probabilities increase.</li></ul><p>Examples (values are totally made up to show the point):</p><pre>probs = [0.089, 0.073, ..., 0.043, 0.038]<br>temp(probs, 0) // [1.0, 0.0, ..., 0.0, 0.0]<br>temp(probs, 0.3) // [0.12, 0.091, ..., 0.035, 0.027]<br>temp(probs, 1.5) // [0.067, 0.052, ..., 0.051, 0.049]</pre><p>As you can see, using <em>Temp = 0</em> will make the token with the highest probability to be the only one possible.</p><p>TopP will remove tokens that have lower probability than the provided value.</p><p>Examples (values are totally made up to show the point):</p><pre>probs = [0.089, 0.079, ..., 0.043, 0.038]<br>topP(probs, 0.03) // [0.089, 0.079, ..., 0.043]<br>topP(probs, 0.07) // [0.089, 0.079]</pre><p>My recommendations on how to use them are:</p><ol><li>Do not use them if you don’t understand what are you doing.</li><li>Focus on writing a good prompt with defaults values, then play with them to check if you can get even better results. Example, your prompt is working correctly 80% of the time, then try lowering Temp and TopP to find out if you get better results.</li><li>If you are facing a creative task and your prompt is already producing the content at your target rate, but it feels not that “creative”, then start increasing Temp and decreasing TopP. This will produce more creative content but it can harm the target rate. You may need to adjust your prompt after increasing Temp.</li><li>Always start by resetting TopP to 1. Then play with Temp and fine tune with TopP.</li></ol><h4>Do not fool yourself</h4><p><em>LLM token generation is heavily based on random numbers, so it is easy to think you nailed your prompt, but it maybe that you just experienced the good results. It can also happens the other way around.</em></p><p>It is really easy to think your prompt and settings are working because you tried it a few times. But this is a big mistake as the generation is based on random numbers (probabilities). This means that you could be just getting reasonably good results by random while you manually test. The opposite is also true, you may be experiencing bad results, but your prompt actually works.</p><p>You should have a good automated test suit that can help you understand how often your prompt works. Never expect it to work 100% of the time, although it may happen for very specific cases. I personally target 90% with resilient parsing.</p><p>Random numbers can be difficult to grasp, but you need to know they can make you think you are an awesome Prompt writer or just reconsider taking those cooking clases.</p><p>To give you an example, lets say my prompt works fine when the LLM internal Random generator output is higher or equal than 6.</p><p>This is an actual Random number generator sequence between 1 and 10: <em>[8, 6, 6, 8, 7]</em>. Given these values, I would experience a full five sequence of prompt executions with correct results. I would then think I’m done.</p><p>But lets see the next 5 in the sequence: <em>[4, 9, 7, 1, 6]</em>. Now I would get 3 correct executions and two wrong. This would let me with a sour feeling.</p><p>And what about the next 5?: <em>[3, 2, 4, 7, 2]</em>. Now only one worked, my prompt is broken, I need to keep iterating.</p><p>Reality is my prompt would work 60% of the time in the long run, but I could experience totally random behavior by just running a few times by hand.</p><p>Having a good test harness that can run your prompt many times and check results is a must have. The more times you run it, the better. You would need to find the right number according to your time, cost and error tolerance budget.</p><h4>Optimization?</h4><p><em>How to optimize if there is no visibility into the LLM?</em></p><p>I left this one to the end because there are no available tools nor clear guidance on how to do this for prompts. Which means it is just a cycle of trial and error.</p><p>In code, you first profile your software, and then you focus on those parts where memory/cpu/io is being used the most. But this depends on a good profiler, which does not exists for LLMs, so how to know what part of your prompt is relevant and what can be removed or simplified?</p><p>Sadly, the only way to do this is by using your common sense (yes, this thing we all have, but it is quite far from being common) and your test harness to make changes and then check if you still get the response you expected at the right target value.</p><h4>End Words</h4><p>I hope this helps you write better prompts for Llama models.</p><p>I have not tested this with other models, but many of the ideas should apply if they are based on the same foundational model.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=404360005560" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>