<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by DahamiKostha on Medium]]></title>
        <description><![CDATA[Stories by DahamiKostha on Medium]]></description>
        <link>https://medium.com/@dahaminirmani2001?source=rss-e380ccac0f42------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*xy9F7ro4W38bD3PWez-e5g.jpeg</url>
            <title>Stories by DahamiKostha on Medium</title>
            <link>https://medium.com/@dahaminirmani2001?source=rss-e380ccac0f42------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 13:49:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dahaminirmani2001/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Elasticsearch Scripting is No Longer “Painful”: A Developer’s Guide to Painless]]></title>
            <link>https://medium.com/pickme-engineering-blog/elasticsearch-scripting-is-no-longer-painful-a-developers-guide-to-painless-c0dee19efef8?source=rss-e380ccac0f42------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0dee19efef8</guid>
            <dc:creator><![CDATA[DahamiKostha]]></dc:creator>
            <pubDate>Mon, 01 Jun 2026 11:31:51 GMT</pubDate>
            <atom:updated>2026-06-02T15:34:14.158Z</atom:updated>
            <content:encoded><![CDATA[<p>If you have been working with Elasticsearch for a while, you eventually reach a point where standard JSON DSL queries are not enough. You may need to perform complex calculations during a search, update documents conditionally without fetching them first, or implement custom scoring and sorting logic.</p><p>In the past, scripting options like Groovy, JavaScript, and Python were available, but they came with serious drawbacks security risks and poor performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FfKcl2gc-y2uUgxQBCW0rw.png" /></figure><p>That’s where <strong>Painless</strong> comes in.</p><p>In this article, we will explore what Painless is, where it is used, common real-world use cases, different script types, and the most important concept that often confuses developers: <strong>script contexts</strong></p><h4>What is Painless?</h4><p>Painless is a scripting language developed by Elastic specifically for Elasticsearch. It is the default scripting language in modern versions (<a href="https://www.elastic.co/blog/painless-a-new-scripting-language">Elasticsearch 5.0</a> and above).</p><p>You can think of Painless as <strong>“Secure, Fast Java for Elasticsearch.”</strong></p><p>If you know Java (or C/Go), the syntax will feel immediately familiar. It is designed to be:</p><ol><li><strong>Safe:</strong> It runs in a sandbox with a limited allow-list. You cannot write a script that creates an infinite loop or deletes files on the server.</li><li><strong>Performance:</strong> It compiles directly into native Java bytecode, making it significantly faster than interpreted languages like JavaScript or Python.</li><li><strong>Flexibility: </strong>Developers can execute complex logic directly inside Elasticsearch without needing to move large amounts of data to external applications.</li></ol><h4>Where Can We Use Painless Scripts?</h4><p>Before diving into specific examples, it’s helpful to understand the <strong>main areas where Painless scripts are commonly used in Elasticsearch</strong>.</p><p>Most Painless scripting use cases fall into three broad categories:</p><ol><li><strong>Search Enhancement</strong></li><li><strong>Data Processing</strong></li><li><strong>Operational Automation</strong></li></ol><p>Within these areas, developers commonly use Painless for tasks such as:</p><ul><li><strong>Data Enrichment</strong> — adding or modifying fields based on existing data</li><li><strong>Custom Sorting and Ranking</strong> — implementing business-specific ranking logic for search results</li><li><strong>Runtime Computation</strong> — calculating values dynamically during query execution</li><li><strong>Document Transformation</strong> — modifying documents during indexing or updates</li><li><strong>Bulk Data Updates</strong> — automating large-scale updates using _update_by_query</li></ul><p>These capabilities allow Elasticsearch to perform <strong>real-time computation, data transformation, and automated operations directly inside the cluster</strong>, reducing network overhead.</p><h4>Common Painless Script Use Cases</h4><p>Let’s explore practical use cases based on the categories above.</p><h4>1. Search Enhancement</h4><p>One of the most common uses of Painless is improving how search results are calculated and returned. Sometimes the default Elasticsearch scoring mechanisms are not enough to support real-world business requirements.</p><p><strong>1.1 Custom Scoring using script_score</strong></p><p>Default Elasticsearch scoring is based mainly on text relevance. However, real-world applications often require business-specific ranking logic to combine multiple factors (ratings, reviews, popularity) or override default behavior.</p><p>Example: Boost products with higher ratings only if they have enough reviews</p><pre>{<br>  &quot;query&quot;: {<br>    &quot;script_score&quot;: {<br>      &quot;query&quot;: {<br>        &quot;match_all&quot;: {}<br>      },<br>      &quot;script&quot;: {<br>        &quot;source&quot;: &quot;&quot;&quot;<br>          if (doc[&#39;review_count&#39;].value &gt; 50) {<br>            return doc[&#39;rating&#39;].value * 1.5;<br>          }<br>          return doc[&#39;rating&#39;].value;<br>        &quot;&quot;&quot;<br>      }<br>    }<br>  }<br>}</pre><p><strong>1.2 Custom Sorting Logic</strong></p><p>Sometimes sorting by a single field (e.g., price or date) is not sufficient.</p><p>Example: Sort by profit margin</p><pre>doc[&#39;price&#39;].value - doc[&#39;cost&#39;].value</pre><p>This allows dynamic ranking based on calculated values instead of static fields.</p><p><strong>1.3 Runtime Fields (Calculated Fields)</strong></p><p>Runtime fields allow you to calculate values during query execution without storing them in the index. This means no need to re-index data, keeping your schema flexible and dynamic.</p><h4>2. Data Processing</h4><p>Painless is incredibly useful for <strong>processing and transforming data inside Elasticsearch</strong>.</p><p><strong>2.1 Document Updates using Update API</strong></p><p>Modify documents directly inside Elasticsearch without retrieving them.</p><pre>ctx._source.status = &#39;active&#39;;</pre><p><strong>2.2 Bulk Updates using _update_by_query</strong></p><p>Perform updates on large datasets efficiently.</p><p>Example: Mark high-stock products as surplus</p><pre>if (ctx._source.stock_count &gt; 100) {<br>  ctx._source.status = &#39;surplus&#39;;<br>}</pre><p><strong>2.3 Data Transformation in Ingest Pipelines</strong></p><p>Transform data at the time of indexing. Common use cases include adding derived fields, cleaning/normalizing text, or extracting structured values from raw logs.</p><h4>3. Operational Automation</h4><p>Another important use of Painless is <strong>automating operational tasks within Elasticsearch</strong>.Scripts can be used to automatically update or enrich documents without manually processing each record.</p><ul><li><strong>Bulk Data Operations:</strong> Automate large-scale updates.</li><li><strong>Data Cleanup:</strong> Fix inconsistent or messy data formats.</li><li><strong>Field Migrations:</strong> Update or rename fields without needing a full re-index.</li></ul><p>This makes Painless a powerful tool for <strong>maintaining and managing large datasets efficiently</strong>.</p><h3>Script Types in Elasticsearch</h3><p>When we talk about “types,” we usually refer to how the script is stored and executed. There are two main ways to implement Painless.</p><blockquote><strong><em>1. Inline Scripts</em></strong></blockquote><p>These are scripts you write directly inside your search query JSON. They are perfect for testing, one-off tasks, or dynamic logic that changes per request.</p><p><strong>Scenario:</strong> We have a product index, and we want to calculate a “Summer Sale” price on the fly without actually changing the database.</p><p><strong>Step 1: Add Dummy Data</strong></p><p>First, let’s create a product so we have something to test.</p><pre>POST /products/_doc/1<br>{<br>  &quot;name&quot;: &quot;Gaming Laptop&quot;,<br>  &quot;category&quot;: &quot;Electronics&quot;,<br>  &quot;price&quot;: 1000,<br>  &quot;cost&quot;: 600,<br>  &quot;stock_count&quot;: 150,<br>  &quot;review_count&quot;: 60,<br>  &quot;rating&quot;: 4.5,<br>  &quot;status&quot;: &quot;new&quot;<br>}</pre><p><strong>Step 2: Run the Script</strong></p><p>We will use a script_score query to calculate a custom score. Let’s say we want to discount the price by 20% just for the ranking score.</p><pre>GET /products/_search<br>{<br>  &quot;query&quot;: {<br>    &quot;script_score&quot;: {<br>      &quot;query&quot;: { &quot;match_all&quot;: {} },<br>      &quot;script&quot;: {<br>        &quot;source&quot;: &quot;doc[&#39;price&#39;].value * (1 - params.discount_rate)&quot;,<br>        &quot;lang&quot;: &quot;painless&quot;,<br>        &quot;params&quot;: {<br>          &quot;discount_rate&quot;: 0.2<br>        }<br>      }<br>    }<br>  }<br>}</pre><p>Pro Tip: Always use <em>params</em> (like <em>discount_rate </em>above) instead of hardcoding numbers. If you write * 0.8 directly in the string, Elasticsearch has to recompile the script every time you change the discount. With <em>params</em>, it compiles once and caches it forever.</p><blockquote><strong><em>2. Stored Scripts</em></strong></blockquote><p>Stored scripts are saved inside the cluster state with a unique ID. These are best for production systems where you reuse the same logic across many queries. It keeps your app code clean and allows logic updates without redeploying.</p><p><strong>Step 1: Save the Script We save our logic with the ID</strong></p><p><em>calculate_discount</em></p><pre>POST _scripts/calculate_discount<br>{<br>  &quot;script&quot;: {<br>    &quot;lang&quot;: &quot;painless&quot;,<br>    &quot;source&quot;: &quot;doc[&#39;price&#39;].value * (1 - params.discount_rate)&quot;<br>  }<br>}</pre><p><strong>Step 2: Use it by ID</strong></p><pre>GET /products/_search<br>{<br>  &quot;query&quot;: {<br>    &quot;script_score&quot;: {<br>      &quot;query&quot;: { &quot;match_all&quot;: {} },<br>      &quot;script&quot;: {<br>        &quot;id&quot;: &quot;calculate_discount&quot;,<br>        &quot;params&quot;: {<br>          &quot;discount_rate&quot;: 0.5<br>        }<br>      }<br>    }<br>  }<br>}</pre><h4>The “Context” Trap (Read This Carefully)</h4><p>The most confusing part of Painless isn’t the syntax.It’s the <strong>Context</strong>.</p><p>A script running in an <strong>Update</strong> API behaves completely differently from a script running in a <strong>Search</strong> API. If you try to use the wrong syntax, you will get null pointer exceptions or missing field errors.</p><blockquote><strong><em>Context A: The Update / Ingest Context (Write Access)</em></strong></blockquote><p>Used in _update, _update_by_query, and Ingest Pipelines.</p><ul><li><strong>Goal:</strong> You want to <em>change</em> the document.</li><li><strong>Variable:</strong> You use ctx._source.</li><li><strong>Behavior:</strong> You are accessing the raw JSON source map. It is slower, but you can modify it.</li></ul><p><strong>Example: Update a status only if it’s currently 0</strong></p><pre>if (ctx._source.current_status == 0) {<br>    ctx._source.current_status = 1;<br>}</pre><p><strong>Let’s look at a real-world “Write” example.</strong></p><p><strong>Scenario</strong>: We have a surplus of “Gaming Laptops” and we want to apply a permanent 15% discount to all of them in our index.</p><p><strong>Step 1: Add Product Data</strong></p><p>Let’s make sure we have a product to update.</p><pre>POST /products/_doc/1<br>{<br>  &quot;name&quot;: &quot;Gaming Laptop&quot;,<br>  &quot;category&quot;: &quot;Electronics&quot;,<br>  &quot;price&quot;: 1000,<br>  &quot;cost&quot;: 600,<br>  &quot;stock_count&quot;: 150,<br>  &quot;review_count&quot;: 60,<br>  &quot;rating&quot;: 4.5,<br>  &quot;status&quot;: &quot;new&quot;<br>}</pre><p><strong>Step 2: Update by Query</strong></p><p>We use ctx._source because we are modifying the actual JSON source. We will decrease the price and mark them as “clearance”.</p><pre>POST /products/_update_by_query<br>{<br>  &quot;script&quot;: {<br>    &quot;source&quot;: &quot;ctx._source.price *= (1 - params.discount); ctx._source.status = &#39;clearance&#39;&quot;,<br>    &quot;lang&quot;: &quot;painless&quot;,<br>    &quot;params&quot;: {<br>      &quot;discount&quot;: 0.15<br>    }<br>  },<br>  &quot;query&quot;: {<br>    &quot;term&quot;: {<br>      &quot;category&quot;: &quot;Electronics&quot;<br>    }<br>  }<br>}</pre><blockquote><strong><em>Context B: The Search / Score Context (Read-Only)</em></strong></blockquote><p>Used in script_score, script_fields, sort.</p><ul><li><strong>Goal:</strong> You want to <em>calculate</em> something on the fly (e.g., sorting by distance, custom scoring).</li><li><strong>Variable:</strong> You use doc[&#39;field_name&#39;].</li><li><strong>Behavior:</strong> You are accessing <strong>Doc Values</strong> (the columnar store). This is incredibly fast (in-memory), but it is <strong>Read-Only</strong>. You cannot change the document here.</li></ul><p><strong>Example:</strong> Calculate a custom sort value</p><pre>// WRONG: ctx._source.price (Will fail in search!)<br>// CORRECT:<br>doc[&#39;price&#39;].value * params.multiplier</pre><h4>The Performance Reality: “Doesn’t this kill the CPU?”</h4><p>This is the most common question. And the honest answer is: <strong>Yes.</strong></p><p>Running a script is computationally more expensive than a native query. It forces the CPU to execute logic for every document hit. So, why do we use it?</p><ol><li><strong>Network vs. CPU:</strong> Fetching 50,000 documents to your app, looping through them, and sending them back takes seconds (or minutes) of network transfer. Painless does it on the server in milliseconds.</li><li><strong>Safety:</strong> Unlike Groovy or Python, Painless is sandboxed and strictly typed. It won’t crash your node.</li><li><strong>Impossible Queries:</strong> Sometimes, you can’t express your logic in standard JSON. If you need to sort by (votes / views) * decay_factor, scripting is the only way.</li></ol><pre>// The script creates this value purely for the sort<br>doc[&#39;votes&#39;].value / doc[&#39;views&#39;].value</pre><h3>Summary</h3><p>Painless is a powerful tool that turns Elasticsearch from a simple search engine into a computation engine.</p><ul><li><strong>Use Inline Scripts</strong> for development and dynamic queries (always use params).</li><li><strong>Use Stored Scripts</strong> for production stability and code reuse.</li><li><strong>Use </strong>ctx._source when you need to update data.</li><li><strong>Use </strong>doc[&#39;field&#39;].value when you need fast, read-only access for searching and sorting.</li></ul><p>Don’t be afraid of the syntax. If you can write a if statement in Java or Go, you are already 90% of the way there. Happy scripting!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0dee19efef8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/pickme-engineering-blog/elasticsearch-scripting-is-no-longer-painful-a-developers-guide-to-painless-c0dee19efef8">Elasticsearch Scripting is No Longer “Painful”: A Developer’s Guide to Painless</a> was originally published in <a href="https://medium.com/pickme-engineering-blog">The PickMe Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>