<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sai Ram Naragoni on Medium]]></title>
        <description><![CDATA[Stories by Sai Ram Naragoni on Medium]]></description>
        <link>https://medium.com/@naragonisairam?source=rss-cb0bc9c0ad12------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*A5j6soL1OqycYBrGkeOIcQ.jpeg</url>
            <title>Stories by Sai Ram Naragoni on Medium</title>
            <link>https://medium.com/@naragonisairam?source=rss-cb0bc9c0ad12------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 11:21:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@naragonisairam/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AWS AI League 2025 — How I fine tuned my way to 2nd place, Part I]]></title>
            <link>https://naragonisairam.medium.com/aws-ai-league-2025-how-i-fine-tuned-my-way-to-2nd-place-part-i-c3c3f170d9ee?source=rss-cb0bc9c0ad12------2</link>
            <guid isPermaLink="false">https://medium.com/p/c3c3f170d9ee</guid>
            <category><![CDATA[lora]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Sai Ram Naragoni]]></dc:creator>
            <pubDate>Sun, 23 Nov 2025 04:17:10 GMT</pubDate>
            <atom:updated>2025-11-23T04:17:10.826Z</atom:updated>
            <content:encoded><![CDATA[<h3>AWS AI League 2025 — How I fine tuned my way to 2nd place, Part I</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_CVQ-FyAoomwaeCG1o8rMg.png" /><figcaption>By Rogue Nation</figcaption></figure><p>A while back, I was obsessed with making a tiny car finish a virtual circuit faster than everyone else — and that’s how I ended up <em>winning the AWS Deepracer World Championship in 2021</em>.</p><p>This time, the vehicle was a 3B parameter Llama model.<br>The track ? The AWS AI League.<br>And somehow, I managed to place 2nd.</p><p>Going from reward functions and racing lines to loss curves and LoRA ranks was a very different kind of fun, but core feeling was the same — iterate, tune, break, optimze to squeeze out that last bit of performance.</p><p>This is <strong>Part 1</strong> of a technical deep dive, focused on the engine that drives the whole system. Here, I break down the engineering backbone: the competition setup, how LoRA works under the hood, the hyperparameter tuning, and the log analysis that ties it all together.</p><p>(Stay tuned for Part 2, which will cover the synthetic data generation pipeline)</p><h3>The Challenge: David vs. Goliath</h3><p><strong>Fine-tune a Llama 3.2 3B model to outperform a Llama 70B model.</strong> The overall format was similar to the AWS AI League structure you’ll see in most public write-ups — leverage a Low Rank Adaptation (LoRA) to fine-tune a foundation model.</p><h3>But why use LoRA to fine-tune ?</h3><p>Why can’t we simply update every parameter in the model instead ? Let’s break down the math.</p><p>A Llama 3.2 3B parameter model has 3.21 Billion parameters. In a standard FP32 (32-bit float) precision based training, every parameter takes about 4 bytes of memory. During training, several stages require their own memory allocations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*TFLSZ5_O29g6daO8ZpCx2w.png" /><figcaption>Memory Requirement for FFT (Full Fine-Tuning) for 3B model</figcaption></figure><p>To fully fine-tune a small 3B model, you would need an A100 or a H100 (80GB) GPU or a carefully orchestrated multi-GPU setup with a tensor/model parallelism. And this requirement only grows more demanding as the model size increases.</p><h3>LoRA (Low-Rank Adaptation)</h3><p>This is where PEFT (Parameter-Efficient Fine-Tuning) and LoRA come in. It’s a clever and beautifully simple trick — instead of updating the large weight matrix W, it freezes the original model weights and introduce <em>two smaller trainable matrices</em>, A(<strong>r</strong> x d) and B(d x <strong>r</strong>), where <strong>r</strong> is a low rank we choose as a hyperparameter. The updated weights are then computed as:</p><blockquote>W’ = W + (α/r) * B * A</blockquote><p>where <strong>α</strong> is another hyperparameter.</p><p>Because <strong>r</strong> is very small (eg., 8, 16 or 64) compared to the model dimension d (eg., 4096), the number of trainable parameters drops drastically.</p><p><strong>Example:</strong> The following example uses a 2×2 weight matrix to show the calculations. However, in the next example, you see the real power of LoRA, where we train only about <strong>0.39%</strong> of the total parameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*WZhTg4pgE7w3LNyN0a_4ew.png" /><figcaption>LoRA calculation example</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*sDnMSFiLrotrNyNXnn_e8w.png" /><figcaption>LoRA PEFT with rank=8</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*wB6hQAEuL4JRKMESkMgYvQ.png" /><figcaption>Memory Requirement for PEFT using FP32 &amp; LoRA for 3B model</figcaption></figure><p>With LoRA, the memory requirement to fine-tune a Llama 3.2 3B model in FP32 drops to a little over 20GB. However, keep in mind this is a baseline estimate; in real setups, hyperparameters like batch size and sequence length will increase memory demand.</p><p>LoRA is just one tool in the efficiency toolbox. In practice, it’s often paired with other techniques that can drive memory usage down even further, such as:</p><ol><li>Mixed-precision training (FP16 / BF16)</li><li>Distributed training strategies, e.g., Fully Sharded Data Parallel (FSDP)</li><li>Quantized LoRA (QLoRA)</li><li>Activation/Gradient checkpointing</li><li>FlashAttention and other optimized attention kernels</li><li>ZeRO (Zero Redundancy Optimizer or DeepSpeed)</li></ol><p>In the AWS AI League, we fine-tuned the model on SageMaker JumpStart, which already incorporates several of these techniques out of the box, including mixed-precision training, FSDP, and QLoRA. I won’t go into each of them in this blog, but they’re all great rabbit holes to explore if you’re interested in scaling-efficient fine-tuning.</p><h3>Hyperparameter Tuning</h3><p>While LoRA gives you the mechanism, Hyperparameters give you the finesse. To maintain the integrity of the competition for future participants, I will not disclose the exact values I used. Instead, I will share the <strong>hints, strategic reasoning, key observations and recommendations </strong>that guided me toward the optimal configuration.</p><p>For fine-tuning, we were required to use an ml.g5.2xlarge instance, which provides a single 24 GB of GPU VRAM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/855/1*kMpg4dasKue1Rf4d6LXixA.png" /><figcaption>Knobs that actually mattered in the League</figcaption></figure><p><strong>Batch Size: </strong>The number of training samples processed before the model’s weights are updated. Larger batches — smoother gradients, faster convergence but higher memory usage. Smaller batches — noisier gradients, potential for better generalization, but slower effective training.</p><p>I targeted the largest batch size that comfortably fit within the GPU memory while keeping training stable. It’s also worth increasing the evaluation batch size, since a larger eval batch can modestly speed up the evaluation steps between epochs.</p><p>My recommendation: Find the limit, start small, try 2, then 4, then 8 on smaller datasets and keep increasing until you hit an out-of-memory error, then step back down to the last stable value.</p><p><strong>Int 8 Quantization: </strong>Enables 8-bit integer quantization via QLoRA, further reducing memory usage with minimal loss in training precision. For a 3B model, I didn’t find it necessary. The model trained comfortably on an ml.g5.2xlarge instance without pushing memory limits. However, if I were training a larger model that wouldn’t fit on the instance even with a batch size of 1, Int8 quantization would be one of the first techniques I’d consider.</p><p>The table below summarizes the typical accuracy loss observed when fine-tuning an LLM with QLoRA across different precision settings. Refer to <a href="https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization">a visual guide to quantization</a> for a deeper, more intuitive understanding of how it works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/585/1*_BL2Hrbeb1Vd3qJxpd5nfw.png" /><figcaption>Typical Accuracy Loss in QLoRA</figcaption></figure><blockquote>1% loss in accuracy for 75% reduction in memory for INT8 Quantization — That’s not a compromise, it’s a win.</blockquote><p><strong>FSDP (Fully Sharded Data Parallel): </strong>FSDP shards model parameters, gradients, and optimizer states across multiple GPUs. Instead of every GPU holding a full copy of the model, each one stores only a shard that is reconstructed on the fly when needed. For a 3B model, this is largely overkill — and in our case, the ml.g5.2xlarge instance has only a single GPU, so FSDP doesn’t meaningfully come into play.</p><p><strong>Epochs: </strong>Epochs control how many times you pass over the training dataset. With instruction-tuning or synthetic QA-style data, more epochs are not always better: synthetic data is usually high-signal but low-diversity compared to the internet, so small models can memorize its patterns very quickly.</p><p>My recommendation: Below is a plot of training and evaluation loss versus epochs. You can see that after around epoch 6 or 7, the eval loss curve flattens out — there’s very little additional benefit from further training beyond that point, and the risk of overfitting only increases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*WwVFvRDFbwMI8-dUEfTeVA.png" /><figcaption>Loss vs Epoch</figcaption></figure><p><strong>Learning Rate: </strong>The learning rate is the step size the optimizer takes while minimizing the loss. In general, a high LR can destabilize training and degrade performance, while a low LR makes training very slow and may underfit within the time and compute constraints of the competition.</p><p>LoRA behaves differently from full fine-tuning — because we’re only updating a small fraction of the parameters, we often need a slightly more aggressive learning rate to have a meaningful effect.</p><p>My recommendation: start with the default value and then tune up or slightly down based on stability and validation performance.</p><p><strong>LoRA R &amp; Alpha:</strong> As shown in the previous section, LoRA introduces two low-rank matrices, <strong>A</strong> and <strong>B</strong>. The <strong>LoRA rank (R)</strong> is the dimension we choose for this low-rank decomposition, while <strong>alpha</strong> is the scaling factor applied to the LoRA update.</p><p>LoRA <strong>R</strong> controls the capacity of the adapter: a higher rank → more expressive updates, but also more parameters and a higher risk of overfitting. I aimed for a rank that resulted in training roughly <strong>0.3–0.5%</strong> of the model’s parameters, tuned in relation to the size and diversity of the dataset.</p><p><strong>Alpha</strong> acts as a scaling factor, effectively controlling how “loud” the LoRA update is relative to the frozen base weights. A common rule of thumb is to set alpha = 2 × R, but in my experiments I saw better results with alpha = 4 × R.</p><p><strong>Lora Modules: </strong>Which parts of the transformer block should we actually attach adapters to? By default, SageMaker JumpStart and many libraries in general, apply LoRA to the Query and Value projections. Extending LoRA to additional modules can boost expressiveness, but it also increases the parameter count and the risk of overfitting.</p><p>My recommendation: stick with the default first. Once your model is performing well and begins to plateau, experiment with adding LoRA to other modules.</p><p>The diagram below shows a simplified transformer block and the main modules where LoRA can be applied, along with what each one is conceptually doing:</p><p><strong>The Attention Mechanism (Context)</strong><br><strong>Query(q_proj): </strong>Transforms input into “What am I looking for ?”<br><strong>Key(k_proj):</strong> Transforms input into “What information do I contain ?”<br><strong>Value(v_proj):</strong> Transforms input into “What information will I contribute ?”<br><strong>Output(o_proj):</strong> The Synthesizer. It mixes the insights from all attention heads into a coherent result.</p><p><strong>The Feed-Forward Network (Reasoning)<br>FFN — W1(Up/Gate):</strong> The Processor. Allows the model to expand dimensions and learn complex feature combinations.<br><strong>FFN — W2(Down):</strong> The Summarizer. Projects the processed features back down to the model dimension.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/444/1*pvnuWq-MO8O0e-ZffK-eFw.png" /><figcaption>Simplified Transformer Block highlighting LoRA target modules.</figcaption></figure><p><strong>LoRA Dropout: — </strong>randomly drops some adapter activations during training, similar to standard dropout. This helps regularize the adapters and reduce overfitting to the training set, especially when the dataset is small or highly synthetic.</p><p>My recommendation: It’s worth experimenting with different non-zero values to find a good balance.</p><h3>Log Analysis — The Feedback loop</h3><p>Keep in mind these hyperparameter recommendations are based on my dataset. Results will vary depending on the complexity, diversity, and size of your own data, and you may find different optimal values.</p><p>So the real question is: how do you know whether your configuration is actually working? <strong>If Hyperparameters are the hypothesis; Log Analysis is the observation.</strong> You cannot validate the former without analyzing the latter.</p><p>But, the training logs you see during fine-tuning are pretty limited, and there wasn’t an easy way to download them. So I wrote a small JavaScript snippet that you can run in the Chrome console — it scrolls through the entire logs panel from start to finish and saves the full log data as a file.</p><p>In it’s current state, the log analysis notebook provides the following information (NOTE: Below logs are not from my final submission):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*sFejfkxfc0JxbJbJbFHXYg.png" /><figcaption>Statistics of the trainable parameters as based on your hyperparameter configurations.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*EahBrTa617RSiPosilB4Lg.png" /><figcaption>Statistics of training time</figcaption></figure><p>Since the competition enforces compute limits, these statistics are vital for estimating the runtime of future experiments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*zYs_AixmqHLTc38xX6Jaxg.png" /><figcaption>Training Summary</figcaption></figure><p>This little <strong>training summary</strong> is my before/after snapshot of the run. It answers, at a glance, two key questions:</p><p><strong>Did the model actually learn anything? — </strong>The <em>Initial vs Final Train Loss</em> and <em>Loss Improvement (%)</em> tell you how much the model’s objective went down during training. The <em>Train Perplexity</em> numbers are the same story in a more interpretable scale: lower perplexity means the model is less “surprised” by the data.</p><p><strong>Did it generalize beyond the training set? — </strong>The <em>Initial vs Final Eval Loss</em> shows whether the improvements on the training set carried over to unseen data. If eval loss dropped along with train loss (and didn’t bounce back up), that’s a good sign you’re not just memorizing. If eval loss starts to climb as epochs increase, that’s a clear signal of performance degradation and overfitting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/519/1*c5OjpBg6paPln0GWB7miyA.png" /><figcaption>Training &amp; Evaluation Loss Vs Epochs</figcaption></figure><p>This was probably the most important graph. In a healthy training run, both train and eval loss should decrease together; if eval loss starts to climb while train loss keeps dropping, that’s a classic sign of overfitting.</p><p>Here, eval loss falls quickly in the first few epochs — <strong>the model learns ~90% of what it’s going to learn by around epoch 5, and ~95% by epochs 7–8.</strong> After that, eval loss basically plateaus while train loss continues to decline, meaning the model is just memorizing the training set without gaining meaningful performance on the validation set.</p><p>This curve let me pinpoint the “sweet spot” epoch where the model is neither underfitting nor overfitting, so I can stop training there and avoid wasting compute on redundant epochs in future runs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/520/1*PsCmE-qSpDsyaiK7y-Jyog.png" /><figcaption>Perplexity Vs Epochs</figcaption></figure><p>Perplexity is another way of expressing loss, in a form that’s a bit more intuitive for language models. It measures how well the model predicts the next token — essentially how uncertain or “confused” it is when generating text. Lower perplexity means the model is more confident and better aligned with the training data.</p><p>This graph basically tells the same story as the Loss vs. Epochs plot.</p><p>LLM Judge has inherent uncertainty and won’t always return the same score; there’s noticeable variance between runs. From my experiments, that variance seems correlated with how well the model is trained.</p><p><strong>My Observation &amp; hopothesis:</strong> When the model has <strong>high perplexity</strong>, I’ve seen LLM Judge scores swing wildly: one evaluation might land around 35, and another on the <em>same</em> model can jump to 60. In contrast, models with <strong>low perplexity (below ~3.5)</strong> tend to produce much more stable scores, usually staying within about 7 points of the mean score across submissions.</p><p>That stability at low perplexity is often a sign the model is overfitting and becoming more deterministic in its responses. If your training data is very high quality, this can be great, LLM Judge will reliably give you strong scores. But if the dataset is noisy or contains even a few bad examples, that same determinism can consistently drag performance down.</p><p>With <strong>higher perplexity</strong>, you effectively get more “spread” in behavior: results are less consistent, but you occasionally benefit from lucky runs, and with enough submissions you might land on an especially good score. Try to strike the right balance by picking an appropriate number of training epochs.</p><p>If you’d like to run your own log analysis, feel free to fork my repository: <a href="https://github.com/SairamNaragoni/aws-ai-league-log-analysis">https://github.com/SairamNaragoni/aws-ai-league-log-analysis</a></p><h3>Where the AWS AI League fell short (And what I’d Love to See Next Time)</h3><ol><li><strong>The “All or Nothing” Termination (Lack of Graceful Shutdown) </strong>— Right now if you stop a training job midway, it just kills the run and throws away all progress. There’s no graceful shutdown that saves the last checkpoint that can be used. This would be very useful to actively monitor logs and stop the training based on Log Analysis.</li><li><strong>No Iterative Fine-Tuning (The “One-Shot” Constraint) — </strong>Once you’ve fine-tuned a model, you can’t just take that checkpoint and keep training on new or improved data. You basically have to start over from the base model every time, which is both inefficient and discourages iterative experimentation.</li><li><strong>Limited access to logs and no simple download option — </strong>The logging on UI is useful for quick checks, but there’s no straightforward way to download the full logs for deeper analysis. That’s why I ended up hacking together a small script to scrape them — something that really should be a built-in feature.</li><li><strong>Does not allow your own locally fine-tuned model — </strong>I wish there was an option to bring your own model: train locally (or on your own pipeline with custom schedules, LoRA configs, data preprocessing, etc.) and then upload that checkpoint into the league environment. That would open the door to much richer experimentation while still benchmarking on a common evaluation setup.</li></ol><p>The league has only just begun, so I’m sure more features and challenges are on the way — but I really hope these points get addressed as the platform evolves.</p><h3>Conclusion — The “Engine vs. Fuel”</h3><p>At this point, we’ve looked at a highly tuned engine. We’ve optimized the mechanics with LoRA, dialed in the hyperparameters for finesse, and used log analysis to confirm that the model is actually learning efficiently. But even a Ferrari will lose a race if it’s running on low-grade fuel — the model can only learn what’s present in the data. If the signal isn’t there, no amount of clever scheduling or hyperparameter tweaking can magically invent it. Garbage in, garbage out still applies.</p><p>In <strong>Part 2</strong>, we’ll dive into the <strong>Synthetic Data Generation Pipeline</strong>: how I automated the process, engineered prompts, filtered outputs, and curated high-quality examples so that a “small” model could punch way above its weight and actually compete at the top of the leaderboard.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c3c3f170d9ee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeepRacer for Cloud (DRFC) configuration]]></title>
            <link>https://awstip.com/deepracer-for-cloud-drfc-configuration-4a785494423c?source=rss-cb0bc9c0ad12------2</link>
            <guid isPermaLink="false">https://medium.com/p/4a785494423c</guid>
            <category><![CDATA[ec2]]></category>
            <category><![CDATA[configuration]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[deepracer]]></category>
            <category><![CDATA[deepracer-for-cloud]]></category>
            <dc:creator><![CDATA[Sai Ram Naragoni]]></dc:creator>
            <pubDate>Wed, 13 Apr 2022 11:41:11 GMT</pubDate>
            <atom:updated>2022-09-02T17:27:27.571Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y4S2EG5gh4Oaa7JCfMytYg.jpeg" /><figcaption>By RogueNation</figcaption></figure><p>In this blog, we’ll review some of the important configurations involved in <a href="https://aws-deepracer-community.github.io/deepracer-for-cloud/reference.html">DRFC</a>. Although most of the properties are self-explanatory, we’ll take a look at few most used configurations from <strong>system.env</strong> followed by <strong>run.env</strong>.</p><h3>system.env</h3><p><strong>DR_SAGEMAKER_IMAGE : </strong>The <a href="https://hub.docker.com/r/awsdeepracercommunity/deepracer-sagemaker">sagemaker docker image</a> provides the environment for Machine Learning and Deep Learning framework of Deepracer. It comes primarily in 2 variants — CPU and GPU, depending on how you want to proceed with training. GPU is only used during policy update. This image is not as frequently updated but it’s better to keep an eye out in the community for when there is a new version available. To upgrade to the latest, pull the latest image from docker hub and point this property to its corresponding version.</p><p><strong>DR_ROBOMAKER_IMAGE : </strong>The <a href="https://hub.docker.com/r/awsdeepracercommunity/deepracer-robomaker">robomaker docker image</a> provides the simulation of physics — friction, gravity, track etc., for deepracer to explore and exploit. A new version of image is built whenever new tracks are released by AWS which is usually every month during the season. So, every month, once a new version is available, pull the image from docker hub and update this property to point to latest version if you want to train on new tracks. Read through docker hub overview section to understand differences in avx, avx2 etc. This image comes in different variants but primarily cpu, cpu-gl, gpu, gpu-gl where gl stands for openGL. GL images are said to perform a little better but it might get <a href="https://aws-deepracer-community.github.io/deepracer-for-cloud/opengl.html">a little tricky to configure them</a>.</p><p><strong>DR_COACH_IMAGE : </strong>The <a href="https://hub.docker.com/r/awsdeepracercommunity/deepracer-rlcoach/tags">RL Coach docker image</a> contains the implementations of the Reinforcement Learning Algorithms (Proximal Policy Optimization and Soft Actor Critic) on which deepracer trains on. This is not as frequently updated but better to keep an eye out for any upgrades in the community.</p><p><strong>DR_UPLOAD_S3_BUCKET</strong> <strong>:</strong> Name of the bucket on AWS S3 to upload the locally trained model.</p><p><strong>DR_LOCAL_S3_BUCKET :</strong> Name of the bucket/folder on your machine where you want your trained models to be saved.</p><h3>run.env</h3><p><strong>DR_WORLD_NAME :</strong> Track name on which you want to train your model on (not the one you see on console). Refer the track name from <a href="https://github.com/aws-deepracer-community/deepracer-race-data/tree/main/raw_data/tracks">this github repo</a>. Example : To train on Rogue Raceway, use <em>2022_march_pro </em>as the world name.</p><p><strong>DR_RACE_TYPE</strong> : Takes one of the following : TIME_TRIAL/OBJECT_AVOIDANCE/HEAD_TO_BOT depending on the format you want to train for.</p><p><strong>DR_ENABLE_DOMAIN_RANDOMIZATION :</strong> Trains the model in different lighting settings to make it more generalized. Extremely useful when being trained for physical racing and not of much use for virtual racing.</p><p><strong>DR_TRAIN_CHANGE_START_POSITION</strong> : Decide if you want to change the start position of the car for each episode during the training. If True, then make sure you maintain the <a href="https://aws-deepracer-community.github.io/deepracer-for-cloud/multi_worker.html">equality</a> given below. For physical racing, it’s better to train from different positions on the track. For virtual racing, although you should start from different positions initially, eventually you can set this configuration to False for better results.</p><pre>(num_episodes_between_training / DR_WORKERS) * DR_TRAIN_ROUND_ROBIN_ADVANCE_DIST = 1.0</pre><p><strong>DR_TRAIN_ROUND_ROBIN_ADVANCE_DIST : </strong>Distance to advance the start position for each episode when <em>DR_TRAIN_CHANGE_START_POSITION </em>is True.</p><p><strong>DR_TRAIN_START_POSITION_OFFSET : </strong>Decide from what percentage of the track you want to start every episode instead of from 0. Extremely useful when you notice your agent doing over or under steering at any part of the track or always going off at a specifc point on track. The above 3 configurations including this one work together in combinations.</p><p><strong>DR_TRAIN_MIN_EVAL_TRIALS :</strong> Minimum number of evaluations to perform after every iteration. Setting it to a minimum number (1 or 2) during initial training should reduce the training time to some extent. Best checkpoint is determined based on the performance of the model during evaluation between iterations. So, if you plan to use best checkpoint for racing instead of last checkpoint, once the model starts exploiting the environment, its better to increase this value to 5 atleast.</p><p><strong>DR_LOCAL_S3_MODEL_PREFIX :</strong> Name of the model that is currently being trained.</p><p><strong>DR_LOCAL_S3_PRETRAINED :</strong> If you want to train on top of a model which is already trained give this as True.</p><p><strong>DR_LOCAL_S3_PRETRAINED_PREFIX</strong> <strong>:</strong> Name of the model on top of which you want to proceed with current training.</p><p><strong>sample system.env</strong></p><pre>DR_CLOUD=local<br>DR_AWS_APP_REGION=us-east-1<br>DR_UPLOAD_S3_PROFILE=default<br>DR_UPLOAD_S3_BUCKET=rogue-minio<br>DR_UPLOAD_S3_ROLE=to-be-defined<br>DR_LOCAL_S3_BUCKET=rogue-minio<br>DR_LOCAL_S3_PROFILE=minio<br>DR_GUI_ENABLE=False<br>DR_KINESIS_STREAM_NAME=<br>DR_KINESIS_STREAM_ENABLE=True<br>DR_SAGEMAKER_IMAGE=5.0.0-gpu<br>DR_ROBOMAKER_IMAGE=5.0.1-cpu-avx2<br>DR_ANALYSIS_IMAGE=cpu<br>DR_COACH_IMAGE=5.0.0<br>DR_WORKERS=1<br>DR_ROBOMAKER_MOUNT_LOGS=False<br>DR_CLOUD_WATCH_ENABLE=False<br>DR_DOCKER_STYLE=swarm<br>DR_HOST_X=False<br>DR_WEBVIEWER_PORT=8100<br># DR_DISPLAY=:99<br># DR_REMOTE_MINIO_URL=<a href="http://mynas:9000/">http://mynas:9000</a><br># CUDA_VISIBLE_DEVICES=0</pre><p><strong>sample run.env</strong></p><pre>DR_RUN_ID=0<br>DR_WORLD_NAME=reInvent2019_track<br>DR_RACE_TYPE=TIME_TRIAL<br>DR_CAR_NAME=FastCar<br>DR_CAR_BODY_SHELL_TYPE=deepracer<br>DR_CAR_COLOR=Red<br>DR_DISPLAY_NAME=$DR_CAR_NAME<br>DR_RACER_NAME=$DR_CAR_NAME<br>DR_ENABLE_DOMAIN_RANDOMIZATION=False<br>DR_EVAL_NUMBER_OF_TRIALS=3<br>DR_EVAL_IS_CONTINUOUS=True<br>DR_EVAL_MAX_RESETS=100<br>DR_EVAL_OFF_TRACK_PENALTY=5.0<br>DR_EVAL_COLLISION_PENALTY=5.0<br>DR_EVAL_SAVE_MP4=False<br>DR_EVAL_CHECKPOINT=last<br>DR_EVAL_OPP_S3_MODEL_PREFIX=rl-deepracer-sagemaker<br>DR_EVAL_OPP_CAR_BODY_SHELL_TYPE=deepracer<br>DR_EVAL_OPP_CAR_NAME=FasterCar<br>DR_EVAL_OPP_DISPLAY_NAME=$DR_EVAL_OPP_CAR_NAME<br>DR_EVAL_OPP_RACER_NAME=$DR_EVAL_OPP_CAR_NAME<br>DR_EVAL_DEBUG_REWARD=False<br>DR_EVAL_RESET_BEHIND_DIST=1.0<br>#DR_EVAL_RTF=1.0<br>DR_TRAIN_CHANGE_START_POSITION=True<br>DR_TRAIN_ALTERNATE_DRIVING_DIRECTION=False<br>DR_TRAIN_START_POSITION_OFFSET=0.0<br>DR_TRAIN_ROUND_ROBIN_ADVANCE_DIST=0.05<br>DR_TRAIN_MULTI_CONFIG=False<br>DR_TRAIN_MIN_EVAL_TRIALS=2<br>#DR_TRAIN_RTF=1.0<br>DR_LOCAL_S3_MODEL_PREFIX=rl-deepracer-sagemaker<br>DR_LOCAL_S3_PRETRAINED=False<br>DR_LOCAL_S3_PRETRAINED_PREFIX=rl-sagemaker-pretrained<br>DR_LOCAL_S3_PRETRAINED_CHECKPOINT=last<br>DR_LOCAL_S3_CUSTOM_FILES_PREFIX=custom_files<br>DR_LOCAL_S3_TRAINING_PARAMS_FILE=training_params.yaml<br>DR_LOCAL_S3_EVAL_PARAMS_FILE=evaluation_params.yaml<br>DR_LOCAL_S3_MODEL_METADATA_KEY=$DR_LOCAL_S3_CUSTOM_FILES_PREFIX/model_metadata.json<br>DR_LOCAL_S3_HYPERPARAMETERS_KEY=$DR_LOCAL_S3_CUSTOM_FILES_PREFIX/hyperparameters.json<br>DR_LOCAL_S3_REWARD_KEY=$DR_LOCAL_S3_CUSTOM_FILES_PREFIX/reward_function.py<br>DR_LOCAL_S3_METRICS_PREFIX=$DR_LOCAL_S3_MODEL_PREFIX/metrics<br>DR_UPLOAD_S3_PREFIX=$DR_LOCAL_S3_MODEL_PREFIX<br>DR_OA_NUMBER_OF_OBSTACLES=6<br>DR_OA_MIN_DISTANCE_BETWEEN_OBSTACLES=2.0<br>DR_OA_RANDOMIZE_OBSTACLE_LOCATIONS=False<br>DR_OA_IS_OBSTACLE_BOT_CAR=False<br>DR_OA_OBJECT_POSITIONS=<br>DR_H2B_IS_LANE_CHANGE=False<br>DR_H2B_LOWER_LANE_CHANGE_TIME=3.0<br>DR_H2B_UPPER_LANE_CHANGE_TIME=5.0<br>DR_H2B_LANE_CHANGE_DISTANCE=1.0<br>DR_H2B_NUMBER_OF_BOT_CARS=3<br>DR_H2B_MIN_DISTANCE_BETWEEN_BOT_CARS=2.0<br>DR_H2B_RANDOMIZE_BOT_CAR_LOCATIONS=False<br>DR_H2B_BOT_CAR_SPEED=0.2<br>DR_H2B_BOT_CAR_PENALTY=5.0</pre><p>I have only covered the information which needs to be kept on mind when modifying different configurations. Full reference can be found here — <a href="https://aws-deepracer-community.github.io/deepracer-for-cloud/reference.html">https://aws-deepracer-community.github.io/deepracer-for-cloud/reference.html</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4a785494423c" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/deepracer-for-cloud-drfc-configuration-4a785494423c">DeepRacer for Cloud (DRFC) configuration</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeepRacer for Cloud (DRFC) Local setup]]></title>
            <link>https://awstip.com/deepracer-for-cloud-drfc-local-setup-3c6418b2c75a?source=rss-cb0bc9c0ad12------2</link>
            <guid isPermaLink="false">https://medium.com/p/3c6418b2c75a</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[local]]></category>
            <category><![CDATA[deepracer]]></category>
            <category><![CDATA[nvidia]]></category>
            <dc:creator><![CDATA[Sai Ram Naragoni]]></dc:creator>
            <pubDate>Fri, 08 Apr 2022 06:17:12 GMT</pubDate>
            <atom:updated>2022-04-13T09:47:43.756Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wxiA3CQOHdRCtVOcHv8J1Q.jpeg" /><figcaption>By RogueNation</figcaption></figure><p>Step by step guide to set up <a href="https://aws-deepracer-community.github.io/deepracer-for-cloud/">DRFC</a> on your local machine.</p><p><strong>Step 1 (only for windows OS): </strong>Make sure you have wsl2 installed on your machine. If not, follow this guide to set up wsl on your machine — <a href="https://docs.microsoft.com/en-us/windows/wsl/install">https://docs.microsoft.com/en-us/windows/wsl/install</a></p><p><strong>Step 2 (only for windows OS): </strong>Install ubuntu 20.04 from microsoft store and run the application. Set up root username and password for ubuntu.</p><p><strong>Step 3 (only for windows OS): </strong>Install required packages and set up docker-ce in the wsl-ubuntu.</p><pre>sudo apt-get install jq awscli python3-boto3 docker-compose</pre><pre>curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -</pre><pre>sudo add-apt-repository    &quot;deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable&quot;</pre><pre>sudo apt-get update &amp;&amp; sudo apt-get install -y --no-install-recommends docker-ce docker-ce-cli containerd.io</pre><p><strong>Step 4(only for Nvidia GPU)</strong>: If you have a dedicated Nvidia GPU, follow the below steps to install <a href="https://docs.nvidia.com/cuda/wsl-user-guide/index.html">CUDA 11.x and Nvidia Docker 2</a>. If you don’t own a dedicated Nvidia GPU, skip the below steps. You can use CPU instead.</p><p>Make sure you have the latest Nvidia driver for your graphics card and then execute the below commands.</p><pre>curl https://get.docker.com | sh</pre><pre>distribution=$(. /etc/os-release;echo $ID$VERSION_ID)<br><br>curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -<br><br>curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list</pre><pre>sudo apt-get update<br><br>sudo apt-get install -y nvidia-docker2</pre><pre>sudo service docker stop<br><br>sudo service docker start</pre><pre>cat /etc/docker/daemon.json | jq &#39;del(.&quot;default-runtime&quot;) + {&quot;default-runtime&quot;: &quot;nvidia&quot;}&#39; | sudo tee /etc/docker/daemon.json </pre><pre>sudo usermod -a -G docker $(id -un)</pre><p><strong>Step 5:</strong> Start the docker service (if not already running) and clone the DRFC repo.</p><pre>sudo service docker start</pre><pre>git clone <a href="https://github.com/aws-deepracer-community/deepracer-for-cloud">https://github.com/aws-deepracer-community/deepracer-for-cloud</a></pre><pre>cd deepracer-for-cloud</pre><p><strong>Step 6: </strong>From the deepracer-for-cloud directory, run the below command to install required packages and pull the docker images. If gpu is not detected, the script swiches to cpu or you can explicitly specify cpu as arch type instead of gpu if you don’t own any nvidia GPU.</p><pre>bin/init.sh -a gpu -c local</pre><p>Restart your machine. WSL users can run the below command in the powershell to restart ubuntu.</p><pre>wsl --shutdown </pre><pre>or </pre><pre>wsl -t Ubuntu</pre><p>On wsl ubuntu shutdown/restart, docker will be stopped as well. So after you log back into ubuntu run the below command to start the docker. For a permanent fix, refer known issues section here — <a href="https://aws-deepracer-community.github.io/deepracer-for-cloud/windows.html">https://aws-deepracer-community.github.io/deepracer-for-cloud/windows.html</a></p><pre>sudo service docker start</pre><p>If any of the docker commands throws permission denied exceptions, run the below command to fix it.</p><pre>sudo chmod 666 /var/run/docker.sock</pre><p><strong>Step 7 : </strong>We’ll be using minio in local as an s3 equivalent on cloud. To configure minio we would need accesskey_id and secret_key from your AWS account. Go to IAM -&gt; security_credentials and create a new accessid_key. Once you have both of them handy, run the below command and give accessid_key and secret_key when prompted. Leave the rest of fields empty.</p><pre>aws configure --profile minio</pre><p>Then execute the below script to start minio server and register deepracer for cloud commands on the terminal. If after restart, you are unable to run dr-* commands, it can be fixed by executing the below script.</p><pre>source bin/activate.sh</pre><p>Now that the DRFC set up is done. Its time to start training the model. Before starting the training, always upload the files in custom_files directory to minio and update the env variables using the below command.Then, use <strong><em>dr-start-training</em></strong> to start the training.</p><pre>dr-update &amp;&amp; dr-update-env &amp;&amp; dr-upload-custom-files</pre><pre>dr-start-training</pre><p>I’ll be covering in detail in the following blogs on how to customize training parameters.</p><p>For any help on the above process, please join Global Deepracer Slack Community @<a href="https://deepracing.io/">https://deepracing.io/</a></p><p><strong>External References :</strong></p><ul><li><a href="https://aws-deepracer-community.github.io/deepracer-for-cloud/">Introduction</a></li><li><a href="https://medium.com/@shi.pan218/setting-up-deepracer-local-training-on-windows-with-wsl2-and-gpu-passthrough-cc3d6b40e983">Setting up Deepracer Local Training on Windows 11 with WSL2 and GPU passthrough.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3c6418b2c75a" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/deepracer-for-cloud-drfc-local-setup-3c6418b2c75a">DeepRacer for Cloud (DRFC) Local setup</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Upsert and Upsert all with JDBCTemplate and Cockroach DB]]></title>
            <link>https://naragonisairam.medium.com/upsert-and-upsert-all-with-jdbctemplate-and-cockroach-db-8f1a017478ea?source=rss-cb0bc9c0ad12------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f1a017478ea</guid>
            <category><![CDATA[jdbc-template]]></category>
            <category><![CDATA[spring]]></category>
            <category><![CDATA[cockroachdb]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[upsert]]></category>
            <dc:creator><![CDATA[Sai Ram Naragoni]]></dc:creator>
            <pubDate>Sat, 05 Feb 2022 16:31:57 GMT</pubDate>
            <atom:updated>2022-02-05T17:15:51.599Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WlW19jN89cxYzCKXIiCjgA.jpeg" /><figcaption>By RogueNation</figcaption></figure><p>In my <a href="https://naragonisairam.medium.com/upsert-all-with-spring-data-jpa-and-cockroach-db-668d068a68f6">previous blog</a>, I have covered the custom implementation of upsert and upsertAll operations for all the repositories in the application using spring-data-jpa and reviewed the performance difference. In this blog, I am going address the same but using JDBC-Template. I will not be using any in-built repository such as CrudRepository, for comparing upsert and upsertAll performance since they give out similar result as seen in JPA. Instead, I will be using JDBCTemplate to execute queries for the save() and saveAll() operations.</p><h4>Intuition !</h4><p>The intuition here is same as seen with JPA implementation where I used the concept of reflection on the annotations of the entity class to frame the UPSERT query, except that the annotations change along with the execution flow where the generated query is executed using JDBCTemplate.</p><blockquote>For this demonstation, I used ‘users’ table in the cockroach db with a single node local cluster on the ‘movr’ database.</blockquote><h4>Let’s get started !</h4><p>To begin with, create a custom base repository interface to declare upsert() and upsertAll() methods. All the repositories defined in the project should be extending the custom base repository as shown below.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/715e879593c9044c892ea72ea414ad36/href">https://medium.com/media/715e879593c9044c892ea72ea414ad36/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2fc39d791cc66bd4701e36bf4c978700/href">https://medium.com/media/2fc39d791cc66bd4701e36bf4c978700/href</a></iframe><p>Now comes the important part — implementing the upsert and upsertAll query generation and execution. To frame the query, we would need 3 things — table_name, column_names in the table and the data_values for the given corresponding column_names. Fetching table_name would be simple with the use of reflection on the entity class but fetching the column_names from the entity is a bit more complicated because of superclasses (if any — also note that there are no embedded-Ids or join annotations involved here like in JPA). Brief explanation of the functions in the QueryGenerator can be found below.</p><pre><strong>1. getColumnFields</strong> - Fetch all the declared fields in the Entity class and its super classes(if any) that are annotated with <em>Column.class</em></pre><pre><strong>2. formInitialQuery </strong>- Forms the initial query &#39;UPSERT INTO table_name (column_names) VALUES &#39;</pre><pre><strong>3. populateTableColumns </strong>- Populates the column_names in the above formed initial query.</pre><pre><strong>4. populateColumnParams (and values) </strong>- For a given entity, fetch the value of the column_name. The output of this function would look something like - &quot;(value1,value2,value3,value4,value5),&quot;. There is no need to parameterize here cause the column_name and its corresponding value is what needed.</pre><pre><strong>6. upsert and upsertAll </strong>- upsert for a single entity and upsert all for a list of entites. POSTGRE SQL has a variable limit in a single query as 32767, hence any query statement during upsertAll with variable count more than this limit will be split into batches and generated.</pre><pre>Find below the source code for the above described functions :</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/023a755e013a25f4ec9b0bdeaf0c621f/href">https://medium.com/media/023a755e013a25f4ec9b0bdeaf0c621f/href</a></iframe><p>Now, It’s time to implement a concrete class for UserReposity interface. There is one additional operation defined in the custom base respository — upsertAllParallel(). I am using parallel Streams from Java-8 to execute upsert queries in parallel which gives a significant performance boost for upserting records in orders of millions as shown later below.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/310e276c5239a46568d8aae944289898/href">https://medium.com/media/310e276c5239a46568d8aae944289898/href</a></iframe><h4>Performance comparision of save/saveAll vs upsert/upsertAll/upsertAllParallel</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/09cd12596101c27944579725d400c256/href">https://medium.com/media/09cd12596101c27944579725d400c256/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/176fa673a967cc49a908926a9daa06ea/href">https://medium.com/media/176fa673a967cc49a908926a9daa06ea/href</a></iframe><p>All the attached tests are for upsert operations (for saveAll batch_size used is 250). The time to execute the query is more for upsert() than save() but its quite the opposite for upsertAll() and saveAll(). Unlike what is observed in JPA though, the difference can be seen starting with entities more than or equal to 10. The time to execute the query is reduced by almost 60 times in upsertAll() and 200 times in upsertAllParallel(). The improvement in terms of percentage is also very huge 5000% for upsertAll() and 20000% for upsertAllParallel().</p><h4>Conclusion</h4><p>The catch here is the same as stated in my <a href="https://naragonisairam.medium.com/upsert-all-with-spring-data-jpa-and-cockroach-db-668d068a68f6">previous blog</a>. But Regardless, for less frequent bulk database operations, this approach gives a huge advantage.</p><p>You can find the complete source code for this post on my github.</p><p><a href="https://github.com/SairamNaragoni/cockroach-db-upsert-jdbc">GitHub - SairamNaragoni/cockroach-db-upsert-jdbc: This project demonstates how to add custom JPA methods for upsert and upsertAll operations for all JPA repositories.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f1a017478ea" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Upsert and Upsert all with Spring Data JPA and Cockroach DB]]></title>
            <link>https://naragonisairam.medium.com/upsert-all-with-spring-data-jpa-and-cockroach-db-668d068a68f6?source=rss-cb0bc9c0ad12------2</link>
            <guid isPermaLink="false">https://medium.com/p/668d068a68f6</guid>
            <category><![CDATA[spring]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[upsert]]></category>
            <category><![CDATA[jpa]]></category>
            <category><![CDATA[cockroachdb]]></category>
            <dc:creator><![CDATA[Sai Ram Naragoni]]></dc:creator>
            <pubDate>Sun, 14 Nov 2021 09:01:11 GMT</pubDate>
            <atom:updated>2022-02-05T12:05:11.221Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FAdKCfLRojEUaoSfTlHwYw.jpeg" /><figcaption>By RogueNation</figcaption></figure><p>JPA is a powerful ORM but its performance varies depending on the use case and configurations. Similar to JPA’s save() method which is generally used for both INSERT and UPDATE operations, most of the modern databases provide a similar functionality at the database layer called UPSERT. Letting the database handle the INSERT or UPDATE operation through UPSERT would be more reliable and efficient than JPA firing 2 queries to first do a select operation and then update/insert through save() method, especially when your use-case revolves around bulk inserts or updates. This post demonstrates on how to configure upsert() and upsertAll() methods similar to save() and saveAll() methods for all the JPA repositories in the current project.</p><h4>Intuition !</h4><p>Although there are couple of ways to do it, to make it more generic, I’ll be using the concept of reflection on the entity class and its annotations to frame the query — “UPSERT INTO TABLE_NAME () VALUES () ()…..” and execute the query using the entityManager.</p><blockquote>For this demonstation, I used ‘users’ table in the cockroach db with a single node local cluster on the ‘movr’ database.</blockquote><h4>Let’s get started !</h4><p>To begin with, create a custom JPA repository interface extending the JPA repository to declare upsert() and upsertAll() methods. All the repositories defined in the project should be extending the custom repository instead of JPA repository as shown below.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a92fb9c314c1342cf7d5b6750b7da5e8/href">https://medium.com/media/a92fb9c314c1342cf7d5b6750b7da5e8/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/86b248c2c86f9b5aef29f255e0d8e5f8/href">https://medium.com/media/86b248c2c86f9b5aef29f255e0d8e5f8/href</a></iframe><p>Now comes the important part — implementing the custom JPA repository. To frame the query, we would need 3 things — table_name, column_names in the table and the data_values for the given corresponding column_names. Fetching table_name would be simple with the use of reflection on the entity class but fetching the column_names from the entity is a bit more complicated because of superclasses or embedded id’s. To tackle this, I’ll be using recursion to fetch the required values. Brief explanation of the functions in the custom JPA repository implementation can be found below.</p><pre><strong>1. getColumnFields</strong> - Fetches all the declared fields in the Entity class and its super classes(if any) that are annotated with either of <em>Column.class</em> or <em>JoinColumn.class</em> or <em>EmbeddedId.class</em></pre><pre><strong>2. formInitialQuery </strong>- Forms the initial query &#39;UPSERT INTO table_name (column_names) VALUES &#39;</pre><pre><strong>3. populateTableColumns </strong>- Populates the column_names in the above initial query. Recursion is used for <em>EmbeddedId.class.</em></pre><pre><strong>4. populateColumnParams </strong>- For a given entity, parameterize each column_name. The output of this function would look something like - &quot;:id0,:city0,:name0,:address0,:credit_card0&quot;, where 0 is the passed index.</pre><pre><strong>5. populateColumnValues</strong> - For each parameterized column from the above function, set the column_value in the native query.</pre><pre><strong>6. upsert and upsertAll </strong>- upsert for a single entity and upsert all for a list of entites. POSTGRE SQL has a variable limit in a single query as 32767, hence any query statement during upsertAll with variable count more than this limit will be split into batches and executed.</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cb94329482db7e50fac22a786c6583b8/href">https://medium.com/media/cb94329482db7e50fac22a786c6583b8/href</a></iframe><p>Finally, we have to let the Spring know that it should use the custom JPA repository implementation as a repository base class instead of default repository base class with the following class level annotion.</p><pre>@EnableJpaRepositories(repositoryBaseClass = ExtendedJpaRepositoryImpl.<em>class</em>)</pre><h4>Performance comparision of save/saveAll vs upsert/upsertAll</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6fd571216bb59d14f579894341949ea2/href">https://medium.com/media/6fd571216bb59d14f579894341949ea2/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dc807d520a6d7d0c92d91f68b00ae936/href">https://medium.com/media/dc807d520a6d7d0c92d91f68b00ae936/href</a></iframe><p>All the attached tests are for insert operations with a batch_size of 250 and on cockroach db. I did perform the same for updates with different batch sizes and the results are the same as above in comparision. As it is evident from the above tables, performance is more or less the same for entities upto 100 but for bulk inserts there is a reduction in time to execute the query by 60–80 % for JPA entity implementing the persistable interface and almost 99% over traditional saveAll. If we speak in terms of performance improvement, upsertAll gives almost 200–400% improvement with persistable interface and 30000+% improvement over traditional saveAll.</p><h4>Conclusion</h4><p>Although the results are quite overwhelming, the catch with this implementation is the memory requirement, especially when the application is deployed on cloud with 1Gig memory. As we keep on increasing the number of entities, the memory requirement to execute the query also increases for forming and storing large strings. It can be optimized for memory but for now its up to the developer to develop the application in a way that at any given point in time there won’t be an upsertAll operation for 50k or 60k records at once.</p><p>I encountered similar results with spring data JDBC as well but more on that in the coming posts. You can find the complete source code for this post on my github.</p><p><a href="https://github.com/SairamNaragoni/cockroach-db-upsert">GitHub - SairamNaragoni/cockroach-db-upsert: This project demonstates how to add custom JPA methods for upsert and upsertAll operations for all JPA repositories</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=668d068a68f6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Openstack Ansible AIO]]></title>
            <link>https://naragonisairam.medium.com/openstack-ansible-aio-8355ef663cef?source=rss-cb0bc9c0ad12------2</link>
            <guid isPermaLink="false">https://medium.com/p/8355ef663cef</guid>
            <category><![CDATA[devstack]]></category>
            <category><![CDATA[error-handling]]></category>
            <category><![CDATA[openstack]]></category>
            <category><![CDATA[ansible]]></category>
            <dc:creator><![CDATA[Sai Ram Naragoni]]></dc:creator>
            <pubDate>Tue, 19 Mar 2019 19:32:27 GMT</pubDate>
            <atom:updated>2022-02-05T14:17:27.998Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Problems with Devstack :</strong></p><ul><li>Devstack is made for development purposes. It is not suitable to be used in production environment.</li><li>It has a single script which installs all the required services. In case after installation, any openstack service goes down due to some fatal error, the only way to recover is to reinstall each and every service overwriting the existing configuration.</li><li>Even if the system crashes or reboots, most of the devstack services break and wont’t work properly like for example glance and swift.</li><li>Devstack must be used for proof of concept of openstack or for quick development purpose.</li></ul><p><strong>Why Openstack Ansible ?</strong></p><ul><li>Openstack ansible uses the power of ansible playbooks to deploy openstack.</li><li>It installs the openstack components in the LXC containers , which provides complete OS Level isolation between the openstack services.</li><li>Unlike devstack, ansible is used to deploy openstack in production environment.</li><li>Maintaining of openstack deployed using ansible is very easy and a takes a of couple of minutes to reinstall any service during maintainence.</li><li>Changing the configuration of a service is also feasible with openstack-ansible deployment.</li></ul><p><strong>OPENSTACK — ANSIBLE AIOINSTALLATION GUIDE</strong></p><p><strong>Minimum required configuration :</strong></p><ul><li>8 vCPUs</li><li>50GB free disk space on the root partition</li><li>8GB RAM</li></ul><p><strong>Recommended Configuration :</strong></p><ul><li>CPU/motherboard that supports hardware-assisted virtualization</li><li>8 CPU Cores</li><li>80GB free disk space on the root partition, or 60GB+ on a blank secondary disk.</li><li>16GB RAM</li></ul><p><strong>Installation :</strong></p><p><strong>Prepare the Host :</strong></p><p>All the packages on the host machine needs to be upgraded, so execute the following commands as a root user. Please note that this installation guide is for Ubuntu distribution.</p><pre><strong># apt-get dist-upgrade<br># reboot</strong></pre><p><strong>Bootstrap Ansible and the required roles :</strong></p><p>Clone the <strong>openstack-ansible</strong> repository into the <strong>/opt/openstack_ansible</strong> directory.</p><pre><strong># git clone </strong><a href="https://git.openstack.org/openstack/openstack-ansible"><strong>https://git.openstack.org/openstack/openstack-ansible</strong></a><strong> /opt/openstack-ansible</strong></pre><pre><strong># cd /opt/openstack-ansible</strong></pre><p>Checkout the stable branch of your requirement or by using any tag</p><p>For Ex :</p><pre><strong># git checkout stable/rocky</strong></pre><p>Run the following script :</p><pre><strong># scripts/bootstrap-ansible.sh</strong></pre><p>Bootstrap the AIO configuration.</p><pre><strong># scripts/bootstrap-aio.sh</strong></pre><p>The above command creates the following path <strong>/etc/openstack_deploy</strong></p><p>Now, if you want to add any other services over the default services, For example, in order to add <strong>horizon, sahara and swift</strong> services run the following script</p><pre><strong># cp etc/openstack_deploy/conf.d/{horizon,sahara,swift}.yml.aio /etc/openstack_deploy/conf.d/</strong></pre><pre><strong>for f in $(ls -1 /etc/openstack_deploy/conf.d/*.aio); do mv -v ${f} ${f%.*}; done</strong></pre><p><strong>Run the playbooks :</strong></p><pre><strong># cd /opt/openstack-ansible/playbooks</strong></pre><pre><strong># openstack-ansible setup-hosts.yml</strong></pre><pre><strong># openstack-ansible setup-infrastructure.yml</strong></pre><pre><strong># openstack-ansible setup-openstack.yml</strong></pre><ul><li>The <strong>setup-hosts.yml</strong>, creates the containers for each service that is to be installed on the host machine. It also installs the common requirements in the containers.</li><li>The <strong>setup-infrastructure.yml</strong>, configures the infrastructure services required for the openstack. All the required git repositories of the openstack services are cloned and stored in the repository server. The galera server for sql database management is configured. Rabbitmq server and rsyslog are also installed in this playbook.</li><li>The <strong>setup-openstack.yml</strong>, this playbook installs the openstack services that are specified earlier one by one in the corresponding containers.</li></ul><p>If everything goes as intended without any errors, the openstack horizon dashboard should be available at your host IP.</p><p>During installation of the above three playbooks, in case any of the playbook throws an error — then note which container/service is throwing the error and reinstall that particular faulty container/service , then re-run the playbook.</p><p><strong>How to reinstall a container :</strong></p><p>Say, there is an error in the horizon container, get the container ID by running the following command</p><pre><strong># lxc-ls</strong></pre><p>then run the below commands for the corresponding container to clean it out.</p><pre><strong># lxc-stop -n aio1_horizon_container-1c29b052</strong></pre><pre><strong># lxc-destroy -n aio1_horizon_container-1c29b052</strong></pre><pre><strong># rm -rf /openstack/aio1_horizon_container-1c29b052/*</strong></pre><p>To re-configure the horizon container and necessary dependencies again, run the below command.</p><pre><strong># openstack-ansible/playbooks</strong></pre><pre><strong># openstack-ansible setup-hosts.yml -l aio1 -l aio1_horizon_container-1c29b052</strong></pre><pre><strong># openstack-ansible setup-infrastructure.yml — limit aio1_horizon_container-1c29b052</strong></pre><p>Ansible allows users to install any particular service individually as well. In case of Horizon, run the following playbook.</p><pre><strong># openstack-ansible os-horizon-install.yml</strong></pre><p>At any stage, the error status of the above playbooks during installation or health-check of the system hosts/infrastructure/services after installation can be verified by running the following playbooks</p><p><strong>healthcheck-hosts.yml</strong></p><p><strong>healthcheck-infrastructure.yml</strong></p><p><strong>healthcheck-openstack.yml</strong></p><p>Ex :</p><pre><strong># openstack-ansible healthcheck-hosts.yml </strong></pre><p>The above command checks for any faulty containers and displays the error accordingly.</p><p>Every time, the system is rebooted the galera cluster needs to be reinstalled. Before reinstalling make sure the <strong>haproxy </strong>service is up and running. This is applicable even during installation, after running the playbook <strong>setup-infrastructure.yml </strong>if the system gets shut down due to any unforeseen reason, then run the below commands before going for re-installing openstack services. If required, install haproxy service then install galera.</p><pre><strong># cd openstack-ansible/playbooks</strong></pre><pre><strong># systemctl start haproxy.service</strong></pre><pre><strong># openstack-ansible -e galera_ignore_cluster_state=true galera-install.yml</strong></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8355ef663cef" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>