<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Deepak Arora on Medium]]></title>
        <description><![CDATA[Stories by Deepak Arora on Medium]]></description>
        <link>https://medium.com/@arora.deepak?source=rss-2e7b808abac9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*j70AV1dnraMTopGT-1frdg.jpeg</url>
            <title>Stories by Deepak Arora on Medium</title>
            <link>https://medium.com/@arora.deepak?source=rss-2e7b808abac9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 05:43:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@arora.deepak/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Race to Faster and Cost-Efficient LLM Training: Trends, Techniques, and AWS Innovations]]></title>
            <link>https://medium.com/@arora.deepak/the-race-to-faster-and-cost-efficient-llm-training-trends-techniques-and-aws-innovations-fe4943ff6b73?source=rss-2e7b808abac9------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe4943ff6b73</guid>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[deepseek]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[publicis-sapient]]></category>
            <dc:creator><![CDATA[Deepak Arora]]></dc:creator>
            <pubDate>Tue, 28 Jan 2025 10:34:05 GMT</pubDate>
            <atom:updated>2025-01-28T10:34:05.619Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DFGfwxf15aeiBkSO4Z9LQQ.png" /></figure><p>Large Language Models (LLMs) have revolutionized the AI landscape; however, their training process remains a significant challenge due to high costs and resource-intensive requirements. DeepSeek’s breakthrough with its R1 model, which claims to achieve a <a href="https://arbisoft.com/blogs/deep-seek-r1-the-chinese-ai-powerhouse-outperforming-open-ai-s-o1-at-95-less-cost">95% reduction in training costs</a>, offers an early glimpse into the future of low-cost, faster LLM training. LLM creators and tech companies are heavily investing in innovative solutions to reduce the time, financial costs, and environmental impact associated with training these models.</p><p>This post explores advancements in the industry and outlines primary approaches for customers leveraging LLMs across a set of scenarios.</p><p>The key advancements here are around following approaches: -</p><p>· Hardware Innovations involving tweaking and streamlining of the training hardware such as <a href="https://www.theverge.com/2023/11/15/23960345/microsoft-cpu-gpu-ai-chips-azure-maia-cobalt-specifications-cloud-infrastructure">custom chips from Microsoft</a></p><p>· Software optimisation techniques with a focus on getting better training performance with existing hardware by ensuring effective memory utilisation. Techniques such as <a href="https://docs.nvidia.com/deeplearning/performance/mixed-precision-training/index.html">Mixed precision training</a> and <a href="https://fairscale.readthedocs.io/en/latest/deep_dive/activation_checkpointing.html">Activation checkpointing</a> have proven to be effective in this category</p><p>· Distributed model training by either leveraging multiple GPUs in parallel to train and/or using data parallelism technique designed to optimize memory usage and scalability during the LLM training by leveraging libraries such as <a href="https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html">FSDP</a>.</p><p>Optimised training frameworks such as <a href="https://github.com/ServiceNow/Fast-LLM?tab=readme-ov-file#readme">Fast-LLM</a> from ServiceNow, a opensource library developed for training models of all sizes employs a mix of above techniques (mixed-precision, distributed training and hardware optimisations) to maximise throughput and minimize training time.</p><p>· In scenarios where a smaller model is a better fit, techniques like <a href="https://arxiv.org/pdf/1503.02531">Knowledge distillation</a> where in a smaller “student” model is trained to replicate the performance of a larger “teacher” model can be effective. This allows the student model to benefit from the teacher’s knowledge without the extensive computational resources required to train the larger model from scratch.</p><p>· Shift from traditional supervised fine-tuning (SFT) to reinforcement learning (RL)-first approach as used for DeepSeek-R1. Most large language models (LLMs) begin their training with supervised fine-tuning (SFT) to establish foundational knowledge. However, DeepSeek-R1-Zero deviated from this traditional approach by skipping the SFT stage entirely. Instead, it was trained exclusively using reinforcement learning (RL), allowing the model to independently develop advanced reasoning capabilities, such as chain-of-thought (CoT) reasoning. This innovative RL-first strategy enabled the model to explore and refine its reasoning processes without relying on pre-labeled data.</p><p>Organizations looking to leverage LLMs (Large Language Models) for specific domains or use cases should adopt one or more of the following strategies to accelerate training and reduce costs:</p><p>· Transfer learning and Fine-tuning — this involves starting with a pre-trained model and fine-tuning on domain-specific data. By leveraging pre-trained models and knowledge from existing tasks, transfer learning significantly reduces training time, data requirements, and computational costs. There is also ongoing research and development in this area to further enhance transfer learning capabilities.</p><p>· Domain-specific Pre-training — In this approach, a relatively smaller model is pre-trained specifically on domain-relevant data rather than undergoing general pre-training. This method can lead to better performance in the target domain compared to general pre-training followed by fine-tuning. However, this approach requires sufficient domain-specific data to be effective. A potential solution to this challenge is using smaller, specialized LLMs that perform well on tasks within a specific domain while disregarding performance in unrelated domains. Small models benefit from comparatively fast inference times, lower latency, and less expensive training. Their reasonably small size allows them to be hosted on local, consumer-grade hardware.</p><p>· Mixed-Domain Pre-Training is an effective strategy when there is insufficient domain-specific data for the model to develop a strong understanding of natural language. In such cases, the model can first be pre-trained on a large general dataset to learn foundational language patterns and then fine-tuned on the smaller in-domain dataset. This approach typically yields better performance compared to training solely on a limited domain-specific dataset.</p><p>· Another efficient method for domain-specific modelling is the <a href="https://arxiv.org/pdf/1312.4314">Mixture of Experts (MoE)</a> approach. MoE divides the training workload across multiple specialized sub-networks, activating only the relevant sub-networks for specific tasks. This not only improves efficiency but also accelerates training times by optimizing resource utilization.</p><p>Depending on the use case and available resources, research suggests that domain-specific and mixed-domain pre-training can be a viable and preferable alternative to general pretraining. <a href="https://arxiv.org/html/2407.14076v1?t">Referencing here</a> a great piece of research done for medical use case to compare and benchmark different approaches.</p><p>AWS with its global scale across industries, is innovating rapidly across the above areas and have industry leading capabilities. AWS has lot of capabilities to offer across generic training, continued-pretraining, fine-tuning and domain-specific training. Listing down a few key ones here for GenAI application creators/engineers to explore:</p><p>· Amazon SageMaker provides managed <a href="https://docs.aws.amazon.com/en_jp/sagemaker/latest/dg/train-model.html">training capabilities</a>, enabling users to scale workloads efficiently without the need to manage infrastructure. Features like Amazon SageMaker HyperPod allow users to train foundation models at scale with thousands of accelerators, reducing training time while ensuring resiliency and automated recovery from failures.</p><p>· Additionally, <a href="https://aws.amazon.com/cn/blogs/machine-learning/training-large-language-models-on-amazon-sagemaker-best-practices/">Amazon SageMaker Model Training</a> supports Activation checkpointing and distributed training libraries that automatically split large datasets and models across GPUs, simplifying the process for developers.</p><p>· The <a href="https://aws.amazon.com/ec2/instance-types/p5/">Amazon EC2 P5</a> instances deliver up to 6x faster training times compared to previous generations. These instances also support <a href="https://aws.amazon.com/blogs/machine-learning/how-fp8-boosts-llm-training-by-18-on-amazon-sagemaker-p5-instances/">FP8 precision</a>, which boosts performance by up to 18%, making them ideal for cost-effective LLM training.</p><p>· For domain-specific use cases, AWS provides curated model libraries and recipes through Amazon SageMaker HyperPod, allowing customers to fine-tune popular models like Llama or Mistral in minutes.</p><p>· For orchestration at scale, <a href="https://aws.amazon.com/blogs/machine-learning/accelerate-deep-learning-training-and-simplify-orchestration-with-aws-trainium-and-aws-batch/">AWS Trainium, integrated with AWS Batch</a>, offers massive scalability and cost efficiency. This combination allows users to train models across tens of thousands of accelerators while simplifying job scheduling and resource management.</p><p>Together, these tools empower organizations to train LLMs faster, more efficiently, and at a lower cost, enabling innovation across diverse domains.</p><p>Deepak Arora</p><p>CTO and Head of Engineering, Publicis Sapient Middle East</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe4943ff6b73" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Advancing Enterprise AI: Framework for Implementing GenAI through LLMOps]]></title>
            <link>https://medium.com/@arora.deepak/advancing-enterprise-ai-framework-for-implementing-genai-through-llmops-7b71cf8f50a6?source=rss-2e7b808abac9------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b71cf8f50a6</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[generative-ai-consulting]]></category>
            <category><![CDATA[technical-strategy]]></category>
            <category><![CDATA[publicis-sapient]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Deepak Arora]]></dc:creator>
            <pubDate>Mon, 19 Feb 2024 14:50:26 GMT</pubDate>
            <atom:updated>2024-02-19T14:50:26.180Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AJ1sV_mIZM-ZRLQ4lSDfZg.png" /><figcaption>Generated through Dall-E</figcaption></figure><p>The launch of OpenAI’s ChatGPT marked a watershed moment in AI technology, igniting an unprecedented surge of corporate interest in the vast potential of large language models (LLMs). Beyond mere fascination, businesses across various sectors — from marketing, finance and healthcare to customer service and content creation — are now keenly exploring how LLMs can drive innovation, enhance efficiency, and redefine customer engagement. However, this enthusiasm is at times tempered by considerations around data privacy, ethical AI deployment, and the challenge of integrating LLMs into existing technological infrastructures.</p><p>Amidst this backdrop, Big Tech companies are democratizing access to LLMs, empowering enterprises to unlock the full potential of these technologies. Yet, as organizations strive to tap into the capabilities of LLMs, they encounter significant hurdles. To truly harness LLMs at scale and ensure they operate free from bias, a deep understanding of the technology and its implications is essential. This realization has ushered in the need for a comprehensive approach to managing LLMs — from their initial development and fine-tuning to deployment and ongoing monitoring in production environments.</p><p>Enter the world of LLMOps — a paradigm that emphasizes the end-to-end workflow essential for building, fine-tuning, and deploying LLMs effectively. LLMOps represents a strategic framework that enables organizations to navigate the complexities of LLM integration, ensuring these powerful tools deliver on their promise while aligning with ethical standards and operational requirements.</p><p>This paper outlines the core principles and essential components of the LLMOps ecosystem. In addition, the below infographic lists the criteria for selecting or developing optimal solutions and lists the top available options in the field. In this paper, my focus is on exploring the optimal solution path characterized by best-of-breed approaches. However, it’s important to note that a cloud-native solution path remains a viable alternative. The foundational principles and goals discussed here apply to both options.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3wE_Cx9JDWQ0YKwqEdspvA.png" /></figure><p><strong>LLMs — </strong>When it comes to choosing an LLM, there are typically three broad paths: building your model from scratch, fine-tuning a pre-trained model, or using an off-the-shelf model. Most enterprises will end up leveraging a pre-built and at times pre-trained model to start with and the focus will be on contextualising and customising them for a specific use case.</p><p><strong>Storage — </strong>Invariably the best option here is to select a Vector database that can efficiently store large volumes of vector datasets and help provide a rich query capability.</p><p><strong>Training data Curation and Annotation -</strong>Training Data Creation tools can help create, select and quality check the data for bias, labelling errors etc. Data Annotation tools can annotate text, images, and videos. The other option is using human annotators however that comes with obvious limitations.</p><p><strong>Prompt engineering — </strong>Prompt engineering often involves a combination of techniques and practices applied directly within the platforms that host LLMs. There is an increase in the adoption of tools to assist in developing, testing, and chaining prompts.</p><p><strong>Response parsing</strong> in the context of LLMs involves interpreting and processing the output generated by the model. This step is crucial for applications that use LLMs to ensure the generated responses align with user queries or tasks, enhancing the utility and applicability of the model’s output in real-world applications.</p><p><strong>Reinforcement learning</strong> — Given the qualitative nature of the output from the GenAI application, human feedback becomes the primary feedback loop for performance improvement or fine-tuning of models. This requires another set of tools to be explored and integrated into the workflow. We are seeing the adoption of reinforcement learning from AI or tools that are pre-trained on human values and expectations. This is key to deploying these models at scale.</p><p><strong>Model deployment</strong> — Here the leading cloud-based deployment and management solution will be more prominent especially if hosting the solution on the cloud.</p><p><strong>Model monitoring</strong> — is the continuous evaluation of a deployed model’s performance to ensure it operates as expected. This includes tracking its accuracy, efficiency, and how it adapts to new data or environments over time. Monitoring is essential for maintaining the reliability and relevance of GenAI applications, especially as data distributions and user expectations evolve.</p><p><strong>Auditing, Risk Management and Security</strong> — Ensuring trust, reliability, and safety is increasingly vital for the competitive edge of early generative AI products. Safeguarding datasets and models against compromised data, cyber-attacks, and unauthorized access is essential for maintaining integrity and user confidence.</p><p><strong>Runtime Infrastructure Efficiency Management</strong> — Running or Training an LLM requires intensive computing therefore the runtime environment optimisation becomes critical for LLM deployments. The objective here is to ensure that LLM deployments are not only cost-effective but also achieve desired performance levels. This involves leveraging advanced hardware acceleration techniques, optimizing data processing pipelines, and adopting scalable cloud resources.</p><p>The domain of comprehensive low-code <strong>AI Development Platforms</strong> is undergoing rapid evolution, progressively enhancing capabilities to support the entire AI project lifecycle. These platforms are becoming more sophisticated, offering integrated solutions that span from initial data preparation and model training to the deployment and ongoing monitoring of AI applications. In addition to leading cloud providers, the example here is Vellum which is a comprehensive development platform tailored for creating LLM applications, equipped with tools for prompt engineering, semantic search, version control, testing, and monitoring.</p><p>Hope this helps!</p><p>Deepak Arora, CTO at Publicis Sapient, Middle East</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b71cf8f50a6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying the ‘How’ of Customer Data Platforms]]></title>
            <link>https://medium.com/engineered-publicis-sapient/demystifying-the-how-of-customer-data-platforms-1b3c111e4378?source=rss-2e7b808abac9------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b3c111e4378</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[customer-data-platform]]></category>
            <dc:creator><![CDATA[Deepak Arora]]></dc:creator>
            <pubDate>Tue, 12 May 2020 06:52:22 GMT</pubDate>
            <atom:updated>2020-05-12T06:52:22.272Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BDn4kjyXIK28ihU2lbvFGA.jpeg" /></figure><p>In today’s business world, even large corporations can fall into decline, thanks to disruption from competitors, emerging technologies and changes in consumer behavior. Businesses that succeed adopt a fearless commitment to change, organize their business around their customers’ obsession and utilize data to effectively respond to market and consumer demands. It’s not news that this consumer-first mindset should carry over into a company’s digital presence, but knowing <strong>how to do that is a challenge many businesses face</strong>. The leaders in this space have had their priorities clear:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/1*FIVL4R-cHmGsiwKvZ9LwRg.png" /></figure><p>These priorities often center on getting a single view on customers across the enterprise, an idea that is not new. The single customer view has been a priority for CMOs since 2012, yet 80% have not yet achieved value. The Harvard Business Review has found that the single customer view remains one of the biggest challenges to customer experience management. The primary reason for that, I believe, it’s a moving target due to:</p><p>1. The pace of emerging technologies</p><p>2. The rate that new data points emerge</p><p>3. Shifting consumer expectations</p><p>In most of the cases, businesses already have many of the elements required for a single customer view in place. They are just silo-ed across the business. A way to achieve the single customer view is with a customer data platform (CDP). A CDP resides conceptually between existing data systems and existing marketing technologies. It consolidates and processes the data from existing systems, and then makes it available for marketing technologies. In most scenarios, implementing a CDP does not require the replacement of existing systems. It merely resides in an additional conceptual layer, and can be implemented without disturbing business functions. Unfortunately, it’s not as easy as cobbling data together and hoping for the best.</p><p>When considering the use of a CDP, solution leaders often face two main questions:</p><p>· Should we consider hosting CDP on the Cloud or on the premises? (After all, it is the customer data we are talking about here.)</p><p>· Should we build or buy? (Yes, the famous dilemma, yet again.)</p><p>While weighing in on the famous on-Cloud versus on-prem debate, I won’t bore you with staggering numbers around enterprise-wide Cloud adoption or attempt to alleviate all concerns regarding Cloud security. However, I would say that these are exaggerated fears that result in lost opportunity and inappropriate spending. Gartner reports that by 2025, 99% of all security breaches will be the fault of customers who fail to appropriately manage Cloud security controls, not the providers of the Cloud service. The challenge exists not in the security of the Cloud itself, but in the policies and technologies for security and control of the technology.</p><p>Once you get over the hurdle of convincing your organization that the Cloud is the way to go, the next question is whether your CDP should be a Cloud-agnostic or Cloud-native solution. That debate, in this case, is linked to the question: Should we build or buy? In a buy scenario, the landscape of vendors offering CDP solutions is expanding fast. In 2017, there were five vendors who offered a single customer view solution. In 2019, that number approached 100. Many vendors with services within the peripherals of use cases where a single customer view would be useful, are now expanding their products to offer a CDP as a solution on top of their existing CRM, tag management, analytics and personalization solutions. The increase in vendor solutions is creating a confusing landscape where it is hard to understand the positioning of the solutions, what use cases they can support, what are their security policies, and how liability for a data breach is handled, to name a few. Choosing a third-party solution requires due diligence beyond a use-case and requirements evaluation.</p><p>All that being said, ‘buy’ is totally a valid option if that aligns with the existing IT and marketing strategy of the company, assuming appropriate qualification is done on functional and non-functional factors. (This deserves a separate article of its own. However, <a href="https://www.cdpinstitute.org/"><em>CDP Institute</em></a><em> </em>provides a good vendor comparison for the interested ones.)</p><p>I get very excited on seeing how the Cloud ecosystem has evolved to support something like CDP by leveraging Cloud-native application frameworks. Don’t get me wrong — I am a big fan of Cloud-agnostic solutions, but in this case, I feel that leveraging Cloud-native offerings does more justice to security than many people assume. Here is an overview on how a CDP can leverage Cloud-native options with your choice of Cloud offerings:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*bWN21HmN8-xTAFWhD9svDg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*4J8FgJ01n95IY3NHArAs8w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*MlKBWtneum4dCamSDTES9Q.png" /></figure><p>I personally prefer leveraging the above frameworks as accelerators to get started, and quickly scale up an enterprise-wide CDP. But even though we now have potential answers to the two key questions above, we are not out of the woods yet. How do you go about actually configuring and building your CDP? Here are two key considerations in my opinion:</p><ol><li><strong>Take an E2E Customer Journey Lens: </strong>By mapping out the core customer journeys, we can understand how users interact with the business, what their goals and intentions are, and how successful they are in achieving their goals. Once we understand the journey, we can identify where the customer profile and data resides within our systems, then consolidate that information to inform our initial CDP requirements.</li><li><strong>Think Product, Not Project:</strong> The world moves too fast for 9- to 18-month, big-bang project launches. By the time the big launch is ready, the initiative could have lost its effectiveness or worse, be completely irrelevant. Take a product approach that allows for a thin end-to-end slice of customer use cases and product features to get to market quickly. Learn from customer feedback and expand from there.</li></ol><p>Written by:</p><p><strong>Deepak Arora</strong>,<strong> </strong>VP Engineering, Publicis Sapient and <strong>Jakes Lamprecht</strong>, Director of Technology, Publicis Sapient</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b3c111e4378" width="1" height="1" alt=""><hr><p><a href="https://medium.com/engineered-publicis-sapient/demystifying-the-how-of-customer-data-platforms-1b3c111e4378">Demystifying the ‘How’ of Customer Data Platforms</a> was originally published in <a href="https://medium.com/engineered-publicis-sapient">Engineered @ Publicis Sapient</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>