<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ashish Narasimham on Medium]]></title>
        <description><![CDATA[Stories by Ashish Narasimham on Medium]]></description>
        <link>https://medium.com/@ashish.narasimham?source=rss-619543df86d1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*W9Bjo8t1y0JMqmb_.</url>
            <title>Stories by Ashish Narasimham on Medium</title>
            <link>https://medium.com/@ashish.narasimham?source=rss-619543df86d1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 13:10:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ashish.narasimham/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Financial Analysis Across a 10-K Document Corpus: Language Fine Tuning with TPU and MaxText]]></title>
            <link>https://medium.com/@ashish.narasimham/financial-analysis-across-a-10-k-document-corpus-language-fine-tuning-with-tpu-and-maxtext-6d57a3d0eda2?source=rss-619543df86d1------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d57a3d0eda2</guid>
            <category><![CDATA[aiml]]></category>
            <category><![CDATA[inference]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[tpu]]></category>
            <dc:creator><![CDATA[Ashish Narasimham]]></dc:creator>
            <pubDate>Thu, 19 Mar 2026 14:57:17 GMT</pubDate>
            <atom:updated>2026-03-19T14:57:17.780Z</atom:updated>
            <content:encoded><![CDATA[<h3>Written by Ashish Narasimham and Vijay John-Britto, Members of the AI Infrastructure Technical Practice Community at Google Cloud</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uUAM_2HIL9MCPTS7PNxing.png" /></figure><h3>Intro</h3><p>Financial analysis is an area that is rich with information; there are so many small but important signals hidden inside of paragraphs and paragraphs of qualitative text — risk factors, company description and goals, regionally focused analyses, and others. Due to this same richness however, it takes analysts hours to read through the dense information contained within these industry documents such as 10-Ks. Further, the human mind gets tired; by the 25th 10-K document we may miss something that we would have caught were it the 3rd document in the list.</p><p>This use case is just about the ideal job for an LLM — crunching through large amounts of text and providing a task-specific interpretation with machine-level consistency. Pre-trained models are armed with vast knowledge, but don’t yet understand the nuances of the answers we’d like to receive as a financial analyst — we want brief, dense insights that ultimately help us generate investment return (alpha). We decided to run full fine-tuning on a small model, Qwen3–8B, to realize the value of task-specific training through an approachable example.</p><h3>The Stack: JAX, MaxText, and TPUs</h3><p>We’ll use a TPU-focused stack to accomplish this task. TPU chips are Google-built hardware specifically designed for matrix multiplications, the exact math that LLMs and neural networks more broadly use to both make predictions and learn to better predict. TPUs provide great price-to-performance for model pre- and post-training tasks, pairing with a compiled functional programming model on the software side to squeeze all the performance out of training jobs. JAX is the write-once, deploy-any Python framework used indirectly through MaxText, a library of models written in JAX and optimized to run in the TPU environment.</p><p>Here’s a simple diagram to illustrate the stack and how the components fit together:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7ypmPPoonTvB8Cq0SY5kZg.png" /></figure><h3>The Workflow</h3><p>Following is an overview of the high-level workflow. First get the10-K documents and break that into text chunks, convert that into FAQs using Gemini and stored in a cloud storage bucket. Subsequently, the Gwen3–8B model is fine-tuned with FAQs on TPU v6e using JAX and MaxText, with the resulting weights saved back to the cloud storage bucket. In the final stage, inference is performed with vLLM on TPU with the fine-tuned weights and In the final stage, we perform inference on TPUs using vLLM and the fine-tuned weights. These are then assessed against financial prompts to determine their accuracy improvements over the base model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/953/1*KsCzy_HaYRULKSYipo8OTQ.png" /></figure><h3>Synthetic Data Generation with Gemini</h3><p>We put together a ~110K record dataset based on 10-K documents from all of the companies in the S&amp;P 500. We broke up the 10-K documents into text chunks, sent those chunks to Gemini to create question &amp; answer pairs, and then saved those Q&amp;A pairs into a JSONL file. This process took a lot of Gemini calls; Flash 3.0 handled them with ease and we had a synthetic dataset ready to use quickly. The Qwen3–8B model uses a thinking block, so we incorporated this into our fine-tuning dataset as well.</p><h3>The Preprocessing Step: Data Engineering</h3><p>This resulting JSONL file created a problem — MaxText expects input data to be in either TFRecord or ArrayRecord format. ArrayRecord format is ideal, supporting random access of large data blocks. This happens to be the exact workflow that retrieving randomized training samples takes.</p><p>With help from Gemini we built a script that converts from JSONL to ArrayRecord format, splitting the data into a 50-part fileset. A many-part dataset is ideal for the parallel ingestion that MaxText will perform for fine-tuning; the 50 parts allow JAX to distribute the file loading across the chips so that there is no bottleneck in the loading process.</p><p>We then had to convert the ArrayRecord data into model-specific tokenized strings that will be fed into the TPU chips for training specific models. Llama 3.1 for example has a different start-of-turn token than Qwen, and using one model’s token in the other one’s training loop is a way to ensure you do not increase task accuracy! We used the model’s Hugging Face tokenizer to perform this conversion smoothly for us.</p><p>These tokenized strings in addition have to take a specific key-value pair form, where the key has to be ‘text’. These are the types of nuances we worked through to fine-tune; all code discussed is available on GitHub so that you can take a running start at fine-tuning yourself.</p><h3>Running the Fine-Tuning Job</h3><p>Following the data pre-processing stage, we transfer the data to a GCS bucket. The subsequent steps require TPU VM and a GKE cluster.</p><ol><li>GCS Bucket to store the prepared datasets.</li><li>TPU VM (v6e) utilized for</li></ol><ul><li>Fine tuning with Jax and Maxtext</li><li>Converting checkpoints and</li><li>vLLM inference</li></ul><p>The following commands detail the high-level steps for setting up and running the fine-tuning job:</p><p>1. Set up GCS Bucket, for detailed steps refer to our github repo.</p><p>2. Configure TPU VM Workstation for MaxText</p><p>Provision a TPU VM to handle tasks that cannot run in a cloud shell, specifically checkpoint conversion, running vLLM inference, and providing the xpk environment to manage the GKE TPU Cluster.</p><ul><li><strong>Create the TPU VM and ssh into the TPU VM.</strong></li></ul><pre>export TPU_VM_NAME=finan-ft-v6e-vm<br>gcloud compute tpus tpu-vm create ${TPU_VM_NAME} \<br> - project=${PROJECT_ID} \<br> - zone=${ZONE} \<br> - accelerator-type=${ACCELERATOR_TYPE} \<br> - version=${RUNTIME_VERSION}<br>gcloud compute tpus tpu-vm ssh ${TPU_VM_NAME} \<br> - zone=us-central1-a \<br> - project=&lt;project_id&gt;</pre><ul><li><strong>Set up GCS Bucket</strong></li></ul><p>Create a Google Cloud Storage (GCS) bucket to hold the prepared dataset, logs, and checkpoints for training and copy the datasets.</p><pre>export BUCKET_NAME=maxtext-xpk-tptft-finance<br>gcloud storage buckets create gs://$BUCKET_NAME - location=${REGION}</pre><ul><li><strong>Install the required packages</strong></li></ul><pre>Set noninteractive mode for Debian package installer<br>sudo sed -i.bak -e &quot;s/^#*\s*\$nrconf{restart}\s*=\s*[&#39;\&quot;][il][&#39;\&quot;];/\$nrconf{restart} = &#39;a&#39;;/&quot; /etc/needrestart/needrestart.conf<br>export NEEDRESTART_SUSPEND=1<br>export NEEDRESTART_MODE=1<br>export DEBIAN_FRONTEND=noninteractive</pre><ul><li><strong>Install python3.12 as MaxText requires python&gt;=3.12</strong></li></ul><pre>sudo apt update &amp;&amp; sudo apt install software-properties-common -y &amp;&amp; sudo add-apt-repository ppa:deadsnakes/ppa &amp;&amp; sudo apt update &amp;&amp; sudo apt install python3.12 python3.12-venv -y</pre><ul><li><strong>Create a venv environment and install the necessary packages</strong></li></ul><pre># create virtualenv &amp; activate<br>python3.12 -m venv ~/venv/maxtext &amp;&amp; source ~/venv/maxtext/bin/activate</pre><p><strong>Clone MaxText</strong></p><pre>git clone https://github.com/AI-Hypercomputer/maxtext.git &amp;&amp; cd maxtext/</pre><p>5. Prepare Checkpoint</p><p>Convert the pre-trained model checkpoint (e.g., Qwen3–8b) into the MaxText-compatible format.</p><ul><li><strong>Convert the checkpoint:</strong></li></ul><pre>python -m MaxText.utils.ckpt_conversion.to_maxtext MaxText/configs/base.yml \<br>model_name=$MODEL_NAME \<br>hf_access_token=$HF_TOKEN</pre><p>6. Run Fine-Tuning Job</p><p>Set up the env variables in the TPU VM for the fine tuning job on the curated FAQs of the 10k docs from the cloud storage bucket.</p><ul><li><strong>Setup the environment variables</strong></li></ul><pre>export HF_TOKEN=&lt;HF_TOKEN&gt;<br>export PROJECT_ID=$(gcloud config get-value project)<br>export REGION=us-east5<br>export ZONE=us-east5-a<br>export ACCELERATOR_TYPE=v6e-8<br>export RUNTIME_VERSION=v2-alpha-tpuv6e<br>export MODEL_NAME=qwen3–8b<br>export TOKENIZER_CONFIG=&quot;tokenizer_path=Qwen/Qwen3–8B&quot;<br>export BASE_OUTPUT_DIR=gs://$BUCKET_NAME/output-qwen<br>export DATASET_TYPE=grain<br>export TRAIN_FILES=~/data/qwen_array_record/qwen-*.array_record<br>LOAD_PARAMETERS_PATH=gs://$BUCKET_NAME/pretrained-ckpt-orbax-qwen/0/items<br>export STEPS=700<br>export CHECKPOINT_PERIOD=350</pre><p><strong>Launch the fine-tuning workload:</strong></p><pre>python3 -m MaxText.train MaxText/configs/base.yml model_name=${MODEL_NAME} base_output_directory=${BASE_OUTPUT_DIR} load_parameters_path=${LOAD_PARAMETERS_PATH} learning_rate=2e-5 attention=flash num_vocab_tiling=8 dataset_type=${DATASET_TYPE} grain_file_type=arrayrecord hf_access_token=${HF_TOKEN} grain_train_files=${TRAIN_FILES} ${TOKENIZER_CONFIG} per_device_batch_size=4 remat_policy=full max_target_length=4096 ici_fsdp_parallelism=1 ici_tensor_parallelism=-1 run_name=qwen_finetune steps=${STEPS} checkpoint_period=${CHECKPOINT_PERIOD} async_checkpointing=true scan_layers=true</pre><p>We used a per device batch size of 4 here; what’s interesting is that Grain bin-packs the input examples together up to the 4096 sequence length that we set, allowing the TPU to improve its efficiency quite a bit. We expected to take ~3500 steps to train 1 epoch without bin-packing, but only had to run 700 steps in the end.</p><p>7. Convert fine-tuned Orbax checkpoint into Safetensors checkpoint</p><p>The results of the fine-tuning steps are stored in the GCS bucket at <strong>output/my_finetune_runner/checkpoints/</strong>, which has the full sequence of Orbax checkpoints captured at 10-step intervals. To facilitate the subsequent inference stage, we isolate the terminal checkpoint — specifically the iteration produced at step 30 — for conversion. The resulting Safetensors checkpoint is then exported to the local directory <strong>/dev/shm/my-finetuned-model-safetensor </strong>for inference with vLLM inference.</p><pre>python3 -m MaxText.utils.ckpt_conversion.to_huggingface MaxText/configs/base.yml \<br>model_name=$MODEL_NAME \<br>hf_access_token=$HF_TOKEN \<br>load_parameters_path=gs://$BUCKET_NAME/output/my_finetune_runner/checkpoints/29/items \<br>base_output_directory=/dev/shm/my-finetuned-model-safetensor \ scan_layers=true</pre><h3>High-performance Inference using vLLM</h3><p>Inference refers to the process of deploying a trained machine learning model to a production environment, where it can be used for inference. Inference is supported on TPU v5e, v6e and newer versions. High-performance inference is driven by <a href="https://docs.vllm.ai/en/latest/">vLLM</a>, an open-source library designed for rapid model serving. Cloud TPU integrates with this stack through the tpu-inference plugin, providing native support for models developed in both JAX and PyTorch.</p><p>In this step, we would be installing the vLLM on TPU v6e VM.</p><pre>pip install vllm-tpu<br>Then start the vLLM with the fine tuned weights from the previous step.<br>vllm serve /dev/shm/my-finetuned-model-safetensor - max-model-len=4096 &gt; ~/vllm.out 2&gt;&amp;1 &amp;<br>tail -f ~/vllm.out<br>…..<br>INFO: Started server process [1]<br>INFO: Waiting for application startup.<br>INFO: Application startup complete.</pre><p>Once the vLLM starts successfully, validate with a simple prompt to validate the response as below.</p><pre>USER_PROMPT=&quot;What is the tallest building in Seoul?&quot;<br>curl -X POST http://localhost:8000/v1/completions \<br>-H &quot;Content-Type: application/json&quot; \<br>-d @- &lt;&lt;EOF<br>{<br>&quot;prompt&quot;: &quot;&lt;|im_start|&gt;user\n${USER_PROMPT}&lt;|im_end|&gt;\n&quot;,<br>&quot;temperature&quot;: 0,<br>&quot;max_tokens&quot;: 128,<br>&quot;top_p&quot;: 1.0<br>}<br>EOF</pre><h3>Results: Base vs Fine-tuned</h3><p>We didn’t necessarily know what to expect, but the results were surprisingly stark. They showed that fine-tuning does indeed nudge the model to interpret, rather than summarize, much as a financial analyst would. It produced text that reduced verbosity while packing in insight.</p><p>One specific prompt we tried:</p><pre>Analyze this segment from PAYX_10k.md:<br>Data Security and Privacy Leaks: We collect, use, and retain increasingly large amounts of personal information about our clients, employees of our clients, our employees, and other third parties, including: bank account, credit card, and social security numbers, tax return information, health care information, retirement account information, payroll information, system and network passwords, and other sensitive personal and business information. At the same time, the continued occurrence of high-profile cyber and ransomware attacks and data breaches provides evidence of an external environment increasingly hostile to information security. We may be particularly targeted for cyberattack because of the amount and type of personal and business information that we collect, use, and retain, as well as during and after periods in which we acquire other companies. Vulnerabilities, threats, and more sophisticated and targeted computer crimes pose a risk to the security of our systems and networks, and the confidentiality, availability, and integrity of our data. Furthermore, if any of our solutions contain a software vulnerability, the vulnerability may be exploited to obtain access to our data or our clients&#39; data.</pre><p>Here’s the base model’s response:</p><pre>&lt;think&gt;<br>Okay, let&#39;s take a look at this segment from the PAYX_10k.md. The user wants an analysis of this part. First, I need to understand what the text is about. It&#39;s talking about data security and privacy leaks. The company collects a lot of personal and sensitive information from various parties. They mention things like bank account details, credit cards, social security numbers, tax returns, healthcare info, retirement accounts, payroll data, passwords, and other sensitive info.<br>Then they talk about the external environment being hostile due to high-profile cyberattacks and ransomware. They note that the company might be a target because of the volume and type of data they handle, especially during acquisitions. They also mention vulnerabilities, threats, and sophisticated cybercrimes as risks. Plus, software vulnerabilities in their solutions could lead to data breaches.<br>So, the user probably wants an analysis of the company&#39;s data security risks and how they&#39;re addressing them. Maybe they&#39;re looking for a summary of the key points, the implications, or how this affects the company&#39;s operations and compliance. They might also be interested in the potential consequences of these risks, like financial loss, reputational damage, legal issues, or regulatory penalties.<br>I should break down the main points: the types of data collected, the external threats, the reasons for being a target, the risks from vulnerabilities, and the potential consequences. Also, maybe the company&#39;s acknowledgment of these risks and any measures they might be taking, though the text doesn&#39;t mention specific mitigation strategies. The user might be looking for an evaluation of the company&#39;s risk management in this area.<br>I need to make sure the analysis covers all these aspects. Maybe start by summarizing the key elements, then discuss the implications, and perhaps touch on the regulatory landscape (like GDPR, CCPA) even though it&#39;s not mentioned here. Also, note that the company is aware of the risks and is likely taking steps to mitigate them, even if not detailed in the text. The user might be a stakeholder, investor, or someone assessing the company&#39;s security posture.<br>&lt;/think&gt;<br>The segment from PAYX_10k.md highlights the company&#39;s acknowledgment of significant **data security and privacy risks** in its operations. Here&#39;s a structured analysis of the key points and implications:<br>- <br>### **1. Scope of Data Collected**<br>- **Sensitive Information Types**: The company handles a wide range of **highly sensitive data**, including:<br>- Financial data (bank accounts, credit cards, tax returns).<br>- Personal identifiers (social security numbers, health care information).<br>- Business-related data (retirement accounts, payroll, system passwords).<br>- Third-party data (clients, employees, and other stakeholders).<br>- **Implication**: This extensive data collection increases the **potential impact of a breach**, as attackers could exploit this information for identity theft, financial fraud, or other malicious activities.<br>- <br>### **2. External Threat Landscape**<br>- **Cybersecurity Risks**: The text emphasizes the **increasing frequency and sophistication of cyberattacks**, ransomware, and data breaches. These threats are described as part of a \&quot;hostile external environment.\&quot;<br>- **Targeted Vulnerabilities**: The company is **particularly vulnerable** due to:<br>- The **volume and sensitivity** of data it processes.<br>- **Acquisition activities**, which may introduce new vulnerabilities or expand attack surfaces.<br>- **Software vulnerabilities** in its solutions, which could be exploited to access internal or client data.<br> - -<br>### **3. Risks to Systems and Data**<br>- **Threats to Security Pillars**: The segment underscores risks to the **confidentiality, integrity, and availability** of data:<br>- **Confidentiality**: Unauthorized access to sensitive information.<br>- **Integrity**: Data tampering or corruption.<br>- **Availability**: System outages or ransomware disrupting operations.<br>- **Operational and Reputational Impact**: A breach could lead to **financial losses**, **legal liabilities**, **regulatory penalties**, and **loss of customer trust**.<br> - -<br>### **4. Regulatory and Compliance Considerations**<br>- **Implicit Risks**: While not explicitly stated, the mention of sensitive data (e.g., health care, social security numbers) suggests potential **regulatory scrutiny** under laws like:<br>- **GDPR** (for EU data subjects).<br>- **CCPA** (for California residents).<br>- **HIPAA** (for health care data).<br>- **Compliance Challenges**: The company must ensure **data protection measures** align with evolving regulations, which could require significant investment in security infrastructure and incident response protocols.<br> - -<br>### **5. Mitigation and Risk Management**<br>- **Acknowledgment of Risks**: The company explicitly recognizes these risks, indicating a **proactive stance** in identifying vulnerabilities.<br>- **Potential Mitigation Strategies** (implied but not detailed):<br>- **Enhanced cybersecurity measures** (e.g., encryption, multi-factor authentication).<br>- **Regular vulnerability assessments** and patch management.</pre><p>Here’s the fine-tuned model’s response:</p><pre>&lt;think&gt;<br>To analyze this 10-K segment, I first identified the core business function described: the collection and retention of highly sensitive, multi-jurisdictional data (SSNs, tax, health, and payroll). I then evaluated the external threat landscape, noting the explicit mention of ransomware and the company&#39;s specific vulnerability as a high-value target due to its data volume. I connected the mention of \&quot;acquisitions\&quot; to the operational risk of integrating legacy systems, which often introduces technical debt and security gaps. Finally, I synthesized these factors to determine the material impact on the company&#39;s risk profile, focusing on the potential for regulatory penalties, reputational damage, and the financial burden of remediation following a breach.<br>&lt;/think&gt;<br>**Investment Memo: Data Security &amp; Cybersecurity Risk Profile**<br>**Key Financial Drivers**<br>* **Data Sensitivity:** Paychex&#39;s business model relies on the collection and retention of high-value, sensitive data, including SSNs, tax returns, health care information, and retirement account details. The integrity of this data is a primary driver of client trust and long-term retention.<br>* **M&amp;A Integration:** The company&#39;s growth strategy through acquisitions introduces significant financial risk, as integrating legacy systems and data sets often reveals latent vulnerabilities that require immediate remediation and capital expenditure.<br>**Risks**<br>* **Targeted Cyberattacks:** Due to the volume and type of sensitive information held, Paychex is a high-value target for ransomware and sophisticated cybercriminals.<br>* **Systemic Vulnerabilities:** The company faces risks from both internal software vulnerabilities and external threats to its network infrastructure, which could compromise the confidentiality, availability, and integrity of client data.<br>* **Operational Disruption:** A successful breach could lead to significant reputational damage, regulatory fines, and the potential loss of clients, particularly during periods of system instability.<br>**Material Changes**<br>* **Evolving Threat Landscape:** Management notes an increasingly hostile external environment characterized by high-profile, sophisticated ransomware attacks.<br>* **Heightened Acquisition Risk:** There is a specific emphasis on the heightened risk profile associated with post-acquisition periods, where the integration of disparate IT environments may temporarily weaken the company&#39;s security posture.&quot;</pre><p>Even from the shorter thinking block, you can see the brevity in the fine-tuned response while interpreting rather than summarizing the input data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*sN-zTyMJxCbVJk-qzG7DmA.png" /></figure><p>Our fine-tuning efforts yielded a model precisely calibrated for the nuances of financial analysis, demonstrating a marked superiority over the base architecture in both contextual relevance and structural fidelity. By distilling qualitative risk disclosures into dense, <strong>actionable investment insights</strong>, the model transcends simple summarization to provide the <strong>domain-specific lens</strong> necessary for generating alpha. While the base model maintains a degree of faithfulness, it ultimately fails to capture the professional rigor and financial materiality inherent in the fine-tuned iteration.</p><h3>Conclusion</h3><p>We ran fine-tuning on a small, open-source model using a rather large dataset, all on efficient TPU hardware. We’ve achieved an adjusted output style, tailored towards brevity yet impactful information; this is the brief that a financial analyst would be interested in. We could have equally gone the few-shot prompting route; we chose fine-tuning here to “bake in the examples”. This will reduce latency; the model doesn’t have to process many hundreds more tokens of few-shot prompting. As a result we can achieve high throughput when scanning hundreds of 10-K documents to, for example, summarize the broader view of the corpus that the analyst may be reviewing.</p><p>The overview here along with our GitHub repository should give you a great starting point if you choose to fine-tune for a specific task, such as qualitative financial analysis.</p><p>References:</p><ul><li><a href="https://github.com/anarasimham2/tpu-financial-fine-tuning">GitHub</a></li><li><a href="https://docs.cloud.google.com/tpu/docs/tpu-inference">Inference on TPUs</a></li><li><a href="https://developers.googleblog.com/building-production-ai-on-google-cloud-tpus-with-jax/">Building production AI on Google Cloud TPUs with JAX</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d57a3d0eda2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Neural Network from Scratch: What I Learned]]></title>
            <link>https://medium.com/@ashish.narasimham/building-a-neural-network-from-scratch-what-i-learned-44ded382d0e0?source=rss-619543df86d1------2</link>
            <guid isPermaLink="false">https://medium.com/p/44ded382d0e0</guid>
            <category><![CDATA[generative]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Ashish Narasimham]]></dc:creator>
            <pubDate>Mon, 30 Sep 2024 17:22:38 GMT</pubDate>
            <atom:updated>2024-09-30T17:22:38.737Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aUIN-x5nbjTXQ1B-_Txz_g.jpeg" /><figcaption>Created by Twistedpoly as part of the Visualising AI project launched by Google DeepMind.</figcaption></figure><h3>Intro</h3><p>Have you ever scanned a PDF and then tried to search it?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*StnEiJ_Zc9dpjzY7" /><figcaption>Example of a page scanned from physical to digital</figcaption></figure><p>Typically a machine cannot read the content of that document because it now “looks like an image”, rather than text. Reading the document as you intended was a hard problem for machines to solve until neural networks entered the scene. We are now able to train complex models that can understand these documents and essentially reinterpret the image back into machine-readable text. What’s more — these same concepts can be used for a broad variety of tasks including anomaly detection, natural language generation, and time series forecasting. The same technology has such diversity in application that there are use cases for it no matter where you look. Let’s go into how neural networks actually work so that you the reader can come away with a deeper understanding of what powers the massive models that we indirectly interact with more and more every day.</p><h3>Rooted in Mathematics and Statistics</h3><p>Neural networks — are they Computer Science? Tangentially yes; at the surface level we need to leverage the massive compute infrastructure we as a society have already built (CPUs, GPUs, and TPUs), and we do that through code. But more to the point, neural networks are a form of applied statistics. This fact certainly surprised me as I learned more about them. A neural network is a series of linear and non-linear transformations applied to a given data input (or batch of inputs). As data flows through the network, it gets multiplied by the weights of the network. Let’s take an example where we are classifying images into one of multiple classes, an example we’ll come back to throughout this post:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*B2zLZUjyLUAJtolF" /><figcaption>General structure of a Convolutional Neural Network, the type of network that can learn from images</figcaption></figure><p>This is a typical example of how a network flows. They work based on the statistical principle of Maximum Likelihood Estimation (MLE) — using all of the examples in the training dataset, our best way to predict the label of a novel example is to “take the average of what we’ve seen”. In practice it is more involved than taking averages since there are complex non-linearities at work, but this is a good mental model to work with.</p><p>Back-propagation is where the learning happens — this process seemed like magic to me before understanding the math. Back-propagation is computational math at its finest — the process is comprised of repeatedly and recursively taking partial derivatives of a massively multivariate equation by leveraging the chain rule. The chain rule allows us to decompose functions of functions one segment at a time, allowing for a compositional approach that modern algorithms like PyTorch’s computation graph use for optimized processing. Going back to our example of classifying images, let’s look at the math.</p><p>Mathematically, a network can be represented as one <em>loong</em> equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/0*eURDrfevDObjwnAt" /></figure><p>To compute the change in weight, we need to take the partial derivative of the loss with respect to that weight:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/0*g4IVRI-mt_lS82EM" /></figure><p>We can continue this process step by step backwards through the network to compute the change in loss with respect to each parameter in the network. Computing the actual derivatives is beyond the scope of this blog post, so I will leave that as an exercise for the reader.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*m_9wvbds-j4dP2mM" /><figcaption>Sample network architecture and flow, forwards and backwards</figcaption></figure><p>The process of back-propagation generates a set of gradients — one for each tunable parameter, or weight, in the network. This gradient signifies the direction of travel along each dimension that maximizes the slope of the function. We can now understand how much the loss will change and in what direction when we change each parameter in the network — there can sometimes be billions for larger models, so this is great knowledge to have. This information is essential for us to nudge the model in the direction of the objective, typically exactly opposite to the way the gradient points. Next time through the network, we’ll be just a little closer to the outcome that we desire.</p><h3>Your Choice — Train Models… or Skip Training</h3><p>Training models doesn’t have to be a frustrating process in trial-and-error. Although iterative improvement is an unavoidable (and often enjoyable) part of the model deployment process, you can leverage managed services and pre-infused domain expertise to make things easier on your Data Science team. Google Cloud has distilled the best of our AI knowledge and learnings into our Data Science experimentation platform, Vertex AI. We’ve made complex model architectures available (Tabular Wide &amp; Deep, Tabular for TabNet, Tabular for AutoML: <a href="https://cloud.google.com/vertex-ai/docs/tabular-data/tabular-workflows/overview">more info on these here</a>), put visualizations of <a href="https://cloud.google.com/vertex-ai/docs/training/overview">training</a> and <a href="https://cloud.google.com/vertex-ai/docs/general/deployment">deployment</a> processes together with a point of view on the <a href="https://cloud.google.com/vertex-ai/docs/pipelines/introduction">MLOps pipeline</a>, and pre-trained a <a href="https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models">solid set of foundational models</a> including the multimodal Gemini 1.5 Pro that should serve a broad variety of use cases well.</p><p>Taking it a step further, Google’s AutoML technology does what the name implies — it automates your ML model building. <a href="https://cloud.google.com/vertex-ai/docs/beginner/beginners-guide">AutoML</a> can perform classification, forecasting, and regression on tabular data; it can additionally operate on images, text, or video; and also serves the edge with <a href="https://cloud.google.com/vertex-ai/docs/training/automl-edge-console">AutoML Edge</a>. Give it a training budget and a little bit of labeled data, and it will find the best model architecture and hyperparameters for you.</p><p>Use a method above to build a model from scratch to learn the ins and outs, then scale your knowledge with Google Cloud technologies and let us worry about the details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*mJqVZsp4yxgsfHLj" /><figcaption>Leverage Google Cloud’s Vertex AI Platform for models, training, MLOps, serving, and governance</figcaption></figure><p>Or — don’t even worry about training a model. Use our managed offerings to leverage prebuilt assets and get going on the business innovation right away:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*tnEd1d-XvhCKZ5gx" /><figcaption>Go up the Vertex AI stack and don’t worry about training a model — use existing ones and accelerate your work</figcaption></figure><p>Conclusion</p><p>Neural networks underlie LLMs and many other AI applications today — they are widely applicable and create powerful internal representations of the data without the operator having to provide much beyond the raw data. Research will continue to rapidly advance everything from self-driving cars to reconstructing barely-visible images of distant galaxies; this stuff isn’t going away anytime soon! Given this, it’s in your best interest (you, the reader) to skill up in this area and understand how they work so that you can come to your next AI conversation fully informed of the capabilities.</p><p>My parting call to action to you: I challenge you to train your own neural network, whether from scratch or using existing libraries, to learn more about how they work. Use <a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html">this PyTorch tutorial</a> to get started.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=44ded382d0e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CPGs and First-Party Data — A Cross-Organization Priority]]></title>
            <link>https://medium.com/@ashish.narasimham/cpgs-and-first-party-data-a-cross-organization-priority-c79dbf37a44e?source=rss-619543df86d1------2</link>
            <guid isPermaLink="false">https://medium.com/p/c79dbf37a44e</guid>
            <category><![CDATA[cpg]]></category>
            <category><![CDATA[thought-leadership]]></category>
            <category><![CDATA[first-party-data]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Ashish Narasimham]]></dc:creator>
            <pubDate>Tue, 20 Sep 2022 02:23:41 GMT</pubDate>
            <atom:updated>2022-09-20T03:13:14.076Z</atom:updated>
            <content:encoded><![CDATA[<h3>CPGs and First-Party Data — A Cross-Organization Priority</h3><p><a href="https://www.linkedin.com/in/anarasimham/">Ashish Narasimham</a>, Customer Engineering Architect, CPG, Google Cloud</p><p><a href="https://www.linkedin.com/in/georgenyako/">George Nyako</a>, Industry Solutions Architect, CPG, Google Cloud</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1pPWKntJnE8ZxbduiaDkEw.jpeg" /></figure><h3>Introduction</h3><p>Companies in the Consumer Packaged Goods (CPG) industry are challenged today by external factors like inflation and supply chain issues. COVID temporarily shifted trends in a big way, but those effects are fading — manufacturers are seeing a return to normal in consumer purchase patterns. Data however is sticking around; CPGs have started to recognize that it’s a company asset that’s grown to be one of the most important sources of business insight in recent decades. Stretching the value of data is critical for CPGs given they typically don’t have a direct relationship with the consumer.</p><p>As Architects supporting Google Cloud’s CPG industry, we often get asked how can we leverage this valuable information?</p><p>We should first look at a CPG’s goals.</p><p>CPGs want to:</p><ul><li>Build a closer relationship with their consumers</li><li>Innovate with new product development</li><li>Maintain profitability in a rapidly changing landscape</li></ul><p>Let’s look at how business leaders in CPG can exceed their objectives and accelerate their business with first party data (1PD).</p><h3>What’s the buzz about?</h3><p>Let’s go back to the goals outlined above — the first two, innovation and relationship building, center around the consumer. Consumers today are shaped by brand innovation, and brands in turn are galvanized to build on their progress and create an ever-easier experience. It’s a tight feedback loop, and it’s all about meeting consumers where they are. That’s where first party data comes in.</p><p><em>Email, place of residence, personal preferences, phone numbers, and other attributes </em>make up the world of first party data. These are simple facts with powerful purposes: organizations can tailor their content, messaging, and overall interactions in big ways based on these pieces of data.</p><p>Imagine there’s a big July 4th event coming up in New York City. As a CPG in Food &amp; Beverage, you might offer a version of your product that is colored red, white, and blue. Leveraging first party data, you’d be able to reach out to those people that you know would enjoy this spin on the product based on their dietary preferences, purchase history, or any number of other attributes. As a result you grow the relationship for those that you are engaging with. You could take this approach with multiple groups of consumers, ultimately creating smaller groups of interaction with high relevance rather than a generic message for a single, large group.</p><h3>Why Do CPGs struggle with first party data?</h3><p>Given that CPGs in large part sell to other businesses, the B2B interaction doesn’t make for the easiest time in engaging with their end consumers. Traditionally, CPGs have resigned themselves to this fact and done what they could without a deep understanding of their consumers. This might look like a mass marketing campaign hoping to effectively engage with a small subset of that audience. However, consumer demands are changing: they’re responding to outreach when they feel personally invested; they’re engaging with brands that show they understand them; and they’re buying products that align with their values.</p><blockquote><em>Consumer demands are changing: they’re responding to outreach when they feel personally invested; they’re engaging with brands that show they understand them; and they’re buying products that align with their values.</em></blockquote><p>CPGs are realizing this and starting the journey. It’s a big project, but a worthwhile one that will drive key business outcomes. CPGs are starting to build their relationship with the consumer where one didn’t exist through D2C channels, online content creation, and ad-hoc campaign/product personalization. The goal is to drive a <em>value exchange</em> with the consumer that will help power their strategy for the next decade.</p><h3>1PD and the Core Marketing Cycle</h3><p>As we discuss powering a long-term strategy, the heart of this will continue to be the core marketing cycle:</p><ul><li>Run a marketing campaign, getting feedback from consumers in the form of engagement, conversions, and page views</li><li>Extract that data to your analytics layer, structuring it for deep analysis and solving for relevancy and better consumer understanding</li><li>As your analytics matures, create custom audience segments based on the data analysis and infuse predictive capability</li><li>Bring these audiences back to your campaigns, activating against them and closing the personalization loop with more relevant messaging</li><li>Track progress through key metrics</li></ul><p>Every organization is engaging daily in this feedback loop and gaining valuable knowledge about their audience and how best to interact with them.</p><p>And as you embark on your analytics maturity journey, you can take a phased approach by tackling your high value use cases first; get quick, impactful results; and then rinse and repeat to build out your full platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ttB5Zew9Qgb8meklou5v3g.png" /></figure><p>A sample of the use cases you can solve effectively with Google Cloud’s Consumer Data Intelligence Platform (CDIP) is shown above. CDIP is a packaged solution offering, which balances quick time to value with extensibility for advanced analytics solutions to solve your media/marketing, digital commerce and innovation use cases.</p><p>Now that we’ve covered the fundamentals, let’s take that to the next level and talk about transformational impact.</p><h3>How can CPGs create transformational impact?</h3><p>First, what does transformational impact look like? It’s different for every CPG; it could be new concepts for product development, creatives customized to events or demographics, or timely product packaging that taps into a trend. The possibilities are broad, but if you are building the fundamentals correctly then it’s hard <em>not</em> to get value out of first party data.</p><p>As you scope this massive effort, the biggest challenge is often aligning teams on this new way of thinking. Tried and true processes are, for good reason, how the human mind works and succeeds. We find the nearest hill and get to the top. But what if there’s a mountain to climb just out of sight that will get us even higher? We won’t know until we go and look. In this case experimentation is how we peek around corners, so that we can start to build the threads of collaboration across different parts of the company. Create small experiments, fail fast, and learn from them.</p><p>This human collaboration will lead to a similar effect on the data side, bringing data silos together and allowing the marketing analytics cycle to take on a life of its own. Choose a few (2–3) priority data sources and start to bring them together, allowing your analysts to enrich existing data and think about new ways of looking at the problem.</p><p>Once these foundational elements are in place, organizations can take on larger and larger efforts. They will move up on the maturity curve from fledgling to leader, creating the previously described transformative impacts across their organization along the way.</p><h3>Where Can Google Cloud Help?</h3><p>The marketing analytics world contains a variety of challenges to overcome before realizing value; analytics doesn’t have to be one of them. Google Cloud is a leader in the data &amp; analytics space, working with many of the largest CPG &amp; Retail organizations globally and helping them focus on their insights rather than their infrastructure. Building your personalization engine on Google Cloud means having technology that automates away the work that you <em>don’t</em> <em>want</em> to be doing and drives focus to the innovation that you <em>need</em> to be doing.</p><h4>Data analytics</h4><ul><li>BigQuery is a fully managed, serverless data warehouse that can be deployed across your business units, eliminating data silos and broadening the scope of activities that your analysts can engage in</li><li>As you mature in your journey you’ll leverage BQML, or BigQuery Machine Learning</li><li>This enables anyone with the SQL skillset to build powerful predictive models that forecast current and future trends</li><li>In the marketing analytics use case, you’ll use this to build predictive audiences</li><li>As you customize and advance your knowledge, you’ll branch out into bespoke use cases. Technologies such as Vertex AI, AutoML, Pub/Sub, and DataFlow can help you build real-time, predictive pipelines that help you engage with your consumers at the right time and with the right message.</li></ul><h4>The ecosystem</h4><ul><li>Google’s unique datasets allow you to enrich your data and spot new trends, opening up more opportunities. You can access these datasets, bring different datasets together, and externally share data in a controlled manner using Analytics Hub</li><li>Google’s seamless integration across Cloud, Ads, and Marketing Platform allows you to not only minimize the time you spend stitching technology together, but also gain deeper insights across your marketing analytics use cases and unify your strategy</li></ul><p>Google, having built 9 products that each have over 1 billion users, knows the consumer well. That experience has been weaved into Google Cloud’s product portfolio, and a key part of that portfolio is GCP analytics. It is meant to be powerful yet flexible, allowing your organization to find the next gear in their productivity and the next gear in their understanding of the consumer.</p><h3>Conclusion</h3><p>First party data is a powerful asset to any organization, and it’s only growing in importance. As consumers invite brands into their homes and demand an experience that fits with their lives, brands that deliver on this will be the ones that rise to the top of consumers’ minds, shopping lists, and profitability.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c79dbf37a44e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>