<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[AlamedaDev - Medium]]></title>
        <description><![CDATA[We build tailored digital products that drive innovation and create sustainable value. - Medium]]></description>
        <link>https://medium.com/alameda-dev?source=rss----e4980a567584---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>AlamedaDev - Medium</title>
            <link>https://medium.com/alameda-dev?source=rss----e4980a567584---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 15:48:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/alameda-dev" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Diagnostics with Multi-Agent AI Systems]]></title>
            <link>https://medium.com/alameda-dev/diagnostics-with-multi-agent-ai-systems-dbd0f82a3465?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/dbd0f82a3465</guid>
            <category><![CDATA[multi-agent-systems]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[heath]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Wed, 19 Feb 2025 10:27:15 GMT</pubDate>
            <atom:updated>2025-02-19T10:26:43.811Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eTJ3hpk6H5IvRsNhe0veFw.jpeg" /></figure><p>A private healthcare provider sought to optimize diagnostic workflows using AI-driven solutions. Managing high patient volumes while ensuring timely detection of complex cases posed significant operational challenges that traditional systems struggled to address. By leveraging multi-agent AI, the provider aimed to enhance diagnostic accuracy, reduce physician workload, and improve interdisciplinary collaboration.</p><h3>Existing System Limitations</h3><p><strong>Diagnostic Accuracy: </strong>Traditional single-stage AI tools lack the ability to handle complex cases requiring multi-specialty collaboration, leading to limited reliability in nuanced diagnoses.</p><p><strong>Consultation Times: </strong>Physicians face extended consultation durations due to manual data review, impacting efficiency and overall patient experience.</p><p><strong>Resource Allocation: </strong>Legacy diagnostic systems lack adaptive resource distribution, making it difficult to dynamically assign specialists based on evolving patient needs.</p><h3>Technical Overview</h3><p>To address these challenges, we collaborated with the client to develop a proof of concept for a multi-agent AI system, exploring its feasibility and benefits in medical diagnostics.</p><h4>Key Components</h4><p><strong>AI Doctor Agent: </strong>Conducts initial consultations, collects symptoms, and generates preliminary diagnoses before engaging specialist agents when necessary.</p><p><strong>Specialist Agents: </strong>Simulated domain-specific specialists (e.g., cardiology, neurology) refine initial diagnoses and provide second opinions based on specialized knowledge.</p><p><strong>Dispute Resolution Mechanism: </strong>Facilitates consensus-building among AI agents when conflicting diagnoses arise, ensuring robust and reliable recommendations.</p><p><strong>Scalable Architecture: </strong>Designed to incorporate additional AI agents and adapt to emerging medical specialties as needed.</p><h4>Concept Development &amp; Scoping</h4><ul><li>Identified key challenges and defined project scope.</li><li>Established objectives, including enhancing diagnostic accuracy and improving consultation efficiency.</li></ul><h4>Data Preparation</h4><ul><li>Sourced patient records and relevant medical literature.</li><li>Generated synthetic datasets to simulate complex diagnostic cases.</li></ul><h4>Core System Features</h4><ul><li>Developed specialized agents utilizing machine learning models tailored for diagnostic roles such as symptom analysis, anomaly detection, and treatment recommendation.</li><li>Implemented NLP models to analyze patient records and doctor notes, enabling context-aware interpretations.</li><li>Optimized agent interactions using Reinforcement Learning (RL), allowing them to refine decision-making strategies through iterative feedback loops.</li><li>Built a robust API layer to ensure seamless communication between AI agents, supporting real-time data exchange and collaborative decision-making.</li><li>Integrated explainability mechanisms using SHAP (Shapley Additive Explanations) to enhance physician trust in AI-generated diagnoses.</li><li>Designed an intuitive, role-specific user interface ensuring minimal cognitive load for medical professionals, with clear dashboards, real-time case updates, and smooth navigation.</li></ul><h4>Simulation &amp; Testing</h4><ul><li>Evaluated system performance in controlled clinical environments.</li><li>Assessed diagnostic accuracy, interaction quality, and decision-making efficiency.</li><li>Refined the system iteratively based on expert feedback and real-world test results.</li></ul><h4>Evaluation &amp; Feedback</h4><ul><li>Measured diagnostic accuracy through precision-recall analysis.</li><li>Tracked consultation time reduction compared to baseline physician workflows.</li><li>Conducted qualitative assessments on physician trust in AI recommendations.</li><li>Identified areas for further optimization and potential real-world deployment.</li></ul><h3>Results</h3><ul><li>Achieved a 23% reduction in consultation times, improving patient throughput without compromising diagnostic accuracy.</li><li>Increased diagnostic agreement rates by 18% through multi-agent collaboration, reducing inconsistencies in specialist recommendations.</li><li>Enhanced physician trust in AI-generated recommendations, with 85% of doctors reporting improved workflow efficiency.</li><li>Identified 12% more complex cases requiring additional specialist intervention compared to traditional systems.</li></ul><h3>Next Steps</h3><ul><li>Implementing convolutional neural networks (CNNs) to analyze medical images, enabling AI-assisted radiology and pathology assessments.</li><li>Further refining NLP models to interpret complex medical histories and improve diagnostic recommendations.</li></ul><h3>Other Applications</h3><p>The multi-agent AI system developed for medical diagnostics can be adapted to various industries and use cases.</p><p><strong>Customer Support: </strong>Enhancing multi-department interactions for efficient query resolution.</p><p><strong>Financial Advisory: </strong>Providing comprehensive investment and financial planning advice.</p><p><strong>Education: </strong>Powering interactive AI tutors for personalized learning experiences.</p><p>AlamedaDev provides full-service end-to-end software. Experts in modern software development and AI solutions.</p><p>From #barcelona</p><p>Website: <a href="http://www.alamedadev.com/">www.alamedadev.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dbd0f82a3465" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/diagnostics-with-multi-agent-ai-systems-dbd0f82a3465">Diagnostics with Multi-Agent AI Systems</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Whisper and Speech Recognition]]></title>
            <link>https://medium.com/alameda-dev/whisper-and-speech-recognition-fdfd62101b9a?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/fdfd62101b9a</guid>
            <category><![CDATA[speech-recognition]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[speech]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Thu, 30 Jan 2025 14:39:01 GMT</pubDate>
            <atom:updated>2025-01-30T14:38:48.156Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V_sOVCyXCF3ba2eUMVCUeg.jpeg" /></figure><p>Developed by OpenAI, Whisper has been trained on a diverse range of internet-collected data, enabling it to understand context better and recognize speech nuances more effectively than many of its predecessors. OpenAI’s Whisper model represents a significant advancement in speech recognition technology, offering remarkable accuracy in transcribing audio to text across multiple languages and in various noisy environments.</p><p>The model’s versatility makes it suitable for a wide array of applications, from assisting in accessibility features to enhancing user interaction in software applications. This introduction aims to highlight the potential of Whisper in revolutionizing speech recognition and its practical applications in our digital world.</p><h3>Setting Up the Project Environment</h3><p>To use Whisper, we’ll need to rely on Python for our backend. Whisper also needs the command-line tool <a href="https://ffmpeg.org/">ffmpeg</a>, which enables our application to record, convert, and stream both audio and video.</p><h4>1. ReactJS Application Setup:</h4><p>Initialize a new ReactJS project using Create React App for a quick start.<br>Install necessary dependencies such as Axios for making HTTP requests to the backend.</p><h4>2. Python Flask Backend Setup:</h4><p>Create a new Flask project to act as the backend server.</p><p>Install Whisper and other necessary Python libraries.</p><pre>brew install ffmpeg</pre><h3>Backend Setup</h3><ul><li>Use Python 3.7 or newer and create a virtual environment within your project directory to isolate your project’s dependencies.</li></ul><pre>mkdir openaiWisper &amp;&amp; cd openaiWisper &amp;&amp; mkdir backend &amp;&amp; cd backend</pre><ul><li>Use the below command in the terminal window to create the `venv` folder. This folder holds our dependencies.</li></ul><pre>python3 -m venv openaiWisper</pre><ul><li>Your requirements.txt at the root of the backend directory should include Flask, Flask-CORS for handling cross-origin requests, Whisper, and other necessary libraries like numpy, torch, and ffmpeg-python.</li></ul><pre>#requirements.txt<br>numpy<br>tqdm<br>transformers&gt;=4.19.0<br>ffmpeg-python==0.2.0<br>pyaudio<br>SpeechRecognition<br>pydub<br>git+https://github.com/openai/whisper.git<br>--extra-index-url https://download.pytorch.org/whl/cu113<br>torch<br>flask<br>flask_cors</pre><ul><li>Create a bash script, install_dependencies.sh, to activate the virtual environment and install dependencies listed in requirements.txt.</li></ul><pre># install and run backend<br>cd backend &amp;&amp; python3 -m venv openaiWisper<br>source openaiWisper/bin/activate<br>brew install portaudio<br><br>pip install wheel<br>pip install -r requirements.txt</pre><ul><li>In your Flask application app.py, define a /transcribe endpoint that accepts POST requests with audio files. Use Whisper to transcribe the audio file and return the transcription as a response.</li></ul><pre>touch backend/app.py<br>code backend/app.py</pre><pre>import os<br>import flask<br>from flask import request<br>from flask_cors import CORS<br>import whisper<br><br>app = flask.Flask(__name__)<br>CORS(app)<br><br><br>@app.route(&#39;/transcribe&#39;, methods=[&#39;POST&#39;])<br>def transcribe():<br>    if request.method == &#39;POST&#39;:<br>        audio_model = whisper.load_model(&quot;small&quot;)<br>        audio_file = request.files[&#39;audio_data&#39;]<br>        print(audio_file)<br><br>        save_path = os.path.join(&#39;./&#39;, &#39;test.wav&#39;)<br>        wav_file = request.files[&#39;audio_data&#39;]<br>        wav_file.save(save_path)<br><br>        result = audio_model.transcribe(save_path)<br>        print(result)<br><br>        return result[&#39;text&#39;]<br>    else:<br>        return &quot;This endpoint only processes POST wav blob&quot;</pre><ul><li>Start the Flask application on a specified port to listen for incoming transcription requests from your ReactJS app</li></ul><pre>cd backend<br>flask run -p 8000</pre><ul><li>Optionally, use tools like <a href="https://ngrok.com/">ngrok</a> to expose your local development server to the internet, enabling your ReactJS app to communicate with the Flask backend.</li></ul><pre>ngrok http 8000</pre><h3>Frontend Setup</h3><ul><li>First, create a new React application</li></ul><pre>npx create-react-app frontend<br>cd frontend</pre><ul><li>Capturing Audio Input: Implement a component to capture audio input from the user. This can involve using the navigator.mediaDevices.getUserMedia API to record audio. Our Flask server expects the file in .wav format.</li></ul><pre> const startRecording = async () =&gt; {<br>  setRecordingStatus(&quot;recording&quot;);<br>    setAudioBlob(null);<br>    setAudioURL(null);<br>    setTranscription(null);<br><br>  const media = new MediaRecorder(stream, { type: mimeType });<br>  mediaRecorder.current = media;<br>  mediaRecorder.current.start();<br>  let localAudioChunks = [];<br><br>  mediaRecorder.current.ondataavailable = (event) =&gt; {<br>   if (typeof event.data === &quot;undefined&quot;) return;<br>   if (event.data.size === 0) return;<br>   localAudioChunks.push(event.data);<br>  };<br><br>  setAudioChunks(localAudioChunks);<br> };<br><br> const stopRecording = () =&gt; {<br>    setTranscription(null);<br>  setRecordingStatus(&quot;inactive&quot;);<br>  mediaRecorder.current.stop();<br><br>  mediaRecorder.current.onstop = () =&gt; {<br>   const audioBlob = new Blob(audioChunks, { type: mimeType });<br>   const audioUrl = URL.createObjectURL(audioBlob);<br><br>      setAudioBlob(audioBlob);<br>   setAudioURL(audioUrl);<br><br>   setAudioChunks([]);<br>  };<br> };</pre><ul><li>Sending Audio to Backend Create a function to send the audio blob to your Flask backend.</li></ul><pre>const sendAudioToBackend = async () =&gt; {<br>    setTranscription(null);<br>    const formData = new FormData();<br>    formData.append(&#39;audio_data&#39;, audioBlob);<br><br>    try {<br>      const response = await fetch(&#39;http://localhost:8000/transcribe&#39;, {<br>        method: &#39;POST&#39;,<br>        body: formData<br>      });<br>      if (!response.ok) throw new Error(&#39;Network response was not ok.&#39;);<br>      const data = await response.text();<br>      return setTranscription(data);<br>    } catch (error) {<br>      console.error(&#39;Error sending audio to backend:&#39;, error);<br>    }<br>  };</pre><ul><li>Displaying Transcription Finally, display the transcription received from the backend:</li></ul><pre>import { useState, useRef } from &quot;react&quot;;<br><br>const mimeType = &quot;audio/webm&quot;;<br><br>const AudioRecorder = () =&gt; {<br> const mediaRecorder = useRef(null);<br> const [permission, setPermission] = useState(false);<br> const [recordingStatus, setRecordingStatus] = useState(&quot;inactive&quot;);<br> const [stream, setStream] = useState(null);<br> const [audioBlob, setAudioBlob] = useState(null);<br>  const [audioURL, setAudioURL] = useState(null);<br> const [audioChunks, setAudioChunks] = useState([]);<br>  const [transcription, setTranscription] = useState(null);;<br><br> const getMicrophonePermission = async () =&gt; {<br>  if (&quot;MediaRecorder&quot; in window) {<br>   try {<br>    const mediaStream = await navigator.mediaDevices.getUserMedia({<br>     audio: true,<br>     video: false,<br>    });<br>    setPermission(true);<br>    setStream(mediaStream);<br>   } catch (err) {<br>    alert(err.message);<br>   }<br>  } else {<br>   alert(&quot;The MediaRecorder API is not supported in your browser.&quot;);<br>  }<br> };<br><br> const startRecording = async () =&gt; {<br>  setRecordingStatus(&quot;recording&quot;);<br>    setAudioBlob(null);<br>    setAudioURL(null);<br>    setTranscription(null);<br><br>  const media = new MediaRecorder(stream, { type: mimeType });<br>  mediaRecorder.current = media;<br>  mediaRecorder.current.start();<br>  let localAudioChunks = [];<br><br>  mediaRecorder.current.ondataavailable = (event) =&gt; {<br>   if (typeof event.data === &quot;undefined&quot;) return;<br>   if (event.data.size === 0) return;<br>   localAudioChunks.push(event.data);<br>  };<br><br>  setAudioChunks(localAudioChunks);<br> };<br><br> const stopRecording = () =&gt; {<br>    setTranscription(null);<br>  setRecordingStatus(&quot;inactive&quot;);<br>  mediaRecorder.current.stop();<br><br>  mediaRecorder.current.onstop = () =&gt; {<br>   const audioBlob = new Blob(audioChunks, { type: mimeType });<br>   const audioUrl = URL.createObjectURL(audioBlob);<br><br>      setAudioBlob(audioBlob);<br>   setAudioURL(audioUrl);<br><br>   setAudioChunks([]);<br>  };<br> };<br><br>  const sendAudioToBackend = async () =&gt; {<br>    setTranscription(null);<br>    const formData = new FormData();<br>    formData.append(&#39;audio_data&#39;, audioBlob);<br><br>    try {<br>      const response = await fetch(&#39;http://localhost:8000/transcribe&#39;, {<br>        method: &#39;POST&#39;,<br>        body: formData<br>      });<br>      if (!response.ok) throw new Error(&#39;Network response was not ok.&#39;);<br>      const data = await response.text();<br>      return setTranscription(data);<br>    } catch (error) {<br>      console.error(&#39;Error sending audio to backend:&#39;, error);<br>    }<br>  };<br><br> return (<br>  &lt;div&gt;<br>   &lt;h2&gt;Audio Recorder + Whisper Transcription&lt;/h2&gt;<br>   &lt;main&gt;<br>    &lt;div className=&quot;audio-controls&quot;&gt;<br>     {!permission &amp;&amp; (<br>      &lt;button onClick={getMicrophonePermission} type=&quot;button&quot;&gt;<br>       Get Microphone<br>      &lt;/button&gt;<br>     )}<br>     {permission &amp;&amp; recordingStatus === &quot;inactive&quot; &amp;&amp; (<br>      &lt;button onClick={startRecording} type=&quot;button&quot;&gt;<br>       Start Recording<br>      &lt;/button&gt;<br>     )}<br>     {recordingStatus === &quot;recording&quot; &amp;&amp; (<br>      &lt;button onClick={stopRecording} type=&quot;button&quot;&gt;<br>       Stop Recording<br>      &lt;/button&gt;<br>     )}<br>    &lt;/div&gt;<br>    {audioBlob &amp;&amp; (<br>     &lt;div className=&quot;audio-player&quot;&gt;<br>      &lt;audio src={audioURL} controls&gt;&lt;/audio&gt;<br>      &lt;a download className=&quot;download-link&quot; href={audioURL}&gt;<br>       Download Recording<br>      &lt;/a&gt;<br>            &lt;button onClick={sendAudioToBackend} type=&quot;button&quot;&gt;<br>              Send Audio To Whisper<br>      &lt;/button&gt;<br>     &lt;/div&gt;<br>    )}<br>        {audioBlob &amp;&amp; transcription &amp;&amp; (<br>          &lt;div className=&quot;transcription&quot;&gt;<br>            {transcription}<br>          &lt;/div&gt;<br>        )}<br>   &lt;/main&gt;<br>  &lt;/div&gt;<br> );<br>};<br><br>export default AudioRecorder;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/1*xh5shekR-sTZu8RrCctKAA.png" /></figure><p>The <a href="https://github.com/uokesita/openaiWhisper">project repository</a> is publicly available. We have included a sample audio file for testing.</p><h3>Conclusion</h3><p>This article guided us through integrating OpenAI’s Whisper for speech-to-text in ReactJS applications, highlighting its potential to revolutionize app interactivity and accessibility. We explored creating a dictation app and discussed the possibilities for developers to further leverage Whisper in enhancing web and mobile device functionalities. The focus is on practical implementation and the anticipation of Whisper’s broader impact on technology use.</p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><p>AlamedaDev provides full-service end-to-end software. Experts in modern software development and AI solutions.</p><p>From #barcelona</p><p>Website: <a href="http://www.alamedadev.com/">www.alamedadev.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fdfd62101b9a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/whisper-and-speech-recognition-fdfd62101b9a">Whisper and Speech Recognition</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Semantic Search Works in Real Projects]]></title>
            <link>https://medium.com/alameda-dev/how-semantic-search-works-in-real-projects-6e5ba094d7b9?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/6e5ba094d7b9</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[semantic-search]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Sat, 23 Nov 2024 07:51:16 GMT</pubDate>
            <atom:updated>2024-11-23T07:51:16.142Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V2vU0ANPNqjrwO2YV6gGdA.png" /></figure><p>Semantic search is transforming the way we find and interact with information. It’s no longer just about finding exact keyword matches — it’s about understanding the meaning behind what someone is looking for. This evolution marks a significant leap for AI, enabling machines to process and respond to human language in a way that feels intuitive and contextually relevant.</p><p>For artificial intelligence, semantic search is more than just a technical improvement — it’s a shift toward understanding. By leveraging advanced models like transformers and embeddings, AI systems can comprehend relationships, intent, and nuances in a query. This makes it possible to deliver results that go beyond the surface, paving the way for smarter applications across industries.</p><h3>Traditional vs. Semantic Search: Why Meaning Matters</h3><p>Traditional search engines often work like this: you type in a word, and the system hunts for documents that contain that exact word. It’s efficient but limited. What happens if someone searches for ‘good Italian food,’ but your database has ‘great pasta spots’? A keyword-based search might miss the match.</p><p>Semantic search, on the other hand, focuses on the meaning behind the words. By using language models, it creates connections between phrases, synonyms, and even the overall intent of the query. This makes the results feel more human, less robotic.</p><h3>Example: An E-commerce Platform That Understands Customers</h3><p>Imagine you’re working on an e-commerce platform. A customer types in a query like “comfortable shoes for running” into the search bar. A traditional search engine might only return products with the exact keywords “comfortable,” “shoes,” and “running” in their descriptions. This could miss related items like “lightweight trainers” or “breathable athletic footwear,” even though these are a perfect match for what the customer is looking for.</p><p>With semantic search, the platform goes beyond matching words. It understands the intent behind the query: the customer wants shoes suitable for running, prioritizing comfort. The system can recognize that terms like “lightweight,” “cushioned,” or “designed for athletes” are highly relevant. It could even surface products with reviews or descriptions mentioning “best for running” or “great comfort for long-distance jogs.”</p><h4>How It Could Work: Practical Implementation</h4><p><strong>Data Preparation: </strong>Encode all product titles, descriptions, and customer reviews into embeddings (vector representations of meaning).</p><p><strong>Query Matching: </strong>When a user searches, encode their query into an embedding.</p><p><strong>Similarity Scoring: </strong>Compare the query embedding to all product embeddings using cosine similarity.</p><p><strong>Ranking and Display: </strong>Rank the results based on similarity and display the most relevant products.</p><h3>A Practical Example 1: Using Semantic Search</h3><p>Here, we’ll show a simple and focused example of semantic search using an open-source model like sentence-transformers. Keep in mind that this is just a small demonstration to help you understand the basics. If you’re planning to work with real-world data, you’ll need to spend time preparing your dataset and setting up a more robust infrastructure.</p><p>We’ve also included a link to a Google Colab notebook at the end, so you can experiment with the code and modify the prompts to fit your needs.</p><pre>from sentence_transformers import SentenceTransformer, util<br>import torch<br><br># Cargar el modelo preentrenado<br>model = SentenceTransformer(&#39;all-MiniLM-L6-v2&#39;)<br><br># Descripciones de productos de ejemplo<br>products = [<br>    &quot;Lightweight running shoes with breathable mesh fabric.&quot;,<br>    &quot;Comfortable sneakers perfect for long-distance running.&quot;,<br>    &quot;Cushioned trainers ideal for jogging and casual wear.&quot;,<br>    &quot;Formal leather shoes for office and evening events.&quot;,<br>]<br><br># Codificar las descripciones de productos<br>product_embeddings = model.encode(products, convert_to_tensor=True)<br><br># Consulta del usuario<br>query = &quot;comfortable shoes for running&quot;<br>query_embedding = model.encode(query, convert_to_tensor=True)<br><br># Calcular la similitud coseno<br>similarities = util.cos_sim(query_embedding, product_embeddings)<br><br># Mostrar los mejores resultados<br>top_k = 3  # Número de resultados a mostrar<br>results = torch.topk(similarities, k=top_k)<br><br>print(f&quot;User Query: {query}&quot;)<br>print(&quot;Recommended Products:&quot;)<br>for idx in results[1][0]:<br>    print(f&quot;- {products[idx]} (Similarity: {similarities[0][idx]:.4f})&quot;)</pre><pre>User Query: comfortable shoes for running<br>Recommended Products:<br>- Comfortable sneakers perfect for long-distance running. (Similarity: 0.8823)<br>- Cushioned trainers ideal for jogging and casual wear. (Similarity: 0.8417)<br>- Lightweight running shoes with breathable mesh fabric. (Similarity: 0.8129)</pre><p>Try It Yourself</p><p>To make it easy for you to experiment, we’ve created a <a href="https://colab.research.google.com/drive/1j-wMjL_mhpwwyaUsMuOn2NLmr6W87t_C?authuser=3#scrollTo=VJTDe-Nijbf3">Google Colab notebook</a> with the full code. Open it, run the cells, and modify the query to test your search scenarios.</p><h3>A Practical Example 2: Document Search for a Legal Firm</h3><p>Semantic search provides a powerful way for legal professionals to retrieve relevant documents by analyzing the intent and context behind their queries, rather than relying on exact keyword matches. This example offers a simplified demonstration of the concept. However, in production systems, implementing semantic search typically involves more sophisticated steps, such as preprocessing large datasets, scaling search capabilities with vector databases, and fine-tuning models on domain-specific legal language.</p><p>Imagine you’re building a tool for lawyers to search legal case summaries. A lawyer might type something like “cases about intellectual property theft”, but the database might describe it as “court ruling on intellectual property disputes.” Traditional search may not pick up this match, but semantic search understands the intent behind the query and retrieves relevant results.</p><pre>from sentence_transformers import SentenceTransformer, util<br>import torch<br><br># Load a pre-trained model<br>model = SentenceTransformer(&#39;all-MiniLM-L6-v2&#39;)<br><br># Example dataset of legal case summaries<br>cases = [<br>    &quot;A case involving copyright infringement on a digital platform.&quot;,<br>    &quot;Legal analysis of a patent dispute in the tech industry.&quot;,<br>    &quot;A trademark conflict between two major brands.&quot;,<br>    &quot;Court ruling on intellectual property theft in design.&quot;,<br>]<br><br># Encode the case summaries into embeddings<br>case_embeddings = model.encode(cases, convert_to_tensor=True)<br><br># User&#39;s query<br>query = &quot;cases about intellectual property theft&quot;<br>query_embedding = model.encode(query, convert_to_tensor=True)<br><br># Compute cosine similarity between the query and case embeddings<br>similarities = util.cos_sim(query_embedding, case_embeddings)<br><br># Display top matches<br>top_k = 3  # Number of results to display<br>results = torch.topk(similarities, k=top_k)<br><br>print(f&quot;User Query: {query}&quot;)<br>print(&quot;Recommended Cases:&quot;)<br>for idx in results[1][0]:<br>    print(f&quot;- {cases[idx]} (Similarity: {similarities[0][idx]:.4f})&quot;)</pre><pre>Console Output:<br>User Query: cases about intellectual property theft<br>Recommended Cases:<br>- Court ruling on intellectual property theft in design. (Similarity: 0.95)<br>- Legal analysis of a patent dispute in the tech industry. (Similarity: 0.87)<br>- A trademark conflict between two major brands. (Similarity: 0.72)</pre><p>Semantic search is changing the way we find information, focusing on the meaning behind what people are looking for rather than just matching exact words. Whether it’s helping lawyers retrieve relevant case documents or improving product recommendations in e-commerce, semantic search creates a smoother and more intuitive experience.</p><h3>Looking Ahead</h3><p>While we’ve focused on a couple of examples here, semantic search can do so much more:</p><p><strong>Healthcare and Research: </strong>Help doctors and researchers quickly find the latest studies or treatment guidelines. For example, a query like “new treatments for lung cancer” can surface relevant and recent studies, even if they use technical terminology or synonyms.</p><p><strong>Education Platforms: </strong>Connect students with learning resources tailored to their needs. For instance, if a student searches for “easy examples of photosynthesis,” semantic search could retrieve tutorials, videos, or simplified articles that match their learning level.</p><p><strong>Entertainment Recommendations: </strong>Recommend movies, shows, or music based on someone’s preferences. A user searching for “movies like Inception” might get results for psychological thrillers or films with mind-bending plots, even if the exact term isn’t used in the database.</p><p><strong>IoT in Industry: </strong>Equipment Troubleshooting: A technician looking for “error code 501 solutions for conveyor belts” can get tailored results, even if the database describes it differently.</p><p>If you’re thinking about implementing <strong>Semantic Search</strong> or other <strong>AI-powered solutions</strong> in your project, let’s talk! At <a href="https://www.alamedadev.com/">AlamedaDev</a>, we specialize in building smart, scalable AI solutions tailored to your needs. Visit our site to learn more about what we do and how we can help bring your ideas to life.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e5ba094d7b9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/how-semantic-search-works-in-real-projects-6e5ba094d7b9">How Semantic Search Works in Real Projects</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Canopy, a RAG framework by Pinecone]]></title>
            <link>https://medium.com/alameda-dev/using-canopy-a-rag-framework-by-pinecone-e49d930aadcc?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/e49d930aadcc</guid>
            <category><![CDATA[pinecone]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Sat, 31 Aug 2024 15:12:21 GMT</pubDate>
            <atom:updated>2024-02-02T15:52:32.293Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RpXzVnK3My4UA2-yFhUNCQ.png" /><figcaption>Image generated with DALL-E</figcaption></figure><p>The <a href="https://www.pinecone.io/blog/canopy-rag-framework">Canopy</a> RAG Framework is a tool developed by the <a href="https://www.pinecone.io/">Pinecone</a> team to simplify the process of building and managing robust Retrieval-Augmented Generation (RAG), which are often complex to build. Key features of Canopy include the ability to chat within a terminal to compare RAG and non-RAG outputs, thus evaluating the performance of RAG pipelines efficiently.</p><p>To use the Canopy RAG Framework in the CLI, here is a series of steps:</p><p><strong>Note</strong>: Before starting, ensure you have Python and pip installed on your system, and you have created the necessary API keys for Pine Cone and OpenAI</p><ol><li><strong>Set Up a Virtual Environment (Optional):</strong><br>Create a virtual environment: <br>python3 -m venv canopy-env<br>Activate the environment: <br>source canopy-env/bin/activate</li><li><strong>Set environment variables for Pine Cone and OpenAI API keys:</strong><br>Set PINECONE_API_KEY, PINECONE_ENVIRONMENT, and INDEX_NAME with your actual Pinecone API key and environment.<br>Set OPENAI_API_KEY with your OpenAI API key.</li><li><strong>Install the Canopy SDK:</strong><br> Install the package: pip install canopy-sdk<br> Check that installation is successful, and the environment is set, run:<br>canopy<br>Output should be similar to this:<br>Canopy: Ready<br>Usage: canopy [OPTIONS] COMMAND [ARGS]…<br># rest of the help message</li><li><strong>Create a New Canopy Index:</strong><br>Run canopy new and follow the CLI instructions to create a new Pinecone index configured to work with Canopy.</li><li><strong>Uploading Data:</strong><br>Load data into your Canopy index using the canopy upsert command with the path to your data (supports jsonl, parquet, csv, and .txt formats). Ensure the document fields comply with the required schema (id, text, source, metadata).<br>You can use this Notebook to explore one way to upload data to the index <a href="https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/generation/canopy/00-canopy-data-prep.ipynb">https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/generation/canopy/00-canopy-data-prep.ipynb</a></li><li><strong>Start the Canopy Server:</strong><br> Start the server with the canopy start command. <br>This exposes Canopy’s functionality via a REST API, including document upload, retrieval of relevant documents for queries, and a chat interface.</li></ol><p>You can now start using the <strong>Canopy</strong> server with any chat application that supports a /chat.completion endpoint.</p><p><strong>Evaluate RAG Responses</strong><br>To interactively chat with your text data, run canopy chat in a new terminal window, setting the required environment variables beforehand.</p><p>To compare responses with and without RAG, use canopy chat — no-rag which shows both RAG and non-RAG responses side-by-side​.<br>Test the RAG pipeline with different queries, including complex ones with multiple search terms.<br>Observe how Canopy handles the queries and provides responses with and without RAG.</p><p>In conclusion, the Canopy RAG Framework, an innovation by Pinecone, stands as a game-changer in the realm of Generative AI applications. It empowers developers to easily and rapidly build and host robust, production-ready chat assistants of any scale. Key features include:</p><ul><li><strong>Ease of Use</strong>: Canopy simplifies complex processes like chunking, embedding, and query optimization, allowing developers to focus on building and experimenting with RAG applications.</li><li><strong>Cost-Effectiveness</strong>: It offers free storage for up to 100K embeddings, sufficient for substantial text data, with scalable paid options for larger needs.</li><li><strong>Flexibility and Extensibility</strong>: The framework’s modular nature enables integration into existing applications or development of custom solutions, adapting to diverse use cases.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RpXzVnK3My4UA2-yFhUNCQ.png" /></figure><ul><li><strong>Interactive Evaluation</strong>: The built-in CLI chat application facilitates side-by-side comparison of RAG and non-RAG workflows, enhancing iterative development and evaluation.</li></ul><p>As Canopy evolves, it promises to support more data formats, LLMs, embedding models, and other advancements, making it an essential tool for developers in the rapidly expanding field of Generative AI​.</p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><p>AlamedaDev provides full-service end-to-end software. Experts in modern software development and AI solutions.</p><p>Founded in #barcelona</p><p>Website: <a href="http://www.alamedadev.com/">www.alamedadev.com</a></p><p>Alameda-AI: <a href="http://www.alamedadev.com/">www.alameda.dev</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e49d930aadcc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/using-canopy-a-rag-framework-by-pinecone-e49d930aadcc">Using Canopy, a RAG framework by Pinecone</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Serverless AI Applications with Amazon Bedrock: A Step-by-Step Guide]]></title>
            <link>https://medium.com/alameda-dev/ai-serverless-applications-with-amazon-bedrock-a-step-by-step-guide-05fdb3df1ce9?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/05fdb3df1ce9</guid>
            <category><![CDATA[ai-integration]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[amazon-bedrock]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Fri, 26 Apr 2024 16:48:55 GMT</pubDate>
            <atom:updated>2024-04-29T15:08:50.128Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building serverless AI applications with Amazon Bedrock</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kWGsGBbLySCLNAqWAKpdew.jpeg" /></figure><p>With serverless computing, you don’t have to worry about the heavy lifting of running servers or managing resources. It’s all about focusing on your app’s features. Amazon Bedrock and other services we’ll help you use powerful AI models to do things like understanding what’s said in an audio file or responding to user queries, all without needing to be an expert in AI.</p><p>In this guide, we’ll cover the basics like setting up your project, to more advanced stuff like turning speech into text and summarizing conversations. By the end, you’ll know how to make apps that can interact with users, using Amazon’s cloud services.</p><h4>What is Amazon Bedrock?</h4><p>Amazon Bedrock lets you easily use large language models without worrying about the underlying infrastructure. In this tutorial, we will use Amazon Bedrock to interact with these models. First, we’ll set up a simple environment by creating a new directory for our project.</p><pre>mkdir bedrock<br>cd bedrock/</pre><h4>Basic response for generating a paragraph</h4><p>Here, you’ll learn how to generate a paragraph from a model by writing a Python script that uses Amazon Bedrock. We’ll start with a simple task: asking the model to summarize a topic in one sentence. This example will help you understand how to interact with the model and process its responses.</p><pre>import boto3<br>import json<br><br><br>bedrock_runtime = boto3.client(&#39;bedrock-runtime&#39;, region_name=&#39;us-east-1&#39;)<br>prompt = &quot;Write a one sentence summary of Las Vegas.&quot;<br><br>kwargs = {<br>    &quot;modelId&quot;: &quot;amazon.titan-text-express-v1&quot;,<br>    &quot;contentType&quot;: &quot;application/json&quot;,<br>    &quot;accept&quot;: &quot;*/*&quot;,<br>    &quot;body&quot; : json.dumps(<br>        {<br>            &quot;inputText&quot;: prompt,<br>            &quot;textGenerationConfig&quot;: {<br>                &quot;maxTokenCount&quot;: 100,<br>                &quot;temperature&quot;: 0.7,<br>                &quot;topP&quot;: 0.9<br>            }<br>        }<br>    )<br>}<br>response = bedrock_runtime.invoke_model(**kwargs)<br>response_body = json.loads(response.get(&#39;body&#39;).read())<br><br>generation = response_body[&#39;results&#39;][0][&#39;outputText&#39;]<br>print(generation)<br><br>#&gt; Las Vegas is a famous city known for its gambling, entertainment, and nightlife. It is located in Nevada and is the largest city within the Mojave Desert. Las Vegas is home to several iconic landmarks, including the Las Vegas Strip, the Bellagio, and the MGM Grand. The city is a popular destination for tourists from around the world, who come to enjoy the luxurious casinos, hotels, restaurants, and shows. Las Vegas is also known for its world-class entertainment, including live</pre><h4>Summarising the transcript file of a call</h4><p>Now, we’ll take it a step further by summarizing the transcript of a conversation. This involves reading a text file, sending it to the model, and asking it to condense the conversation into key points. This part of the tutorial will show you how to process larger pieces of text and use models to identify and summarize important information.</p><pre>with open(&#39;transcript.txt&#39;, &quot;r&quot;) as file:<br>    dialogue_text = file.read()<br><br>prompt = f&quot;&quot;&quot;The text between the &lt;transcript&gt; XML tags is a transcript of a conversation. <br>Write a short summary of the conversation.<br><br>&lt;transcript&gt;<br>{dialogue_text}<br>&lt;/transcript&gt;<br><br>Here is a summary of the conversation in the transcript:&quot;&quot;&quot;<br><br><br>kwargs = {<br>    &quot;modelId&quot;: &quot;amazon.titan-text-lite-v1&quot;,<br>    &quot;contentType&quot;: &quot;application/json&quot;,<br>    &quot;accept&quot;: &quot;*/*&quot;,<br>    &quot;body&quot;: json.dumps(<br>        {<br>            &quot;inputText&quot;: prompt,<br>            &quot;textGenerationConfig&quot;: {<br>                &quot;maxTokenCount&quot;: 512,<br>                &quot;temperature&quot;: 0,<br>                &quot;topP&quot;: 0.9<br>            }<br>        }<br>    )<br>}<br><br>response = bedrock_runtime.invoke_model(**kwargs)<br>response_body = json.loads(response.get(&#39;body&#39;).read())<br>generation = response_body[&#39;results&#39;][0][&#39;outputText&#39;]<br>print(generation)<br><br>#&gt; Alex is looking to book a room for his 10th wedding anniversary at the Crystal Heights Hotel in Singapore. The hotel offers several room types that offer stunning views of the city skyline and the fictional Sapphire Bay. The special diamond suite even comes with exclusive access to the moonlit pool and star deck. The package includes breakfast, complimentary access to the moonlit pool and star deck, a one-time spa treatment for two, and a special romantic dinner at the cloud nine restaurant. A preauthorization amount of $1000 will be held on the card, which will be released upon checkout. There is a 10% service charge and a 7% fantasy tax applied to the room rate.</pre><h4>Summarising an audio file from a call</h4><p>Next, we’ll work with audio files. You’ll learn how to upload an audio recording to AWS S3, transcribe it into text using Amazon Transcribe, and then summarize the conversation using the model. This section covers the entire process from handling audio files, transcribing them, and summarizing the content, which is a common workflow in processing audio data.</p><p>For the next sample, we will use the following process:<br>1. Import packages and load the audio file.<br>2. Setup:<br> a. S3 client.<br> b. Transcribe client.<br>3. Upload the audio file to S3.<br>4. Create the unique job name.<br>5. Build the transcription response.<br>6. Access the needed parts of the transcript.<br>7. Setup Bedrock runtime.<br>8. Create the prompt template.<br>9. Configure the model response.<br>10. Generate a summary of the audio transcript.</p><pre>import os<br>import boto3<br>import uuid<br>import time<br>from IPython.display import Audio<br><br>bedrock_runtime = boto3.client(&#39;bedrock-runtime&#39;, region_name=&#39;us-east-1&#39;)<br>audio = Audio(filename=&quot;dialog.mp3&quot;)<br>bucket_name = os.environ[&#39;AI_LEARN_BEDROCK_BUCKETNAME&#39;]<br><br># Setup s3 and upload audio file<br>s3_client = boto3.client(&#39;s3&#39;, region_name=&#39;us-east-1&#39;)<br>file_name = &#39;dialog.mp3&#39;<br>s3_client.upload_file(file_name, bucket_name, file_name)<br><br># Setup transcribe and generate job id<br>transcribe_client = boto3.client(&#39;transcribe&#39;, region_name=&#39;us-east-1&#39;)<br>job_name = &#39;transcription-job-&#39; + str(uuid.uuid4())<br>response = transcribe_client.start_transcription_job(<br>    TranscriptionJobName=job_name,<br>    Media={&#39;MediaFileUri&#39;: f&#39;s3://{bucket_name}/{file_name}&#39;},<br>    MediaFormat=&#39;mp3&#39;,<br>    LanguageCode=&#39;en-US&#39;,<br>    OutputBucketName=bucket_name,<br>    Settings={<br>        &#39;ShowSpeakerLabels&#39;: True,<br>        &#39;MaxSpeakerLabels&#39;: 2<br>    }<br>)<br>while True:<br>    status = transcribe_client.get_transcription_job(TranscriptionJobName=job_name)<br>    if status[&#39;TranscriptionJob&#39;][&#39;TranscriptionJobStatus&#39;] in [&#39;COMPLETED&#39;, &#39;FAILED&#39;]:<br>        break<br>    time.sleep(2)<br>print(status[&#39;TranscriptionJob&#39;][&#39;TranscriptionJobStatus&#39;])</pre><p><strong>Upload the audio file to s3</strong></p><p>Build a string for the job (transcription is an async process, so we need a job to be able to know its state and know when it is ready)<br>We need to poll the transcription job to see if it&#39;s complete or failed for some reason.<br>This returns the status, but not the actual transcription.</p><p><strong>Access the needed parts of the transcript</strong></p><p>In this next step, you will access the transcription result, extract the necessary parts of the conversation, and format it for summarization. This teaches you how to navigate and use the output from Amazon Transcribe, preparing it for further processing with Amazon Bedrock.</p><pre>import json<br><br>if status[&#39;TranscriptionJob&#39;][&#39;TranscriptionJobStatus&#39;] == &#39;COMPLETED&#39;:<br>    # Load the transcript from S3.<br>    transcript_key = f&quot;{job_name}.json&quot;<br>    transcript_obj = s3_client.get_object(Bucket=bucket_name, Key=transcript_key)<br>    transcript_text = transcript_obj[&#39;Body&#39;].read().decode(&#39;utf-8&#39;)<br>    transcript_json = json.loads(transcript_text)<br><br>    output_text = &quot;&quot;<br>    current_speaker = None<br><br>    items = transcript_json[&#39;results&#39;][&#39;items&#39;]<br><br>    for item in items:<br><br>        speaker_label = item.get(&#39;speaker_label&#39;, None)<br>        content = item[&#39;alternatives&#39;][0][&#39;content&#39;]<br><br>        # Start the line with the speaker label:<br>        if speaker_label is not None and speaker_label != current_speaker:<br>            current_speaker = speaker_label<br>            output_text += f&quot;\n{current_speaker}: &quot;<br><br>        # Add the speech content:<br>        if item[&#39;type&#39;] == &#39;punctuation&#39;:<br>            output_text = output_text.rstrip()<br><br>        output_text += f&quot;{content} &quot;<br><br>    # Save the transcript to a text file<br>    with open(f&#39;{job_name}.txt&#39;, &#39;w&#39;) as f:<br>        f.write(output_text)</pre><p><strong>Setup Bedrock runtime, Create the prompt template</strong></p><p>In this step, you’ll set up the Bedrock runtime and create a template for the prompt that will be sent to the model. This part involves reading the formatted transcript, creating a structured prompt that guides the model in generating a summary, and configuring the model response. It’s a crucial step in translating the raw transcript into a format that the model can understand and respond to effectively.</p><pre>bedrock_runtime = boto3.client(&#39;bedrock-runtime&#39;, region_name=&#39;us-west-2&#39;)<br>with open(f&#39;{job_name}.txt&#39;, &quot;r&quot;) as file:<br>    transcript = file.read()<br><br>with open(&#39;prompt_template.txt&#39;, &quot;r&quot;) as file:<br>    template_string = file.read()<br>data = {<br>    &#39;transcript&#39; : transcript<br>}<br>template = Template(template_string)<br>prompt = template.render(data)<br>print(prompt)<br><br>###<br>I need to summarize a conversation. The transcript of the <br>conversation is between the &lt;data&gt; XML like tags.<br><br>&lt;data&gt;<br><br>spk_0: Hi, is this the Crystal Heights Hotel in Singapore? <br>spk_1: Yes, it is. Good afternoon. How may I assist you today? <br>spk_0: Fantastic, good afternoon. I was looking to book a room for my 10th wedding anniversary. Ive heard your hotel offers exceptional views and services. Could you tell me more? <br>spk_1: Absolutely, Alex and congratulations on your upcoming anniversary. Thats a significant milestone and wed be honored to make it a special occasion for you. We have several room types that offer stunning views of the city skyline and the fictional Sapphire Bay. Our special diamond suite even comes with exclusive access to the moonlit pool and star deck. We also have in house spa services, world class dining options and a shopping arcade. <br>spk_0: That sounds heavenly. I think my spouse would love the moonlit pool. Can you help me make a reservation for one of your diamond suites with a sapphire bay view? <br>spk_1: Of course. May I know the dates you planning to visit? <br>spk_0: Sure. It would be from October 10th to 17th. <br>spk_1: Excellent. Let me check the availability. Ah It looks like we have a diamond suite available for those dates. Would you like to proceed with the reservation? <br>spk_0: Definitely. Whats included in the package? <br>spk_1: Wonderful. The package includes breakfast, complimentary access to the moonlit pool and star deck. A one time spa treatment for two and a special romantic dinner at our cloud nine restaurant. <br>spk_0: You making it impossible to resist. Lets go ahead with the booking. <br>spk_1: Great. I&#39;ll need some personal information for the reservation. Can I get your full name, contact details and a credit card for the preauthorizations? <br>spk_0: Certainly. My full name is Alexander Thompson. My contact number is 12345678910. And the credit card is, wait, did you say pre authorization? How much would that be? <br>spk_1: Ah, I should have mentioned that earlier. My apologies. A pre authorization. A mt of $1000 will be held on your card which would be released upon checkout <br>spk_0: $1000. That seems a bit excessive. Don&#39;t you think <br>spk_1: I understand your concern, Alex. The pre authorization is a standard procedure to cover any incidental expenses you may incur during your stay. However, I assure you its only a hold and not an actual charge. <br>spk_0: Thats still a lot. Are there any additional charges that I should know about? <br>spk_1: Well, there is a 10% service charge and a 7% fantasy tax applied to the room rate. <br>spk_0: Mm. You know what its a special occasion. So lets go ahead. <br>spk_1: Thank you, Alex for understanding. Will ensure that your experience at Crystal Heights is well worth it. <br>&lt;/data&gt;<br><br>The summary must contain a one word sentiment analysis, and <br>a list of issues, problems or causes of friction<br>during the conversation. The output must be provided in <br>JSON format shown in the following example. <br><br>Example output:<br>{<br>    &quot;sentiment&quot;: &lt;sentiment&gt;,<br>    &quot;issues&quot;: [<br>        {<br>            &quot;topic&quot;: &lt;topic&gt;,<br>            &quot;summary&quot;: &lt;issue_summary&gt;,<br>        }<br>    ]<br>}<br><br>Write the JSON output and nothing more.<br><br>Here is the JSON output:</pre><p><strong>Configure the model response</strong></p><p>Finally, you will configure the model’s response to your prompt. This includes setting parameters for the text generation and invoking the model with your prepared prompt. This last step completes the process of summarizing audio transcripts by extracting meaningful insights and presenting them in a structured format. This demonstrates the full potential of integrating various AWS services to process and understand audio data.</p><pre>kwargs = {<br>    &quot;modelId&quot;: &quot;amazon.titan-text-lite-v1&quot;,<br>    &quot;contentType&quot;: &quot;application/json&quot;,<br>    &quot;accept&quot;: &quot;*/*&quot;,<br>    &quot;body&quot;: json.dumps(<br>        {<br>            &quot;inputText&quot;: prompt,<br>            &quot;textGenerationConfig&quot;: {<br>                &quot;maxTokenCount&quot;: 512,<br>                &quot;temperature&quot;: 0,<br>                &quot;topP&quot;: 0.9<br>            }<br>        }<br>    )<br>}<br>response = bedrock_runtime.invoke_model(**kwargs)<br>response_body = json.loads(response.get(&#39;body&#39;).read())<br>generation = response_body[&#39;results&#39;][0][&#39;outputText&#39;]<br>print(generation)<br><br>###<br><br>{<br>    &quot;sentiment&quot;: &quot;Positive&quot;,<br>    &quot;issues&quot;: [<br>        {<br>            &quot;topic&quot;: &quot;Hotel services&quot;,<br>            &quot;summary&quot;: &quot;The hotel offers exceptional views and services.&quot;<br>        },<br>        {<br>            &quot;topic&quot;: &quot;Room booking&quot;,<br>            &quot;summary&quot;: &quot;The hotel has several room types that offer stunning views of the city skyline and the fictional Sapphire Bay.&quot;<br>        },<br>        {<br>            &quot;topic&quot;: &quot;Diamond suite&quot;,<br>            &quot;summary&quot;: &quot;The diamond suite comes with exclusive access to the moonlit pool and star deck.&quot;<br>        },<br>        {<br>            &quot;topic&quot;: &quot;Spa services&quot;,<br>            &quot;summary&quot;: &quot;The hotel has in-house spa services, world-class dining options, and a shopping arcade.&quot;<br>        },<br>        {<br>            &quot;topic&quot;: &quot;Reservation process&quot;,<br>            &quot;summary&quot;: &quot;The reservation process includes breakfast, complimentary access to the moonlit pool and star deck, a one-time spa treatment for two, and a special romantic dinner at the cloud nine restaurant.&quot;<br>        },<br>        {<br>            &quot;topic&quot;: &quot;Pre-authorization&quot;,<br>            &quot;summary&quot;: &quot;A pre-authorization of $1000 is held on the credit card, which is released upon checkout.&quot;<br>        },<br>        {<br>            &quot;topic&quot;: &quot;Additional charges&quot;,<br>            &quot;summary&quot;: &quot;There is a 10% service charge and a 7% fantasy tax applied to the room rate.&quot;<br>        }<br>    ]<br>}</pre><h4>Conclusion</h4><p>By now, you should have a solid foundation in using Amazon Bedrock alongside services like Amazon Transcribe and AWS Lambda. These skills are not just limited to the examples shown, but can be extended to a wide range of applications, from automated customer support to data analysis and beyond. The key takeaway is the ease with which we can integrate various AWS services to build powerful, serverless applications that understand and interact with human language, opening up a world of possibilities for developers and businesses alike.</p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><p>AlamedaDev provides full-service end-to-end software. Experts in modern software development and AI solutions.</p><p>From #barcelona</p><p>Website: <a href="http://www.alamedadev.com/">www.alamedadev.com</a><br>Alameda-AI: <a href="http://www.alamedadev.com/">www.alameda.dev</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=05fdb3df1ce9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/ai-serverless-applications-with-amazon-bedrock-a-step-by-step-guide-05fdb3df1ce9">Serverless AI Applications with Amazon Bedrock: A Step-by-Step Guide</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlocking the Power of Retrieval Augmented Generation (RAG) with LangChainJS]]></title>
            <link>https://medium.com/alameda-dev/unlocking-the-power-of-retrieval-augmented-generation-rag-with-langchainjs-9212c2f5e0e7?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/9212c2f5e0e7</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Tue, 06 Feb 2024 10:32:56 GMT</pubDate>
            <atom:updated>2024-02-13T11:51:31.982Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2jA9OxQa_DcW6J6O2MpdEA.jpeg" /></figure><p>This is an overview focused on learning the basics of Large Language Model (LLM) applications using LangChainJS, a tool for building and operating LLMs.</p><p>LangChain.js is particularly suitable for building applications across multiple platforms, such as browser extensions, mobile apps with React Native, and desktop apps with Electron. Given the extensive use of JavaScript among developers, the ease of deployment, scaling features, and advanced tools available in this ecosystem.</p><p>LangChain employs a special language to compose chains of components called runnables. These runnables define core methods, input and output types, and allow for common LLM application functionalities like invoke, stream, batch, and runtime parameter modification.</p><p>Retrieval Augmented Generation, or RAG, is an exciting application of Large Language Models (LLMs) that is gaining widespread popularity. In this blog post, we will delve into what RAG is and explore some tools offered by LangChainJS, the JavaScript version of LangChain, to simplify its implementation, including document loaders and text splitters.</p><h3>Basic Example:</h3><pre>import &quot;dotenv/config&quot;;<br>import { ChatOpenAI } from &quot;@langchain/openai&quot;;<br>import { HumanMessage } from &quot;@langchain/core/messages&quot;;<br><br>const model = new ChatOpenAI({<br>    modelName: &quot;gpt-3.5-turbo-1106&quot;<br>});<br>​<br>await model.invoke([<br>    new HumanMessage(&quot;Tell me a joke.&quot;)<br>]);<br><br>### RESPONSE:<br><br>AIMessage {<br>  lc_serializable: true,<br>  lc_kwargs: {<br>    content: &quot;Why don&#39;t skeletons fight each other?\nThey don&#39;t have the guts!&quot;,<br>    additional_kwargs: { function_call: undefined, tool_calls: undefined }<br>  },<br>  lc_namespace: [ &quot;langchain_core&quot;, &quot;messages&quot; ],<br>  content: &quot;Why don&#39;t skeletons fight each other?\nThey don&#39;t have the guts!&quot;,<br>  name: undefined,<br>  additional_kwargs: { function_call: undefined, tool_calls: undefined }<br>}</pre><h4>Prompt templates</h4><p>Prompt templates in programming, particularly for Large Language Models (LLMs) like those used in LangChain, are standardized formats or structures used for creating prompts. They incorporate placeholders for variable input, making them reusable and adaptable for different queries. These templates streamline the process of prompt creation, enhance readability, and improve the maintainability of the code by centralizing changes to prompt logic.</p><p>In LangChain, prompt templates are implemented using classes like `PromptTemplate` and `ChatPromptTemplate`, which provide methods to create and format prompts with input variables. These templates support different templating engines and offer features like validation of input variables and flexibility in specifying input values.</p><pre>import { ChatPromptTemplate } from &quot;@langchain/core/prompts&quot;;<br>​<br>const prompt = ChatPromptTemplate.fromTemplate(<br>    `What are three good names for a company that makes {product}?`<br>)<br>await prompt.format({<br>    product: &quot;colorful socks&quot;<br>});<br><br>##&gt;&gt; &quot;Human: What are three good names for a company that makes colorful socks?&quot;<br><br>await prompt.formatMessages({<br>    product: &quot;colorful socks&quot;<br>});<br><br>##&gt; [<br>  HumanMessage {<br>    lc_serializable: true,<br>    lc_kwargs: {<br>      content: &quot;What are three good names for a company that makes colorful socks?&quot;,<br>      additional_kwargs: {}<br>    },<br>    lc_namespace: [ &quot;langchain_core&quot;, &quot;messages&quot; ],<br>    content: &quot;What are three good names for a company that makes colorful socks?&quot;,<br>    name: undefined,<br>    additional_kwargs: {}<br>  }<br>]</pre><h4>LangChain Expression Language (LCEL)</h4><p>LCEL is designed to connect different components, referred to as “runnables,” in a sequence. This allows for the creation of sophisticated workflows where the output of one component becomes the input for another. These are essentially building blocks that define core methods and are designed to accept specific input types and produce defined output types. This makes it easier to integrate different functionalities in a seamless manner.</p><p>Runnables come with common methods like ‘invoke’, ‘stream’, and ‘batch’. These methods are essential in LLM applications, allowing for various operations such as invoking the runnable to execute its function, streaming data for continuous processing, and batch processing for handling multiple inputs simultaneously.</p><pre>const chain = prompt.pipe(model);<br>await chain.invoke({<br>    product: &quot;colorful socks&quot;<br>});<br><br>##&gt; AIMessage {<br>  lc_serializable: true,<br>  lc_kwargs: {<br>    content: &quot;1. Rainbow Soles\n2. Vivid Footwear Co.\n3. Chromatic Sockworks&quot;,<br>    additional_kwargs: { function_call: undefined, tool_calls: undefined }<br>  },<br>  lc_namespace: [ &quot;langchain_core&quot;, &quot;messages&quot; ],<br>  content: &quot;1. Rainbow Soles\n2. Vivid Footwear Co.\n3. Chromatic Sockworks&quot;,<br>  name: undefined,<br>  additional_kwargs: { function_call: undefined, tool_calls: undefined }<br>}</pre><h4>Output Parser</h4><p>In certain scenarios, it’s more convenient to work with the raw string value of the output rather than its default format (e.g., an AI message object). LangChain addresses this need through the use of an ‘output parser’. An output parser is a tool within LangChain that transforms the chat model output into a different format, such as a simple string.</p><pre>import { StringOutputParser } from &quot;@langchain/core/output_parsers&quot;;<br><br>const outputParser = new StringOutputParser();<br>const nameGenerationChain = prompt.pipe(model).pipe(outputParser);<br><br>await nameGenerationChain.invoke({<br>    product: &quot;fancy cookies&quot;<br>});<br><br>##&gt; &quot;1. Gourmet Cookie Creations\n&quot; +<br>  &quot;2. Delicate Delights Bakery\n&quot; +<br>  &quot;3. Heavenly Sweet Treats Co.&quot;</pre><h4>Stream</h4><p>Runnables in LangChain, as well as sequences of runnables, are equipped with a `.stream` method. This method is particularly useful for handling LLM responses that take a long time to generate. It returns the output in an iterable stream, allowing for a more responsive presentation of data, especially in situations where rapid feedback is important.</p><p>In this example, the chain is executed with a different product name (“really cool robots”), and the async iterator syntax (`for await…of`) is used to loop over and display the stream chunks. This results in getting individual string chunks of the output, such as names of companies, which can be displayed more quickly in a frontend application.</p><pre>const stream = await nameGenerationChain.stream({<br>  product: &quot;really cool robots&quot;,<br>});<br><br>for await (const chunk of stream) {<br>    console.log(chunk);<br>}</pre><h4>Batch</h4><p>The `batch` method is useful for performing multiple operations concurrently. This method allows for handling multiple inputs simultaneously, leading to efficient generation of responses. Given where a set of inputs, such as “large calculators” and “alpaca-wool-sweaters”, are processed using the `.batch` method. This results in multiple string outputs corresponding to different company names for each product type. This showcases the method’s ability to handle and generate responses for multiple queries at once.</p><pre>const inputs = [<br>    { product: &quot;large calculators&quot; },<br>    { product: &quot;alpaca wool sweaters&quot; }<br>];<br><br>await nameGenerationChain.batch(inputs);<br><br>##&gt; [<br>  &quot;1. GiantCalc Co.\n2. MegaMath Devices\n3. JumboCalculations Inc.&quot;,<br>  &quot;1. Alpaca Luxe\n2. Sweater Alpaca\n3. Woolly Alpaca Co.&quot;<br>]</pre><h3>Understanding Retrieval Augmented Generation (RAG)</h3><p>RAG, short for Retrieval Augmented Generation, is a technique that leverages the capabilities of LLMs to generate text while keeping contextual information in mind. It follows a basic workflow:</p><h4>Document Loading with LangChainJS</h4><p>LangChainJS offers a powerful suite of document loaders that can collect data from various sources, including the web and proprietary sources, specifically tailored for JavaScript applications. For instance, you can use the GitHub loader provided by LangChainJS to access JavaScript repositories and retrieve files. LangChainJS even allows you to customize the loading process by specifying parameters like file paths and exclusions.</p><p>To load a GitHub repository like LangChain.js using LangChainJS, you can initialize the loader, set options, and load the data. This flexibility makes it easy to incorporate JavaScript-based data from diverse sources into your RAG applications using LangChainJS.</p><pre>##&gt; Loading from Github repo:<br><br>import { GithubRepoLoader } from &quot;langchain/document_loaders/web/github&quot;;<br>// Peer dependency, used to support .gitignore syntax<br>import ignore from &quot;ignore&quot;;<br><br>// Will not include anything under &quot;ignorePaths&quot;<br>const loader = new GithubRepoLoader(<br>  &quot;https://github.com/langchain-ai/langchainjs&quot;,<br>  { recursive: false, ignorePaths: [&quot;*.md&quot;, &quot;yarn.lock&quot;] }<br>);<br><br>const docs = await loader.load();<br>console.log(docs.slice(0, 3));<br><br>##&gt; [<br>  Document {<br>    pageContent: &quot;coverage:\n&quot; +<br>      &quot;  status:\n&quot; +<br>      &quot;    project:\n&quot; +<br>      &quot;      default:\n&quot; +<br>      &quot;        informational: true\n&quot; +<br>      &quot;    patch:\n&quot; +<br>      &quot;      default&quot;... 151 more characters,<br>    metadata: {<br>      source: &quot;.codecov.yml&quot;,<br>      repository: &quot;https://github.com/langchain-ai/langchainjs&quot;,<br>      branch: &quot;main&quot;<br>    }<br>  },<br>  Document {<br>    pageContent: &quot;# top-most EditorConfig file\n&quot; +<br>      &quot;root = true\n&quot; +<br>      &quot;\n&quot; +<br>      &quot;# Unix-style newlines with a newline ending every file\n&quot; +<br>      &quot;[*]&quot;... 17 more characters,<br>    metadata: {<br>      source: &quot;.editorconfig&quot;,<br>      repository: &quot;https://github.com/langchain-ai/langchainjs&quot;,<br>      branch: &quot;main&quot;<br>    }<br>  },<br>  Document {<br>    pageContent: &quot;* text=auto eol=lf&quot;,<br>    metadata: {<br>      source: &quot;.gitattributes&quot;,<br>      repository: &quot;https://github.com/langchain-ai/langchainjs&quot;,<br>      branch: &quot;main&quot;<br>    }<br>  }<br>]</pre><pre>## Loading from PDF<br><br>// Peer dependency<br>import * as parse from &quot;pdf-parse&quot;;<br>import { PDFLoader } from &quot;langchain/document_loaders/fs/pdf&quot;;<br><br>const loader = new PDFLoader(&quot;./data/MachineLearning-Lecture01.pdf&quot;);<br><br>const rawCS229Docs = await loader.load();<br>console.log(rawCS229Docs.slice(0, 5));<br><br>##&gt;<br>[<br>  Document {<br>    pageContent: &quot;MachineLearning-Lecture01  \n&quot; +<br>      &quot;Instructor (Andrew Ng): Okay. Good morning. Welcome to CS229, the machin&quot;... 2999 more characters,<br>    metadata: {<br>      source: &quot;./data/MachineLearning-Lecture01.pdf&quot;,<br>      pdf: {<br>        version: &quot;1.10.100&quot;,<br>        info: {<br>          PDFFormatVersion: &quot;1.4&quot;,<br>          IsAcroFormPresent: false,<br>          IsXFAPresent: false,<br>          Title: &quot;&quot;,<br>          Author: &quot;&quot;,<br>          Creator: &quot;PScript5.dll Version 5.2.2&quot;,<br>          Producer: &quot;Acrobat Distiller 8.1.0 (Windows)&quot;,<br>          CreationDate: &quot;D:20080711112523-07&#39;00&#39;&quot;,<br>          ModDate: &quot;D:20080711112523-07&#39;00&#39;&quot;<br>        },<br>        metadata: Metadata { _metadata: [Object: null prototype] },<br>        totalPages: 22<br>      },<br>      loc: { pageNumber: 1 }<br>    }<br>  },<br>  ...</pre><h4>Splitting Documents for Contextual Clarity</h4><p>To ensure that the generated content is coherent and contextually relevant within JavaScript applications, it’s crucial to split documents sensibly. LangChainJS offers different strategies for splitting data, specifically designed for JavaScript content.</p><p>For example, when dealing with JavaScript code snippets from GitHub repositories, LangChainJS allows you to split documents at code-specific delimiters, ensuring that related code remains together. This approach makes it easier for LLMs to understand and generate code snippets accurately in a JavaScript context.</p><p>In contrast, for textual content in JavaScript applications, you can use a recursive character text splitter provided by LangChainJS. This splitter intelligently splits text into paragraphs, maintaining the flow of ideas and helping LLMs generate coherent text within the JavaScript environment.</p><pre>import { RecursiveCharacterTextSplitter } from &quot;langchain/text_splitter&quot;;<br><br>const splitter = RecursiveCharacterTextSplitter.fromLanguage(&quot;js&quot;, {<br>  chunkSize: 32,<br>  chunkOverlap: 0,<br>});<br>const code = `function helloWorld() {<br>console.log(&quot;Hello, World!&quot;);<br>}<br>// Call the function<br>helloWorld();`;<br><br>await splitter.splitText(code);<br><br>##&gt; [<br>  &quot;function helloWorld() {&quot;,<br>  &#39;console.log(&quot;Hello, World!&quot;);\n}&#39;,<br>  &quot;// Call the function&quot;,<br>  &quot;helloWorld();&quot;<br>]<br><br><br>import { CharacterTextSplitter } from &quot;langchain/text_splitter&quot;;<br><br>const splitter = new CharacterTextSplitter({<br>  chunkSize: 32,<br>  chunkOverlap: 0,<br>  separator: &quot; &quot;<br>});<br><br>await splitter.splitText(code);<br><br>##&gt; [<br>  &quot;function helloWorld()&quot;,<br>  &#39;{\nconsole.log(&quot;Hello,&#39;,<br>  &#39;World!&quot;);\n}\n// Call the&#39;,<br>  &quot;function\nhelloWorld();&quot;<br>]<br><br>const splitter = RecursiveCharacterTextSplitter.fromLanguage(&quot;js&quot;, {<br>  chunkSize: 64,<br>  chunkOverlap: 32,<br>});<br><br>await splitter.splitText(code);<br><br>##&gt; [<br>  &#39;function helloWorld() {\nconsole.log(&quot;Hello, World!&quot;);\n}&#39;,<br>  &#39;console.log(&quot;Hello, World!&quot;);\n}\n// Call the function&#39;,<br>  &quot;}\n// Call the function\nhelloWorld();&quot;<br>]<br><br>const splitter = new RecursiveCharacterTextSplitter({<br>  chunkSize: 512,<br>  chunkOverlap: 64,<br>});<br>const splitDocs = await splitter.splitDocuments(rawCS229Docs);<br>console.log(splitDocs.slice(0, 5));<br><br>##&gt; [<br>  Document {<br>    pageContent: &quot;MachineLearning-Lecture01  \n&quot; +<br>      &quot;Instructor (Andrew Ng): Okay. Good morning. Welcome to CS229, the machin&quot;... 352 more characters,<br>    metadata: {<br>      source: &quot;./data/MachineLearning-Lecture01.pdf&quot;,<br>      pdf: {<br>        version: &quot;1.10.100&quot;,<br>        info: {<br>          PDFFormatVersion: &quot;1.4&quot;,<br>          IsAcroFormPresent: false,<br>          IsXFAPresent: false,<br>          Title: &quot;&quot;,<br>          Author: &quot;&quot;,<br>          Creator: &quot;PScript5.dll Version 5.2.2&quot;,<br>          Producer: &quot;Acrobat Distiller 8.1.0 (Windows)&quot;,<br>          CreationDate: &quot;D:20080711112523-07&#39;00&#39;&quot;,<br>          ModDate: &quot;D:20080711112523-07&#39;00&#39;&quot;<br>        },<br>        metadata: Metadata { _metadata: [Object: null prototype] },<br>        totalPages: 22<br>      },<br>      loc: { pageNumber: 1, lines: { from: 1, to: 6 } }<br>    }<br>  },<br>  ...</pre><p>Now, we move on to the next critical step: embedding these chunks and storing them in a vector database for efficient retrieval based on user queries.</p><h3>Leveraging Text Embedding Models</h3><p>To enable the retrieval of relevant chunks from your vector database, you’ll need a text embedding model. We’re using OpenAI’s hosted model, but you can easily replace it with an embedding provider of your choice. The vector database is a specialized database equipped with natural language search capabilities.</p><p>When a user submits a query, the system searches the vector database for an embedding similar to the query. This search results in a set of relevant chunks that the LLM uses to generate its final output.</p><p>To demonstrate how this works, we start with the first step: document ingestion. This process involves using the embeddings model, a specialized machine learning model, to convert document contents into vectors. We use OpenAI’s hosted embeddings for this demonstration.</p><h4>Document Ingestion with LangChainJS</h4><p>In LangChainJS, you’ll import the embeddings model and instantiate it. It’s essential to set up your environment variables to obtain the required authentication key for the hosted embeddings. For this demo, we use an in-memory vector store, which is not suitable for production but serves our illustrative purposes well.</p><p>The result of embedding a query is a vector, represented as an array of numbers. These numbers capture various abstract features of the embedded text, enabling efficient searching within the vector database.</p><pre>import { OpenAIEmbeddings } from &quot;@langchain/openai&quot;;<br><br>const embeddings = new OpenAIEmbeddings();<br><br>await embeddings.embedQuery(&quot;This is some sample text&quot;);<br><br>##&gt; [<br>   -0.010464199,  0.0023175413,   -0.000743401,  -0.010936564,  -0.011474536,<br>     0.02293595,  -0.014735167,   0.0017451267,   -0.01760872,   -0.01938009,<br>   0.0051369704,    0.03411526,   -0.012274932,  0.0019403052,  0.0046383627,<br>     0.01308845,    0.02470732,   0.0018664983,  0.0044677868,  -0.006327724,<br>     ...</pre><h4>Searching for Relevant Chunks</h4><p>To illustrate the search process, we calculate the similarity between different embeddings using Cosine Similarity. We compare a vector generated from a query to two different vectors: one related and one unrelated. The similarity score helps us determine the relevance of the content.</p><p>Once we have prepared our documents by splitting them into smaller chunks, we initialize the vector store and add these chunks to it. This action populates a searchable vector store.</p><p>LangChainJS provides a convenient interface for searching directly with natural language queries. We use the similarity search method with a query like “What is Deeplearning?” and retrieve four related documents from our vector store. These documents contain content related to Deep learning, machine learning, and learning algorithms.</p><pre>import { similarity } from &quot;ml-distance&quot;;<br><br>const vector1 = await embeddings.embedQuery(<br>    &quot;What are vectors useful for in machine learning?&quot;<br>);<br>const unrelatedVector = await embeddings.embedQuery(<br>    &quot;A group of parrots is called a pandemonium.&quot;<br>);<br><br>similarity.cosine(vector1, unrelatedVector);<br>## &gt; 0.6962143564715657<br><br><br>const similarVector = await embeddings.embedQuery(<br>    &quot;Vectors are representations of information.&quot;<br>);<br><br>similarity.cosine(vector1, similarVector);<br>## &gt; 0.8590074042429146</pre><h4>The Power of Retrievers</h4><p>Now, let’s explore retrievers. Retrievers are a broader abstraction within LangChainJS that retrieve documents related to a given natural language query. Unlike vector stores, retrievers implement the invoke method and are expression language runnables. This means they can be seamlessly integrated into chains with other modules, such as LLMs, after parsers, and prompts.</p><p>We can easily instantiate a retriever from an existing vector store with a simple function call, making it a versatile tool for constructing retrieval chains.</p><pre>// Peer dependency<br>import * as parse from &quot;pdf-parse&quot;;<br>import { PDFLoader } from &quot;langchain/document_loaders/fs/pdf&quot;;<br>import { <br>    RecursiveCharacterTextSplitter<br>} from &quot;langchain/text_splitter&quot;;<br>​import { MemoryVectorStore } from &quot;langchain/vectorstores/memory&quot;;<br><br>const loader = new PDFLoader(&quot;./data/MachineLearning-Lecture01.pdf&quot;);<br>​<br>const rawCS229Docs = await loader.load();<br>​<br>const splitter = new RecursiveCharacterTextSplitter({<br>  chunkSize: 128,<br>  chunkOverlap: 0,<br>});<br>​<br>const splitDocs = await splitter.splitDocuments(rawCS229Docs);<br><br>const vectorstore = new MemoryVectorStore(embeddings);<br><br>await vectorstore.addDocuments(splitDocs);<br><br>const retrievedDocs = await vectorstore.similaritySearch(<br>    &quot;What is deep learning?&quot;, <br>    4<br>);<br><br>const pageContents = retrievedDocs.map(doc =&gt; doc.pageContent);<br><br>pageContents<br><br>##&gt; [<br>  &quot;piece of research in machine learning, okay?&quot;,<br>  &quot;are using a learning algorithm, perhaps without even being aware of it.&quot;,<br>  &quot;some of my own excitement about machine learning to you.&quot;,<br>  &quot;of the class, and then we&#39;ll start to talk a bit about machine learning.&quot;<br>]<br><br>const retriever = vectorstore.asRetriever();<br>await retriever.invoke(&quot;What is deep learning?&quot;)<br><br>##&gt; [<br>  Document {<br>    pageContent: &quot;piece of research in machine learning, okay?&quot;,<br>    metadata: {<br>      source: &quot;./data/MachineLearning-Lecture01.pdf&quot;,<br>      pdf: {<br>        version: &quot;1.10.100&quot;,<br>        info: {<br>          PDFFormatVersion: &quot;1.4&quot;,<br>          IsAcroFormPresent: false,<br>          IsXFAPresent: false,<br>          Title: &quot;&quot;,<br>          Author: &quot;&quot;,<br>          Creator: &quot;PScript5.dll Version 5.2.2&quot;,<br>          Producer: &quot;Acrobat Distiller 8.1.0 (Windows)&quot;,<br>          CreationDate: &quot;D:20080711112523-07&#39;00&#39;&quot;,<br>          ModDate: &quot;D:20080711112523-07&#39;00&#39;&quot;<br>        },<br>        metadata: Metadata { _metadata: [Object: null prototype] },<br>        totalPages: 22<br>      },<br>      loc: { pageNumber: 8, lines: { from: 2, to: 2 } }<br>    }<br>  },<br>  ...</pre><h3>Constructing the Retrieval Chain</h3><p>Now, it’s time to build our retrieval chain, which will consist of several steps:</p><h4>1. Document Retrieval Wrapper</h4><p>We’ll create a step called for Document Retrieval in a Chain. While retrievers take string inputs, we prefer our chains to work with objects for flexibility. This step will accept an object with a “question” field as input and format the resulting document’s page content as strings.</p><pre>import { RunnableSequence } from &quot;@langchain/core/runnables&quot;;<br>import { Document } from &quot;@langchain/core/documents&quot;;<br><br>const convertDocsToString = (documents: Document[]): string =&gt; {<br>  return documents.map((document) =&gt; {<br>    return `&lt;doc&gt;\n${document.pageContent}\n&lt;/doc&gt;`<br>  }).join(&quot;\n&quot;);<br>};<br><br>/*<br>{<br>question: &quot;What is deep learning?&quot;<br>}<br>*/<br><br>const documentRetrievalChain = RunnableSequence.from([<br>    (input) =&gt; input.question,<br>    retriever,<br>    convertDocsToString<br>]);<br><br>const results = await documentRetrievalChain.invoke({<br>  question: &quot;What are the prerequisites for this course?&quot;<br>});<br>console.log(results);<br><br>##&gt; &lt;doc&gt;<br>course information handout. So let me just say a few words about parts of these. On the <br>third page, there&#39;s a section that says Online Resources.  <br>Oh, okay. Louder? Actually, could you turn up the volume? Testing. Is this better? <br>Testing, testing. Okay, cool. Thanks.<br>&lt;/doc&gt;<br>&lt;doc&gt;<br>of this class will not be very programming intensive, although we will do some <br>programming, mostly in either MATLAB or Octave. I&#39;ll say a bit more about that later.  <br>I also assume familiarity with basic probability and statistics. So most undergraduate <br>statistics class, like Stat 116 taught here at Stanford, will be more than enough. I&#39;m gonna <br>assume all of you know what random variables are, that all of you know what expectation <br>is, what a variance or a random variable is. And in case of some of you, it&#39;s been a while <br>since you&#39;ve seen some of this material. At some of the discussion sections, we&#39;ll actually <br>go over some of the prerequisites, sort of as a refresher course under prerequisite class.<br>...</pre><h4>2. Input and Output Coercion</h4><p>To bridge the gap between retrievers’ string input and chains’ object input, we’ll use a runnable map. This map will handle the conversion of inputs and outputs, allowing us to seamlessly pass the question through the chain.</p><pre>import { ChatPromptTemplate } from &quot;@langchain/core/prompts&quot;;<br><br>const TEMPLATE_STRING = `You are an experienced researcher, <br>expert at interpreting and answering questions based on provided sources.<br>Using the provided context, answer the user&#39;s question <br>to the best of your ability using only the resources provided. <br>Be verbose!<br><br>&lt;context&gt;<br><br>{context}<br><br>&lt;/context&gt;<br><br>Now, answer this question using the above context:<br><br>{question}`;<br><br>const answerGenerationPrompt = ChatPromptTemplate.fromTemplate(<br>    TEMPLATE_STRING<br>);<br><br>import { RunnableMap } from &quot;@langchain/core/runnables&quot;;<br><br>const runnableMap = RunnableMap.from({<br>  context: documentRetrievalChain,<br>  question: (input) =&gt; input.question,<br>});<br><br>await runnableMap.invoke({<br>    question: &quot;What are the prerequisites for this course?&quot;<br>})<br><br>##&gt; {<br>  question: &quot;What are the prerequisites for this course?&quot;,<br>  context: &quot;&lt;doc&gt;\n&quot; +<br>    &quot;course information handout. So let me just say a few words about parts of these. On the \n&quot; +<br>    &quot;third&quot;... 3063 more characters<br>}</pre><h4>3. Augmented Generation</h4><p>In this step, we’ll use a chat prompt template to generate human-readable responses. We’ll define a prompt and wrap it in a chat prompt template. To ensure the question is passed through the chain, we’ll use the runnable map we created earlier.</p><pre>import { ChatOpenAI } from &quot;@langchain/openai&quot;;<br>import { StringOutputParser } from &quot;@langchain/core/output_parsers&quot;;<br><br>const model = new ChatOpenAI({<br>    modelName: &quot;gpt-3.5-turbo-1106&quot;<br>});<br><br>const retrievalChain = RunnableSequence.from([<br>  {<br>    context: documentRetrievalChain,<br>    question: (input) =&gt; input.question,<br>  },<br>  answerGenerationPrompt,<br>  model,<br>  new StringOutputParser(),<br>]);<br><br>const answer = await retrievalChain.invoke({<br>  question: &quot;What are the prerequisites for this course?&quot;<br>});<br><br>console.log(answer);<br><br>##&gt; The prerequisites for this course include familiarity with basic probability and statistics, as well as basic linear algebra. The instructor assumes that most undergraduate statistics and linear algebra classes would provide sufficient background knowledge. Specifically, students are expected to already have an understanding of random variables, expectation...</pre><h4>Handling Follow-up Questions</h4><p>Now that we have our retrieval chain in place, let’s see how it performs with a follow-up question. We’ll ask, “Can you list them referring to the prerequisites of the course in bullet point form?” However, we encounter a limitation. The response states that the information doesn’t specify a specific list in bullet point form. This happens because LLMs lack inherent memory, and without chat history context, they struggle to understand references like “them.”</p><p>To address this issue and enhance the model’s ability to handle follow-up questions effectively, you can modify the prompt to consider chat history. However, another challenge arises. The vector store needs to return relevant documents based on the reference in the follow-up query. Currently, the vector store has no knowledge of what “them” refers to, creating a gap in context.</p><pre>const followupAnswer = await retrievalChain.invoke({<br>  question: &quot;Can you list them in bullet point form?&quot;<br>});<br><br>console.log(followupAnswer);<br><br>##&gt; - No specific list or items were mentioned in the provided context. Therefore, I cannot list them in bullet point form.<br><br>const docs = await documentRetrievalChain.invoke({<br>  question: &quot;Can you list them in bullet point form?&quot;<br>});<br><br>console.log(docs);<br><br>##&gt; &lt;doc&gt;<br>course information handout. So let me just say a few words about parts of these. On the <br>third page, there&#39;s a section that says Online Resources.  <br>Oh, okay. Louder? Actually, could you turn up the volume? Testing. Is this better? <br>Testing, testing. Okay, cool. Thanks.<br>&lt;/doc&gt;<br>&lt;doc&gt;<br>into four major sections. We&#39;re gonna talk about four major topics in this class, the first<br>...</pre><h3>Addressing Conversation History</h3><p>The problem arises when we ask follow-up questions that reference past information. For instance, if we previously asked for prerequisites and later ask, “Can you list them in bullet point form?” the model can’t understand what “them” refers to. To address this, we need to rephrase the follow-up question to make it standalone and reference-free.</p><p>Here’s a high-level overview of the solution:</p><p><strong>1. Save Chat History:</strong> Each time we pass through the chain, we’ll store the user’s question as a human message and the LLM’s response as an AI message. Later, we’ll provide this chat history as additional context in a prompt for the LLM.<br> <br><strong>2. Rephrase the Question:</strong> We’ll use an LLM to rephrase user input that may contain references to past chat history into a standalone question. This standalone question can be understood by both the vector store and the LLM.</p><pre>####&gt; Let&#39;s split and load our documents into a vector store and create a retriever. Then we will convert its output to a string.<br><br>import &quot;dote<br>import &quot;dotenv/config&quot;;<br>import { loadAndSplitChunks } from &quot;./lib/helpers.ts&quot;;<br>​<br>const splitDocs = await loadAndSplitChunks({<br>    chunkSize: 1536,<br>    chunkOverlap: 128<br>});<br>import { initializeVectorstoreWithDocuments } from &quot;./lib/helpers.ts&quot;;<br>​<br>const vectorstore = await initializeVectorstoreWithDocuments({<br>  documents: splitDocs,<br>});<br>const retriever = vectorstore.asRetriever();<br>import { RunnableSequence } from &quot;@langchain/core/runnables&quot;;<br>import { Document } from &quot;@langchain/core/documents&quot;;<br>​<br>const convertDocsToString = (documents: Document[]): string =&gt; {<br>  return documents.map((document) =&gt; {<br>    return `&lt;doc&gt;\n${document.pageContent}\n&lt;/doc&gt;`<br>  }).join(&quot;\n&quot;);<br>};<br>​<br>const documentRetrievalChain = RunnableSequence.from([<br>    (input) =&gt; input.question,<br>    retriever,<br>    convertDocsToString<br>]);<br><br><br>####&gt; now that we have a retriever, lets build a retriever chain.<br><br>import { ChatPromptTemplate } from &quot;@langchain/core/prompts&quot;;<br>​<br>const TEMPLATE_STRING = `You are an experienced researcher, <br>expert at interpreting and answering questions based on provided sources.<br>Using the provided context, answer the user&#39;s question <br>to the best of your ability using only the resources provided. <br>Be verbose!<br>​<br>&lt;context&gt;<br>​<br>{context}<br>​<br>&lt;/context&gt;<br>​<br>Now, answer this question using the above context:<br>​<br>{question}`;<br>​<br>const answerGenerationPrompt = ChatPromptTemplate.fromTemplate(<br>    TEMPLATE_STRING<br>);<br>import { ChatOpenAI } from &quot;@langchain/openai&quot;;<br>import { StringOutputParser } from &quot;@langchain/core/output_parsers&quot;;<br>​<br>const model = new ChatOpenAI({<br>    modelName: &quot;gpt-3.5-turbo-1106&quot;<br>});<br>const retrievalChain = RunnableSequence.from([<br>  {<br>    context: documentRetrievalChain,<br>    question: (input) =&gt; input.question,<br>  },<br>  answerGenerationPrompt,<br>  model,<br>  new StringOutputParser(),<br>]);<br><br>####&gt; Adding history<br><br>import { MessagesPlaceholder } from &quot;@langchain/core/prompts&quot;;<br>​<br>const REPHRASE_QUESTION_SYSTEM_TEMPLATE = <br>  `Given the following conversation and a follow up question, <br>rephrase the follow up question to be a standalone question.`;<br>​<br>const rephraseQuestionChainPrompt = ChatPromptTemplate.fromMessages([<br>  [&quot;system&quot;, REPHRASE_QUESTION_SYSTEM_TEMPLATE],<br>  new MessagesPlaceholder(&quot;history&quot;),<br>  [<br>    &quot;human&quot;, <br>    &quot;Rephrase the following question as a standalone question:\n{question}&quot;<br>  ],<br>]);<br>const rephraseQuestionChain = RunnableSequence.from([<br>      rephraseQuestionChainPrompt,<br>      new ChatOpenAI({ temperature: 0.1, modelName: &quot;gpt-3.5-turbo-1106&quot; }),<br>      new StringOutputParser(),<br>])<br>import { HumanMessage, AIMessage } from &quot;@langchain/core/messages&quot;;<br>​<br>const originalQuestion = &quot;What are the prerequisites for this course?&quot;;<br>​<br>const originalAnswer = await retrievalChain.invoke({<br>  question: originalQuestion<br>});<br>​<br>console.log(originalAnswer);<br>const chatHistory = [<br>  new HumanMessage(originalQuestion),<br>  new AIMessage(originalAnswer),<br>];<br>​<br>await rephraseQuestionChain.invoke({<br>  question: &quot;Can you list them in bullet point form?&quot;,<br>  history: chatHistory,<br>});<br><br>####&gt; Putting it all together<br><br>const convertDocsToString = (documents: Document[]): string =&gt; {<br>  return documents.map((document) =&gt; `&lt;doc&gt;\n${document.pageContent}\n&lt;/doc&gt;`).join(&quot;\n&quot;);<br>};<br>​<br>const documentRetrievalChain = RunnableSequence.from([<br>  (input) =&gt; input.standalone_question,<br>  retriever,<br>  convertDocsToString,<br>]);<br>const ANSWER_CHAIN_SYSTEM_TEMPLATE = `You are an experienced researcher, <br>expert at interpreting and answering questions based on provided sources.<br>Using the below provided context and chat history, <br>answer the user&#39;s question to the best of <br>your ability <br>using only the resources provided. Be verbose!<br>​<br>&lt;context&gt;<br>{context}<br>&lt;/context&gt;`;<br>​<br>const answerGenerationChainPrompt = ChatPromptTemplate.fromMessages([<br>  [&quot;system&quot;, ANSWER_CHAIN_SYSTEM_TEMPLATE],<br>  new MessagesPlaceholder(&quot;history&quot;),<br>  [<br>    &quot;human&quot;, <br>    &quot;Now, answer this question using the previous context and chat history:\n{standalone_question}&quot;<br>  ]<br>]);<br>import { HumanMessage, AIMessage } from &quot;@langchain/core/messages&quot;;<br>await answerGenerationChainPrompt.formatMessages({<br>  context: &quot;fake retrieved content&quot;,<br>  standalone_question: &quot;Why is the sky blue?&quot;,<br>  history: [<br>    new HumanMessage(&quot;How are you?&quot;),<br>    new AIMessage(&quot;Fine, thank you!&quot;)<br>  ]<br>});<br>import { RunnablePassthrough } from &quot;@langchain/core/runnables&quot;;<br>​<br>const conversationalRetrievalChain = RunnableSequence.from([<br>  RunnablePassthrough.assign({<br>    standalone_question: rephraseQuestionChain,<br>  }),<br>  RunnablePassthrough.assign({<br>    context: documentRetrievalChain,<br>  }),<br>  answerGenerationChainPrompt,<br>  new ChatOpenAI({ modelName: &quot;gpt-3.5-turbo&quot; }),<br>  new StringOutputParser(),<br>]);<br>import { RunnableWithMessageHistory } from &quot;@langchain/core/runnables&quot;;<br>import { ChatMessageHistory } from &quot;langchain/stores/message/in_memory&quot;;<br>const messageHistory = new ChatMessageHistory();<br>​<br>const finalRetrievalChain = new RunnableWithMessageHistory({<br>  runnable: conversationalRetrievalChain,<br>  getMessageHistory: (_sessionId) =&gt; messageHistory,<br>  historyMessagesKey: &quot;history&quot;,<br>  inputMessagesKey: &quot;question&quot;,<br>});<br>const originalQuestion = &quot;What are the prerequisites for this course?&quot;;<br>​<br>const originalAnswer = await finalRetrievalChain.invoke({<br>  question: originalQuestion,<br>}, {<br>  configurable: { sessionId: &quot;test&quot; }<br>});<br>​<br>const finalResult = await finalRetrievalChain.invoke({<br>  question: &quot;Can you list them in bullet point form?&quot;,<br>}, {<br>  configurable: { sessionId: &quot;test&quot; }<br>});<br>​<br>console.log(finalResult);<br><br>##&gt; - Familiarity with basic probability and statistics<br>   - Knowledge of random variables, expectation, variance<br>   - Basic understanding of probability theory<br>   - Recommended undergraduate statistics class such as Stat 116<br>- Familiarity with basic linear algebra<br>   - Understanding of matrices, vectors, matrix multiplication<br>   - Knowledge of matrix inversion<br>   - Familiarity with eigenvectors (beneficial but not required)<br>   - Recommended undergraduate linear algebra courses such as Math 51, 103, Math 113, or CS205</pre><h3>Conclusion</h3><p>Retrieval Augmented Generation (RAG) is a powerful application of Large Language Models, especially when working with JavaScript applications using LangChainJS. By effectively loading documents from various sources and splitting them sensibly, you can harness the full potential of LLMs to generate contextually relevant and coherent text.</p><p><strong>Note:</strong> This post specifically focuses on LangChainJS, the JavaScript version of LangChain. For information on the Python version of LangChain, please refer to our separate blog post dedicated to LangChain for Python applications here <a href="https://medium.com/alameda-dev/extract-useful-information-from-your-data-or-content-with-ai-rag-framework-a2310717b4fc">https://medium.com/alameda-dev/extract-useful-information-from-your-data-or-content-with-ai-rag-framework-a2310717b4fc</a></p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><p>AlamedaDev provides full-service end-to-end software. Experts in modern software development and AI solutions.</p><p>From #barcelona</p><p>Website: <a href="http://www.alamedadev.com/">www.alamedadev.com</a><br>Alameda-AI: <a href="http://www.alamedadev.com/">www.alameda.dev</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9212c2f5e0e7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/unlocking-the-power-of-retrieval-augmented-generation-rag-with-langchainjs-9212c2f5e0e7">Unlocking the Power of Retrieval Augmented Generation (RAG) with LangChainJS</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI in Agriculture: Challenges, Advantages, and Use Cases.]]></title>
            <link>https://medium.com/alameda-dev/ai-in-agriculture-challenges-advantages-and-use-cases-d5f0487e6c93?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/d5f0487e6c93</guid>
            <category><![CDATA[agro-tech]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[agriculture]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Thu, 14 Dec 2023 12:06:26 GMT</pubDate>
            <atom:updated>2024-09-01T19:16:27.846Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>In the evolving landscape of technology, Artificial Intelligence (AI) stands out as a transformative force, particularly in agriculture. The integration of AI in farming practices, a concept now commonly referred to as ‘Agrotech,’ is not just a passing trend but a significant step towards sustainable agriculture. This article delves into the challenges, benefits, and practical use cases of AI in agriculture, highlighting its pivotal role in shaping the future of farming.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cmFsmlRK2rxjiaZSvVmI9A.png" /></figure><h3>The Challenges of Implementing AI in Agriculture</h3><p><strong>Data Collection and Management:</strong> One of the primary challenges is the collection and management of large volumes of data essential for AI algorithms. This includes weather data, soil conditions, crop health, and more.</p><p><strong>Initial Investment: </strong>Implementing AI solutions often requires an initial investment in both technology and training.</p><p><strong>Technological Integration: </strong>Integrating AI with existing agricultural practices and machinery can be complex and requires technical expertise.</p><p><strong>Reliability and Accuracy:</strong> Ensuring the reliability and accuracy of AI predictions is crucial, as incorrect information can lead to detrimental farming decisions.</p><h3>The Benefits of AI in Agriculture</h3><p><strong>Precision Farming:</strong> AI enables precision agriculture, allowing farmers to optimize their inputs like water, fertilizer, and pesticides, thus enhancing crop yield and quality.</p><p><strong>Disease and Pest Detectio</strong>n: AI-powered image recognition can identify crop diseases and pest infestations early, enabling timely intervention.</p><p><strong>Resource Management: </strong>AI assists in efficient resource management, reducing waste, and ensuring sustainable practices.</p><p><strong>Predictive Analytics: </strong>AI algorithms can analyze data to predict various outcomes, such as weather patterns and crop yields, helping farmers make informed decisions.</p><h3>Practical Use Cases of AI in Agriculture</h3><p><strong>Crop Monitoring and Management: </strong>Using drones and satellite imagery, AI algorithms can monitor crop health and growth, providing insights for better crop management.</p><p><strong>Automated Machinery:</strong> AI-driven tractors and harvesters can perform tasks like planting, weeding, and harvesting more efficiently and with minimal human intervention.</p><p><strong>Soil and Water Management: </strong>AI systems can analyze soil conditions and manage irrigation systems to optimize water usage and enhance soil health.</p><p><strong>Supply Chain Optimization:</strong> AI can streamline agricultural supply chains, predicting demand, and optimizing distribution routes to reduce waste and improve market supply.</p><h3>Case Study : Vision Transformers (ViT) in Disease Detection</h3><p><strong>Project Overview</strong>:This project was a testament to the prowess of AI in combating agricultural challenges. We utilized Vision Transformers (ViT), an advanced AI model initially developed for image recognition tasks, to detect diseases in crop leaves. The model’s ability to analyze and interpret complex visual data made it a perfect fit for identifying subtle signs of diseases that are missed by the human eye.</p><p><strong>Implementation: </strong>The ViT model was trained on a vast dataset of crop leaf images, each labeled with specific disease markers. This training enabled the model to learn and identify various disease patterns with high precision. The implementation involved:</p><ul><li><strong>Data Collection: </strong>Gathering and annotating a comprehensive dataset of crop leaf images with various diseases.</li><li><strong>Model Training:</strong> Leveraging Google Brain’s ViT model, we adapted and fine-tuned it to recognize and categorize different plant diseases.</li><li><strong>Real-World Application:</strong> Deploying the model in field trials, where it assisted farmers in early disease detection, saving crops that might have otherwise been lost.</li></ul><p><strong>Practical Implementation</strong></p><p>Let’s delve into a practical example. We’ll use the Hugging Face Trainer API to train our ViT model on the beans dataset. First, we need to define key components like the model, training arguments, data collator, and metrics.</p><p><strong>1.Environment Setup</strong>: First, ensure you have the necessary libraries installed:</p><pre>!pip install transformers datasets</pre><p><strong>2. Load the Dataset: </strong>We will use the ‘beans’ dataset, which contains images of bean leaves categorized based on their health status.</p><pre>from datasets import load_dataset<br>ds = load_dataset(&quot;beans&quot;)</pre><p><strong>3. Preparing the Data: </strong>Utilize the ViTFeatureExtractor to process the images:</p><pre>from transformers import ViTFeatureExtractor<br>feature_extractor = ViTFeatureExtractor.from_pretrained(&#39;google/vit-base-patch16-224-in21k&#39;)</pre><p><strong>4. Data Transformation Function: </strong>Define a function to transform the data into a suitable format for the model:</p><pre>def transform(example_batch):<br>    inputs = feature_extractor(example_batch[&#39;image&#39;], return_tensors=&#39;pt&#39;)<br>    inputs[&#39;labels&#39;] = example_batch[&#39;labels&#39;]<br>    return inputs</pre><p><strong>5. Apply Transformations to Dataset: </strong>Apply the transformation to your dataset:</p><pre>prepared_ds = ds.with_transform(transform)</pre><p><strong>6. Define the Data Collator: </strong>Create a data collator function for batching:</p><pre>import torch<br>def collate_fn(batch):<br>    return {<br>        &#39;pixel_values&#39;: torch.stack([x[&#39;pixel_values&#39;] for x in batch]),<br>        &#39;labels&#39;: torch.tensor([x[&#39;labels&#39;] for x in batch])<br>    }</pre><p><strong>7. Load a Pre-Trained ViT Model: </strong>Load a ViT model pre-trained on a similar task, which can be fine-tuned on your dataset:</p><pre>from transformers import ViTForImageClassification<br>model = ViTForImageClassification.from_pretrained(&#39;google/vit-base-patch16-224-in21k&#39;, num_labels=3)</pre><p><strong>8. Define Training Arguments: </strong>Set up the training arguments:</p><pre>from transformers import TrainingArguments<br>training_args = TrainingArguments(<br>    output_dir=&#39;./results&#39;, <br>    num_train_epochs=3, <br>    per_device_train_batch_size=4, <br>    per_device_eval_batch_size=4, <br>    warmup_steps=500, <br>    weight_decay=0.01, <br>    logging_dir=&#39;./logs&#39;, <br>    logging_steps=10<br>)</pre><p><strong>9. Define Metrics for Evaluation: </strong>Implement a function to compute metrics for evaluation:</p><pre>from sklearn.metrics import accuracy_score<br>def compute_metrics(p):<br>    return {&quot;accuracy&quot;: accuracy_score(p.label_ids, p.predictions.argmax(-1))}</pre><p><strong>10. Initialize the Trainer: </strong>Instantiate the Trainer class with all the components:</p><pre>from transformers import Trainer<br>trainer = Trainer(<br>    model=model,<br>    args=training_args,<br>    train_dataset=prepared_ds[&#39;train&#39;],<br>    eval_dataset=prepared_ds[&#39;validation&#39;],<br>    data_collator=collate_fn,<br>    compute_metrics=compute_metrics<br>)</pre><p><strong>11. Model Training: </strong>Start training the model:</p><pre>trainer.train()</pre><p><strong>11. Evaluate the Model:</strong> After training, evaluate the model’s performance on the validation set:</p><pre>trainer.evaluate()</pre><p><strong>Outcomes: </strong>The project not only improved crop yield and health but also served as a valuable tool for agricultural education and awareness. It demonstrated how advanced AI techniques could be effectively adapted for critical real-world applications in agriculture.</p><h3>Agricultural Applications of ViT</h3><p>Vision Transformers, with their advanced image analysis capabilities, can be utilized in numerous ways to enhance agricultural practices:</p><p><strong>Early Disease Detection: </strong>ViT can be used to detect plant diseases at an early stage, enabling prompt treatment and reducing the spread of infections. This capability is crucial for maintaining crop health and yield.</p><p><strong>Pest Infestation Identification:</strong> Similar to disease detection, ViT models can be trained to identify pest infestations in crops, allowing for timely pest control measures.</p><p><strong>Crop Quality Assessment:</strong> ViT can analyze images to assess the quality of crops, identifying any deficiencies or anomalies. This helps in ensuring the quality of the produce before it reaches the market.</p><p><strong>Weed Detection and Management: </strong>By distinguishing between crops and weeds, ViT can play a significant role in automated weed control, thus aiding in efficient farm management.</p><p><strong>Yield Prediction: </strong>Analyzing crop growth patterns and health, ViT models can contribute to accurate yield predictions, which are essential for supply chain and market planning.</p><p><strong>Soil Health Monitoring: </strong>ViT can be used to analyze images of soil to assess its health and fertility, providing valuable information for optimizing crop planting and soil treatment strategies.</p><p><strong>Irrigation Management:</strong> By detecting variations in crop health and soil moisture levels, ViT can assist in optimizing irrigation schedules and water usage.</p><p><strong>Phenotyping for Crop Breeding:</strong> In crop breeding, ViT can help in phenotyping — the process of measuring and analyzing observable plant characteristics, which is vital for breeding more resilient and productive crop varieties.</p><p><strong>Post-Harvest Analysis: </strong>Post-harvest, ViT can be used to assess the quality and condition of produce, helping in sorting and grading, thus ensuring optimal market value.</p><p><strong>Monitoring Environmental Impact: </strong>By analyzing the impact of various environmental factors on crop health, ViT can provide insights for sustainable farming practices.</p><h3>Explore and Experiment</h3><p>The future of agriculture lies in the synergy between technology and traditional farming practices. Exploring AI applications in agriculture, like the Vision Transformer for disease detection, offers a glimpse into a future where farming is more efficient, sustainable, and productive.</p><h3>What about other industries?</h3><p>ViT’s robust image analysis capabilities make it a versatile tool suitable for numerous applications</p><p><strong>Healthcare and Medical Imaging: </strong>ViT can be used for analyzing medical images, such as X-rays, MRIs, and CT scans, to assist in diagnosing diseases, identifying anomalies, and supporting surgical planning.</p><p><strong>Automotive Industry</strong>: In the automotive sector, ViT can enhance the development of autonomous vehicles by improving object detection and scene understanding in real-time navigation.</p><p><strong>Manufacturing and Quality Control: </strong>ViT can be employed for visual inspection on manufacturing lines, detecting defects in products or components, and ensuring consistent quality.</p><p><strong>Retail and Customer Experience: </strong>In retail, ViT can analyze customer behavior through surveillance cameras, help in inventory management through product recognition, and enhance the shopping experience.</p><p><strong>Environmental Monitoring:</strong> ViT can be used for analyzing satellite and aerial imagery for environmental monitoring, including tracking changes in land use, deforestation, and effects of climate change.</p><p><strong>Security and Surveillance: </strong>Vision Transformers can be applied to enhance security systems through facial recognition, abnormal activity detection, and monitoring crowded areas.</p><p><strong>Entertainment and Media:</strong> In the media industry, ViT can aid in content moderation, enhancing visual effects, and creating personalized user experiences.</p><p><strong>Urban Planning and Development: </strong>ViT can assist in analyzing urban landscapes, aiding in planning and development decisions based on the visual data of cities and infrastructure.</p><p>The potential of Vision Transformers extends across these diverse sectors, demonstrating the technology’s adaptability and its capacity to tackle a wide array of challenges. As AI continues to evolve, we anticipate the applications of ViT to broaden further, providing innovative, human-centric solutions to complex issues in various industries.</p><p>AlamedaDev is a full life cycle software solutions company. We provide Modern Software Development and #AI integrations.</p><p>Founded in #barcelona</p><p>Website: <a href="http://www.alamedadev.com/">www.alamedadev.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d5f0487e6c93" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/ai-in-agriculture-challenges-advantages-and-use-cases-d5f0487e6c93">AI in Agriculture: Challenges, Advantages, and Use Cases.</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Maximizing Enterprise Performance Through AI Interoperability]]></title>
            <link>https://medium.com/alameda-dev/maximizing-enterprise-performance-through-ai-interoperability-767702f61d7a?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/767702f61d7a</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ineroperability]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[enterprise]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Fri, 01 Dec 2023 18:32:07 GMT</pubDate>
            <atom:updated>2023-12-13T08:10:12.659Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KJUDncDRpftSR2uoCX1PIQ.jpeg" /></figure><p><strong>Introduction</strong></p><p>In the fast-evolving landscape of technology, AI interoperability has emerged as a critical concept for forward-thinking enterprises. It’s essential for businesses aiming to leverage the full potential of AI technology effectively and sustainably.</p><p><strong>Understanding AI Interoperability</strong></p><p>AI interoperability is the ability of different AI systems and models to work together cohesively. This capability is crucial in a business ecosystem that increasingly relies on a variety of AI tools for diverse functionalities. Proper integration of these tools can lead to enhanced efficiency, innovation, and competitive advantage.</p><p><strong>Why AI Interoperability Is relevant for Businesses</strong></p><p>For businesses, interoperability means flexibility and resilience. It allows companies to leverage the strengths of <strong>various AI models</strong>, ensuring continuous operation even if one model becomes unavailable or undergoes significant changes. This adaptability is particularly vital for enterprise marketing teams, whose operations are increasingly data-driven and reliant on AI technologies.</p><p><strong>AlamedaDev’s Approach to AI Integration and Interoperability</strong></p><p>AlamedaDev’s approach to AI interoperability involves a detailed process of integrating AI models into existing software systems. This process starts with an in-depth analysis of the client’s current technological infrastructure and business needs.</p><p><strong>Key benefits of this integration include:</strong></p><ul><li><strong>Enhanced Efficiency and Automation</strong>: Streamlining operations by automating routine tasks.</li><li><strong>Data-Driven Insights</strong>: Leveraging AI for advanced data analysis and predictive analytics.</li><li><strong>Customization and Personalization</strong>: Improving the user experience and customer engagement.</li><li><strong>Scalability: </strong>Adapting to evolving business needs and market conditions.</li><li><strong>Cost Optimization</strong>: Reducing operational expenses through smarter resource allocation.</li><li><strong>Risk Management: </strong>Enhancing security and compliance through advanced AI capabilities.</li></ul><p><strong>The Future of AI Interoperability</strong></p><p>As the AI landscape rapidly evolves towards more specialized models tailored for specific applications, the emphasis on effective interoperability grows stronger. This shift underlines the increasing <strong>necessity for enterprises to partner with development agencies capable of integrating, developing, and maintaining these sophisticated AI models continuously</strong>.</p><p>Such collaborations are crucial in ensuring that businesses not only deploy AI solutions that are finely tuned to their unique requirements but also sustain an adaptive and evolving approach in a dynamically changing market.</p><p>This trend towards AI specialization demands a collaborative approach, where a detailed understanding of business needs is matched with the technical skill to craft customized and effective solutions. Ultimately, this enables businesses to not only keep pace with current technology, but also to anticipate and lead future innovations.</p><p>Conclusion</p><p>In conclusion, as the AI landscape continues to evolve, interoperability will play a critical role in ensuring that enterprises can fully leverage the power of AI. At AlamedaDev, we’re committed to leading this evolution, offering free consultancy to help businesses integrate AI in a way that’s not only flexible and secure, but also human-centric and efficient.</p><p>AlamedaDev is a full life cycle software solutions company. We provide Modern Software Development and #AI integrations.</p><p>Founded in #barcelona</p><p>Website: <a href="http://www.alamedadev.com/">www.alamedadev.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=767702f61d7a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/maximizing-enterprise-performance-through-ai-interoperability-767702f61d7a">Maximizing Enterprise Performance Through AI Interoperability</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Extract useful information from your data or content with AI (RAG framework)]]></title>
            <link>https://medium.com/alameda-dev/extract-useful-information-from-your-data-or-content-with-ai-rag-framework-a2310717b4fc?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/a2310717b4fc</guid>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Mon, 27 Nov 2023 15:47:36 GMT</pubDate>
            <atom:updated>2023-12-13T08:10:53.739Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is the RAG framework</h3><p>The RAG (<strong>Retrieval-Augmented Generation</strong>) framework is an advanced method used in AI to enhance the way it understands and responds to questions.</p><p>Imagine AI as a librarian who not only knows a lot, but also knows exactly where to find specific information in a vast library. In the RAG framework, when you ask a question, the AI first acts as a “retrieval,” scanning through vast amounts of data to find relevant information, much like a librarian searching for books that could contain the answer.</p><p>Once it gathers this information, it shifts to a “generator” mode, where it combines and processes the gathered data to provide a comprehensive and accurate response, akin to the librarian summarizing the key points from various books.</p><p>This two-step process allows the AI to provide more detailed, accurate, and contextually relevant answers, making it particularly useful in scenarios where precise and thorough responses are required.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-NQtu9lqWUWIdqqf4e0IHw.jpeg" /></figure><h3>Project description</h3><p>This project aims to showcase the application of the Retrieval-Augmented Generation (RAG) framework to enhance the retrieval and presentation of information from the <a href="https://astro.build/">AstroJS</a> framework documentation.</p><p>Our source of data for this example will be the Astro Framework documentation. (<a href="https://docs.astro.build/en/">https://docs.astro.build/en/</a>). With LangChain we are going to access AstroJS documentation, load it, preprocess and transform it into embeddings (OpenAI).<br>Subsequently, we will save the embeddings in a local store (ChromaDB).<br>We then will access the embeddings using LangChain and recover the texts based on the question the user has prompted.</p><h3>Project setup</h3><p>Requirements</p><pre>chromadb<br>langchain<br>python-dotenv<br>bs4<br>tiktoken<br>streamlit<br>streamlit-chat</pre><p>sources folder</p><pre># In this folder we will have all the content of the documentation we want to use<br># I have stored here all th .hmlt files here<br># since we fdont want to make a lot of requests to the astro server each time we run the program.<br>Getting Started 🚀 Astro Documentation.html<br>Install Astro with the Automatic CLI 🚀 Astro Documentation.html<br>…</pre><h3>Document load</h3><p>Our first method will be for loading the sources documents.<br>This method will load and parse all the links in the file, so we can pass it to the next phase.</p><p>LangChain’s document loaders specialize in importing and transforming data from diverse sources and formats into a uniform structure. They are capable of processing data from a range of origins, including both structured and unstructured environments, such as online platforms, databases, and various proprietary and commercial services. These loaders are compatible with several file types, including documents, spreadsheets, and presentations. They convert these varied data inputs into a consistent document format, complete with content and relevant metadata.</p><p>LangChain offers an extensive collection of over 80 different document loaders we can choose based on our content, as seen in their Document Loaders page <a href="https://python.langchain.com/docs/integrations/document_loaders">https://python.langchain.com/docs/integrations/document_loaders</a>.</p><pre>def load_sources(self):<br>    docs = []<br>    if not os.path.exists(&#39;./sources&#39;):<br>        raise ValueError(&quot;The specified directory does not exist&quot;)<br><br>    # Construct the search pattern<br>    search_pattern = os.path.join(&#39;./sources&#39;, &#39;*.html&#39;)<br><br>    # Use glob to find all files that match the pattern<br>    html_files = glob.glob(search_pattern)<br><br>    for source in html_files:<br>      docs.extend(BSHTMLLoader(source).load())<br>      self.sources = docs</pre><h3>Document splitting</h3><p>Next, we need to split the document in chunks,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p2vGhelgvVjmHV0bAB3HHA.jpeg" /></figure><p>In database management, particularly with vector operations, processing and storing large documents can be resource-intensive.</p><p>Splitting these documents into smaller chunks enhances manageability and efficiency. This approach significantly reduces computational complexity and memory usage, making data handling more streamlined and effective.</p><p>Smaller vectors, derived from these segments, are easier and faster to process, optimizing both speed and resource utilization in the database system.</p><pre>def split_documents(self):<br>    splitter = RecursiveCharacterTextSplitter(<br>        chunk_size=500,<br>        chunk_overlap=100,<br>        separators=[&quot;\n\n&quot;, &quot;\n&quot;, &quot;(?&lt;=\. )&quot;, &quot; &quot;, &quot;&quot;]<br>    )<br>    self.splits = splitter.split_documents(self.sources)</pre><h3>Vector Stores and Embeddings</h3><p>After splitting a document into smaller chunks, we need to index them for easy retrieval during question-answering. For this, LangChain utilizes embeddings and vector stores.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l9ktQZLHuCnAXYGFlHZB3Q.jpeg" /></figure><p>Embeddings convert text into numerical vectors, enabling comparison of semantically similar content. Post text splitting, these embeddings are stored in a vector store, a specialized database for finding similar vectors. When answering a question, embeddings of the question are compared with stored vectors to identify the most relevant document segments.</p><p>These segments are then analyzed by a language model to generate an answer. This efficient process hinges on the effective use of embeddings and vector storage for quick and relevant data retrieval.</p><p>For this, we will use ChromaDB and OpenAI embeddings.</p><pre>def store_vectors(self):<br>    self.vector_store = Chroma.from_documents(<br>      documents=self.splits,<br>      embedding=OpenAIEmbeddings(),<br>      persist_directory=PATH_VECTOR_STORE<br>    )</pre><h3>Content Retrieval</h3><p>In the retrieval phase of LangChain’s document handling process, the primary goal is to efficiently locate and extract the document chunks most relevant to a given query, typically a question.</p><p>The query embedding is compared against the embeddings of all document chunks stored in the vector store. The comparison aims to identify which chunks have the closest semantic relationship to the query.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bxu-m-aYFmSQR9WMuqtaHg.jpeg" /></figure><p>Two key techniques used in this retrieval process are Maximum Marginal Relevance (MMR) and similarity scoring.</p><p><strong>Similarity scoring is straightforward, </strong>it involves calculating a numerical score that represents how closely the embeddings of the query and a document chunk match, often using methods like cosine similarity. The higher the score, the more relevant the chunk is deemed to be.</p><p>On the other hand, <strong>MMR</strong> introduces an additional layer. It not only considers the similarity between the query and the document chunks but also evaluates the <strong>uniqueness</strong> of the information each chunk provides. This technique helps in balancing the relevance of the document regarding the query while also ensuring diversity among the retrieved chunks. By preventing the retrieval of chunks that are too similar to each other, MMR enhances the comprehensiveness and richness of the information retrieved, making it particularly useful in scenarios where a broad understanding of a topic is desired.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/1*467mlBCv5ucrf7DV9l22og.jpeg" /></figure><p>In our case, we will use the MMR technique.</p><pre>def init_qa_retrieval(self):<br>    self.retrieval = RetrievalQA.from_chain_type(<br>        llm=OpenAI(),<br>        chain_type=&quot;map_reduce&quot;,<br>        retrieval=self.vector_store.as_retrieval(<br>          search_type=&quot;mmr&quot;<br>        )<br>    )</pre><p>The resulting UI will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EL3a7wlj0VVZMimbjq3BQQ.png" /></figure><h3>Other Use Cases</h3><p>Each of these use cases demonstrates the versatility of the RAG framework in processing and generating valuable insights across diverse domains:</p><p><strong>1. Customer Support Automation</strong>: RAG can be instrumental in enhancing customer support systems. By retrieving and analyzing previous customer queries and resolutions, the framework can generate informed responses to new customer inquiries. This application improves response accuracy and speed, significantly enhancing the customer experience.<br> <br><strong>2. Product Recommendation Systems</strong>: In e-commerce, RAG can be utilized to refine product recommendation engines. By evaluating a user’s browsing and purchase history, the system can retrieve relevant product information and generate personalized product suggestions, thereby increasing the likelihood of customer satisfaction and sales.<br> <br><strong>3. Medical Diagnosis Assistance:</strong> In healthcare, RAG can assist in diagnosing diseases by analyzing patient symptoms and medical history. By retrieving and processing vast amounts of medical data and research, it can generate potential diagnoses, aiding healthcare professionals in making more informed decisions.<br> <br><strong>4. Legal Document Analysis</strong>: For legal professionals, RAG can be used to analyze and summarize lengthy legal documents. By retrieving relevant legal precedents and articles, it can generate concise summaries or highlight key legal points in complex documents, saving time and enhancing the efficiency of legal research.</p><h3>Conclusion</h3><p>In summary, with AI’s advanced capabilities, coupled with the robust structure of the RAG framework, businesses can achieve a new level of insight and efficiency. This combination is key in navigating the complexities of today’s data-driven landscape, enabling smarter decision-making and fostering innovation.</p><p>If this intersection of AI and the RAG framework sparks your interest, especially in how it can be applied to your data sets, don’t hesitate to get in touch with us. We’re excited to discuss how these cutting-edge tools can be custom-fitted to your business needs, transforming your data into a powerhouse of strategic value.</p><p>Reach out for a chat, and let’s explore the possibilities together.</p><p>AlamedaDev is a full life cycle software solutions company. We provide Modern Software Development and #AI integrations.</p><p>Founded in #barcelona</p><p>Website: <a href="http://www.alamedadev.com/">www.alamedadev.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2310717b4fc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/extract-useful-information-from-your-data-or-content-with-ai-rag-framework-a2310717b4fc">Extract useful information from your data or content with AI (RAG framework)</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Use AI to create a story from an image]]></title>
            <link>https://medium.com/alameda-dev/use-ai-to-create-a-story-from-an-image-8ec5d8c50c0e?source=rss----e4980a567584---4</link>
            <guid isPermaLink="false">https://medium.com/p/8ec5d8c50c0e</guid>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[artificalintelligence]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[AlamedaDev]]></dc:creator>
            <pubDate>Fri, 03 Nov 2023 22:33:51 GMT</pubDate>
            <atom:updated>2023-12-13T08:09:38.755Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TYx9aWPzayqVsu2__RFPbw.jpeg" /></figure><p>Every image has a tale to tell, but what if we could unveil those hidden narratives with the power of artificial intelligence? We’re here to explore the intersection of computer vision and natural language processing. This tutorial delves into the practical application of Hugging Face AI models to turn images into captivating stories.</p><p>As we navigate this tutorial, you’ll discover how AI can discern the subtle nuances within an image, extract meaningful information, and weave it into eloquent narratives.</p><p><strong>Here is a video of the use case:</strong></p><p><a href="https://www.youtube.com/shorts/PrQQ2sLm8FI">Use AI to create a story from an image</a></p><h3>Understanding Hugging Face Models</h3><p>Hugging Face, a prominent name in the realm of artificial intelligence, is your gateway to a wide array of pre-trained language models and transformer architectures. Before we delve into the mechanics of creating stories from images, it’s crucial to understand why Hugging Face models are pivotal for this task.</p><p><strong>Versatility and Popularity:</strong> Hugging Face stands out for its versatility. The platform offers access to various pre-trained models that can tackle diverse natural language processing tasks. These models are widely recognized and adopted across a multitude of applications.<br> <br><strong>Community Collaboration:</strong> Hugging Face adopts an open-source and community-driven approach, allowing AI enthusiasts, developers, and researchers to contribute to model development and share their innovations. This collaborative spirit ensures that the models continuously evolve and improve.<br> <br><strong>Ease of Integration: </strong>Hugging Face models are designed for user-friendliness. They provide straightforward and intuitive APIs, catering to both experienced developers and newcomers to the world of AI. Whether you’re an expert in Python or just getting started with coding, working with Hugging Face models is relatively straightforward.<br> <br><strong>Top-Notch Performance: </strong>Hugging Face models have set benchmarks in various natural language understanding and generation tasks. They are renowned for their outstanding performance, making them a preferred choice for achieving high-quality results.</p><h3>Components for Image-to-Story Conversion</h3><p>In the journey to create stories from images using Hugging Face AI models, we’ll be breaking down the process into three crucial components. These components form the backbone of our image-to-story conversion pipeline:</p><ol><li><strong>Image to Text: Deciphering the Visual Scenario</strong><br>Our first step is to teach the machine to understand the scenario presented by an image. This process, known as image captioning, involves translating visual content into text descriptions. It’s the foundation upon which we build our storytelling magic.<br> <br> To achieve this, we’ll leverage computer vision models that can analyze the image’s content and generate descriptive text. These models have been trained on vast datasets, allowing them to recognize objects, scenes, and context within images. The result is a textual representation of the image, which serves as the starting point for our storytelling journey.</li><li><strong>Language Model Magic: Crafting Stories with LLMs</strong><br>With the image content converted into text, we move on to the storytelling phase. Here, we introduce the star of the show — Large Language Models (LLMs). These models are the creative minds behind the stories we’ll be generating.<br> <br>LLMs are massive neural networks that have been trained on extensive text corpora, making them masters of language. They can take a textual prompt, like the description of an image, and continue the narrative with engaging and coherent text. In our case, they will transform the image caption into a short story.<br> <br>The beauty of LLMs lies in their ability to generate contextually relevant and imaginative text. They provide a canvas upon which we can paint captivating narratives, all powered by the magic of artificial intelligence.</li><li><strong>Text to Speech: Giving Voice to Our Story</strong><br>Our storytelling journey doesn’t stop at written words. We want our stories to come to life, and that’s where text-to-speech models enter the scene. These models can convert our written stories into spoken words, generating audio that adds an extra layer of immersion to the experience.<br> <br>Text-to-speech models have made remarkable strides recently, offering human-like voice synthesis. They take the text output from our language model and turn it into an audio file that can be played and enjoyed.<br> <br>With this component, we bridge the gap between written and spoken storytelling, bringing our narratives to the ears of our audience.</li></ol><h3>Practical Example</h3><p>For the first part image to text, we are going to use the model Salesforce/blip-image-captioning-large <a href="https://huggingface.co/Salesforce/blip-image-captioning-large">https://huggingface.co/Salesforce/blip-image-captioning-large</a><br>The Salesforce/blip-image-captioning-large model is a powerful tool for image captioning. It’s been pre-trained on extensive datasets, making it adept at recognizing objects, scenes, and context within images. This model serves as our bridge between the visual and textual worlds.</p><p>You’ll need a Hugging Face account and a token for accusing the API.</p><p>First create a .env file and put your Hugging Face token there:<br>HUGGINGFACEHUB_API_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxx</p><p>Create an app.py file where we will be working:</p><pre>from dotenv import find_dotenv, load_dotenv<br>from transformers import pipeline<br>load_dotenv(find_dotenv())<br><br># image to text<br>def image_2_text(url):<br>    image_to_text = pipeline(&quot;image-to-text&quot;, model=&quot;Salesforce/blip-image-captioning-base&quot;)<br>    text = image_to_text(url)<br>    print(text)<br>    return text<br><br>image_2_text(&#39;photo.jpg&#39;)</pre><p>When executing this, you should be able to see a small description of the photo:</p><pre>[{&#39;generated_text&#39;: &#39;a castle with a tower&#39;}]</pre><p>For the second part, we’ll use the HuggingFaceH4/zephyr-7b-alpha <a href="https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha">https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha</a> model to generate a short story based on the text:</p><p>Add to the app.py file:</p><pre>def generate_story(scenario):<br>    pipe = pipeline(&quot;text-generation&quot;, model=&quot;HuggingFaceH4/zephyr-7b-alpha&quot;, torch_dtype=torch.float16, device_map=&quot;auto&quot;)<br><br>    messages = [<br>        {<br>            &quot;role&quot;: &quot;system&quot;,<br>            &quot;content&quot;: &quot;You are a story teller; You can generate a short story based on a single narrative, the story should be no more than 30 words long&quot;,<br>        },<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: str(scenario)},<br>    ]<br>    prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)<br>    outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)<br>    print(outputs[0][&quot;generated_text&quot;])<br>    return outputs[0][&quot;generated_text&quot;]</pre><p>For me, it returns something along this:</p><pre>In a far-off land, stood a castle grand,<br>With a tower so high, it touched the sky&#39;s command.</pre><p>For the last part Text to Speech, we are using espnet/kan-bayashi_ljspeech_vits <a href="https://huggingface.co/espnet/kan-bayashi_ljspeech_vits">https://huggingface.co/espnet/kan-bayashi_ljspeech_vits</a></p><pre>def substring_after(s, delim):<br>    return s.partition(delim)[2]<br><br>def text_to_speech(message):<br>    API_URL = &quot;https://api-inference.huggingface.co/models/espnet/kan-bayashi_ljspeech_vits&quot;<br>    headers = {&quot;Authorization&quot;: f&quot;Bearer {HUGGINGFACEHUB_API_TOKEN}&quot;}<br>    payloads = {<br>        &quot;inputs&quot;: substring_after(message, &quot;&lt;|assistant|&gt;&quot;)<br>    }<br>    response = requests.post(API_URL, headers=headers, json=payloads)<br><br>    with open(&#39;audio.flac&#39;, &#39;wb&#39;) as file:<br>        file.write(response.content)</pre><p>The resulting audio, it is like this: <br><a href="https://share.zight.com/8Lu6obd5">https://share.zight.com/8Lu6obd5</a></p><p>To execute the code via <a href="https://streamlit.io/">Streamlit</a>, here is the complete code, or in our repository here: <a href="https://github.com/uokesita/ImageToStory">https://github.com/uokesita/ImageToStory</a></p><pre>from dotenv import find_dotenv, load_dotenv<br>from transformers import pipeline<br>from langchain import PromptTemplate, LLMChain, OpenAI<br>import torch<br>import requests<br>import os<br>import re<br>import streamlit as st<br><br>load_dotenv(find_dotenv())<br>HUGGINGFACEHUB_API_TOKEN = os.getenv(&quot;HUGGINGFACEHUB_API_TOKEN&quot;)<br># image to text<br><br>def image_2_text(url):<br>    image_to_text = pipeline(&quot;image-to-text&quot;, model=&quot;Salesforce/blip-image-captioning-base&quot;)<br>    text = image_to_text(url)<br><br>    # print(text)<br>    return text<br><br># text to speech<br>def generate_story(scenario):<br>    pipe = pipeline(&quot;text-generation&quot;, model=&quot;HuggingFaceH4/zephyr-7b-alpha&quot;, torch_dtype=torch.float16, device_map=&quot;auto&quot;)<br><br>    messages = [<br>        {<br>            &quot;role&quot;: &quot;system&quot;,<br>            &quot;content&quot;: &quot;You are a story teller; You can generate a short story based on a single narrative, the story should be no more than 30 words long&quot;,<br>        },<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: str(scenario)},<br>    ]<br>    prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)<br>    outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)<br>    print(substring_after(outputs[0][&quot;generated_text&quot;], &quot;&lt;|assistant|&gt;&quot;))<br><br>    return substring_after(outputs[0][&quot;generated_text&quot;], &quot;&lt;|assistant|&gt;&quot;)<br><br>def substring_after(s, delim):<br>    return s.partition(delim)[2]<br><br>def text_to_speech(message):<br>    API_URL = &quot;https://api-inference.huggingface.co/models/espnet/kan-bayashi_ljspeech_vits&quot;<br>    headers = {&quot;Authorization&quot;: f&quot;Bearer {HUGGINGFACEHUB_API_TOKEN}&quot;}<br>    payloads = {<br>        &quot;inputs&quot;: message<br>    }<br>    response = requests.post(API_URL, headers=headers, json=payloads)<br><br>    with open(&#39;audio.flac&#39;, &#39;wb&#39;) as file:<br>        file.write(response.content)<br><br><br>def main():<br>    st.set_page_config(page_title=&quot;Use AI to create a story from an image&quot;)<br>    st.header(&quot;Use AI to create a story from an image&quot;)<br><br>    uploaded_file = st.file_uploader(&quot;Choose an image&quot;, type=&quot;jpg&quot;)<br><br>    if uploaded_file is not None:<br>        print(uploaded_file)<br>        bytes_data = uploaded_file.getvalue()<br>        with open(uploaded_file.name, &quot;wb&quot;) as file:<br>            file.write(bytes_data)<br>        st.image(uploaded_file, caption=&quot;Uploaded Image&quot;, use_column_width=True)<br>        scenario = image_2_text(uploaded_file.name)<br>        story = generate_story(scenario)<br>        text_to_speech(story)<br><br>        with st.expander(&quot;scenario&quot;):<br>            st.write(scenario)<br>        with st.expander(&quot;story&quot;):<br>            st.write(story)<br><br>        st.audio(&quot;audio.flac&quot;)<br><br>if __name__ == &#39;__main__&#39;:<br>    main()</pre><p>You can run it with:<br>streamlit run app.py</p><h3>Examples and Use Cases</h3><p>We’ll explore real-world examples and use cases of image-to-story conversion using Hugging Face AI models and the Salesforce/blip-image-captioning-large model.</p><p><strong>Enhancing Social Media Engagement: </strong>Imagine you’re a social media manager for a travel company. You can use image-to-story conversion to automatically generate captivating captions and stories for your travel photos. This saves time and increases engagement with your audience.<br> <br><strong>Accessibility and Inclusivity</strong>: Image-to-text conversion has profound implications for accessibility. By providing textual descriptions of images, we make digital content more inclusive for visually impaired individuals who rely on screen readers to access information.<br> <br><strong>Content Creation and Marketing:</strong> Content creators and marketers can leverage image-to-story conversion to breathe life into their visual content. Whether it’s generating product descriptions from images or crafting engaging narratives for advertising campaigns, the possibilities are vast.<br> <br><strong>Educational Resources:</strong> Educators can use this technology to create educational materials that bridge the gap between visual and textual content. Images in textbooks, for example, can be accompanied by automatically generated explanations.<br> <br><strong>News and Journalism:</strong> Journalists can use image captioning to briefly summarize the content of news images, making it easier to search and categorize visual content in news archives.<br> <br><strong>Artificial Intelligence in Art:</strong> In the realm of art, this technology can be used to generate artistic descriptions for paintings and visual artworks, adding a layer of interpretation to the viewer’s experience.</p><p>These are just a few examples of how image-to-text conversion can be applied across various domains. The versatility of this technology makes it a valuable tool in the hands of creators, educators, businesses, and anyone looking to enhance their content with AI-generated narratives.</p><h3>Conclusion</h3><p>As we wrap up this journey into the fascinating world of image-to-story conversion, let’s take a moment to recap what we’ve learned and the possibilities that lie ahead.</p><p><strong>Empowering Creativity:</strong> With the help of Hugging Face AI models like, we’ve unlocked the power to transform images into captivating stories. This technology empowers us to unleash our creativity and bring stories to life in ways we couldn’t have imagined before.<br> <br><strong>Real-World Impact:</strong> Image-to-text conversion has a significant impact on accessibility, content creation, marketing, education, journalism, and more. It’s a versatile tool that opens up new possibilities in various fields, making it easier to bridge the gap between visuals and text.<br> <br><strong>Continuous Exploration:</strong> The world of AI and natural language processing is continuously evolving. As you embark on your own image-to-story adventures, remember that innovation knows no bounds. Explore, experiment, and push the boundaries of what’s possible with AI.</p><p>— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —</p><p>AlamedaDev is a full life cycle software solutions company. We provide Modern Software Development and #AI integrations.</p><p>Founded in #barcelona</p><p>Website: <a href="http://www.alamedadev.com/">www.alamedadev.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8ec5d8c50c0e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alameda-dev/use-ai-to-create-a-story-from-an-image-8ec5d8c50c0e">Use AI to create a story from an image</a> was originally published in <a href="https://medium.com/alameda-dev">AlamedaDev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>