<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yoshihiro Ochi on Medium]]></title>
        <description><![CDATA[Stories by Yoshihiro Ochi on Medium]]></description>
        <link>https://medium.com/@yoshihiroochi?source=rss-716e06f019ac------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*yPxCoUrMnvwxxNus.</url>
            <title>Stories by Yoshihiro Ochi on Medium</title>
            <link>https://medium.com/@yoshihiroochi?source=rss-716e06f019ac------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 14:36:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yoshihiroochi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Using LangChain to Export Summaries of PDF Files on Google Drive to Google Spreadsheets]]></title>
            <link>https://medium.com/@yoshihiroochi/using-langchain-to-export-summaries-of-pdf-files-on-google-drive-to-google-spreadsheets-fcded0b28ff8?source=rss-716e06f019ac------2</link>
            <guid isPermaLink="false">https://medium.com/p/fcded0b28ff8</guid>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[google-drive]]></category>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[google-spreadsheets]]></category>
            <category><![CDATA[pdf]]></category>
            <dc:creator><![CDATA[Yoshihiro Ochi]]></dc:creator>
            <pubDate>Tue, 11 Apr 2023 08:20:02 GMT</pubDate>
            <atom:updated>2023-04-11T23:33:40.570Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>Background and Motivation</strong></h4><p>The original reason for creating this:</p><ul><li>I had accumulated many downloaded research papers in PDF format on my Drive</li><li>It was cumbersome to open each PDF and check the contents individually</li><li>Skimming through English content takes time</li><li>While ChatPDF is useful, it’s tedious to add each PDF individually</li></ul><p>On the other hand, I found a YouTube video that explains how to use LangChain to load PDFs and interactively perform QA using code, so I decided to give it a try.</p><h4><strong>Requirements</strong></h4><ul><li>Google account (assuming operation on Google Colab)</li><li>OpenAI API key (paid)</li><li>PDF files saved on Google Drive</li></ul><h4><strong>Preparation</strong></h4><ol><li>Create a folder named “pdf” directly under My Drive in Google Drive, and place the PDF files you want to summarize inside.</li><li>The screenshot below shows four PDF files placed as samples.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/423/0*BDrTD5PIXeItajKm" /></figure><h4><strong>Code</strong></h4><p>Run the following code in order on Google Colab. It will take a few minutes to install the packages.</p><pre># Install required packages<br>!pip install langchain openai pypdf chromadb tiktoken gspread</pre><p>Next, import the necessary libraries.</p><pre># Import libraries<br>from langchain.document_loaders import PyPDFLoader<br>from langchain.embeddings import OpenAIEmbeddings<br>from langchain.text_splitter import RecursiveCharacterTextSplitter<br>from langchain.vectorstores import Chroma<br>from langchain.chains import RetrievalQA<br>from langchain.chat_models import ChatOpenAI<br>import os</pre><p>Enter the obtained OpenAI API key in the “**********” part.</p><pre># Set up OPENAI_API_KEY<br>os.environ[&quot;OPENAI_API_KEY&quot;] = &quot;**********&quot;</pre><p>The prompt part to have GPT create sentences.</p><pre># Set query<br>query = &quot;&quot;&quot;Please output the given text in Japanese in the following format.```<br>・Summary<br><br>・Technical highlights<br> ・Highlight 1<br> ・Highlight 2<br> ・Highlight 3<br><br>・Classification keywords<br> ・Keyword 1<br> ・Keyword 2<br> ・Keyword 3<br>```&quot;&quot;&quot;</pre><p>Specify the location of the PDF files in the path.<br>It is set to target PDFs in subfolders under MyDrive/pdf/.<br>Press “Connect to Google Drive” when the confirmation dialog “Do you want to allow this notebook access to Google Drive files?” appears, then select the account and allow it. It may take several seconds to mount the drive.</p><pre># Load PDF files<br>from google.colab import drive<br>drive.mount(&#39;/content/gdrive/&#39;)<br>path = &#39;/content/gdrive/MyDrive/pdf/&#39;<br><br>import glob<br>files = glob.glob(path + &#39;**/*.pdf&#39;, recursive=True)</pre><p>Here’s the crucial part.<br>Load the PDF files one by one, split them into appropriate lengths, and give them to GPT to get answers to the query. Currently, the GPT model is set to gpt-3.5-turbo. It takes a few tens of seconds per PDF, so wait for a while.<br>A message “WARNING:chromadb:Using embedded DuckDB without persistence” may appear in the middle, but it seems to be harmless, so ignore it.</p><pre># GPT-based text generation<br>data_table = []<br><br>for file in files:<br>  data_record = []<br>  print(file.split(&quot;/&quot;)[-1].strip(&quot;.pdf&quot;))<br>  data_record.append(file.split(&quot;/&quot;)[-1])<br><br>  loader = PyPDFLoader(file)<br>  documents = loader.load()<br><br>  text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)<br>  texts = text_splitter.split_documents(documents)<br>  embeddings = OpenAIEmbeddings()<br>  vectordb = Chroma.from_documents(texts, embeddings)<br><br>  qa = RetrievalQA.from_chain_type(llm=ChatOpenAI(model_name=&quot;gpt-3.5-turbo&quot;), chain_type=&quot;stuff&quot;, retriever=vectordb.as_retriever())<br><br>  answer = qa.run(query)<br>  data_record.append(answer)<br><br>  data_table.append(data_record)</pre><p>Save the obtained answers to a Google Spreadsheet.<br>Allow access to Google authentication information when the confirmation dialog “Do you want to allow this notebook access to Google authentication information?” appears.<br>A file named “pdf_summary” will be created under the root of My Drive on Google Drive.</p><pre># Code for authentication<br>from google.colab import auth<br>auth.authenticate_user()<br><br>import gspread<br>from google.auth import default<br>creds, _ = default()<br><br>gc = gspread.authorize(creds)<br><br># Write to Google Spreadsheet<br>filename = &quot;pdf_summary&quot;<br>ss = gc.create(filename)<br>ss.values_append(&quot;シート1&quot;, {&quot;valueInputOption&quot;: &quot;USER_ENTERED&quot;}, {&quot;values&quot;: data_table})</pre><p>The following is the appearance of the generated Google Spreadsheet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/0*9LbgSTl7zWAqZdhb" /></figure><h4><strong>Caution</strong></h4><p>Since this uses the OpenAI API, there will be a cost involved. When testing with 50 papers, the cost was about the price of a canned coffee.</p><h4><strong>Reference Site</strong></h4><p>This very informative and easy-to-understand video was a big help, and most of the crucial parts were utilized directly from it.<br><a href="https://www.youtube.com/watch?v=eCtHVmXcXMs">https://www.youtube.com/watch?v=eCtHVmXcXMs</a></p><h4><strong>Moving Forward</strong></h4><p>This time, we created a basic tool that performs the desired task. Below are some ideas for making the tool even more convenient in the future.</p><ul><li>Support for non-PDF office documents, images, and video files</li><li>Automatically run when a file is saved to the drive</li><li>Crawl related papers</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fcded0b28ff8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image classification by Raspberry Pi Zero W + VisionBonnet accessory board + Camera Module Version…]]></title>
            <link>https://medium.com/@yoshihiroochi/image-classification-by-raspberry-pi-zero-w-visionbonnet-accessory-board-camera-camera-module-ef74b3da81a9?source=rss-716e06f019ac------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef74b3da81a9</guid>
            <category><![CDATA[visionbonnet]]></category>
            <category><![CDATA[raspberry-pi]]></category>
            <category><![CDATA[vision-kit]]></category>
            <dc:creator><![CDATA[Yoshihiro Ochi]]></dc:creator>
            <pubDate>Tue, 23 Jan 2018 13:10:24 GMT</pubDate>
            <atom:updated>2018-01-23T13:15:04.154Z</atom:updated>
            <content:encoded><![CDATA[<h3>Image classification by Raspberry Pi Zero W + VisionBonnet accessory board + Camera Module Version 1</h3><p>Implement camera image recognition using deep learning on Raspberry Pi Zero W (Pi Zero). By attaching VisionBonnet accessory board of <a href="https://aiyprojects.withgoogle.com/vision/">Vision Kit</a> to Pi Zero, almost real time classification processing is realized. Camera Module V2 is required for assembling the Vision Kit, but this time we will implement it using Camera Module V1.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F-ZhVdgR4h6w%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D-ZhVdgR4h6w&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F-ZhVdgR4h6w%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d7a8e8702acfdb5369b3fc3a836f30a/href">https://medium.com/media/6d7a8e8702acfdb5369b3fc3a836f30a/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef74b3da81a9" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>