<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Julian Shen on Medium]]></title>
        <description><![CDATA[Stories by Julian Shen on Medium]]></description>
        <link>https://medium.com/@jlnshn?source=rss-b2c7083a1bdf------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*8w3e7KWn_CiS55BC0VuZxQ.jpeg</url>
            <title>Stories by Julian Shen on Medium</title>
            <link>https://medium.com/@jlnshn?source=rss-b2c7083a1bdf------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 02:26:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jlnshn/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Translate eBooks Locally with Ollama + TranslateGemma]]></title>
            <link>https://medium.com/juliansplayground/translate-ebooks-locally-with-ollama-translategemma-fc657a8391b1?source=rss-b2c7083a1bdf------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc657a8391b1</guid>
            <dc:creator><![CDATA[Julian Shen]]></dc:creator>
            <pubDate>Sat, 02 May 2026 14:48:11 GMT</pubDate>
            <atom:updated>2026-08-31T15:39:21.372Z</atom:updated>
            <content:encoded><![CDATA[<p>So you’ve got an ebook… maybe it’s in English, maybe Japanese, maybe Chinese — and you just want it in <em>your</em> language. No cloud APIs, no fees, no weird privacy concerns.</p><p>Good news: you can do the whole thing locally with Calibre + a plugin + Ollama + a model called TranslateGemma.</p><p>It sounds fancy, but it’s actually pretty straightforward. Let’s walk through it.</p><h3>1) Install the Ebook Translator Plugin</h3><p>First, you need the plugin:</p><p>👉 <a href="https://github.com/bookfere/Ebook-Translator-Calibre-Plugin">https://github.com/bookfere/Ebook-Translator-Calibre-Plugin</a></p><h3>Quick steps</h3><ul><li>Open Calibre</li><li>Go to <strong>Preferences → Plugins</strong></li><li>Click <strong>“Load plugin from file”</strong></li><li>Select the downloaded .zip</li><li>Restart Calibre</li></ul><p>That’s it. You should now see <strong>Ebook Translator</strong> somewhere in your UI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/110/1*x2ePXL0z2ZlJHOga5IRL9A.png" /></figure><h3>2) What’s TranslateGemma + Ollama?</h3><p>Let’s keep this simple.</p><ul><li>Ollama = runs AI models locally on your machine</li><li>TranslateGemma = a model that’s <em>really good at translation</em></li></ul><p>Together, they basically replace Google Translate… but fully local and customizable.</p><h3>Install &amp; run</h3><pre># install ollama<br>curl -fsSL https://ollama.com/install.sh | sh</pre><pre># download the model<br>ollama pull translategemma</pre><pre># try it<br>ollama run translategemma</pre><p>Ollama will expose a local API at:</p><pre>http://127.0.0.1:11434</pre><p>We’ll hook Calibre into this next.</p><h3>3) Set Up a Custom Engine (the important bit)</h3><p>Now we connect the plugin to Ollama.</p><h3>Where to go</h3><p>Inside Calibre:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*vMhHz5jnIfcCg3duyilPlg.png" /></figure><ul><li>Open <strong>Ebook Translator</strong></li><li>Go to <strong>Settings / Engines</strong></li><li>Add a <strong>Custom Engine</strong></li></ul><p>Then paste this config:</p><pre>{<br>    &quot;name&quot;: &quot;Ollama - TranslateGemma&quot;,<br>    &quot;languages&quot;: {<br>        &quot;source&quot;: {<br>            &quot;English&quot;: &quot;English (en)&quot;,<br>            &quot;Chinese&quot;: &quot;Chinese (zh-Hans)&quot;, <br>            &quot;Japanese&quot;: &quot;Japanese (ja)&quot;<br>        },<br>        &quot;target&quot;: { <br>            &quot;English&quot;: &quot;English (en)&quot;,<br>            &quot;Chinese&quot;: &quot;Chinese (zh-TW)&quot;,<br>            &quot;Spanish&quot;: &quot;Spanish (es)&quot;,<br>            &quot;Japanese&quot;: &quot;Japanese (ja)&quot;<br>        }<br>    },<br>    &quot;request&quot;: {<br>        &quot;url&quot;: &quot;http://127.0.0.1:11434/api/generate&quot;,<br>        &quot;method&quot;: &quot;POST&quot;,<br>        &quot;headers&quot;: {<br>            &quot;Content-Type&quot;: &quot;application/json&quot;<br>        },<br>        &quot;data&quot;: {<br>            &quot;model&quot;: &quot;translategemma&quot;,<br>            &quot;stream&quot;: false,<br>            &quot;prompt&quot;: &quot;You are a professional &lt;source&gt; to &lt;target&gt; translator. Your goal is to accurately convey the meaning and nuances of the original &lt;source&gt; text while adhering to &lt;target&gt; grammar, vocabulary, and cultural sensitivities. Produce only the &lt;target&gt; translation, without any additional explanations or commentary. Please translate the following &lt;source&gt; text into &lt;target&gt;:\n\n\n&lt;text&gt;&quot;<br>        }<br>    },<br>    &quot;response&quot;: &quot;response[&#39;response&#39;]&quot;<br>}</pre><h3>What’s going on here</h3><ul><li>Calibre sends text → Ollama API</li><li>Ollama runs <strong>TranslateGemma</strong></li><li>The model translates it</li><li>Calibre inserts the result back into your ebook</li></ul><p>No magic. Just clean plumbing.</p><h3>4) Pick a Book and Go</h3><p>Now the fun part.</p><h3>What to start with</h3><p>If this is your first try, don’t go straight into a 1000-page novel 😅</p><p>Try:</p><ul><li>Short novels</li><li>Light novels</li><li>Public domain books from Project Gutenberg</li><li>EPUB format (works best)</li></ul><h3>Translate it</h3><ul><li>Import your book into Calibre</li><li>Right-click → <strong>Translate Book</strong></li><li>Choose:</li><li>Engine: <em>Ollama — TranslateGemma</em></li><li>Source language</li><li>Target language</li><li>Hit go 🚀</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc657a8391b1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/juliansplayground/translate-ebooks-locally-with-ollama-translategemma-fc657a8391b1">Translate eBooks Locally with Ollama + TranslateGemma</a> was originally published in <a href="https://medium.com/juliansplayground">Julian’s Playground</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[2024 奈良追紅葉快閃 （1）]]></title>
            <link>https://medium.com/%E6%9C%B1%E9%9A%B8%E5%AE%89%E7%9A%84%E6%97%85%E9%81%8A%E8%A8%98%E4%BA%8B/2024-%E5%A5%88%E8%89%AF%E8%BF%BD%E7%B4%85%E8%91%89%E5%BF%AB%E9%96%83-1-b8ac79d0e151?source=rss-b2c7083a1bdf------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8ac79d0e151</guid>
            <category><![CDATA[travel]]></category>
            <dc:creator><![CDATA[Julian Shen]]></dc:creator>
            <pubDate>Sat, 21 Dec 2024 16:24:52 GMT</pubDate>
            <atom:updated>2024-12-21T16:24:52.121Z</atom:updated>
            <content:encoded><![CDATA[<p>其實原本今年並沒這趟的規劃的，最早主要是看到日本有一個Japan Trusted Traveler Program，有了一個想法，覺得如果有了這個，過關速度變快，加上日本又近，那或許機票便宜時，就可以來個週五晚上出發，上班前回來的快閃，所以就跑去申請了，先說，目前我還是還沒拿到，已經是一肚子氣了（後面再來補充），不是很推薦申請，光第一階段就等了快三個月吧，第二階段要去日本面試，我八月多收到第一階段通過的通知，才想說生日（十一月）來去進行第二階段，順便來個快閃獨旅追紅葉吧。</p><p>最近迷上拍底片，想說這時節拍紅葉剛好，自己一個人開著車跑也比較方便拍，比較沒有顧慮，就想說計畫生日時候去，老婆也說，我生日自己去沒問題。生日是十一月底，北方應該都差不多了吧，所以想說依據往年資料，關西差不多見傾（但今年並沒有orz），所以就規劃了一個從奈良到京都三重加上滋賀的超精實行程（想說自己跑，精實點沒關係，不用顧慮團員，結果事實自己跑也沒辦法那麼精實XD）。</p><p>既然是拍照，事前的準備不能少，一開始覺得自己很猛很會拍，傳了大概八卷底片吧，也帶了數位單眼，兩顆鏡頭（本來還想多帶微單，不過，根本一時失心瘋呀），另外想說三腳架不是很方便，所以又去淘寳找了隻SIRUI的單腳架</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-7i6IuhgpnawmiDDnHFBeA@2x.jpeg" /></figure><p>我是蠻喜歡這隻腳架的，方便，收合快，略重但穩定，用來當登山杖應該也沒問題，所以特地還拿這段來介紹XD 不過，因為大部分的景點是禁用腳架的，不管幾隻腳都一樣，結果我大概只有在天理的陸橋拿來「空拍」用，其他地方都是只能丟在車上QQ</p><p>目的是申請JTTP，定位是快閃行，加上生日是週一，所以正好就安排個週五晚上七點的航班（虎航），週一晚上回來的行程。實際就差不多玩兩天。第一天抵達時已經是日本時間十一點了，這邊先誇讚一下虎航第一排的位子，空間寬敞，我腳不會侷促，又是第一個出機艙門，完全搶前面去通關，行李也早出，所以不到12點我已經入境完畢了。由於這時候進市區也已經沒交通工具了，加上租車公司也下班了，本來就打定主意住機場的膠囊First cabin，上次去韓國也是半夜到去住機場膠囊，還算可以。First cabin就在關西空港的Aeroplaza內，所以很近，下機後就可以找到休息的地方，但一定要先預定，很容易滿。First cabin採男女分開，有公共衛浴可用，房間就像這樣</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FRR8sBmJxG6sWFIqHNGD4g@2x.jpeg" /></figure><p>空間還可以，就一個人睡還可以，我本以為我是訂比較大的結果是我記錯了，但就一個晚上，這樣還算可以。這種地方就不用期待好好睡，晚上大家進進出出的，也沒隔音，不能鎖門，雖然櫃檯有免費耳塞拿，但也沒啥用，我睡到一半都掉出來了，反正就一個晚上的棲身之處，還行。因為在Aeroplaza內，起來後就找的到吃的，我八點也就準時到租車公司櫃檯報到（也在同一棟）。</p><p>這次租車，我選擇了Orix這家租車公司，櫃檯小姐英文不錯，如果把該填的事先在台灣都填完了，那整個流程很快。我大概花了十分鐘就取到車了吧！這次想說沒開過k car，就租了台Daihatsu Tento，小巧方方的，是有點cute</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RtMwxL3ww_I1uMjiYUjEXQ@2x.jpeg" /></figure><p>但我下次不會租這種車了，車身輕，動力又不太夠，在出空港的大橋上，感覺差點飛上天，強風一來，有種快被吹走的感覺。只好趕快雙手抓緊方向龐。不過一個人加上一個行李箱，這樣的大小的確是蠻夠的了。</p><p>既然租了車，就不打算跑很多台灣觀光客會去的地方，第一天（實質上的第二天）計畫要跑的是奈良縣內，自然不想跑啥東大寺這類的地方。原本，打算至少安排岡寺，談山神社，長谷寺三個點，但其實這樣有點緊，只要有些狀況，就很難跑全，而且事實上也發生了一些狀況，害我不得不放棄長谷寺，甚至隔天行程全部重新安排。</p><p>第一站是在明日香村（沒二號機？）的岡寺，又稱龍蓋寺。是有義淵僧正在1300年建立的。之所以叫龍蓋寺，就是相傳有一隻惡龍大鬧地方，被僧正封印於此。也是日本最初的厄除靈場。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ml-V0GHDZugjYRVraJigTQ@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5kjrKoDguCtSoZJxDWnLhA@2x.jpeg" /></figure><p>這邊觀光客也真的不太多，整個地方有種寧靜的氣氛。但一來到這邊，就覺得有點不妙！說好的見傾呢？怎麼還那麼多綠油油的？</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CqLcWxZukBcuJru9960j1A@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IZqS6owHVzZO5tJD0gkmbg@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W-frUEQEnOmDu_5R50cEfg@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FlcxbQ7a-WXNT2Rj2S097Q@2x.jpeg" /></figure><p>雖說是也有一些紅的蠻漂亮的，但就是比想像中差很多，可惡的暖冬！</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h84ZfNUSGxEi361vViR-Qw@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Mpl1idcSXdlBJlBeuHPgmA@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0_4-sIiepV0AgU2yhd41wA@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4izhLu3QBZXLDnaDz7PJUg@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*65sEK5weHxNRfl8HbkOnzQ@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*40sDZSzi6MJq3Qh76j3deA@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YHMgMtxmNwFQVHXcEOZWiw@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pJBv_l7MMXxnUTqV8pybNw@2x.jpeg" /></figure><p>這地方已經有1300歷史，也蠻有特色的。入口的御手水，裝飾的五彩繽紛的。另外還有一個供奉著彌勒的石窟，一個小小的天然石窟內，就供奉著彌勒石像。不過洞口有點小，加上裡面光線也沒很好，我就大概洞口看了一下，到此一遊了。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pp2knMjgChDGWi-Yddk51g@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JD2ycG2T7QzvIwtYzVT5ow@2x.jpeg" /></figure><p>離開了龍蓋寺後，才發現，原本規劃的路線其實不太理想，接下來想走的兩個地點，一個是長谷寺，一個是談山神社，岡寺剛好在兩個的中間。如果先去其他兩邊，其實應該比較順路。不過，反正自己一個跑，自由自在，沒差，所以就選一個往山上走的先，先往談山神社去。</p><p>雖然都是偏僻，但比起岡寺，談山神社似乎是一個更加熱門的景點。我上到快到目的地前，就已經看到一大堆車在排隊了，排到進到停車場，我想大概排了有三四十分鐘。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H7hAGBPfd1XJoZHfhWHR1g@2x.jpeg" /></figure><p>在這種上坡的地方排隊，對K car似乎不太友善呀，放開煞車時都怕往下滑下去。不過還好，這個景點沒讓我失望。</p><p>談山神社是一座世界上唯一擁有一座木造十三重塔的神社。建於西元678年，是大化革新的重要人物藤原鎌足與中大兄皇子（後來天智天皇）暢談大化革新的地點，因而叫談山。大化革新是日本唐化一個重要的歷史，所以這也是一個相當具有歷史意義的地點。</p><p>這邊的人潮就多很多，入神社前的坂，雖然不長，但還是有些商店街，賣一些土特產。由於奈良這邊算是清酒的發源地吧，所以也不少商店賣酒，看到比較多的是神酒，可能是剛好買來供奉給神？我是不知道有沒比較好喝，反正順手買了一隻叫做「談峯」的酒，開車不能試喝，所以就盲買了XD</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EaK9zVgnMirQBeff-xBRtw@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hEJa5f4G2u3bRIkVv5wJAA@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xAAsL9EdolMTnt9qmCp9ug@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*61Q2HNRQxXhKB6lduUd-BQ@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P-6A3wyCvj1B1aDZO4-JKg@2x.jpeg" /></figure><p>這邊和前面岡寺差不多，也是沒到滿片紅葉，但多少有好了有好一點，十三重塔果然也名不虛傳。這邊的景色，真的不枉我排了這麼久停車了。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dT9MKWaWXg_E2N2hHK_VlQ@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C_mlFLb1UI_ue_ZqbyA49A@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZDw6ts8QsTO-7BPatF4_QQ@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P-6A3wyCvj1B1aDZO4-JKg@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q_LjrZyvaYGcFsgifKxbBQ@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TuulaXfFg3jPU9Hyl4yk5Q@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EMSBZ92rnE_v4e7sLNZVOA@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kvlFAOVTrut9vsREVQ7kTQ@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*osKw9ZxjDF9ytXGJ_s73_Q@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zewcmPsfpPEpaNtPdU6ctw@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7-GRyce_9NL4cTYMBYXJFw@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4CK3ecYAEcMc7PanCFxPKA@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4aoREIxeGJxc0mzW1Au2rw@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kCgqUXg_SaNX-HAwJHCb8Q@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OMDadW5tJPdy4jj8pFDzgA@2x.jpeg" /></figure><p>這邊還有另一個有名的就是它有一個戀愛神社。走過了恋の道，就會來到了會帶給女生一生幸福的恋神社。這個神社供奉的是鏡女王，一看到這個名字，就覺得好酷，鏡女王耶。不過Google了一下，才知道，這不是啥神話傳說來的，鏡女王其實就是藤原鎌足的妻子，也叫鏡王女或是鏡姬王。之所以有這個名字，是因為是鏡王的女兒。她寫作的和歌甚至還有收錄到万葉集，可說是個才女。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3oa71MGugesk85iMC7QEkA@2x.jpeg" /></figure><p>玩到這邊其實都還蠻開心的，可以拍到很多漂亮的風景，心想，規劃這些比較沒有台灣觀光客來的點真的是太好了！不過那時沒想到，接下來發生的烏龍事件，會一整個打亂了我原本排的很緊湊的行程，也讓我一度有點鬱卒，不過這就容我下篇再說了。</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8ac79d0e151" width="1" height="1" alt=""><hr><p><a href="https://medium.com/%E6%9C%B1%E9%9A%B8%E5%AE%89%E7%9A%84%E6%97%85%E9%81%8A%E8%A8%98%E4%BA%8B/2024-%E5%A5%88%E8%89%AF%E8%BF%BD%E7%B4%85%E8%91%89%E5%BF%AB%E9%96%83-1-b8ac79d0e151">2024 奈良追紅葉快閃 （1）</a> was originally published in <a href="https://medium.com/%E6%9C%B1%E9%9A%B8%E5%AE%89%E7%9A%84%E6%97%85%E9%81%8A%E8%A8%98%E4%BA%8B">Ｊ聞錄.</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Crawling Dynamic Websites Using chromedp in Go]]></title>
            <link>https://medium.com/juliansplayground/crawling-dynamic-websites-using-chromedp-in-go-660feee9126a?source=rss-b2c7083a1bdf------2</link>
            <guid isPermaLink="false">https://medium.com/p/660feee9126a</guid>
            <category><![CDATA[web-clawer]]></category>
            <category><![CDATA[chromedp]]></category>
            <category><![CDATA[web]]></category>
            <dc:creator><![CDATA[Julian Shen]]></dc:creator>
            <pubDate>Sat, 29 Jun 2024 18:47:28 GMT</pubDate>
            <atom:updated>2024-06-29T18:47:28.185Z</atom:updated>
            <content:encoded><![CDATA[<p>(This article is translated from <a href="https://blog.jln.co/Shi-Yong-Chromedp-Pa-Qu-Dong-Tai-Wang-Zhan-Zi-Liao/">https://blog.jln.co/Shi-Yong-Chromedp-Pa-Qu-Dong-Tai-Wang-Zhan-Zi-Liao/</a> with ChatGPT)</p><p>In modern web development, JavaScript-driven dynamic websites are becoming increasingly common, presenting challenges for traditional HTML parsing tools. Traditional methods are no longer suitable, as the content of web pages is generated only after executing JavaScript. Parsing such websites requires more front-end knowledge.</p><p>chromedp is a toolset written in Go language that performs headless browser automation via Chrome&#39;s DevTools protocol. This allows developers to programmatically control the Chrome browser, making it easy to crawl and parse dynamically generated content. This article will introduce how to use chromedp to create a simple web crawler.</p><p>chromedp mainly functions by communicating with the Chrome browser through the DevTools protocol. This enables developers to simulate user actions, such as navigating to web pages, clicking buttons, filling out forms, and extracting dynamically loaded content. These operations are performed in headless mode, meaning the browser interface is not visible, thus improving performance and resource efficiency. Through this approach, chromedp can handle scenarios that traditional HTML parsing tools cannot, especially when dealing with dynamically generated content.</p><h3>Simple Example Code</h3><p>Here is a simple example of using chromedp to crawl website data. This example demonstrates how to navigate to a website, select some elements, submit a form, and extract the required data:</p><pre>package main<br><br>import (<br>    &quot;context&quot;<br>    &quot;fmt&quot;<br>    &quot;log&quot;<br>    &quot;time&quot;<br><br>    &quot;github.com/chromedp/chromedp&quot;<br>)<br><br>func main() {<br>    // Create context<br>    ctx, cancel := chromedp.NewContext(context.Background())<br>    defer cancel()<br><br>    // Allocate browser<br>    ctx, cancel = chromedp.NewExecAllocator(ctx, chromedp.DefaultExecAllocatorOptions[:]...)<br>    defer cancel()<br><br>    // Create context with a 30-second timeout<br>    ctx, cancel = context.WithTimeout(ctx, 30*time.Second)<br>    defer cancel()<br><br>    // Execute tasks<br>    var res string<br>    err := chromedp.Run(ctx,<br>        chromedp.Navigate(&quot;https://example.com&quot;),<br>        chromedp.WaitVisible(`body`, chromedp.ByQuery),<br>        chromedp.SendKeys(`input[name=&quot;q&quot;]`, &quot;chromedp&quot;, chromedp.ByQuery),<br>        chromedp.Click(`input[type=&quot;submit&quot;]`, chromedp.ByQuery),<br>        chromedp.WaitVisible(`#result-stats`, chromedp.ByQuery),<br>        chromedp.Text(`#result-stats`, &amp;res, chromedp.ByQuery),<br>    )<br><br>    if err != nil {<br>        log.Fatal(err)<br>    }<br><br>    fmt.Println(&quot;Search Result Stats:&quot;, res)<br>}</pre><p>In this example, we create a chromedp task that navigates to example.com, waits for the page to load, enters &quot;chromedp&quot; in the search box, clicks the submit button, then waits for the search result stats element to be visible, and extracts that information. This is a basic example showing how to use chromedp for basic web interaction and data extraction. You can expand this example as needed to implement more complex crawlers.</p><p>Next, we will use a more practical case. The following example demonstrates how to use chromedp to crawl the schedule of the Chinese Professional Baseball League (CPBL), which is implemented using Vue.js.</p><h3>Analyze the document</h3><p>The URL of the CPBL schedule website is <a href="https://cpbl.com.tw/schedule">https://cpbl.com.tw/schedule</a>. First, we use “View Page Source” to inspect the web page and find that there is no schedule data in it. Additionally, we can find the following piece of code:</p><pre>var app = new Vue({<br>    el: &quot;#Center&quot;,<br>    mixins: [mixin],</pre><p>Another piece of code for obtaining schedule information is as follows:</p><pre>$.ajax({<br>    url: &#39;/schedule/getgamedatas&#39;,<br>    type: &#39;POST&#39;,<br>    data: filterData,<br>    headers: {<br>        RequestVerificationToken: &#39;PzmpuUOvS4z2zH_QhwgFQYTzVC82b0n2QH30wEOJ12kOWA6zeq0Yn7_6d2v_o-ZTWuNPe3HjrqsMqAHp9sL0F5KB4KM1:5jgubJ0tGDTK3cLm2JU7_bCw9JqLOG8j8yeNiWDhR4nnTACLXerDqmzB5chZv-iqY8m1ep6IirI3hAwRCPfNTU6jO_E1&#39;<br>    },<br>    success: function(result) {<br>        if (result.Success) {<br>            _this.gameDatas = JSON.parse(result.GameDatas);<br>            _this.getGames();<br>        }<br>    },<br>    error: function(res) {<br>        console.log(res);<br>    },<br>    complete: function () {<br>        $(&quot;body&quot;).unblock()<br>    }<br>});</pre><p>It is clear that this is a web page written in Vue.js. We could try hitting its API, but seeing that token string, it might have some protection in place, so using chromedp might be simpler.</p><p>How do we start parsing? Using ChatGPT might be a good method. Open Chrome’s developer tools, go to the Elements tab where the final rendered webpage is displayed, save it as a file, and ask ChatGPT:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xzjdJwBtkvraW2sCCSLcIw.png" /><figcaption>Asking chatgpt to analyze the document (in Chinese)</figcaption></figure><p>You can also ask it to design the data structure:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N07lb1BwFiCwn4oyPwHrSQ.png" /><figcaption>Generate data structure with chatgpt (in Chinese)</figcaption></figure><p>This can only serve as an initial reference. You can later ask it to write the code for you. However, I tried it, and it only produces an example, not the correct result, but it’s useful as a foundation for modification.</p><p>First, define the requirements. We need to write a function that can input the year, month, and type of game to obtain the schedule information.</p><p>Based on this information, let’s write the first rough version to experiment with:</p><pre>type Game struct {<br>    No       int    `json:&quot;no&quot;`<br>    Year     int    `json:&quot;year&quot;`<br>    Month    int    `json:&quot;month&quot;`<br>    Day      int    `json:&quot;day&quot;`<br>    Home     string `json:&quot;home&quot;`<br>    Away     string `json:&quot;away&quot;`<br>    Ballpark string `json:&quot;ballpark&quot;`<br>}<br><br>func getGameNodes(nodes *[]*cdp.Node) chromedp.Action {<br>    return chromedp.ActionFunc(func(ctx context.Context) error {<br>        ctxWithTimeout, cancel := context.WithTimeout(ctx, 900*time.Millisecond)<br>        defer cancel()<br><br>        chromedp.Nodes(&quot;div.game&quot;, nodes).Do(ctxWithTimeout)<br>        for _, n := range *nodes {<br>            dom.RequestChildNodes(n.NodeID).WithDepth(6).Do(ctxWithTimeout)<br>        }<br><br>        return nil<br>    })<br>}<br><br>func selectMonth(month string) chromedp.QueryAction {<br>    return chromedp.SetValue(&quot;div.item.month select&quot;, month, chromedp.ByQueryAll)<br>}<br><br>func selectYear(year string) chromedp.QueryAction {<br>    return chromedp.SetValue(&quot;div.item.year select&quot;, year, chromedp.ByQueryAll)<br>}<br><br>func selectGameType(gtype string) chromedp.Action {<br>    return chromedp.SetValue(&quot;div.item.game_type select&quot;, gtype, chromedp.ByQueryAll)<br>}<br><br>func fetchGamesByMonth(ctx context.Context, year string, month string) ([]Game, error) {<br>    chromedp.Run(ctx, selectMonth(month),<br>        chromedp.WaitVisible(&quot;div.ScheduleGroup&quot;),<br>        chromedp.Sleep(800*time.Millisecond),<br>    )<br><br>    var nodes []*cdp.Node<br>    var mn string<br><br>    chromedp.Run(ctx,<br>        chromedp.Text(&quot;.date_selected .date&quot;, &amp;mn),<br>        getGameNodes(&amp;nodes),<br>    )<br><br>    var games []Game = make([]Game, len(nodes))<br>    for i, node := range nodes {<br>        games[i].No, _ = strconv.Atoi(strings.Trim(node.Children[0].Children[0].Children[0].Children[1].Children[0].NodeValue, &quot; &quot;))<br>        games[i].Ballpark = node.Children[0].Children[0].Children[0].Children[0].Children[0].NodeValue<br>        games[i].Year, _ = strconv.Atoi(year)<br>        monthInt, _ := strconv.Atoi(month)<br>        games[i].Month = monthInt + 1<br>        dataDate := node.Parent.Children[0].AttributeValue(&quot;data-date&quot;)<br>        day, _ := strconv.Atoi(dataDate)<br>        games[i].Day = day<br>        games[i].Away = node.Children[0].Children[0].Children[1].Children[0].Children[0].AttributeValue(&quot;title&quot;)<br>        games[i].Home = node.Children[0].Children[0].Children[1].Children[2].Children[0].AttributeValue(&quot;title&quot;)<br>    }<br><br>    return games, nil<br>}</pre><p>This version of the code is rough but usable, heavily relying on NodeValue and AttributeValue to fetch values. The problem with this code is that each chromedp call requires a call to Chrome, achieved through the Chrome DevTools Protocol. The Chrome DevTools Protocol uses WebSocket for communication, making frequent round-trips inefficient and less stable.</p><p>Here is an optimized version based on methods learned from ChatGPT:</p><pre>// FetchSchedule fetches the schedule from CPBL website based on the year, month, and game type<br>func FetchSchedule(year int, month int, gameType string) ([]GameSchedule, error) {<br>    ctx, cancel := chromedp.NewContext(context.Background())<br>    defer cancel()<br><br>    var schedules []GameSchedule<br><br>    // Define the URL<br>    url := &quot;https://cpbl.com.tw/schedule&quot;<br><br>    // Run chromedp tasks<br>    err := chromedp.Run(ctx,<br>        chromedp.Navigate(url),<br>        chromedp.WaitReady(`.ScheduleTableList`), // Wait for year select to be ready<br>        chromedp.Evaluate(fmt.Sprintf(&quot;document.querySelector(&#39;#Center&#39;).__vue__.filters.kindCode = &#39;%s&#39;&quot;, gameType), nil),<br>        chromedp.Evaluate(fmt.Sprintf(&quot;document.querySelector(&#39;#Center&#39;).__vue__.calendar.year = %d&quot;, year), nil),<br>        chromedp.WaitReady(`.ScheduleTableList`), // Wait for year select to be ready<br>        chromedp.Evaluate(fmt.Sprintf(&quot;document.querySelector(&#39;#Center&#39;).__vue__.calendar.month = %d&quot;, month-1), nil),<br>        chromedp.Evaluate(`document.querySelector(&#39;#Center&#39;).__vue__.getGameDatas()`, nil), // Wait for table to be visible<br>        chromedp.Sleep(2*time.Second), // Wait for table to load<br>        chromedp.Evaluate(`<br>            (() =&gt; {<br>                let schedules = [];<br>                document.querySelectorAll(&#39;.ScheduleTable tbody .date&#39;).forEach(dateDiv =&gt; {<br>                    let date = dateDiv.innerText.trim();<br>                    let parent = dateDiv.parentNode;<br>                    parent.querySelectorAll(&#39;.game&#39;).forEach(gameDiv =&gt; {<br>                        let location = gameDiv.querySelector(&#39;.place&#39;) ? gameDiv.querySelector(&#39;.place&#39;).innerText.trim() : &#39;&#39;;<br>                        let game_no = gameDiv.querySelector(&#39;.game_no&#39;) ? gameDiv.querySelector(&#39;.game_no&#39;).innerText.trim() : &#39;&#39;;<br>                        let away_team = gameDiv.querySelector(&#39;.team.away span&#39;) ? gameDiv.querySelector(&#39;.team.away span&#39;).title.trim() : &#39;&#39;;<br>                        let home_team = gameDiv.querySelector(&#39;.team.home span&#39;) ? gameDiv.querySelector(&#39;.team.home span&#39;).title.trim() : &#39;&#39;;<br>                        let score = gameDiv.querySelector(&#39;.score&#39;) ? gameDiv.querySelector(&#39;.score&#39;).innerText.trim() : &#39;&#39;;<br>                        let remark = gameDiv.querySelector(&#39;.remark .note div&#39;) ? gameDiv.querySelector(&#39;.remark .note div&#39;).innerText.trim() : &#39;&#39;;<br>                        schedules.push({ date, location, game_no, away_team, home_team, score, remark });<br>                    });<br>                });<br>                return schedules;<br>            })()<br>        `, &amp;schedules),<br>    )<br><br>    if err != nil {<br>        return nil, err<br>    }<br><br>    return schedules, nil<br>}</pre><p>This version extensively uses chromedp.Evaluate to embed JavaScript code directly in the web page, modifying Vue&#39;s internal values and directly returning the execution results to the Go program. This improves readability and avoids excessive communication with Chrome.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=660feee9126a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/juliansplayground/crawling-dynamic-websites-using-chromedp-in-go-660feee9126a">Crawling Dynamic Websites Using chromedp in Go</a> was originally published in <a href="https://medium.com/juliansplayground">Julian’s Playground</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[2016 博多祇園山笠祭]]></title>
            <link>https://medium.com/%E6%9C%B1%E9%9A%B8%E5%AE%89%E7%9A%84%E6%97%85%E9%81%8A%E8%A8%98%E4%BA%8B/2016-%E5%8D%9A%E5%A4%9A%E7%A5%87%E5%9C%92%E5%B1%B1%E7%AC%A0%E7%A5%AD-6560a0b22efc?source=rss-b2c7083a1bdf------2</link>
            <guid isPermaLink="false">https://medium.com/p/6560a0b22efc</guid>
            <category><![CDATA[博多]]></category>
            <category><![CDATA[fukuoka]]></category>
            <category><![CDATA[祇園山笠]]></category>
            <category><![CDATA[japan]]></category>
            <category><![CDATA[travel]]></category>
            <dc:creator><![CDATA[Julian Shen]]></dc:creator>
            <pubDate>Sun, 16 Jul 2017 07:42:38 GMT</pubDate>
            <atom:updated>2017-07-16T07:42:38.984Z</atom:updated>
            <content:encoded><![CDATA[<p>從去年就一直想總不能老寫些無趣的技術呀! 偶爾也需要寫寫一些旅遊的見聞, 不然出去玩一玩回來後好像什麼都沒有了, 說是遊記, 但卻不太想用一天天的方式來記錄, 畢竟自己記性沒那麼好, 另一方面也想試用看看medium這平台, 所以打算以事件或地點當作一個medium story, 不過想歸想, 一段時間過去了, 一直遲遲沒動筆, 直到這一兩天, Facebook提醒了我去年去了福岡看博多祇園山笠祭的事, 剛好這幾天也是博多祇園山笠祭的時間, 就趁等待要去接小孩的空檔把這部分寫一寫吧!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FerTrCCV_T3E%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DerTrCCV_T3E&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FerTrCCV_T3E%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2e1c65256763af515f456248ba517bb1/href">https://medium.com/media/2e1c65256763af515f456248ba517bb1/href</a></iframe><p>其實早在2016年初就去過一次九州了, 年中再去這次, 完全就是衝著博多祇園山笠祭而來的, 這真的得感謝老婆小孩陪我瘋這一次, 關於這個祭典, 從<a href="http://www.hakatayamakasa.com/">官網</a>可以查到一些資料, 但之所以會想去看這祭典, 完全被島耕作給影響的!</p><p>在<a href="https://bookstore.yahoo.co.jp/shoshi-103723/">部長島耕作12</a>中, 島耕作參加了這個祭典, 那時候看到這段內容, 覺得好酷喔, 一堆大男人大清早的穿著丁字褲, 扛著神轎在路上跑, 一整個頗熱血呀!(會嗎?), 那時候就想這輩子一定要去看個一次!</p><p>這個活動在每年的7/1~7/15舉辦, 7/15凌晨五點是高潮時段, 是正式的起跑, 但在7/15之前則會有各式各樣的活動, 跟練習</p><p>練習有可能會在大白天, 尤其是這季節, 根本是大熱天, 所以在衝刺的時候, 旁邊都還會有人去潑水, 要不然即使已經穿著非常”清涼”了, 還是有可能熱昏掉呀!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*76178dMCTNfei-O8sy9fbQ.jpeg" /><figcaption>標準服裝</figcaption></figure><p>當然, 標準服裝就是露出屁屁的丁字褲, 當地人可能習以為常了, 但外地人可能要有一定的恥度才敢穿吧?</p><p>本來覺得這種活動大概是老頭參加的吧, 沒想到其實也有不少的小朋友</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PDYJN4ErM_tiNjxFlctUKg.jpeg" /><figcaption>小朋友也玩得不亦樂乎</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3F45nk8ZfbQLZ6gvy210KA.jpeg" /><figcaption>歐吉桑一聲令下, 大家跟著出發囉!</figcaption></figure><p>山笠分為兩種, 一種是裝飾用的山笠, 極盡所能的大, 正面背面會是不同的風格, 各區域有自己的山笠, 各自不同的故事, 穿梭在街頭找這些山笠也是一件頗有趣的事</p><p>而另一種是真的扛在肩上的山笠, 雖說小多了, 但也是十分沈重, 需要很多大男人輪番接力才能跑完全程的</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/427/1*gsXohF9bd5PDJeF_Q6kjGw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/427/1*sGIebwu0muvw16wHtGljkA.jpeg" /><figcaption>裝飾山笠</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XHvpE6pRL4xyRYTFd6NNug.jpeg" /><figcaption>實際扛著跑的</figcaption></figure><p>實際上真的比賽, 起跑時間是凌晨五點鐘, 但就算是提早了一兩個小時到, 現場也是已經人山人海了, 這個時間, 天都還沒亮, 讓我比較訝異的是, 我發現有外國人也在隊伍的行列內(當然也穿著丁字褲), 可能是被當地人一起邀請去的吧</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u7eB-iHx8JNxQT05YkhR0Q.jpeg" /><figcaption>坐在轎上指揮的看起來都像是比較年長者</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EHAeRO9xArPdti9NxBJJjQ.jpeg" /><figcaption>這裡面就可以看到外國人</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XEY4HItooM7uirkjV6hOwQ.jpeg" /><figcaption>電視台也會來轉播</figcaption></figure><p>實際上跟著一起跑的話, 應該會很有趣, 只是人真的太多了, 沒辦法跟著一站站追, 看完起跑後, 就在漸白的天空下走回旅館找還在睡覺的老婆兒子去了</p><p>下面是一些起跑實際情況:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F3o66afBzsUM%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D3o66afBzsUM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F3o66afBzsUM%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e080742fb5cb3ef9747b1b269160327f/href">https://medium.com/media/e080742fb5cb3ef9747b1b269160327f/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FPYeYkpXXKR8%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DPYeYkpXXKR8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FPYeYkpXXKR8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d60d78d98fd47e11267b0d10bcea3921/href">https://medium.com/media/d60d78d98fd47e11267b0d10bcea3921/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6560a0b22efc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/%E6%9C%B1%E9%9A%B8%E5%AE%89%E7%9A%84%E6%97%85%E9%81%8A%E8%A8%98%E4%BA%8B/2016-%E5%8D%9A%E5%A4%9A%E7%A5%87%E5%9C%92%E5%B1%B1%E7%AC%A0%E7%A5%AD-6560a0b22efc">2016 博多祇園山笠祭</a> was originally published in <a href="https://medium.com/%E6%9C%B1%E9%9A%B8%E5%AE%89%E7%9A%84%E6%97%85%E9%81%8A%E8%A8%98%E4%BA%8B">Ｊ聞錄.</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>