<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lau Dai He on Medium]]></title>
        <description><![CDATA[Stories by Lau Dai He on Medium]]></description>
        <link>https://medium.com/@laudaihe?source=rss-29a4bce64558------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*3mahV0C0ozoLePuf</url>
            <title>Stories by Lau Dai He on Medium</title>
            <link>https://medium.com/@laudaihe?source=rss-29a4bce64558------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 15:35:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@laudaihe/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding the EBS IO1 configuration the right way.]]></title>
            <link>https://laudaihe.medium.com/understanding-the-ebs-io1-configuration-the-right-way-ebadfca27ff3?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/ebadfca27ff3</guid>
            <category><![CDATA[amazon-ebs]]></category>
            <category><![CDATA[fios]]></category>
            <category><![CDATA[io1]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Tue, 19 May 2026 19:57:41 GMT</pubDate>
            <atom:updated>2026-05-19T19:57:41.389Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Author：Lau Dai</blockquote><blockquote>Date：2026/05/19</blockquote><p>Currently, benchmarking the EBS IO1 volume type in the following configuration</p><p>Host: c7g.16xlarge (No instance-side volume throttle issue)</p><pre>|+----------------------------------+------------------+|<br>|||                 EbsOptimizedInfo                  |||<br>||+-------------------------------------+-------------+||<br>|||  BaselineBandwidthInMbps            |  20000      |||<br>|||  BaselineIops                       |  80000      |||<br>|||  BaselineThroughputInMBps           |  2500.0     |||<br>|||  MaximumBandwidthInMbps             |  20000      |||<br>|||  MaximumIops                        |  80000      |||<br>|||  MaximumThroughputInMBps            |  2500.0     |||<br>||+-------------------------------------+-------------+||</pre><p>Two IO1 volumes configured with 2,000 and 48,000 IOPS, respectively.</p><p>fio version: fio-3.32</p><p>FIO configuration file</p><pre>tail -v -n 99 io1-io-testing.cfg<br>==&gt; io1-io-testing.cfg &lt;==<br><br>[global]<br>clocksource=clock_gettime<br>randrepeat=0<br>runtime=600<br>time_based=1<br>direct=1<br>group_reporting=1<br>norandommap=1<br>size=1G<br><br>[phase1-randwrite-16k]<br>description=Phase 1: 16KiB Random Write on 2000 IOPS<br>filename=/dev/nvme1n1<br>ioengine=psync<br>rw=randwrite<br>bs=16k<br>numjobs=16<br>do_verify=0<br><br>[phase2-randwrite-256k]<br>stonewall<br>description=Phase 2: 256KiB Random Write on 2000 IOPS<br>filename=/dev/nvme1n1<br>ioengine=psync<br>rw=randwrite<br>bs=256k<br>numjobs=16<br>do_verify=0<br><br>[phase3-randread-16k]<br>stonewall<br>description=Phase 3: 16KiB Random Read on 48000 IOPS<br>filename=/dev/nvme2n1<br>ioengine=psync<br>rw=randread<br>bs=16k<br>numjobs=4<br>do_verify=0<br><br>[phase4-randread-256k]<br>stonewall<br>description=Phase 4: 256KiB Random Read on 48000 IOPS<br>filename=/dev/nvme2n1<br>ioengine=psync<br>rw=randread<br>bs=256k<br>numjobs=4<br>do_verify=0</pre><p>Encountered an issue where phase3 only reached 8,600 IO/s.</p><pre>[ec2-user@ip-172-31-30-136 ~]$ date -Isec -u &amp;&amp; sudo fio io1-io-testing.cfg<br>2026-05-19T14:07:10+00:00<br>phase1-randwrite-16k: (g=0): rw=randwrite, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=1<br>...<br>phase2-randwrite-256k: (g=1): rw=randwrite, bs=(R) 256KiB-256KiB, (W) 256KiB-256KiB, (T) 256KiB-256KiB, ioengine=psync, iodepth=1<br>...<br>phase3-randread-16k: (g=2): rw=randread, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=1<br>...<br>phase4-randread-256k: (g=3): rw=randread, bs=(R) 256KiB-256KiB, (W) 256KiB-256KiB, (T) 256KiB-256KiB, ioengine=psync, iodepth=1<br>...<br>fio-3.32<br>Starting 40 processes<br>Jobs: 4 (f=4): [_(36),r(4)][100.0%][r=750MiB/s][r=3000 IOPS][eta 00m:00s]<br>phase1-randwrite-16k: (groupid=0, jobs=16): err= 0: pid=5449: Tue May 19 14:47:11 2026<br>  Description  : [Phase 1: 16KiB Random Write on 2000 IOPS]<br>  write: IOPS=2003, BW=31.3MiB/s (32.8MB/s)(18.3GiB/600009msec); 0 zone resets<br>    clat (usec): min=533, max=18348, avg=7985.87, stdev=1740.89<br>     lat (usec): min=533, max=18348, avg=7986.05, stdev=1740.89<br>    clat percentiles (usec):<br>     |  1.00th=[ 3458],  5.00th=[ 4015], 10.00th=[ 5080], 20.00th=[ 7898],<br>     | 30.00th=[ 7963], 40.00th=[ 7963], 50.00th=[ 8029], 60.00th=[ 8029],<br>     | 70.00th=[ 8094], 80.00th=[ 9372], 90.00th=[10028], 95.00th=[10552],<br>     | 99.00th=[12911], 99.50th=[13042], 99.90th=[13042], 99.95th=[13173],<br>     | 99.99th=[13304]<br>   bw (  KiB/s): min=28128, max=82545, per=99.98%, avg=32046.53, stdev=134.74, samples=19184<br>   iops        : min= 1758, max= 5153, avg=2002.61, stdev= 8.41, samples=19184<br>  lat (usec)   : 750=0.07%, 1000=0.12%<br>  lat (msec)   : 2=0.01%, 4=4.19%, 10=84.89%, 20=10.74%<br>  cpu          : usr=0.02%, sys=0.10%, ctx=1202035, majf=1, minf=154<br>  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, &gt;=64=0.0%<br>     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &gt;=64=0.0%<br>     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &gt;=64=0.0%<br>     issued rwts: total=0,1202002,0,0 short=0,0,0,0 dropped=0,0,0,0<br>     latency   : target=0, window=0, percentile=100.00%, depth=1<br>phase2-randwrite-256k: (groupid=1, jobs=16): err= 0: pid=5791: Tue May 19 14:47:11 2026<br>  Description  : [Phase 2: 256KiB Random Write on 2000 IOPS]<br>  write: IOPS=2000, BW=500MiB/s (525MB/s)(293GiB/600009msec); 0 zone resets<br>    clat (usec): min=792, max=18620, avg=7994.29, stdev=1221.30<br>     lat (usec): min=793, max=18621, avg=7995.90, stdev=1221.30<br>    clat percentiles (usec):<br>     |  1.00th=[ 5407],  5.00th=[ 5997], 10.00th=[ 6390], 20.00th=[ 7570],<br>     | 30.00th=[ 7898], 40.00th=[ 7963], 50.00th=[ 7963], 60.00th=[ 8029],<br>     | 70.00th=[ 8094], 80.00th=[ 8160], 90.00th=[ 8979], 95.00th=[10945],<br>     | 99.00th=[11469], 99.50th=[11600], 99.90th=[11994], 99.95th=[12125],<br>     | 99.99th=[13435]<br>   bw (  KiB/s): min=409600, max=740085, per=100.00%, avg=512284.06, stdev=2600.58, samples=19184<br>   iops        : min= 1600, max= 2882, avg=2000.71, stdev=10.15, samples=19184<br>  lat (usec)   : 1000=0.01%<br>  lat (msec)   : 2=0.04%, 4=0.15%, 10=91.37%, 20=8.43%<br>  cpu          : usr=0.04%, sys=0.13%, ctx=1200539, majf=0, minf=146<br>  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, &gt;=64=0.0%<br>     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &gt;=64=0.0%<br>     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &gt;=64=0.0%<br>     issued rwts: total=0,1200528,0,0 short=0,0,0,0 dropped=0,0,0,0<br>     latency   : target=0, window=0, percentile=100.00%, depth=1<br>phase3-randread-16k: (groupid=2, jobs=4): err= 0: pid=6195: Tue May 19 14:47:11 2026<br>  Description  : [Phase 3: 16KiB Random Read on 48000 IOPS]<br>  read: IOPS=8631, BW=135MiB/s (141MB/s)(79.0GiB/600001msec)<br>    clat (usec): min=147, max=19921, avg=463.04, stdev=48.64<br>     lat (usec): min=147, max=19921, avg=463.09, stdev=48.64<br>    clat percentiles (usec):<br>     |  1.00th=[  355],  5.00th=[  429], 10.00th=[  441], 20.00th=[  441],<br>     | 30.00th=[  441], 40.00th=[  445], 50.00th=[  445], 60.00th=[  449],<br>     | 70.00th=[  461], 80.00th=[  529], 90.00th=[  529], 95.00th=[  537],<br>     | 99.00th=[  545], 99.50th=[  553], 99.90th=[  619], 99.95th=[  701],<br>     | 99.99th=[ 1303]<br>   bw (  KiB/s): min=134656, max=140320, per=100.00%, avg=138131.36, stdev=155.02, samples=4796<br>   iops        : min= 8416, max= 8770, avg=8633.12, stdev= 9.69, samples=4796<br>  lat (usec)   : 250=0.01%, 500=76.87%, 750=23.09%, 1000=0.02%<br>  lat (msec)   : 2=0.02%, 4=0.01%, 10=0.01%, 20=0.01%<br>  cpu          : usr=0.13%, sys=0.70%, ctx=5179128, majf=0, minf=54<br>  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, &gt;=64=0.0%<br>     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &gt;=64=0.0%<br>     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &gt;=64=0.0%<br>     issued rwts: total=5179125,0,0,0 short=0,0,0,0 dropped=0,0,0,0<br>     latency   : target=0, window=0, percentile=100.00%, depth=1<br>phase4-randread-256k: (groupid=3, jobs=4): err= 0: pid=6519: Tue May 19 14:47:11 2026<br>  Description  : [Phase 4: 256KiB Random Read on 48000 IOPS]<br>  read: IOPS=3004, BW=751MiB/s (788MB/s)(440GiB/600002msec)<br>    clat (usec): min=396, max=14135, avg=1330.75, stdev=82.12<br>     lat (usec): min=397, max=14135, avg=1330.81, stdev=82.12<br>    clat percentiles (usec):<br>     |  1.00th=[ 1139],  5.00th=[ 1221], 10.00th=[ 1237], 20.00th=[ 1270],<br>     | 30.00th=[ 1303], 40.00th=[ 1319], 50.00th=[ 1336], 60.00th=[ 1352],<br>     | 70.00th=[ 1369], 80.00th=[ 1385], 90.00th=[ 1418], 95.00th=[ 1450],<br>     | 99.00th=[ 1500], 99.50th=[ 1516], 99.90th=[ 1631], 99.95th=[ 1762],<br>     | 99.99th=[ 2966]<br>   bw (  KiB/s): min=754414, max=909312, per=100.00%, avg=769395.83, stdev=3319.61, samples=4796<br>   iops        : min= 2946, max= 3552, avg=3005.38, stdev=12.97, samples=4796<br>  lat (usec)   : 500=0.01%, 750=0.02%, 1000=0.08%<br>  lat (msec)   : 2=99.87%, 4=0.03%, 10=0.01%, 20=0.01%<br>  cpu          : usr=0.05%, sys=0.57%, ctx=1802984, majf=0, minf=293<br>  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, &gt;=64=0.0%<br>     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &gt;=64=0.0%<br>     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &gt;=64=0.0%<br>     issued rwts: total=1802982,0,0,0 short=0,0,0,0 dropped=0,0,0,0<br>     latency   : target=0, window=0, percentile=100.00%, depth=1<br><br>Run status group 0 (all jobs):<br>  WRITE: bw=31.3MiB/s (32.8MB/s), 31.3MiB/s-31.3MiB/s (32.8MB/s-32.8MB/s), io=18.3GiB (19.7GB), run=600009-600009msec<br><br>Run status group 1 (all jobs):<br>  WRITE: bw=500MiB/s (525MB/s), 500MiB/s-500MiB/s (525MB/s-525MB/s), io=293GiB (315GB), run=600009-600009msec<br><br>Run status group 2 (all jobs):<br>   READ: bw=135MiB/s (141MB/s), 135MiB/s-135MiB/s (141MB/s-141MB/s), io=79.0GiB (84.9GB), run=600001-600001msec<br><br>Run status group 3 (all jobs):<br>   READ: bw=751MiB/s (788MB/s), 751MiB/s-751MiB/s (788MB/s-788MB/s), io=440GiB (473GB), run=600002-600002msec<br><br>Disk stats (read/write):<br>  nvme1n1: ios=141/2402198, merge=0/0, ticks=392/19174041, in_queue=19174433, util=99.99%<br>  nvme2n1: ios=6981517/0, merge=0/0, ticks=4760872/0, in_queue=4760872, util=100.00%</pre><p>This does not match what the public documentation states regarding <a href="https://docs.aws.amazon.com/ebs/latest/userguide/provisioned-iops.html#EBSVolumeTypes_piops">linear increase.</a></p><blockquote>io1 volumes provisioned with more than 32,000 IOPS (up to the maximum of 64,000 IOPS) yield a linear increase in throughput at a rate of 16 KiB per provisioned IOPS. For example, a volume provisioned with 48,000 IOPS can support up to 750 MiB/s of throughput (16 KiB per provisioned IOPS × 48,000 provisioned IOPS = 750 MiB/s).</blockquote><p>Ideally, phase 3 IOPS should be 48,000 instead of 8,600.</p><h3>Deep dive into the root cause:</h3><p>Let’s deep dive into the latency of the fio requests.</p><p>You can see that the Phase 3 latency is only 463 us. This is much lower than other phases, such as 7,985 us in phase 1, 2, and 4.</p><pre>Phase 3<br>clat (usec): min=147, max=19921, avg=463.04, stdev=48.64<br> lat (usec): min=147, max=19921, avg=463.09, stdev=48.64<br>---<br>Phase 1<br>clat (usec): min=533, max=18348, avg=7985.87, stdev=1740.89<br> lat (usec): min=533, max=18348, avg=7986.05, stdev=1740.89</pre><p>Thus, the IOPS per 4-job thread configuration could be:</p><pre>1,000,000 us / 463 us = 2159.82721382 per thread</pre><p>Then four threads would yield around 8,600 IOPS, which aligns with our observation.</p><p>The root cause of this issue is the incorrect engine configuration in fio.</p><pre> psync<br>   Basic pread(2) or pwrite(2) I/O. Default on all supported operating systems except for Windows.<br> libaio<br>    Linux native asynchronous I/O. Note that Linux may only support queued behavior with non-buffered I/O (set direct=1 or buffered=0). This engine defines engine specific options.</pre><p><a href="https://fio.readthedocs.io/en/latest/fio_doc.html#i-o-engine">source</a></p><p>The psync engine enforces <a href="https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-iodepth">iodepth=1</a> because it is a synchronous engine.</p><p>Once you modify the engine to libaio, which is an async engine, you can set a higher iodepth.</p><p>How to resolve this issue?</p><p>Modify the engine to an async engine such as libaio to send requests asynchronously.</p><p>After modifying the configuration, we can see that we achieve the maximum volume performance.</p><pre>IO latency<br>avg=5241.51<br>IOPS<br>avg=48088.73</pre><pre>[ec2-user@ip-172-31-30-136 ~]$ date -Isec -u &amp;&amp; sudo fio fix_8600IOPS-io1-testing.cfg<br>2026-05-19T15:06:47+00:00<br>phase3-randread-16k: (g=0): rw=randread, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=libaio, iodepth=64<br>...<br>fio-3.32<br>Starting 4 processes<br>Jobs: 4 (f=4): [r(4)][100.0%][r=750MiB/s][r=48.0k IOPS][eta 00m:00s]<br>phase3-randread-16k: (groupid=0, jobs=4): err= 0: pid=7428: Tue May 19 15:16:47 2026<br>  Description  : [Phase 3: 16KiB Random Read on 48000 IOPS]<br>  read: IOPS=48.1k, BW=751MiB/s (788MB/s)(440GiB/600002msec)<br>    slat (nsec): min=1234, max=3792.4k, avg=82328.73, stdev=246678.82<br>    clat (usec): min=501, max=28362, avg=5241.51, stdev=3304.00<br>     lat (usec): min=503, max=30044, avg=5323.84, stdev=3357.75<br>    clat percentiles (usec):<br>     |  1.00th=[ 1532],  5.00th=[ 1696], 10.00th=[ 1729], 20.00th=[ 1909],<br>     | 30.00th=[ 4555], 40.00th=[ 4686], 50.00th=[ 5211], 60.00th=[ 5276],<br>     | 70.00th=[ 5407], 80.00th=[ 5866], 90.00th=[ 6456], 95.00th=[14222],<br>     | 99.00th=[17695], 99.50th=[18482], 99.90th=[20317], 99.95th=[21103],<br>     | 99.99th=[21890]<br>   bw (  KiB/s): min=332576, max=1781184, per=100.00%, avg=769419.61, stdev=63897.95, samples=4796<br>   iops        : min=20786, max=111324, avg=48088.73, stdev=3993.62, samples=4796<br>  lat (usec)   : 750=0.01%, 1000=0.01%<br>  lat (msec)   : 2=20.67%, 4=3.57%, 10=68.17%, 20=7.41%, 50=0.17%<br>  cpu          : usr=0.81%, sys=3.06%, ctx=4121230, majf=0, minf=1062<br>  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, &gt;=64=100.0%<br>     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, &gt;=64=0.0%<br>     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, &gt;=64=0.0%<br>     issued rwts: total=28849585,0,0,0 short=0,0,0,0 dropped=0,0,0,0<br>     latency   : target=0, window=0, percentile=100.00%, depth=64<br><br>Run status group 0 (all jobs):<br>   READ: bw=751MiB/s (788MB/s), 751MiB/s-751MiB/s (788MB/s-788MB/s), io=440GiB (473GB), run=600002-600002msec<br><br>Disk stats (read/write):<br>  nvme2n1: ios=28844450/0, merge=0/0, ticks=58276760/0, in_queue=58276760, util=100.00%<br>[ec2-user@ip-172-31-30-136 ~]$ tail -v -n 99 fix_8600IOPS-io1-testing.cfg<br>==&gt; fix_8600IOPS-io1-testing.cfg &lt;==<br>[global]<br>clocksource=clock_gettime<br>randrepeat=0<br>runtime=600<br>time_based=1<br>direct=1<br>group_reporting=1<br>norandommap=1<br>size=1G<br><br>[phase3-randread-16k]<br>description=Phase 3: 16KiB Random Read on 48000 IOPS<br>filename=/dev/nvme2n1<br>ioengine=libaio<br>iodepth=64<br>rw=randread<br>bs=16k<br>numjobs=4<br>do_verify=0</pre><h3>Summary:</h3><p>The system call you invoke will produce different results.</p><p>To understand how psync works, please refer to the man 2 <a href="https://linux.die.net/man/2/pwrite">pwrite</a> manual page.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ebadfca27ff3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Figure out the asymmetric routing with the UDP protocol.]]></title>
            <link>https://laudaihe.medium.com/figure-out-the-asymmetric-routing-with-the-udp-protocol-e894c34e4990?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/e894c34e4990</guid>
            <category><![CDATA[asymmetric-routing]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[udp]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Sat, 10 May 2025 12:07:35 GMT</pubDate>
            <atom:updated>2025-06-30T00:05:13.811Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Author：Lau Dai</blockquote><blockquote>Date：2025/05/10</blockquote><p>Before starting this post.</p><p>This is a one thing you need to be aware.</p><p>The most of distro will replace the nc with ncat.</p><p>However, the Ubuntu distro didn’t soft link the nc(netcat) command with the ncat.</p><pre>ubuntu@ip-172-31-39-144:~$ ls -l /etc/alternatives/nc<br>lrwxrwxrwx 1 root root 15 Apr  8  2024 /etc/alternatives/nc -&gt; /bin/nc.openbsd</pre><p>So be sure that the version of nc command you use is.</p><h3>TL; DR;</h3><ol><li>Make sure that you already configure the policy-based routing on the right interface.</li><li>UDP protocol is stateless protocol rather than TCP. So it will return the packet as the OS routing policy without the flag.</li><li>Make sure that the application binds the interface you want to return.</li></ol><p>In this experiment, I bind the UDP server with ncat command on the secondary ENI.</p><p>Make sure that the socket you bind only on the target IP but not all interface as *.</p><pre>ubuntu@ip-172-31-39-144:~$ lsof -i -n<br>COMMAND   PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME<br>ncat    10285 ubuntu    8u  IPv4  40123      0t0  UDP 172.31.44.159:5000</pre><p>You can see that both of them can retrieve the UDP packet back to the original src IP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0ZM4ZNbzHtz4wk1Ifk3I3g.png" /></figure><h3>Appendix</h3><p>Why, I need the option in ncat -e &quot;/usr/bin/cat&quot;</p><p>Before added the option of nc command.</p><p>We can see that the ncat will try to respond the UDP package to the original src.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*pHt6rdeWOzvi-AJYihib_Q.png" /></figure><p>However, it will stuck as the below image for the stdout.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tlh1jyeOLiBCDHC-4-hXZQ.png" /></figure><p>Until you press the enter key, the UDP package will back to the original src.</p><p>However, it will lead to the UDP package timeout.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9iHggkk9V0uPPiAjP76U4w.png" /></figure><p>When you use the exec option, it will redirect to the cat process to deal with it.</p><pre>ubuntu@ip-172-31-39-144:~$ lsof -i -n<br>COMMAND   PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME<br>ncat    10376 ubuntu    4u  IPv4  40334      0t0  UDP 172.31.44.159:5000<br>ncat    10380 ubuntu    3u  IPv4  40302      0t0  UDP 172.31.44.159:5000-&gt;172.31.19.124:52845<br>ncat    10380 ubuntu    4u  IPv4  40332      0t0  UDP 172.31.44.159:5000-&gt;172.31.0.54:54899<br>cat     10381 ubuntu    3u  IPv4  40302      0t0  UDP 172.31.44.159:5000-&gt;172.31.19.124:52845<br>cat     10381 ubuntu    4u  IPv4  40332      0t0  UDP 172.31.44.159:5000-&gt;172.31.0.54:54899<br>ncat    10382 ubuntu    3u  IPv4  40333      0t0  UDP 172.31.44.159:5000-&gt;172.31.46.52:43516<br>ncat    10382 ubuntu    4u  IPv4  40332      0t0  UDP 172.31.44.159:5000-&gt;172.31.0.54:54899<br>cat     10383 ubuntu    3u  IPv4  40333      0t0  UDP 172.31.44.159:5000-&gt;172.31.46.52:43516<br>cat     10383 ubuntu    4u  IPv4  40332      0t0  UDP 172.31.44.159:5000-&gt;172.31.0.54:54899<br>ncat    10384 ubuntu    3u  IPv4  40333      0t0  UDP 172.31.44.159:5000-&gt;172.31.46.52:43516<br>ncat    10384 ubuntu    4u  IPv4  40334      0t0  UDP 172.31.44.159:5000<br>cat     10385 ubuntu    3u  IPv4  40333      0t0  UDP 172.31.44.159:5000-&gt;172.31.46.52:43516<br>cat     10385 ubuntu    4u  IPv4  40334      0t0  UDP 172.31.44.159:5000</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e894c34e4990" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Some distro can’t run user data with Base64 encoding, but how?]]></title>
            <link>https://laudaihe.medium.com/some-distro-cant-run-user-data-with-base64-encoding-but-how-25422e797afc?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/25422e797afc</guid>
            <category><![CDATA[cloud-init]]></category>
            <category><![CDATA[ec2]]></category>
            <category><![CDATA[user-data]]></category>
            <category><![CDATA[aws-ec2]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Wed, 23 Apr 2025 23:32:47 GMT</pubDate>
            <atom:updated>2025-04-23T23:32:47.827Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Author：Lau Dai</blockquote><blockquote>Date：2025/04/23</blockquote><p>In the previous experiment, I observed that the user data with base64 encoding already can run successfully on Ubuntu 20.04 and the AL 2023 but CentOS 7 to specifically CentOS 9.</p><p>This blog will record how to deep dive this issue and find out the root cause.</p><h3>Root cause</h3><p>Because the user data I passed into the base64 encoding to the AWS CLI will cause some of the version clod-init can’t recognise it.</p><p>Brief:</p><ol><li>Some distro can’t recognise the user data as shell script because they can’t read the data which was double encoding in base64.</li><li>The AWS will auto encoding your user data on the console and the CLI. And it will cause the double encoding issue with it. (You are only encoding the base64 via API if <a href="https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html">needed</a>.)</li><li>Amazon Linux have long patch on it for the double base64 encoding. Mentioned it in <a href="https://github.com/canonical/cloud-init/pull/4276#issue-1816443963">link</a>.</li><li>cloud init upstream accept this PR and merge it on <a href="https://github.com/canonical/cloud-init/pull/4276">cloud-init/pull/4276</a></li></ol><h3>Deep Dive</h3><p>You can run instance with CentOS 8 and Amazon Linux 2023.</p><p>Here is some highlight between the cloud-init.log with them.</p><p>Kernel and cloud init version difference.</p><pre>[cloud-user@ip-172-31-42-188 cloudinit]$ uname -a<br>Linux ip-172-31-42-188.eu-west-1.compute.internal 4.18.0-408.el8.x86_64 #1 SMP Mon Jul 18 17:42:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux<br>[cloud-user@ip-172-31-42-188 cloudinit]$ cloud-init --version<br>/usr/bin/cloud-init 22.1-6.el8<br>---<br>[ec2-user@ip-172-31-42-97 cloudinit]$ uname -a<br>Linux ip-172-31-42-97.eu-west-1.compute.internal 6.1.132-147.221.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Apr  8 13:14:54 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux<br>[ec2-user@ip-172-31-42-97 cloudinit]$ cloud-init --version<br>/usr/bin/cloud-init 22.2.2</pre><p>cloud-init.log</p><p>vimdiff c8-init.log al2023-init.log</p><pre>2025-04-18 16:54:05,421 - __init__.py[DEBUG]: {&#39;MIME-Version&#39;: &#39;1.0&#39;, &#39;Content-Type&#39;: &#39;text/x-not-multipart&#39;, &#39;Content-Disposition&#39;: &#39;attachment; filename=&quot;part-001&quot;&#39;}<br>2025-04-18 16:54:05,422 - __init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: &#39;b&#39;IyEvYmluL2Jhc2gKCkRFRkFV&#39;...&#39;<br>---<br>2025-04-18 23:22:20,062 - __init__.py[DEBUG]: {&#39;MIME-Version&#39;: &#39;1.0&#39;, &#39;Content-Type&#39;: &#39;text/x-shellscript&#39;, &#39;Content-Disposition&#39;: &#39;attachment; filename=&quot;part-001&quot;&#39;}<br>2025-04-18 23:22:20,062 - __init__.py[DEBUG]: Calling handler ShellScriptPartHandler: [[&#39;text/x-shellscript&#39;]] (text/x-shellscript, part-001, 2) with frequency always</pre><p>Also, we can observed differences in handlers/__init__.py and Datasource in /var/log/cloud-init-output.log</p><pre>2025-04-18 23:22:27,220 - __init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: &#39;b&#39;IyEvYmluL2Jhc2gKCkRFRkFV&#39;...&#39;<br>Cloud-init v. 22.1-6.el8 running &#39;modules:config&#39; at Fri, 18 Apr 2025 23:22:28 +0000. Up 11.54 seconds.<br>Cloud-init v. 22.1-6.el8 running &#39;modules:final&#39; at Fri, 18 Apr 2025 23:22:28 +0000. Up 12.37 seconds.<br>Cloud-init v. 22.1-6.el8 finished at Fri, 18 Apr 2025 23:22:29 +0000. Datasource DataSourceEc2Local.  Up 12.54 seconds<br>---<br>Cloud-init v. 22.2.2 running &#39;modules:config&#39; at Sat, 19 Apr 2025 23:05:46 +0000. Up 8.88 seconds.<br>Cloud-init v. 22.2.2 running &#39;modules:final&#39; at Sat, 19 Apr 2025 23:05:47 +0000. Up 9.90 seconds.<br>Cloud-init v. 22.2.2 finished at Sat, 19 Apr 2025 23:05:47 +0000. Datasource DataSourceEc2.  Up 10.19 seconds</pre><p>The log show out that both of them can bootup successfully and finished the final boot stage in the cloud init.</p><p>However, the __init__.py didn’t handled the user data and regard it as the non-multipart text.</p><h3>Let’s review the Cloud init Boot stages</h3><p><a href="https://cloudinit.readthedocs.io/en/latest/explanation/boot.html">https://cloudinit.readthedocs.io/en/latest/explanation/boot.html</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/1*5fM8Mz-J4o_SCsVegIlaGw.png" /></figure><p>According to the official document, there are 5 stages when running the cloud init.</p><p>The user data script will handle it on the final stage:</p><p>This stage runs as late in boot as possible. Any scripts that a user is accustomed to running after logging into a system should run correctly here. Things that run here include:</p><ul><li>package installations,</li><li>configuration management plugins (Ansible, Puppet, Chef, salt-minion), and</li><li><strong>user-defined scripts</strong> (i.e., shell scripts passed as user-data).</li></ul><p>So that, we can confirm that the cloud init running successfully.</p><p>It just didn’t execute it because cloud init didn’t how to run the raw text file.</p><p>Finally, let’s analysis of code.</p><p>The cloud init will engage the user_data.py, util.py to handle the user data.</p><p>Related files:</p><p>The latest version merged the code as below.</p><p>It will try decode the user date with base64 decoder.</p><pre>handlers/<br>├── init.py<br>├── pycache<br>│   ├── init.cpython-39.opt-1.pyc<br>│   ├── init.cpython-39.pyc<br>│   ├── boot_hook.cpython-39.opt-1.pyc<br>│   ├── boot_hook.cpython-39.pyc<br>│   ├── cloud_config.cpython-39.opt-1.pyc<br>│   ├── cloud_config.cpython-39.pyc<br>│   ├── jinja_template.cpython-39.opt-1.pyc<br>│   ├── jinja_template.cpython-39.pyc<br>│   ├── shell_script.cpython-39.opt-1.pyc<br>│   ├── shell_script.cpython-39.pyc<br>│   ├── shell_script_by_frequency.cpython-39.opt-1.pyc<br>│   └── shell_script_by_frequency.cpython-39.pyc<br>├── user_data.py<br>├── util.py &lt;---------------------------</pre><pre>def maybe_b64decode(data: bytes) -&gt; bytes:<br>    &quot;&quot;&quot;base64 decode data</pre><pre>If data is base64 encoded bytes, return b64decode(data).<br>    If not, return data unmodified.</pre><pre>@param data: data as bytes. TypeError is raised if not bytes.<br>    &quot;&quot;&quot;<br>    if not isinstance(data, bytes):<br>        raise TypeError(&quot;data is &#39;%s&#39;, expected bytes&quot; % type(data))<br>    try:<br>        return b64decode(data, validate=True)<br>    except binascii.Error:<br>        return data</pre><ul><li><a href="https://github.com/canonical/cloud-init/commit/658837389ee4fa094278f87dbd67b67ed76e63fb">https://github.com/canonical/cloud-init/commit/658837389ee4fa094278f87dbd67b67ed76e63fb</a></li><li><a href="https://github.com/canonical/cloud-init/blob/23.3/cloudinit/util.py">https://github.com/canonical/cloud-init/blob/23.3/cloudinit/util.py</a></li><li><a href="https://github.com/canonical/cloud-init/compare/22.1...23.3">https://github.com/canonical/cloud-init/compare/22.1...23.3</a></li></ul><p>The function in different distro but CentOS 8.</p><p>AL 2023</p><pre>[ec2-user@ip-172-31-42-97 cloudinit]$ grep -rne &quot;maybe_b64decode&quot; .<br>grep: ./__pycache__/util.cpython-39.opt-1.pyc: binary file matches<br>grep: ./__pycache__/util.cpython-39.pyc: binary file matches<br>./sources/DataSourceEc2.py:538:            crawled_metadata[&quot;user-data&quot;] = util.maybe_b64decode(raw_userdata)<br>./sources/DataSourceHetzner.py:87:        self.userdata_raw = util.maybe_b64decode(ud)<br>grep: ./sources/__pycache__/DataSourceEc2.cpython-39.opt-1.pyc: binary file matches<br>grep: ./sources/__pycache__/DataSourceEc2.cpython-39.pyc: binary file matches<br>grep: ./sources/__pycache__/DataSourceHetzner.cpython-39.opt-1.pyc: binary file matches<br>grep: ./sources/__pycache__/DataSourceHetzner.cpython-39.pyc: binary file matches<br>./util.py:148:def maybe_b64decode(data: bytes) -&gt; bytes:</pre><p>CentOS 8</p><pre>[cloud-user@ip-172-31-42-188 cloudinit]$ grep -rne &quot;maybe_b64decode&quot; .<br>./sources/DataSourceHetzner.py:87:        self.userdata_raw = hc_helper.maybe_b64decode(ud)<br>Binary file ./sources/__pycache__/DataSourceHetzner.cpython-36.opt-1.pyc matches<br>Binary file ./sources/__pycache__/DataSourceHetzner.cpython-36.pyc matches<br>Binary file ./sources/helpers/__pycache__/hetzner.cpython-36.opt-1.pyc matches<br>Binary file ./sources/helpers/__pycache__/hetzner.cpython-36.pyc matches<br>./sources/helpers/hetzner.py:30:def maybe_b64decode(data: bytes) -&gt; bytes:</pre><p>Ubuntu 22</p><pre>ubuntu@ip-172-31-0-54:~$ grep -rne &quot;maybe_b64decode&quot; /usr/lib/python3/dist-packages/<br>grep: /usr/lib/python3/dist-packages/cloudinit/__pycache__/util.cpython-310.pyc: binary file matches<br>grep: /usr/lib/python3/dist-packages/cloudinit/sources/__pycache__/DataSourceHetzner.cpython-310.pyc: binary file matches<br>grep: /usr/lib/python3/dist-packages/cloudinit/sources/__pycache__/DataSourceEc2.cpython-310.pyc: binary file matches<br>/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceHetzner.py:90:        self.userdata_raw = util.maybe_b64decode(ud)<br>/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceEc2.py:581:            crawled_metadata[&quot;user-data&quot;] = util.maybe_b64decode(raw_userdata)<br>/usr/lib/python3/dist-packages/cloudinit/util.py:149:def maybe_b64decode(data: bytes) -&gt; bytes:<br>ubuntu@ip-172-31-0-54:~$ lsb_release -a<br>No LSB modules are available.<br>Distributor ID:    Ubuntu<br>Description:    Ubuntu 22.04.5 LTS<br>Release:    22.04<br>Codename:    jammy</pre><h3>Ref</h3><p>Github:<br><a href="https://github.com/canonical/cloud-init/pull/4276">ec2: Support double encoded userdata<strong> #4276</strong></a><br><a href="https://github.com/amazonlinux/amazon-linux-2023/issues/401">[Bug] — Custom cloud init hack for userdata is broken for AL2023 <strong>#401</strong></a><br><a href="https://github.com/canonical/cloud-init/issues/4239">__init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata error <strong>#4239</strong></a></p><p>Doc:<br><a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html#user-data-cloud-init">User data and cloud-init directives</a><br><a href="https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html">RunInstances API</a><br><a href="https://awscli.amazonaws.com/v2/documentation/api/2.0.34/reference/ec2/run-instances.html">run-instances v2</a><br><a href="https://cloudinit.readthedocs.io/en/latest/explanation/format.html#id2">User-data script</a><br><a href="https://cloudinit.readthedocs.io/en/latest/reference/modules.html#mod-cc-scripts-user">Scripts User</a><br><a href="https://docs.aws.amazon.com/zh_cn/linux/al2/ug/amazon-linux-cloud-init.html#supported-user-data-formats">Supported user data formats (AL2 mentioned)</a><br><a href="https://docs.aws.amazon.com/linux/al2023/release-notes/relnotes-2023.1.20230823.html"><strong>Amazon Linux 2023 version 2023.1.20230823 release notes</strong></a></p><blockquote><em>The AMIs for this release were withdrawn shortly after release due to a bug that was found when launching an instance without </em><em>user-data. This issue doesn&#39;t affect containers. A new release without this issue will be available soon.</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25422e797afc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[愛爾蘭生活紀錄]]></title>
            <link>https://laudaihe.medium.com/%E6%84%9B%E7%88%BE%E8%98%AD%E7%94%9F%E6%B4%BB%E7%B4%80%E9%8C%84-a2466a31c318?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/a2466a31c318</guid>
            <category><![CDATA[dublin]]></category>
            <category><![CDATA[ireland]]></category>
            <category><![CDATA[life]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Mon, 07 Oct 2024 09:53:12 GMT</pubDate>
            <atom:updated>2024-10-07T09:53:12.718Z</atom:updated>
            <content:encoded><![CDATA[<p>What’s everything I meet in the Ireland.</p><blockquote>Author：Lau Dai</blockquote><blockquote>Date：2024/10/07 in Dublin</blockquote><h3>開戶</h3><p><em>Record date: 2024/10/07</em></p><p>在一般而言，歐洲開戶是需要帳管費用的</p><p>在愛爾蘭常見的銀行有 AIB, BOI</p><p>如果在無需再帳管費用考量下，可以考慮開立一年期的 <a href="https://www.citizensinformation.ie/en/money-and-tax/personal-finance/banking/standard-bank-account/">Basic Account</a></p><p>AIB 開立方式靈活，且條件簡單，可以參考官方<a href="https://aib.ie/our-products/current-accounts/online-bank-account-opening">介紹</a>。</p><blockquote>New Customers can open a sole Basic Bank Account on the Mobile app if they can prove they:</blockquote><blockquote>Live in the Republic of Ireland</blockquote><blockquote>Are aged 16 and over</blockquote><blockquote>Have a passport <a href="https://aib.ie/our-products/current-accounts/online-bank-account-opening/list-of-eligible-eea-countries">(or other acceptable photographic ID)</a></blockquote><p>BOI 開立<a href="https://personalbanking.bankofireland.com/bank/current-accounts/basic-bank-account/how-to-apply/">方式</a>只有寫要到現場去辦理(如下)</p><p>因此此次主要記錄 BOI 為什麼會被說是難以辦理的原因。</p><blockquote>You can open a Basic Bank Account by visiting any Bank of Ireland branch, where a Branch Adviser will guide you through the application process.</blockquote><ol><li>行員品質不一，在許多分局都會想把人打發走，並不會真的想解決問題。在Dublin City Center，在此推薦 O’Connell St 的女行員以及College Green 分行。她們才是真心會解決客戶的問題</li><li>基於前面四五次的來回，Proof of ID 我們選擇 Passport, Proof of Address 選擇 utility bill，在最後兩次，他們才說 Utility Application form 並不能當作依據，需要 utility bill payment 才可以。</li><li>最後，也是最奇怪的。 BOI 拒絕我們申請 Basic Account，他們說明 Basic Account 是只能給領取 <a href="https://www.citizensinformation.ie/en/social-welfare/">Social Welfare</a> 的人辦理。而這並沒有在前面提到的 <a href="https://www.citizensinformation.ie/en/money-and-tax/personal-finance/banking/standard-bank-account/">Basic Account</a> 以及 BOI 頁面提及。最後也只能作罷辦理 BOI Basi Account</li></ol><p>關於AIB, BOI 的 Regular Account 收費方式，可以參考：</p><p><a href="https://personalbanking.bankofireland.com/bank/current-accounts/personal-current-account/features-and-benefits/">BOI Personal current account</a></p><p><a href="https://aib.ie/content/dam/aib/personal/docs/our-products/current-accounts/personal-current-accounts/personal-information-leaflet.pdf">AIB Personal Bank Account</a></p><h3>租屋</h3><p>…</p><p>此篇受眾主要是在愛爾蘭的華人。</p><p>不論是在愛爾蘭打工度假、語言學校或者是在這工作。</p><p>如果讀者有任何需要討論的部分，歡迎一起討論。</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2466a31c318" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[雜記] JLab Epic Air ANC 耳機使用後心得]]></title>
            <link>https://laudaihe.medium.com/%E9%9B%9C%E8%A8%98-jlab-epic-air-anc-%E8%80%B3%E6%A9%9F%E4%BD%BF%E7%94%A8%E5%BE%8C%E5%BF%83%E5%BE%97-f30b9e03d7f9?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/f30b9e03d7f9</guid>
            <category><![CDATA[jlabs]]></category>
            <category><![CDATA[earbuds]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Sat, 31 Dec 2022 21:42:32 GMT</pubDate>
            <atom:updated>2023-01-02T09:40:33.962Z</atom:updated>
            <content:encoded><![CDATA[<h4>The experience of using JLab Epic Air ANC Earbuds</h4><blockquote>Author：Lau Dai</blockquote><blockquote>Date：2022/12/31 in Dublin</blockquote><p>我是在 igogosport 上購買人生第一隻破 3,000 的耳機</p><h3>TL;DR</h3><p>優點：一分錢一分貨，跟我另外一支 1,600 的漫步者耳機不能相比，不論是音質、降噪、通話或客製化按鍵等功能，都算高上許多檔次</p><p>缺點：個人懷疑產品良率不良</p><ol><li>當初在店面購買現場拆開連接後，發現一支耳朵無法連接，店家懷疑是沒電，所以請我回去充飽電再測試；結果隔天，右耳還是無法連接。因此，隔天只好跑去店家更換。</li><li>更換後返家進行褒機測試，褒機音源檔取自 JLab 官方網站提供，進行褒機後，發現兩隻耳朵的消耗電量不同，同樣燒機 5 個小時，左耳會剩下50% 電量，右耳剩下 30%，測試多次都是這樣的結果。</li></ol><p>因此，又再去店家更換，去更換的時候，店家說可能是因為先拿出的耳機會成為主耳機，因此耗電量會比較高，因此我這次回去再褒機測試看看；最後，在此次褒機測試 40 小時內，再也沒發生此問題。</p><p>直到今日，我已經出國使用三個禮拜了，居住環境室外大約 10~8 度、室內溫度約 17~15 度；溼度大約 70~60% 的都柏林，在此期間發生<strong>兩次</strong>聽歌或通話大約一個半小時，會出現雜音，並且中斷連線，說耳機電量低（Battery Low），到最後只能放入充電艙再取出，重新連線後才正常，心理真的很無法理解這產品到底出了什麼問題。已經無心、無力也無法再去處理此問題了</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*rqCd-cKkNyxJ649YCEA2LA.png" /><figcaption>上圖是今日出現連接異常後，放入充電艙馬上再取出的電量狀態。使用期間全程是兩耳同時使用</figcaption></figure><p>希望有看到此篇文章的讀者或者廠商，可以聊聊你的使用心得或者解釋這是什麼原因...</p><p>※附註：我對此耳機都是十分溫柔對待，絕不手溼、髒情況下碰此耳機與充電艙，輕放無碰撞，並且使用合乎電壓規範的充電裝置。畢竟這耳機要價不菲... 至於另外一對漫步者耳機，還蠻耐摔的，掉了兩次依然正常運作 XD</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f30b9e03d7f9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linux 掛載 swap 分區過程]]></title>
            <link>https://laudaihe.medium.com/linux-%E6%8E%9B%E8%BC%89-swap-%E5%88%86%E5%8D%80%E9%81%8E%E7%A8%8B-d747a287a67d?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/d747a287a67d</guid>
            <category><![CDATA[swap]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Thu, 04 Aug 2022 17:16:23 GMT</pubDate>
            <atom:updated>2022-08-04T17:16:46.292Z</atom:updated>
            <content:encoded><![CDATA[<h4>mount your swap partition on Linux</h4><blockquote>Author：Lau Dai</blockquote><blockquote>Date：2022/08/05</blockquote><p>凡事總有個因，今天會要處理這個swap分區問題，主要是當初 Ubuntu 裝好後，明明有設定 swap 分區，卻沒有開機自動掛載</p><p><strong>今天，用快速的兩分鐘來說明這件事情如何處理</strong></p><h3>過程</h3><p>因此脈絡很簡單，就是把該分區設定成 swap 自動掛載點。在此次紀錄中，因為原本沒有設定此掛載點，預設會採用檔案做 swap。所以才會看到有 2G的 swap 空間。</p><p>透過下圖一，可以看到我當初有設定10G分區，且當初是設定swap使用的</p><pre>lsblk -o name,mountpoint,label,size,rm,ro,type,uuid,partuuid -l {your partation}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OOai1swqCEaLKBBLLU6iqA.png" /><figcaption>圖一：系統當初設定分區</figcaption></figure><p>將 fstab寫入相對應的 config 設定即可，讀者可以依照需求填入自己的設定。並且記得以 UUID 方式寫入會更加保險</p><p>設定後即可 mount 起來，進行非重開機的測試，如下圖二，可以看見已經將該分區掛載</p><pre>sudoedit -e /etc/fstab<br>sudo mount -a</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/1*P2jnt8FQAL1e-7UmyBVT_A.png" /><figcaption>圖二：已將分區掛載</figcaption></figure><p>接下來掛載測試swap是否可以正常使用</p><pre># 掛載 fstab 中的所有swap分區<br>sudo swapon -a</pre><p>如圖三可以看到已經成功被掛載到了，至於 StarShip prompt 因為把原本的2G也算入，所以才會出現如下13G Swap的情況。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*18Up1U8deTxt240HkPljbw.png" /><figcaption>圖三：成功使用swap</figcaption></figure><p>重新啟動後，可以看到 htop成功顯示正確的swap大小</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*duulMu9EJiptr6S9KVnhFQ.png" /><figcaption>圖四：htop顯示正確的swap大小</figcaption></figure><h3>後記</h3><p>會想做這件事是因為最近真的快被 oom 搞死了，沒想到我 24 GB RAM 還是不夠用，看來開五百多個網頁的習慣還是該改掉。</p><p>另外，沒想到操作很快速，但要把過程紀錄下來卻是這個過程的將近6倍時間，想到網路上會留紀錄資料的前輩們，真是萬分的感謝。</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d747a287a67d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kill python subprocess when timeout]]></title>
            <link>https://laudaihe.medium.com/kill-python-subprocess-when-timeout-aea5173713e6?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/aea5173713e6</guid>
            <category><![CDATA[python3]]></category>
            <category><![CDATA[signal]]></category>
            <category><![CDATA[subprocess]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Sun, 12 Jun 2022 15:26:14 GMT</pubDate>
            <atom:updated>2022-09-20T14:17:54.351Z</atom:updated>
            <content:encoded><![CDATA[<h4>在 subprocess timeout 時，終結子進程</h4><blockquote>Author：Lau Dai</blockquote><blockquote>Date：2022/06/12</blockquote><p>在 subprocess 模組中，有個 timeout 的選項，之前在使用的時候會很自然以為調用此模組後，若觸發 timeout 會將子進程一併中斷。 但在實際運作過後才發現並非如此。</p><p>timeout 觸發後會 raise 事件，但原本的子進程會仍在背景執行。</p><p><strong>因此，若要解決此問題，最好的辦法就是在 timeout 後，同時將子進程 kill 即可。</strong></p><p>對此已經有人寫了一篇文章，裡面有十分詳細的說明發生的原因，以及如何處理。文章連結：<a href="https://alexandra-zaharia.github.io/posts/kill-subprocess-and-its-children-on-timeout-python/">Kill a Python subprocess and its children when a timeout is reached</a></p><p>在此文章中有說明 <strong>Process Group</strong>、<strong>Session </strong>等以及調用start_new_session選項。</p><p>對於此程式有興趣的讀者，可以使用下面兩組程式碼進行測試，執行 python檔案時，可以測試保留註解以及拿掉註解後，會有什麼差異。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*Yvn8G5F3lFwSgIFiQCadVQ.png" /><figcaption>單純執行 bash script 結果</figcaption></figure><pre>#!/usr/local/bin bash</pre><pre>testfilename=/tmp/testfile</pre><pre>echo &quot;check $testfilename exist or not?&quot;<br>[ -f $testfilename ] &amp;&amp; echo &quot;$testfilename exists.&quot; || echo &quot;$testfilename not exists&quot;<br>echo</pre><pre>sleep 10</pre><pre>echo &quot;new file $testfilename&quot;<br>echo<br>touch $testfilename</pre><pre>echo &quot;check $testfilename exist or not?&quot;<br>[ -f $testfilename ] &amp;&amp; echo &quot;$testfilename exists.&quot; || echo &quot;$testfilename not exists&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/421/1*njosXGFEKLmKWjbt-zSoWQ.png" /><figcaption>執行 python 結果（保留註解）</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*mmOlUfIzk_LrLoJeqpoS_g.png" /><figcaption>執行 python 結果（移除註解）</figcaption></figure><pre>#!/usr/bin/env python<br># encoding: utf8<br># Author : laudai</pre><pre>import subprocess<br>import os<br>import signal</pre><pre>cmd = [&quot;/usr/bin/env&quot;,&quot;bash&quot;,&quot;/home/laudai/touchfile.sh&quot;]<br>timeout_sec = 5</pre><pre>try:<br>    cp = subprocess.Popen(cmd)<br>    cp.wait(timeout_sec)<br>except subprocess.TimeoutExpired:<br>    print(f&quot;run subprocess timeout {timeout_sec} seconds&quot;)<br>    #print(&quot;will kill subprocess via send signal TERM to subprocess&quot;)<br>    #os.killpg(os.getpgid(cp.pid), signal.SIGTERM)</pre><p>— 2022/09/20 更新 —</p><p>在官方文件中，可以使用下列兩種方式送出 Signal，結束 timeout 的子程序</p><pre>proc.terminate()<br>proc.kill()</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aea5173713e6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linux FHS重點節錄]]></title>
            <link>https://laudaihe.medium.com/linux-fhs%E9%87%8D%E9%BB%9E%E7%AF%80%E9%8C%84-2e20a0a3bbc1?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e20a0a3bbc1</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[fh]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Wed, 01 Jun 2022 16:21:26 GMT</pubDate>
            <atom:updated>2022-06-01T16:21:26.067Z</atom:updated>
            <content:encoded><![CDATA[<h3>Linux FHS 重點節錄</h3><h4>Linux FHS note</h4><blockquote>Author：Lau Dai</blockquote><blockquote>Date：2021/11/27</blockquote><p>此文是依照 Linux Foundation <a href="https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html">FHS 3.0</a> 做資料整理。</p><p>主要依照筆者個人理解方式進行翻譯、整理。若有需要，建議可以直接觀看原文資料。</p><p>FHS全文為Filesystem Hierarchy Standard，主要用於三種使用者：</p><ol><li>給獨立軟體供應商開發時，符合FHS規範</li><li>創建符合FHS的系統</li><li>使用者了解與維護符合FHS規範的系統</li></ol><h3>檔案系統（The Filesystem）</h3><p>在文件系統中，可以透過2x2矩陣來劃分以及區隔：可分享（Shareable）、不可分享（Unshareable）、靜態（Static）、可變動（Variable）。</p><p>舉例來說：user home 屬於可分享，然而 device lock 則不是；而靜態檔案包括二進制檔案（binaries）、函式庫（libraries）、文件（documentation）以及其他在沒有管理權限下無法修改的檔案。</p><p>使用者可以將靜態檔案與動態檔案分開儲存，因為靜態檔案可以儲存在 read-only 的裝置上，而且不用像可變動檔案一樣進行排程備份。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/325/1*h0AMHlg6BInxjVUubYQRIQ.png" /><figcaption>官方舉例</figcaption></figure><h3>根檔案系統（The Root Filesystem）</h3><p>根檔案系統內必修要有足夠的能力去開機、修復、恢復</p><pre>The contents of the root filesystem must be adequate to boot, restore, recover, and/or repair the system.</pre><p>因此根路徑會有應用程式、設定檔、boot loader資訊和其他必要的啟動資料</p><p>/usr, /opt和/var等</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*bgMYQgrI6F6q9t3QgGY20A.png" /><figcaption>根目錄下，各目錄描述</figcaption></figure><p>資料來源：<a href="https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s02.html">https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s02.html</a></p><ul><li>/bin：必要的命令binary檔</li><li>/boot：boot loader的靜態檔案</li><li>/dev：裝置檔案</li><li>/etc：系統設定檔</li><li>/lib：必要的可分享函式庫與核心模組</li><li>/media：可移除裝置的掛載點</li><li>/mnt：檔案系統暫時掛載的掛載點</li><li>/opt：額外的應用程式安裝位置</li><li>/run：正在運行的程序資料</li><li>/sbin：必要的系統binary檔</li><li>/tmp：暫存檔案</li><li>/usr：第二個階層(?</li><li>/var：可變動資料</li></ul><p>在此文件中，將詳細說明各個目錄，並將 /usr與/var另外拉出新章節說明。</p><p>/home、/lib&lt;qual&gt;、/root都是屬於 Optional 的目錄</p><h4>/bin : Essential user command binaries (for use by all users)</h4><blockquote>contains commands that may be used by both the system administrator and by users, but which are required <strong>when no other filesystems are mounted</strong> (e.g. in single user mode). It may also contain commands which are used indirectly by scripts.</blockquote><ol><li>/bin內不能有子目錄</li><li>非必要放入 /bin內的命令檔案都放到 /usr/bin內部</li><li>在目錄中有規定需要的 <a href="https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s04.html#ftn.idm236094534800">command binaries</a></li></ol><p>附註：</p><pre>If <strong>/bin/sh</strong> is not the POSIX compatible shell command itself, it must be a hard or symbolic link to the real shell command.</pre><h4>/boot : Static files of the boot loader</h4><p>此目錄包含除了開機時的設定檔、map installer 之外所有的開機會用到的檔案都在此。 /boot內儲存的都是在執行 kernel前 user-mode 可以使用的程式。可以儲存 master boot sectors 與 sector map 檔案。(個人註記：UEFI 啟動時，有所不同，需要讀者另外參照 EFI 啟動相關資料)</p><ol><li>使開機程式 boot loader 協助可以正常運作的程式必須放在 /sbin</li><li>開機時，boot loader 不需加載的設定檔放在 /etc</li></ol><p>特別規範：</p><p>作業系統的 kernel 必須放在 /或/boot</p><p>某些系統對於 /boot有特別要求的在此不一依列舉</p><h4>/dev : Device files</h4><p>放置特殊檔案或者裝置檔案</p><p>如果需要手動建立 /dev內的檔案，需要有個 MAKEDEV的命令來建立/dev內的檔案。也需要 MAKEDEV.local為了本地設備檔案。</p><blockquote>If required, MAKEDEV must have provisions for creating any device that may be found on the system, not just those that a particular distribution installs.</blockquote><blockquote>這句話無法理解</blockquote><h4>/etc : Host-specific system configuration</h4><p>此 /etc目錄階層主要是放置系統的設定檔。它是本地檔案並且用來控制設定軟體使用；它必須是靜態且並非 binary 檔案。</p><p>主要推薦存放位置在 /etc目錄底下的子目錄，而非直接儲存在/etc底下。</p><p><strong>此目錄要求：</strong></p><ol><li>在此目錄沒有 binaries</li><li>有規定特定的實體目錄或者連結目錄</li></ol><p>如果有裝相對應的子系統，這些的實體目錄或連結目錄必須在 /etc當中。</p><ul><li>X11 Configuration for the X Window system (optional)</li><li>sgml Configuration for SGML (optional)</li><li>xml Configuration for XML (optional)</li></ul><p>也有檔案規定需要放置在 /etc，細節請看此<a href="https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s07.html#ftn.idm236096774784">文件</a></p><p>常見的檔案如：fstab, group, hosts, motd, mtab, passwd, profile等</p><p>其中在Ubuntu 20.04中式採用 systemd作為 init，因此是不會看到如 inetd.conf、inittab等檔案。</p><p><strong>/etc/opt : Configuration files for /opt</strong></p><p>主要是你安裝在/opt的第三方套件的設定檔</p><p><strong>/etc/X11 : Configuration for the X Window System (optional)</strong></p><p>主要是給 X11 使用的設定擋路徑。當 /user 被掛載成 read-only 時，此目錄必須讓本地可以控制。</p><p>xorg.conf X.org 7 之後使用的設定檔</p><p>Xmodmap全局 X11 鍵盤設定檔案</p><ul><li>/etc/sgml : Configuration files for SGML (optional)</li><li>/etc/xml : Configuration files for XML (optional)</li><li>上面這兩個路徑，有興趣的同樣再去翻看<a href="https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s07.html#ftn.idm236096774784">文件</a></li></ul><p>xdm之前設定擋路徑在 /usr/lib/X11/xdm，現在則是/etc/X11/xdm。xdm 局部變動資料儲存在/var/lib/xdm。</p><h4>/home : User home directories (optional)</h4><p>/home 路徑顯然是標準的概念，但並非特殊規範的檔案系統路徑，使用者可能會想把他帳號在不同的地方。因此，程式不該假設有一定有個 /home 路徑而去對它進行訪問。</p><p>當要找user的家目錄時，不應該依靠 /etc/passwd，因為使用者資訊可能儲存在 NIS 伺服器，因次你需要 library function 如fgetpwent中的 getpwent, getpwent_r 。</p><p>在許多環境下，已有其他目錄服務比 NIS 更先進、更安全，例如 <a href="https://zh.wikipedia.org/wiki/LDAP">LDAP</a> 就已經逐漸取代 NIS。<a href="https://zh.wikipedia.org/zh-tw/%E7%B6%B2%E8%B7%AF%E8%B3%87%E8%A8%8A%E6%9C%8D%E5%8B%99">wiki</a></p><p><strong>此目錄要求：</strong></p><p>當需要在使用者家目錄底下為了程式建立特定的設定檔時，需要以 “.”作為開頭（即 dot file 的意思）；若該程式有多個設定檔，則需要以 subdirectory 子目錄方式建立 dot directory 資料夾，但其資料夾內的檔案不該再以“.”作為開頭。</p><p>個人舉例：</p><pre>ls .ssh/<br>config  id_ed25519  id_ed25519.pub  id_rsa  id_rsa.pub  known_hosts</pre><h4>/lib : Essential shared libraries and kernel modules</h4><p>此目錄包含了開機時所需要用到的 shared libraries 以及在 root filesystems 所需要的 command 檔案。</p><p>至少需要下面任一種檔案（可以是檔案或者連結符號）</p><ul><li>libc.so.*The dynamically-linked C library (optional)</li><li>ld*The execution time linker/loader (optional)</li></ul><p>如果 C 語言的預處理器有安裝，/lib/cpp 必須參照它。</p><p>當相對應的子系統安裝時，module 目錄必須在 /lib底下。</p><p>module 中存放可載入的 kernel 模組。</p><h4>/lib&lt;qual&gt; : Alternate format essential shared libraries (optional)</h4><p>此資料夾目的（待確認）</p><pre>There may be one or more variants of the /lib directory on systems which support more than one binary format requiring separate libraries.</pre><p>在32或者64位元的電腦中，此資料夾名稱通常為/lib32、/lib64。</p><p><strong>此目錄要求：</strong></p><p>如果這類型的目錄有多個類似的，那麼需要與一般的/lib目錄內容相同，除了 /lib&lt;qual&gt;/cpp 這檔案非必要相同。</p><h4>/media : Mount point for removable media</h4><p>此目錄的子目錄通常掛載可移除裝置媒體，如：軟碟、光碟、zip disk</p><p>在歷史上有許多掛載在根目錄的可移除裝置，如/cdrom、/mnt、/mnt/cdrom。這要會造成根目錄有許多額外的目錄。</p><p>下列目錄需在/media目錄下：floppy, cdrom, cdrecorder(CD Writer), zip。當目錄下有多個相同裝置時，需要從數字0開始附加在目錄後方，如：/media/cdrom0 和 /media/cdrom1。</p><h4>/mnt : Mount point for a temporarily mounted filesystem</h4><p>主要是提供給系統管理者，暫時掛載檔案系統所需要的。此路徑不應該要影響任何程式的運作。</p><p>此路徑不應該用來安裝程式，下列句子無法理解。</p><pre>a suitable temporary directory not in use by the system must be used instead.</pre><h4>/opt : Add-on application software packages</h4><p>/opt是保留給額外安裝的應用程式</p><p>安裝在/opt的套件，其靜態檔案需放在 /opt/&lt;package&gt; 或 /opt&lt;provider&gt;目錄樹上。其差異為 package 表示描述此軟體名稱，而 provider 是依據LANANA註冊的名字。（LANANA 全名為 The Linux Assigned Names And Numbers Authority，Linux 命名空間認証機構）</p><p>若是此目錄需要的設定檔，儲存位置於 /etc/opt 路徑。套件中的檔案不能放置在 /opt, /var/opt, /etc/opt 之外。除了特殊目的之特殊檔案，舉例為了讓檔案系統運作正常的檔案，如 device lock file 需要位於 /var/lock，而 device檔案需要位於 /dev。</p><h4>/root : Home directory for the root user (optional)</h4><p>雖然 root 帳戶的家目錄可以依照開發者決定，但推薦還是使用此路徑作為root 的家目錄。</p><p>如果 root 帳戶的家目錄不在 root分區當中，則有必要在設定在無法找到時，將 root 家目錄設定為 /目錄。</p><p>root 帳戶不應作為一般非授權帳戶使用，而需將 root 帳戶作為系統管理者角色。基於此原因，不推薦將mail或者其他應用程式子目錄放置 /root 路徑。當需要 mail 給具有管理權限使用者時，使用 root, postmaster, webmaster 等比較恰當。</p><h4>/run : Run-time variable data</h4><p>此目錄結構包含從開機以來，關於作業系統的信息資料。此目錄的資料開機的過程中，需要被清除（removed or truncated as appropriate）。</p><p>過去與此目錄相同角色的路徑為 /var/run ，除非有特別註明，不然遷移到/run路徑的程式應該要停止使用 /var/run 路徑。</p><p><strong>此目錄要求：</strong></p><p>最初放在 /etc的 pid(Process identifier）需要放置在 /etc 中，舉例如：acpi.pid, cron.pid。命名格式為：PID檔案加上名字：&lt;program-pid&gt;.pid</p><p>PID 內檔案規格一致，都是使用 ASCII 編碼十進位數字加上換行符號。</p><p>作業系統中，維護 Unix-domain sockets 的系統程式也會放在此路徑，或者是適當的子目錄。</p><h4>/sbin : System binaries</h4><p>系統管理員或者 root 角色會用到的應用程式會存放在 /sbin, /usr/sbin和/usr/local/bin 路徑。除了 /bin路徑， /sbin 也存放開機、修復、恢復 OS 時所需要的 binary 檔案。本地安裝的系統管理程式應該被安裝至 /usr/local/sbin</p><p><strong>此目錄要求：</strong></p><p>在/sbin中不能有子目錄。</p><p>在/sbin中，需要有 shutdown 命令檔案或連結檔案在此路徑。</p><p>至於當安裝相關套件時，哪些檔案需要在此路徑，可參考原文<a href="https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s16.html">連結</a>。</p><h4>/srv : Data for services provided by this system</h4><p>此目錄提供此系統架站時（透過service），存放會用到的資料</p><p>此路徑主要目的為提供使用者在特定服務中，找到資料存放的位置。以便用來放置 read-only data, writable data 或 script(如cgi script）。如果資料只有特定使用者才有興趣，那應該放在他們的 home 路徑。</p><p>目前因為對於 /srv 下面的子目錄還沒有共識，所以在命名上還沒辦法指定如何使用。不應該有任何程式去依照 /srv 或 /srv 下面的子目錄去進行運作，因為不同主機情況下情況都不同。</p><h4>/tmp : Temporary files</h4><p>/tmp 目錄為提供程式需要臨時檔案時使用</p><p>程式不應該假定在 /tmp 中的檔案可以被保留做使用</p><p>IEEE 標準 POSIX.1–2008 列出與之相似的需求</p><p>雖然儲存在 /tmp 中的檔案會被以某種方式進行管理與刪除，但建議系統在啟動時刪除 /tmp 中的目錄與檔案</p><p>如下文，此規範並非強制，而是根據過往歷史習慣給出的建議。</p><blockquote>FHS added this recommendation on the basis of historical precedent and common practice, but did not make it a requirement because system administration is not within the scope of this standard.</blockquote><h4>/usr Hierarchy</h4><p>/usr 在檔案系統中，是第 2 重要的部份。/usr 是可分享、唯讀檔案資料。這意味著 /usr 可以在不同兼容 FHS 中是可分享且必然無法被寫入。</p><p>任何與主機相關或會隨著時間改變的資訊都儲存在其他地方。</p><p>大型軟體不能使用 /usr 下面的子目錄結構。</p><p><strong>此目錄要求：</strong></p><p>在 /usr 中，需要有下列目錄或目錄連結檔案</p><ul><li>/usr/bin大部份使用者放置命令檔的地方</li><li>/usr/lib檔案庫</li><li>/usr/localLocal hierarchy (empty after main installation)推薦讀此目錄<a href="https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s09.html">原文</a></li><li>/usr/sbin非重要的系統二進制檔案</li><li>/usr/shareArchitecture-independent data；與系統架構無關的唯讀文件資料檔。如：系統手冊文件、顏色管理等。</li></ul><p><strong>可存在目錄：</strong></p><ul><li>/usr/games 遊戲與教育相關二進制檔案</li><li>/usr/includeC語言的header檔案</li><li>/usr/libexec其他程式運行時所需要的二進制檔案</li><li>/usr/lib<em>&lt;qual&gt;</em>Alternate Format Libraries (optional)</li><li>/usr/src原始碼</li></ul><p>X Windows 系統相關的在 /usr 目錄中是個體制中的例外</p><p>直到所有版本都使用/var目錄結構為止之前，下列的目錄會以符號連結（symbolic links）方式存在。</p><pre>/usr/spool -&gt; /var/spool<br>/usr/tmp -&gt; /var/tmp<br>/usr/spool/locks -&gt; /var/lock</pre><p>當系統不需要上述符號連結時，如果需要的話可以移除。</p><p>在 Ubuntu 20.04 中，已經無 /usr/{spool,tmp,spool/locks} 等目錄。</p><h4>/var Hierarchy</h4><blockquote>/var contains variable data files. This includes spool directories and files, administrative and logging data, and transient and temporary files.</blockquote><p>/var 目錄主要包含可變動性的資料，包含如 spool 目錄與檔案、管理 log 資料與短暫性檔案。</p><p>但並非所有資料都是屬於可分享的，如/var/log, /var/lock和 /var/run都是不可分享的。但/var/mail, /var/cache/man, /var/cache/fonts和 /var/spool是屬於可分享的。</p><p>/var 的主要目的是要讓 /usr 可以以 read-only 方式掛載。</p><p>若/var 不能以單獨分區存在，最好將 /var 移除根目錄並放入 /usr分區。</p><p>應用程式不能在 /var 目錄內建立目錄，此目錄下需要以系統有影響層級，並與 FHS 組織有進行 mailing list 討論後才加入。</p><ul><li>/var/account(選擇性)：此目錄保留目前使用程序帳戶的log與程序使用的資料</li><li>/var/cache：暫存（cached）應用程式的資料，此資料會用到大量時間的如 I/O 或計算。應用程式必須可以重新產生或者恢復資料。與 /var/spool內的資料不同，這些暫存資料是可以被刪除的。</li><li>/var/crash（選擇性）：此目錄包含系統故障時的 crash dumps 資料。在此版本 FHS 發佈之時，Linux 不支持此結構，但其他符合 FHS 的系統可能有支援此目錄結構。</li><li>/var/games(選擇性)：任何關於 game 在 /usr 相關的可變動資料應該要放在此目錄。而靜態資料如 help text, levle descriptions 等資料要放在其他目錄如 /usr/share/games。</li><li>/var/lib：此目錄結構包含與應用程式有關的狀態訊息（state information)，此狀態訊息是應用程式運行中與主機運行的資料。使用者不需修改 /var/lib 的配置來修改軟體的設定，而儲存這些資料的目錄不能給一般使用者操作。</li><li>/var/lock：lock files 應該被儲存在 /var/lock 目錄結構，lock file 是給裝置或其他多個應用程式分享資源，如原本在/usr/spool/locks 或/usr/spool/uucp的 serial device lock files，現在必須儲存在 /var/lock 中。</li><li>/var/log：此目錄包含各種 log file，大部分的 log 都寫在此目錄或者其子資料夾。</li></ul><ol><li>/var/log/lastlog 紀錄每個使用者最後登入情況</li><li>/var/log/messages 紀錄systemd的訊息</li><li>/var/log/wtmp 紀錄所有登入登出情況</li></ol><ul><li>/var/mail(選擇性)：mail spool 必須透過此目錄取得每個 &lt;username&gt; 的mail檔案，此 /var/mail 路徑可以是來自其他目錄的鏈結。</li><li>/var/opt：在 /opt 目錄中儲存 package variable data 的地方，其中/var/opt/&lt;subdir&gt;，subdir 這個目錄名稱是來自 package 在 /opt 中儲存靜態資料的目錄名稱。</li><li>/var/run：在過去式存放啟動以來的資料，目前此功能以移至 /run 目錄，現在保留是為了與舊系統與軟體保持兼容性。</li><li>/var/spool：此目錄存放稍候處理的資料。通常在處理後就刪除。</li><li>/var/tmp：此目錄存放的資料是在系統重新啟動前所保留的資料，因此此目錄資料會比 /tmp 保存來的久一點。在開機啟動時，不會刪除此目錄資料。</li><li>/var/yp(選擇性)：存放 NIS（Network Information Service） variable data。廣為人知的是 Sun Yellow Pages (YP) 需放在此目錄。</li></ul><h4>Operating System Specific Annex作業系統特定附件</h4><p>此章節的額外需求只推薦限定在作業系統上面，此段落部份不與前面標準衝突</p><ol><li>/ : Root directory 在 Linux 系統，如果核心檔案放在 / 目錄，那麼會推薦將其命名為 vmlinux或vmlinuz，這在最近的 Linux source package 已這樣使用。</li><li>/bin : Essential user command binaries (for use by all users) 在 Linux 系統中，額外的文件檔放在此目錄如：setserial</li><li>/dev : Devices and special files 下列 3 個檔案必須存在 /dev 目錄中， /dev/null、/dev/zero、/dev/tty。（詳情如下補充說明）</li><li>/etc : Host-specific system configuration 在Linux系統中，額外的文件檔放在此目錄如：lilo.conf</li><li>/proc : Kernel and process information virtual filesystem 在 /prco 檔案系統實際上是 Linux 標準處理程序與系統資訊的方式，而不是 /dev/kmem或其他方式。</li><li>/sbin : Essential system binaries 放置檔案系統維護與 bootloader 管理等的檔案。選擇性靜態檔案： ldconfig、sln、ssync。雜項檔案： ctrlaltdel、kbdrate</li><li>/sys : Kernel and system information virtual filesystem 一些關於檔案系統中的裝置、驅動、部份kernel等資訊記載的地方。底層架構由特定 Linux kernel 決定，反之則未指定。</li><li>/usr/include : Header files included by C programs 在裝了 c 或 c++但並非基於 glibc，需要進行符號鏈結檔案。</li></ol><pre> /usr/include/asm -&gt; /usr/src/linux/include/asm-&lt;arch&gt;<br> /usr/include/linux -&gt; /usr/src/linux/include/linux</pre><p>9. /var/spool/cron : cron and at jobs 此目錄包含 variable data 的 cron 與 at 程式</p><p>※ /dev/null：所有寫入此裝置的資料將會被丟棄，若從此裝置讀取內容將會取得EOF狀態。</p><p>※ /dev/zero：此裝置是 0 產生的來源。所有寫入此裝置的資料會被丟棄，從此裝置可以讀取出特定長度的 byte 資料以 0 作為表示。</p><p>※ /dev/tty：此裝置表示控制 terminal 裝置的同義詞，一旦此裝置被打開，所有讀寫的資料就像實際在 terminal 裝置控制一樣。</p><h3>結語</h3><p>從頭到尾花了 6 個月（2022/06/02初版）才把這 FHS 的部份內容看完，實在是愧對自己。透過了解 FHS 架構，才能對於撰寫程式、尋找檔案的時候，更了解 Linux 規則，也了解 Linux OS 設計之美。</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e20a0a3bbc1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[Linux] PXE Boot Ubuntu 20.04 via Serva on Windows 10]]></title>
            <link>https://laudaihe.medium.com/pxe-boot-ubuntu-20-04-via-serva-on-windows-10-43f98df1f8a5?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/43f98df1f8a5</guid>
            <category><![CDATA[ubuntu-20-04]]></category>
            <category><![CDATA[pxe]]></category>
            <category><![CDATA[serva-software]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Wed, 06 Apr 2022 16:41:18 GMT</pubDate>
            <atom:updated>2022-04-06T16:49:28.790Z</atom:updated>
            <content:encoded><![CDATA[<h4>在 Windows 10 透過 Serva 進行 Ubuntu 20.04 Live Desktop PXE 開機</h4><blockquote>Author：Lau Dai</blockquote><blockquote>Date：2022/03/24</blockquote><p>You can learn more about PXE via Serva’s offical article:</p><p><a href="https://www.vercot.com/~serva/an/WindowsPXE1.html">PXE/BINL — AN01: Windows Network Install — Basis</a></p><p><a href="https://www.vercot.com/~serva/an/NonWindowsPXE3.html">PXE/BINL — AN03: Non-Windows Network Boot/Install</a></p><h4>Requirements</h4><ol><li>Serva software [<a href="https://www.vercot.com/~serva/download.html">Download link</a>], I use <a href="https://www.vercot.com/~serva/download/Serva_Community_v4.6.0-21120715.zip">Serva_Community_v4.6.0</a> in this practice.</li><li>Windows OS which run Serva software</li><li>DHCP server (optional if run proxyDHCP)</li><li>Assumed knowledge: PXE knowledge, Linux basics.</li></ol><h3><strong>PXE Boot tutorial</strong></h3><p><strong>Step 1. Run Serva as administrator and set the TFTP server, DHCP/proxyDHCP server</strong></p><p>In this article I use proxyDHCP service instead of DHCP server via Serva.</p><p>Select the root folder for the TFTP server</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*EB8HfjJsKXKqoy6RD67CkA.png" /></figure><p><strong>Step 2. Setup the proxyDHCP and checked the BINL service</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*Ufwu03N-lwVdWKr6yPA9tw.png" /></figure><p><strong>Step 3. Close the Serva and restart Serva to run the config.</strong></p><p><strong>Step 4. After restart the Serva. Share the NWA_PXE folder as NWA_PXE_SHARE via Advanced Sharing.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/363/1*Yj7SHAWywH-AoNcQfzwE6Q.png" /></figure><p><strong>Step 5. Mount the your </strong><strong>ubuntu20.04-desktop.iso. Copy the file to <em>NWA_PXE/{your distro folder name}</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NnlqZ_WxQYckt8WYGWc6mQ.png" /></figure><p><strong>Step 6. Create ServaAsset.inf in <em>NWA_PXE/{your distro folder name}/ServaAsset.inf</em></strong></p><p>ServaAsset.inf content like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b7dbef980581753efcffb6400bd6de38/href">https://medium.com/media/b7dbef980581753efcffb6400bd6de38/href</a></iframe><p><strong>Step 7. Open SMB/CIFS (you can run </strong><strong>OptionalFeatures command)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*B4H0jYm8VjRXNBFjmgsl0g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/415/1*or1oVkgGFf14OetPW0NlFg.png" /></figure><p><strong>Step 8. Create a username/password account serva/avres</strong></p><p>You can run cmd/powershell as administrator via command net user serva avers /add</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/859/1*xWEz2eWOX4Sa7oSKoUURJA.png" /></figure><p><strong>Step 9. Run your PXE client device to select your OS to boot.</strong></p><p>You can see the log in Serva when you run PXE client.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rFy4u72veuQJAM-fuJB4fw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zkdAGDmVW-B8_lrSVihPPQ.png" /></figure><h3>中文版教學與細節補充：</h3><p>基本上看完官方教學文章，或者按照上面流程就可以正確執行。</p><p>主要分成幾個階段</p><ol><li>設定PXE Server所需的服務如TFTP server, DHCP 與 proxyDHCP server。</li><li>掛載SMB/CIFS服務，將檔案移至該文件區。</li><li>建立後續啟動時，與SMB/CIFS 服務連線使用的帳號密碼。</li><li>設定 ServaAsset.inf 檔案，讓非 Windows 的情況下，Serva 的 NBP 可以認得這些檔案。</li></ol><h4>運作流程：</h4><ol><li>PXE Client 透過廣播方式，取得 IP、TFTP Server 位置</li><li>向 TFTP Server 取得 NBP 檔案</li><li>透過 NBP取得 vmlinuz、initrd檔案</li><li>啟動 initrd後，進行 netboot，嘗試與 SMB/CIFS server 進行連線</li><li>將 live_media 的檔案複製到 ram tmpfs(此測試為在8G ram的檔案中，掛載3461858k的tmpfs)</li></ol><pre>mount -t tmpfs -o size=3461858k /dev/shm /cdrom_swap</pre><p>6. 檔案會先從 SMB/CIFS 中下載到 PXE Client tmpfs 當中（此時會吃硬碟與網路的效能）</p><p>7. 之後就是檢查檔案完整性的流程（已無從SMB/CIFS取得檔案），最後會啟動 Live CD，完成Live Desktop開機作業。</p><h4>特別注意：</h4><ol><li>ServaAsset.inf 不要儲存成文字檔，這不像 Linux 單純看檔名決定檔案屬性。</li><li>目前使用 Ubuntu 20.04.4-desktop.iso 進行此次試驗。Ubuntu 20.04.2-desktop.iso 在相同的軟體、ServaAsset.inf 設定檔、相同的SMB/CIFS 環境，並且驗證過檔案完整性的情況下；在使用PXE Boot的時候，會遇到 kernel nfsmount 掛載參數失敗的問題。</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43f98df1f8a5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[Linux] Ubuntu 20.04 install r8125 driver]]></title>
            <link>https://laudaihe.medium.com/linux-ubuntu-20-04-install-r8125-driver-3a999676d6f7?source=rss-29a4bce64558------2</link>
            <guid isPermaLink="false">https://medium.com/p/3a999676d6f7</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[r8125]]></category>
            <category><![CDATA[realtek]]></category>
            <dc:creator><![CDATA[Lau Dai He]]></dc:creator>
            <pubDate>Thu, 03 Mar 2022 15:41:44 GMT</pubDate>
            <atom:updated>2022-03-03T15:46:23.639Z</atom:updated>
            <content:encoded><![CDATA[<h4>在 Ubuntu 20.04 安裝 r8125 驅動紀錄</h4><blockquote>Author：Lau Dai</blockquote><blockquote>Date：2022/03/03</blockquote><h3>Requirements</h3><p>make需要的binutils可透過指令下載</p><pre>apt install build-essential</pre><h3>安裝紀錄</h3><h4>Step 1.</h4><p>Download the file from website:</p><p><a href="https://www.realtek.com/en/component/zoo/category/network-interface-controllers-10-100-1000m-gigabit-ethernet-pci-express-software">Realtek PCIe FE / GBE / 2.5G / Gaming Ethernet Family Controller Software</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2bhO5O-CE8PwbBcKfjJd_Q.png" /></figure><p>雖然上面寫支援kernel到5.6，但我在kernel 5.11的Ubuntu上還是可以使用</p><h4>Step 2.</h4><p>後續跟著README即可完成</p><p>※注意事項：在Ubuntu中，會 update-initramfs，因此需要確保你 /var/tmp空間是否足夠。</p><pre>#解壓縮你的檔案<br>tar jxvf r8125-9.007.01.tar.bz2</pre><pre>#進入資料夾<br>cd r8125-9.007.01</pre><pre>#使用root或具有sudo權限使用者進行他們的腳本安裝<br>./autorun.sh</pre><p>安裝後若原本使用內建的r8169 driver，會改成使用r8125 driver進行有線網卡的使用。可透過下列指令檢查是否安裝成功</p><pre>lsmod | grep r8125<br># 或者<br>ethtool -i {your ethernet device name}</pre><h3>解析 autorun.sh</h3><p>第7~13行，尋找當前kernel版本realtek module安裝的位置，若無則設定成：</p><pre>/lib/modules/$(uname -r)/kernel/drivers/net</pre><p>第15~26行，移除舊版本已載入的module r8169, r8125</p><p>第31行，將make過程紀錄至log.txt</p><p>第32行，將 module name，移除前綴 src/，移除後綴.ko後，放入module變數內。</p><p>第36~55行，檢查module是否正常，並且檢查原本目標路徑是否有r8169.ko檔案，如果有就依照不同方式重新命名。</p><p>第57~60行，重新產生當前kernel的 .dep map檔案，並且試著載入此模組</p><p>第62~83行，檢查 distribution 是否為 ubuntu, debian，透過檢查/etc/debian_version或/etc/lsb-release是否為readable file，若是上述兩者distribution，則會更新 initramfs</p><p>官方原始檔案：</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3161193af1ba9f90b5f1f9bda3271152/href">https://medium.com/media/3161193af1ba9f90b5f1f9bda3271152/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/35227188a51561e5f5405f0f0fe6eff9/href">https://medium.com/media/35227188a51561e5f5405f0f0fe6eff9/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3a999676d6f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>