<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lorenzo Felletti on Medium]]></title>
        <description><![CDATA[Stories by Lorenzo Felletti on Medium]]></description>
        <link>https://medium.com/@lorenzofelletti?source=rss-94dc0e4baf1a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*qITxbu3rhAAKJk5ekj_LeA.jpeg</url>
            <title>Stories by Lorenzo Felletti on Medium</title>
            <link>https://medium.com/@lorenzofelletti?source=rss-94dc0e4baf1a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 17:22:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@lorenzofelletti/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Improving Kubernetes-Mixin API Server Rules Consistency]]></title>
            <link>https://medium.com/codex/improving-kubernetes-mixin-api-server-rules-consistency-1c0d727e8160?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c0d727e8160</guid>
            <category><![CDATA[mixin]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[prometheus]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Sat, 16 Nov 2024 13:36:47 GMT</pubDate>
            <atom:updated>2024-11-16T13:36:47.368Z</atom:updated>
            <content:encoded><![CDATA[<h4>A journey into troubleshooting an insidious, and subtle, issue that may occur with Prometheus Recording Rules</h4><h3>Preamble</h3><p>A very interesting and widely used project in the Kubernetes space is <a href="https://github.com/kubernetes-monitoring/kubernetes-mixin/">kubernetes-mixin</a>. The project, written in Jsonnet, provides a well-tested and customisable set of Prometheus rules, alerts, and Grafana dashboards.</p><p>An important set of rules are the ones regarding the Kubernetes API Server, providing valuable insights, such as the server’s availability, error budget, workqueues length and more.</p><p>I recently made a small contribution to that project, solving an issue I had been seeing for a while. Solving the issue forced me to deep-dive into the Prometheus rules details, and I am writing this to share what I learned.</p><h3>The Problem</h3><p>What I observed happening in some very large clusters, was that the 30-day Availability and error budget from time to time went (slightly) above 100%.</p><blockquote><strong>GitHub Issue — </strong><a href="https://github.com/kubernetes-monitoring/kubernetes-mixin/issues/975">https://github.com/kubernetes-monitoring/kubernetes-mixin/issues/975</a></blockquote><h3>Root Cause Analysis</h3><p>The problem happened <em>sporadically</em>, especially on large and heavily used clusters.</p><p>The kubernetes-mixin splits the API server availability into three views</p><ul><li>Write availability (verb=”write”), i.e. requests with the verb label matching selector verb=~&quot;POST|PUT|PATCH|DELETE&quot;</li><li>Read availability (verb=”read”), i.e. requests with the verb label matching selector verb=~&quot;LIST|GET&quot;</li><li>Overall (verb=”all”) availability.</li></ul><h4>Maths Come to Rescue</h4><p>The problem affected all three views of the availability, but in the following I will refer only to the write one, for the sake of simplicity, since it has the easiest expression of the three to analyse.</p><p>To understand why the 30d availability was going above 100%, what I did was starting from looking at the PromQL expression, and analyse it.</p><p>The following is the expression (in <em>Jsonnet</em>) for the write 30-day availability</p><pre>1 - (<br>  (<br>    # too slow<br>    sum by (cluster) (cluster_verb_scope:apiserver_request_sli_duration_seconds_count:increase30d{verb=&quot;write&quot;})<br>    -<br>    sum by (cluster) (cluster_verb_scope_le:apiserver_request_sli_duration_seconds_bucket:increase30d{verb=&quot;write&quot;,le=&quot;1&quot;})<br>  )<br>  +<br>  # errors<br>  sum by (cluster) (code:apiserver_request_total:increase%(SLODays)s{verb=&quot;write&quot;,code=~&quot;5..&quot;} or vector(0))<br>)<br>/<br>sum by (cluster) (code:apiserver_request_total:increase%(SLODays)s{verb=&quot;write&quot;})</pre><p>The expression terms can be refactored as follows</p><pre>1 - ((all_writes - fast_enough_writes) + write_errors) / write_requests_total<br>=<br>1 - (slow_writes+write_errors) / write_requests_total<br>=<br>1 - unavailable_writes_percent</pre><p>Refactored terms are simpler to analyse</p><ul><li>To go above 100%, the unavailable_writes_percent term needs to be negative (1 - a is greater than 1 – i.e. &gt;100% – if a is negative)</li><li>unavailable_writes_percent is calculated as a fraction – ((all_writes - fast_enough_writes) + write_errors) / write_requests_total</li><li>Since all terms that concur to the calculation are individually positive (all_writes, fast_enough_writes, write_errors, write_requests_total), and the <em>denominator is positive</em> (as made of one positive term only)</li><li>Then it must be the <em>numerator</em> that evaluates to <em>negative</em> values from time to time</li><li>This means that fast_enough_writes &gt; all_writes + write_errors</li><li>(all_writes - fast_enough_writes) + write_errors &lt; 0</li><li>all_writes + write_errors &lt; fast_enough_writes.</li></ul><p>Usually, write_errors is a very small amount, especially when compared to the other two terms, so let’s assume that it amounts to zero, which leaves us with fast_enough_writes &gt; all_writes.</p><p>all_writes is calculated as</p><ul><li>cluster_verb_scope:apiserver_request_sli_duration_seconds_count:increase30d{verb=&quot;write&quot;}, from now on, I will refer to this as the “<em>30d count</em>” metric, or just the “<em>count</em>”.</li></ul><p>fast_enough_writes is on the other hand calculated from this other recording rule:</p><ul><li>cluster_verb_scope_le:apiserver_request_sli_duration_seconds_bucket:increase30d{verb=&quot;write&quot;, le=&quot;1&quot;}, from now on, I will refer to this as the “<em>30d bucket</em>” metric, or just the “<em>bucket</em>”.</li></ul><blockquote>The term fast_enough_writes filters the bucket metric to get the number of write operations that took less than one second (1s) to complete. When the system is healthy, we can expect this number to be close to the overall number of write requests (i.e. the bucket infinity, le=&quot;+Inf&quot;).</blockquote><h4>Seeking Confirmations</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k2ZX6mfM92mFyox8DFrYPw.png" /><figcaption>Plot of bucket infinity vs count.</figcaption></figure><p>To confirm the hypothesis, I plotted for one of the clusters showing this behaviour cluster_verb_scope_le:apiserver_request_sli_duration_seconds_bucket:increase1h{cluster=&quot;mycluster&quot;, le=&quot;+Inf&quot;, verb=&quot;GET&quot;, scope=&quot;resource&quot;} (the green line in the image) vs cluster_verb_scope:apiserver_request_sli_duration_seconds_count:increase1h{cluster=&quot;mycluster&quot;, verb=&quot;GET&quot;, scope=&quot;resource&quot;} (yellow line), and it is easy to notice from the image that the yellow line (count) is always below or equal the green one (bucket infinity). That was true for all the verbs I’ve tested.</p><h4>The Cause / Digging Into Prometheus Internals</h4><p><strong>Recording Rules Are Not Metrics</strong></p><p>The count and bucket metrics look like they are part of a histogram metric in Prometheus. In Prometheus, we’re guaranteed that in a histogram metric, the count and the bucket infinity (le=”+Inf”) amount to the same value, but for some reason this doesn’t seem to be true in our case.</p><p>The reason is the Kubernetes API server does not expose any 30d metric out-of the box, those are <em>recording rules</em>. As a matter of facts, these two metrics, the 30 count and bucket, are calculated as a recording rule (which in turn uses another pair of recording rules for the 1h count and bucket).</p><p><strong>Prometheus</strong></p><p>By digging a little bit into how Prometheus evaluates recording rules, we discover that recording rules in the same <em>rule group</em> are guaranteed to have the same timestamp, and that they are executed in order, but they are <em>not guaranteed to be executed on the same data</em>.</p><blockquote><strong>Key</strong> — Recording rules in a group are not guaranteed to be executed on the same data.</blockquote><p>By quickly checking how the 30d (and 1h) rules are calculated, I’ve found that they are indeed part of the same rule group, thus they share the same timestamp, but because of what stated above they’re not guaranteed to be executed on the same data (<a href="https://github.com/prometheus/prometheus/issues/12275">reference issue</a>) which seem to be the issue here.</p><p>Interesting is, that in the way they are organised, the bucket metrics (both 1h and 30d) appear after the respective count metrics. This would align with the behaviour observed above: let’s assume that, between the evaluation of the count and bucket recording rules, new data arrives (increasing the total number of observed events). In such a case, the bucket metric is calculated on “<em>fresher</em>” — more — data than the count is. Therefore, the count rule would evaluate to a lower number than the infinity bucket rule (count &lt; bucket{le=&quot;+Inf&quot;}) in this case, which is precisely what was observed.</p><h3>Solution</h3><blockquote><strong>Merged Pull Request — </strong><a href="https://github.com/kubernetes-monitoring/kubernetes-mixin/pull/976">https://github.com/kubernetes-monitoring/kubernetes-mixin/pull/976</a>.</blockquote><p>I’ve though of different possible solutions to the issue, such as</p><ul><li>Introducing an offset to ensure calculation at times t were executed on data from max t - offset, by sizing the offset wisely it would be possible to ensure this issue does not appear, but also your results would be skewed by that offset</li><li>Capping the value at 100%, by accepting the loss in accuracy</li><li>Reordering rules to calculate the bucket first, and the count as the value of the infinity bucket just calculated.</li></ul><p>This last solution was the one with fewer drawbacks, and that is implemented in the PR that was merged. Basically, what it does is reordering the rules in the rule group as such</p><ol><li>Calculate the 1h bucket using its current formula, as is</li><li>Calculate the 1h count by changing the expression to count1h = bucket1h{le=&quot;+Inf&quot;}</li><li>Calculate the 30d bucket with the same formula</li><li>Calculate the 30d count with the expression count30d = bucket30d{le=&quot;+Inf&quot;}.</li></ol><p>This ordering allows ensuring the count is coherent with the bucket, because that is enforced by the expression and rule evaluation order, without trading this off with skewing the data or else.</p><h3>Conclusion</h3><p>The takeaway from this is that when dealing with Prometheus rules, and using them to calculate new aggregation we want to expose as histograms, mimicking the native metrics behaviour, we should take into account that recording rules are not like metrics, and that although unlikely, at scale those differences might cause unexpected behaviour.</p><p>Taking preventive steps, like ensuring in the rules’ expressions assumptions like the count being equal to the bucket infinity, might help us mitigate such issues from occurring.</p><p>Moreover, Prometheus is not a perfect system, and does not aspire to be, so when looking at dashboards based on that, we should never blindly trust the data we see, as (especially with aggregations) small errors may snowball and become big quite fast.</p><p>To conclude, I hope you liked this article, and see you next time!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c0d727e8160" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codex/improving-kubernetes-mixin-api-server-rules-consistency-1c0d727e8160">Improving Kubernetes-Mixin API Server Rules Consistency</a> was originally published in <a href="https://medium.com/codex">CodeX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chatting With ChatGPT On a Sony PSP]]></title>
            <link>https://medium.com/codex/chatting-with-gpt-on-a-sony-psp-939face79c62?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/939face79c62</guid>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[game-development]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Sat, 30 Mar 2024 18:05:19 GMT</pubDate>
            <atom:updated>2024-04-12T08:15:04.868Z</atom:updated>
            <content:encoded><![CDATA[<h4>Say hello to Chat-GPSP, a Rust application for PSP — Sony’s first handheld released in 2005 — to chat with ChatGPT.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*psMLvIwGU1ymv112" /><figcaption>Photo by <a href="https://unsplash.com/@cortes?utm_source=medium&amp;utm_medium=referral">Dennis Cortés</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Foreword</h3><p>Much like those Guinness World Records one wins just because they were the first to attempt such a weird thing, I wanted to be the first to create an application capable of making you chat with the arguably most disruptive technology in years — OpenAI’s GPT — from a Sony PlayStation Portable, my most beloved handheld console to date.</p><h4>Rust on PSP</h4><p>Rust on PSP is possible thanks to the rust-psp crate [5], a wonderful library you can install by simply running cargo install cargo-psp, setting rust version to nightly (rustup override set nightly, in the root of your repository), and adding psp = { version = &quot;0.3.7&quot; } to your <em>Cargo.toml</em>.</p><p>The crate is an SDK for PSP with no dependency on the famous <strong><em>PSPSDK</em></strong> [6] in C++, although the function signatures are almost always the same, making it easier to convert code between the two.</p><p>The SDK is relatively easy to use, and has support for core and alloc, although there is no support for the standard library. The lack of support for std limits you to use only crates that supports no_std, which, fortunately, are not few since it is a widespread constraint in embedded systems, but is still one of the main limitations of the crate.</p><h4>Project Structure</h4><p>To achieve my goal, I split the problem in the following sub-problems</p><ul><li>Reading <em>input</em></li><li>Setting up <em>networking</em> and DNS</li><li>Interacting With OpenAI <em>APIs</em>.</li></ul><p>The GUI leave much to be desired, and is just terminal-like, but it was outside the scope of the project.</p><h3>Reading from OSK — On-Screen Keyboard</h3><p>In the PSP development world, the virtual keyboard is known as the OSK, which stands for On-Screen Keyboard.</p><p>Setting up a working OSK was one of the tasks that took me the longest time, even though it was possible to find some examples on the internet. The main problem, was that I used, for speed of development, the PPSSPP emulator, and only from time to time the actual device. The two devices behaves slightly differently, and that caused me many headaches, and I had to rework the code many times.</p><p>Moreover, rust-psp OSK state management enum was not quite correct (it contained a variant too much), which the code had to account for.<br>I created a PR fixing this that was merged, so versions newer than 0.3.7 should not have the problem.</p><p>Coming to the code, I created a separate module for the OSK, which implements a struct OskState that wraps around SceUtilityOskState — this is done to help cast an i32 to a SceUtilityOskState. Other than that, and some helper functions, the read_from_osk function is the core function to be used to read a string from the OSK.</p><pre>pub fn read_from_osk(params: &amp;mut SceUtilityOskParams) -&gt; Option&lt;String&gt; {<br>    let mut done = false;<br>    let mut osk_state = OskState::new();<br><br>    unsafe {<br>        sceDisplayWaitVblankStart();<br>        sceGuSwapBuffers();<br>        while !done {<br>            sys::sceGuStart(<br>                sys::GuContextType::Direct,<br>                &amp;mut LIST as *mut _ as *mut c_void,<br>            );<br>            sys::sceGuClear(ClearBuffer::COLOR_BUFFER_BIT | ClearBuffer::DEPTH_BUFFER_BIT);<br>            sys::sceGuFinish();<br>            sys::sceGuSync(GuSyncMode::Finish, sys::GuSyncBehavior::Wait);<br>            sceGuClear(ClearBuffer::COLOR_BUFFER_BIT);<br>            <br>            match osk_state.get() {<br>                // TODO: switch to PspUtilityDialogState when it&#39;s implemented<br>                SceUtilityOskState::None =&gt; done = true,<br>                // This is the visible state, but the 0.3.7 enum has a variant too much<br>                SceUtilityOskState::Initialized =&gt; {<br>                    if sceUtilityOskUpdate(1).is_negative() {<br>                        panic!(&quot;cannot update osk&quot;);<br>                    }<br>                }<br>                // same as above, it is the Quit state<br>                SceUtilityOskState::Visible =&gt; {<br>                    if sceUtilityOskShutdownStart().is_negative() {<br>                        panic!(&quot;cannot shutdown osk&quot;);<br>                    }<br>                }<br>                _ =&gt; (),<br>            }<br>            sceDisplayWaitVblankStart();<br>            sceGuSwapBuffers();<br>        }<br>    }<br><br>    // Convert the read text to a string, if any<br>    let osk_data: &amp;SceUtilityOskData = unsafe { params.data.as_ref().unwrap() };<br>    match osk_data.result {<br>        sys::SceUtilityOskResult::Cancelled =&gt; None,<br>        _ =&gt; {<br>            let out_text = unsafe {<br>                mut_ptr_u16_to_vec_char(osk_data.outtext, osk_data.outtextlength as usize)<br>            };<br>            let out_text = String::from_iter(out_text);<br>            Some(out_text)<br>        }<br>    }<br>}</pre><p>Code for the OSK was inspired by <a href="https://github.com/CrossCraft/CrossCraft-Classic/blob/ce932e73468c2868932c3136dfd590473b57a474/src/Player/PlayerInput.cpp#L629">CrossCraft</a> [1].</p><h3>Networking, TLS, and DNS</h3><p>PSP sockets closely resembles Linux ones, thus implementing a UDP and TCP socket abstraction was not too difficult (implementation can be found in the repo).</p><h4>TLS</h4><p>One of the major problems with the PSP is that it does not come with TLS support, so I had to find a Rust-only implementation of it, and luckily that was offered by the embedded-tls crate. In order to make the sockets compatible with it, I had to implement the traits embedded-io::Read and embedded-io::Write from the crate embedded-io. Finally, I used another great crate, embedded-nal, to have a handy way to manage socket addresses and IPs representations.</p><p>Using embedded-tls I was able to make TLS work on the PSP. For the implementation, I started from <a href="https://github.com/sajattack/rust-psp/blob/tls/examples/tls-test/src/main.rs">here</a> [2], and then adapted the implementation to use more up-to-date crates.</p><pre>lazy_static::lazy_static! {<br>    static ref REGEX: Regex = Regex::new(&quot;\r|\0&quot;).unwrap();<br>}<br><br>pub struct TlsSocket&lt;&#39;a&gt; {<br>    tls_connection: TlsConnection&lt;&#39;a, TcpSocket, Aes128GcmSha256&gt;,<br>    tls_config: TlsConfig&lt;&#39;a, Aes128GcmSha256&gt;,<br>}<br><br>impl&lt;&#39;a&gt; TlsSocket&lt;&#39;a&gt; {<br>    pub fn new(<br>        socket: TcpSocket,<br>        record_read_buf: &amp;&#39;a mut [u8],<br>        record_write_buf: &amp;&#39;a mut [u8],<br>        server_name: &amp;&#39;a str,<br>        cert: Option&lt;&amp;&#39;a [u8]&gt;,<br>    ) -&gt; Self {<br>        let tls_config: TlsConfig&lt;&#39;_, Aes128GcmSha256&gt; = match cert {<br>            Some(cert) =&gt; TlsConfig::new()<br>                .with_server_name(server_name)<br>                .with_cert(Certificate::RawPublicKey(cert))<br>                .enable_rsa_signatures(),<br>            None =&gt; TlsConfig::new()<br>                .with_server_name(server_name)<br>                .enable_rsa_signatures(),<br>        };<br><br>        let tls_connection: TlsConnection&lt;TcpSocket, Aes128GcmSha256&gt; =<br>            TlsConnection::new(socket, record_read_buf, record_write_buf);<br><br>        TlsSocket {<br>            tls_connection,<br>            tls_config,<br>        }<br>    }<br><br>    pub fn open(&amp;mut self, seed: u64) -&gt; Result&lt;(), embedded_tls::TlsError&gt; {<br>        let mut rng = ChaCha20Rng::seed_from_u64(seed);<br>        let tls_context = TlsContext::new(&amp;self.tls_config, &amp;mut rng);<br>        self.tls_connection<br>            .open::&lt;ChaCha20Rng, NoVerify&gt;(tls_context)<br>    }<br><br>    pub fn write(&amp;mut self, buf: &amp;[u8]) -&gt; Result&lt;usize, embedded_tls::TlsError&gt; {<br>        self.tls_connection.write(buf)<br>    }<br><br>    pub fn write_all(&amp;mut self, buf: &amp;[u8]) -&gt; Result&lt;(), embedded_tls::TlsError&gt; {<br>        self.tls_connection.write_all(buf)<br>    }<br><br>    pub fn read(&amp;mut self, buf: &amp;mut [u8]) -&gt; Result&lt;usize, embedded_tls::TlsError&gt; {<br>        self.tls_connection.read(buf)<br>    }<br><br>    pub fn read_string(&amp;mut self) -&gt; Result&lt;String, embedded_tls::TlsError&gt; {<br>        let mut buf = Self::new_buffer();<br>        let _ = self.read(&amp;mut buf)?;<br><br>        let text = String::from_utf8_lossy(&amp;buf);<br>        let text = REGEX.replace_all(&amp;text, &quot;&quot;);<br>        Ok(text.into_owned())<br>    }<br><br>    pub fn flush(&amp;mut self) -&gt; Result&lt;(), embedded_tls::TlsError&gt; {<br>        self.tls_connection.flush()<br>    }<br>}</pre><h4>DNS</h4><p>Although the PSP has native DNS, I was not able to make it work on my PSP-3000 (it can as well be that I was using it wrong), thus I had to find a no_std crate I could use to implement my own DNS. Luckily, that was not difficult to find, the dns-protocol crate offers all you need just for this purpose.</p><p>I created a DnsResolver trait (well, to be fair that was implemented in a following iteration, but for storytelling purposes it’s not relevant) as the composition of two other traits ResolveHostname, and ResolveAddr. Types implementing ResolveHostname can resolve a hostname to an IP, types implementing ResolveAddr the opposite. Types implementing both, can be “complete” DNS Resolvers.</p><pre>pub trait ResolveHostname {<br>    type Error: Debug;<br>    fn resolve_hostname(&amp;mut self, hostname: &amp;str) -&gt; Result&lt;SocketAddr, Self::Error&gt;;<br>}<br><br>pub trait ResolveAddr {<br>    type Error: Debug;<br>    fn resolve_addr(&amp;mut self, addr: in_addr) -&gt; Result&lt;String, Self::Error&gt;;<br>}<br><br>pub trait DnsResolver: ResolveHostname + ResolveAddr {}</pre><p>Then I created a type implementing those traits (well, the ResolveAddr is still a todo!() TBH, but it is not needed for this application, I might implement it later on).</p><pre>/// Create a DNS query for an A record<br>pub fn create_a_type_query(domain: &amp;str) -&gt; Question {<br>    Question::new(domain, dns_protocol::ResourceType::A, 1)<br>}<br><br>/// A DNS resolver<br>pub struct DnsResolver {<br>    udp_socket: UdpSocket,<br>    dns: SocketAddr,<br>}<br><br>impl DnsResolver {<br>    /// Create a new DNS resolver<br>    pub fn new(dns: SocketAddr) -&gt; Result&lt;Self, ()&gt; {<br>        let mut udp_socket = UdpSocket::open().map_err(|_| ())?;<br>        udp_socket.bind(Some(dns)).map_err(|_| ())?;<br><br>        Ok(DnsResolver { udp_socket, dns })<br>    }<br><br>    /// Create a new DNS resolver with default settings.<br>    /// The default settings are to use Google&#39;s DNS server at `8.8.8.8:53`<br>    pub fn default() -&gt; Result&lt;Self, ()&gt; {<br>        let dns = GOOGLE_DNS_HOST.clone();<br>        let mut udp_socket = UdpSocket::open().map_err(|_| ())?;<br>        udp_socket.bind(None).map_err(|_| ())?;<br><br>        Ok(DnsResolver { udp_socket, dns })<br>    }<br><br>    pub fn resolve(&amp;mut self, host: &amp;str) -&gt; Result&lt;in_addr, ()&gt; {<br>        // connect to the DNS server, if not already<br>        if self.udp_socket.get_socket_state() != UdpSocketState::Connected {<br>            self.udp_socket.connect(self.dns).map_err(|_| ())?;<br>        }<br><br>        // create a new query<br>        let mut questions = [super::dns::create_a_type_query(host)];<br>        let query = dns_protocol::Message::new(<br>            0x42,<br>            Flags::standard_query(),<br>            &amp;mut questions,<br>            &amp;mut [],<br>            &amp;mut [],<br>            &amp;mut [],<br>        );<br><br>        // create a new buffer with the size of the message<br>        let mut tx_buf = a_vec![0u8; query.space_needed()];<br>        // serialize the message into the buffer<br>        query.write(&amp;mut tx_buf).map_err(|_| ())?;<br><br>        // send the message to the DNS server<br>        let _ = self.udp_socket.write(&amp;tx_buf).map_err(|_| ())?;<br><br>        let mut rx_buf = [0u8; 1024];<br><br>        // receive the response from the DNS server<br>        let data_len = self.udp_socket.read(&amp;mut rx_buf).map_err(|_| ())?;<br><br>        if data_len == 0 {<br>            return Err(());<br>        }<br><br>        // parse the response<br>        let mut answers = [ResourceRecord::default(); 16];<br>        let mut authority = [ResourceRecord::default(); 16];<br>        let mut additional = [ResourceRecord::default(); 16];<br>        let message = dns_protocol::Message::read(<br>            &amp;rx_buf[..data_len],<br>            &amp;mut questions,<br>            &amp;mut answers,<br>            &amp;mut authority,<br>            &amp;mut additional,<br>        )<br>        .map_err(|_| ())?;<br><br>        if message.answers().is_empty() {<br>            return Err(());<br>        }<br>        let answer = message.answers()[0];<br><br>        match answer.data().len() {<br>            4 =&gt; {<br>                let mut octets = [0u8; 4];<br>                octets.copy_from_slice(answer.data());<br>                let addr = in_addr(u32::from_be_bytes(octets));<br>                Ok(addr)<br>            }<br>            _ =&gt; Err(()),<br>        }<br>    }<br>}<br><br>impl traits::dns::ResolveHostname for DnsResolver {<br>    type Error = ();<br><br>    fn resolve_hostname(&amp;mut self, hostname: &amp;str) -&gt; Result&lt;SocketAddr, ()&gt; {<br>        self.resolve(hostname).map(|addr| addr.to_socket_addr())<br>    }<br>}<br><br>impl traits::dns::ResolveAddr for DnsResolver {<br>    type Error = ();<br><br>    fn resolve_addr(&amp;mut self, _addr: in_addr) -&gt; Result&lt;String, ()&gt; {<br>        todo!(&quot;resolve_addr&quot;)<br>    }<br>}<br><br>impl traits::dns::DnsResolver for DnsResolver {}</pre><h3>Chatting With GPT</h3><p>Now that we have implemented a way to prompt the user for input and read it, and have a way to send messages on the network, as well as a way to resolve a hostname to an IP address, all we’re missing is a way to interact with OpenAI APIs, then we will be able to put all the pieces together.</p><h4>Serde</h4><p>OpenAI APIs require the client to keep track of the conversation history, so I implemented a couple types to take care of that</p><pre>#[derive(Debug, Clone, PartialEq)]<br>pub struct Message {<br>    role: String,<br>    pub content: String,<br>}<br><br>impl Message {<br>    pub fn new_user(content: String) -&gt; Self {<br>        Self {<br>            role: &quot;user&quot;.to_owned(),<br>            content,<br>        }<br>    }<br>    pub fn new_assistant(content: String) -&gt; Self {<br>        Self {<br>            role: &quot;assistant&quot;.to_owned(),<br>            content,<br>        }<br>    }<br>}<br><br>impl Display for Message {<br>    // ...<br>}<br><br>impl ChatHistory {<br>    pub fn new(model: String, temperature: f32) -&gt; Self {<br>        Self {<br>            model,<br>            messages: Vec::new(),<br>            temperature,<br>        }<br>    }<br><br>    #[inline]<br>    pub fn new_gpt3(temperature: f32) -&gt; Self {<br>        Self::new(GPT3_MODEL.to_owned(), temperature)<br>    }<br><br>    pub fn clear(&amp;mut self) {<br>        self.messages.clear();<br>    }<br><br>    pub fn add_user_message(&amp;mut self, content: String) {<br>        self.messages.push(Message::new_user(content));<br>    }<br><br>    pub fn add_assistant_message(&amp;mut self, content: String) {<br>        self.messages.push(Message::new_assistant(content));<br>    }<br><br>    pub fn to_string_with_content_length(&amp;self) -&gt; (String, usize) {<br>        let string = self.to_string();<br>        let len = string.len();<br>        (string, len)<br>    }<br>}<br><br>impl Display for ChatHistory {<br>    // ...<br>}</pre><p>Now that we have a way to store our chat (Display implementation is used to serialize the content to valid JSON, so that we can send it in our request), we need a way to parse OpenAI’s responses. Since we don’t have the std crate on our side, we need to use serde with default-features off, and serde-json-core. Also, we need heapless crate for String values deserialisation (because we need to set a max string length a priori).</p><pre>#[derive(Debug, Deserialize)]<br>pub struct Usage {<br>    pub prompt_tokens: i32,<br>    pub completion_tokens: i32,<br>    pub total_tokens: i32,<br>}<br><br>#[derive(Debug, Deserialize)]<br>pub struct ResponseMessage {<br>    pub role: heapless::String&lt;32&gt;,<br>    pub content: heapless::String&lt;1024&gt;,<br>}<br><br>#[derive(Debug, Deserialize)]<br>/// # Note<br>/// This struct does not support the [`Self::logprobs`] field different from `None` yet.<br>pub struct CompletionChoice {<br>    pub message: ResponseMessage,<br>    pub index: i64,<br>    pub finish_reason: heapless::String&lt;32&gt;,<br>    pub logprobs: Option&lt;LogprobResult&gt;,<br>}<br><br>#[derive(Debug, Deserialize)]<br>/// This is a dummy struct, it&#39;s not actually used.<br>pub struct LogprobResult {}<br><br>#[derive(Debug, Deserialize)]<br>/// Completion response from OpenAI.<br>pub struct CompletionResponse&lt;&#39;a&gt; {<br>    pub id: &amp;&#39;a str,<br>    pub object: &amp;&#39;a str,<br>    pub created: i64,<br>    pub model: &amp;&#39;a str,<br>    pub choices: heapless::Vec&lt;CompletionChoice, 3&gt;,<br>    pub usage: Usage,<br>}</pre><p>A lot could be improved on the (de)serialisation side, but this working well enough for the project.</p><h4>OpenAI Interaction</h4><p>To interact with OpenAI APIs, I created two structs, OpenAiContext and OpenAi, the former is used to set up the context in which the latter — the one actually interacting with the APIs — operates.</p><p>Skipping many of the details of the code that you can find in the repo, the main method of the OpenAi struct is ask_gpt which takes a prompt to ask to GPT, adds it to the current history, and send the request to the APIs, parsing the response.</p><p>One of the problems I encountered along the way was that I would often find the connection closed whenever it took me a few seconds too long to type the prompt (which, writing on PSP’s OSK, is not unlikely to happen for messages longer than “hello”). To work around this, I took the decision to open a new connection for each call to ask_gpt, which is probably not a good decision from an engineering POV, but gave me the possibility to keep the use of the struct very simple (the caller do not need to worry about managing the connection).</p><p>Another problem, encountered quite often, was that the first time I called read on the socket, only a partial response was read. To solve this, I implemented a retry mechanism which attempt to read for MAX_RETRIES times (3 by default) a complete response before giving up. I still don’t know what is causing this problem, if it is my code, the crate, the APIs, or else.</p><pre>    pub fn ask_gpt(&amp;mut self, prompt: &amp;str) -&gt; Result&lt;String, OpenAiError&gt; {<br>        self.history.add_user_message(prompt.to_owned());<br><br>        let (request_body, content_length) = self.history.to_string_with_content_length();<br><br>        let mut read_buf = Self::create_new_buf();<br>        let mut write_buf = Self::create_new_buf();<br><br>        let mut tls_socket = Self::open_tls_socket(&amp;mut read_buf, &amp;mut write_buf, self.remote)?;<br><br>        let request = format!(<br>            &quot;POST {} HTTP/1.1\nHost: {}\nAuthorization: Bearer {}\nContent-Type: application/json\nContent-Length: {}\nUser-Agent: Sony PSP\n\n{}\n&quot;,<br>            POST_PATH,<br>            OPENAI_API_HOST,<br>            self.api_key,<br>            content_length,<br>            request_body,<br>        );<br>        let request_bytes = request.as_bytes();<br><br>        let mut response_string = String::new();<br><br>        for _ in 1..=MAX_RETRIES {<br>            tls_socket<br>                .write_all(request_bytes)<br>                .map_err(|e| log_error(&amp;e))?;<br>            tls_socket.flush().map_err(|e| log_error(&amp;e))?;<br><br>            let response_buf = &amp;mut [0u8; 16_384];<br>            tls_socket.read(response_buf).map_err(|e| log_error(&amp;e))?;<br><br>            let text = String::from_utf8_lossy(response_buf)<br>                .to_string().replace([&#39;\r&#39;, &#39;\0&#39;], &quot;&quot;);<br><br>            response_string += &amp;text;<br><br>            let res_code = response_string<br>                .split(&#39;\n&#39;)<br>                .next()<br>                .ok_or(OpenAiError::UnparsableResponseCode)?<br>                .split(&#39; &#39;)<br>                .nth(1)<br>                .ok_or(OpenAiError::UnparsableResponseCode)?;<br>            if res_code != &quot;200&quot; {<br>                return Err(OpenAiError::ResponseCodeNotOk);<br>            }<br><br>            if response_string.ends_with(&quot;}\n&quot;) {<br>                break;<br>            }<br>        }<br><br>        // find for double newline, get the body<br>        let res_body =<br>            response_string<br>                .split(&quot;\n\n&quot;)<br>                .nth(1)<br>                .ok_or(OpenAiError::UnparsableResponseBody(<br>                    &quot;Body not found&quot;.to_owned(),<br>                ))?;<br><br>        let completion_response: CompletionResponse = serde_json_core::from_str(res_body)<br>            .map_err(|e| OpenAiError::UnparsableResponseBody(e.to_string()))?<br>            .0;<br><br>        let assistant_message = completion_response.choices[0].message.content.trim();<br>        self.history<br>            .add_assistant_message(assistant_message.to_owned());<br><br>        Ok(assistant_message.to_owned())<br>    }</pre><p>The checks if the response is complete or not are “empirical” and not very robust, but they seem to work fine in my (limited) testing.</p><h3>Putting All Together</h3><p>Finally, we have all the pieces, and we just need to put them together in the main, and run cargo psp --release.</p><blockquote>For performance reasons, I was advised to always build in release mode.</blockquote><p>To run it yourself, you have to create an OpenAI API key, and export it as an environment variable in the terminal you use to build the program (if you use VS Code as an IDE, I advise you to have it exported already in the terminal when you run it with code .).</p><pre>export OPENAI_API_KEY=my-api-key</pre><p>In the main, which in rust-psp is called psp_main (and you need to add the #![no_main] directive at the top of your <em>main.rs</em>) I need some boilerplate to set up the environment, and then I run a loop of asking the user to write a prompt, asking GPT, printing the results.</p><pre>const OPENAI_API_KEY: &amp;str = core::env!(&quot;OPENAI_API_KEY&quot;);<br><br>fn psp_main() {<br>    psp::enable_home_button();<br><br>    unsafe {<br>        // setup network<br>        net::utils::load_net_modules();<br>        psp::dprintln!(&quot;Initializing network...&quot;);<br>        net::utils::net_init();<br><br>        psp::sys::sceNetApctlConnect(1);<br>        loop {<br>            let mut state: psp::sys::ApctlState = core::mem::zeroed();<br>            psp::sys::sceNetApctlGetState(&amp;mut state);<br>            if let psp::sys::ApctlState::GotIp = state {<br>                break;<br>            }<br>            psp::sys::sceKernelDelayThread(50_000);<br>        }<br><br>        // setup controls<br>        psp::sys::sceCtrlSetSamplingCycle(0);<br>        psp::sys::sceCtrlSetSamplingMode(psp::sys::CtrlMode::Analog);<br>    }<br><br>    psp::dprintln!(&quot;Connected to network!&quot;);<br><br>    let mut resolver = DnsResolver::default().expect(&quot;failed to create resolver&quot;);<br><br>    let openai_context =<br>        OpenAiContext::new(&amp;mut resolver, OPENAI_API_KEY).expect(&quot;failed to create openai context&quot;);<br><br>    let mut input_handler = InputHandler::default();<br><br>    psp::dprintln!(&quot;Press X to start asking GPT-3.5, any other button to exit.&quot;);<br>    if !input_handler.choose_continue() {<br>        unsafe {<br>            psp::dprintln!(&quot;Exiting...&quot;);<br>            sceGuTerm();<br>            sceKernelExitGame();<br>        };<br>    }<br><br>    setup_gu();<br><br>    loop {<br>        let read_text = unsafe {<br>            sceKernelDcacheWritebackAll();<br><br>            let mut out_text: Vec&lt;u16&gt; = Vec::with_capacity(CHAT_MAX_LENGTH_USIZE);<br>            let out_capacity: i32 = out_text.capacity() as i32;<br><br>            let description = str_to_u16_mut_ptr(&quot;Ask GPT\0&quot;);<br>            let mut osk_data = default_osk_data(description, out_capacity, out_text.as_mut_ptr());<br><br>            let params = &amp;mut default_osk_params(&amp;mut osk_data);<br><br>            start_osk(params).expect(&quot;failed to start osk&quot;);<br><br>            read_from_osk(params).unwrap_or_default()<br>        };<br>        let read_text = read_text.replace(&#39;\0&#39;, &quot;&quot;);<br><br>        psp::dprintln!(&quot;User: {}\n&quot;, read_text);<br><br>        let mut openai = OpenAi::new(&amp;openai_context).expect(&quot;failed to create openai&quot;);<br><br>        let answer = openai.ask_gpt(read_text.as_str());<br><br>        if answer.is_err() {<br>            psp::dprintln!(&quot;failed to get answer from openai&quot;);<br>            psp::dprintln!(&quot;Got error: {:?}\n&quot;, answer.err().unwrap());<br>        } else {<br>            psp::dprintln!(&quot;GPT: {}\n&quot;, answer.unwrap());<br>        }<br><br>        psp::dprintln!(&quot;Press X to ask again, any other button to exit.&quot;);<br>        if !input_handler.choose_continue() {<br>            break;<br>        }<br>    }<br><br>    unsafe {<br>        sceGuTerm();<br>        sceKernelExitGame();<br>    };<br>}</pre><h3>Testing And Final Words</h3><h4>Testing</h4><p>The PSP cannot connect to the new password-protected Wi-Fi. The simplest way to get it connected is to create a password-free hotspot with your phone, and connect to it.</p><blockquote>Note, the program as is will try to connect to the saved connection in first position. Thus, beware of having it already set up correctly before you run it.</blockquote><p>To test it on real hardware, copy the <em>EBOOT.PBP</em> in the <em>GAME</em> directory of your PSP.</p><p>I was not able to run the program successfully on the popular PPSSPP emulator, the OSK and networking in particular were giving me troubles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3P2tch6d8KIkrWWEQPaVFg.jpeg" /><figcaption>Chat-GPSP in action. The text in the photo looks kinda blurry, but I wasn’t able to capture a better photo.</figcaption></figure><p>As you can see in the picture above, the UI is not very sophisticated, just a simple terminal-like one.</p><p>To see the complete code, you can have a look at the repo <a href="https://github.com/lorenzofelletti/chat_gpsp">here</a> [3].</p><h4>Final Words</h4><p>Completing this project wasn’t easy — partly because while I was working on it, I completed my master’s degree, was working full-time, and relocated to a different country, partly because the documentation on the topic is limited. I was lucky to find and join the <a href="https://discord.gg/tJ8NXFzDfY">PSP Homebrew Community</a> on Discord [4], they were really helpful whenever I needed, and are a great community in general. I would not have made it without their support, so a big thanks goes to them.</p><h3>References</h3><ol><li>CrossCraft OSK example — <a href="https://github.com/CrossCraft/CrossCraft-Classic/blob/ce932e73468c2868932c3136dfd590473b57a474/src/Player/PlayerInput.cpp#L629">https://github.com/CrossCraft/CrossCraft-Classic/blob/ce932e73468c2868932c3136dfd590473b57a474/src/Player/PlayerInput.cpp#L629</a></li><li>TLS example — <a href="https://github.com/sajattack/rust-psp/blob/tls/examples/tls-test/src/main.rs">https://github.com/sajattack/rust-psp/blob/tls/examples/tls-test/src/main.rs</a></li><li>Chat-GPSP repo — <a href="https://github.com/lorenzofelletti/chat_gpsp">https://github.com/lorenzofelletti/chat_gpsp</a></li><li>PSP Homebrew Community Discord — <a href="https://discord.gg/tJ8NXFzDfY">https://discord.gg/tJ8NXFzDfY</a></li><li>Rust PSP — <a href="https://github.com/overdrivenpotato/rust-psp">https://github.com/overdrivenpotato/rust-psp</a></li><li>PSPSDK — <a href="https://pspdev.github.io/pspsdk/index.html">https://pspdev.github.io/pspsdk/index.html</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=939face79c62" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codex/chatting-with-gpt-on-a-sony-psp-939face79c62">Chatting With ChatGPT On a Sony PSP</a> was originally published in <a href="https://medium.com/codex">CodeX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform Console — A Powerful, Often Overlooked Tool]]></title>
            <link>https://lorenzofelletti.medium.com/terraform-console-a-powerful-often-overlooked-tool-562762d687ef?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/562762d687ef</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[cli]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[iac]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Sun, 18 Feb 2024 09:21:30 GMT</pubDate>
            <atom:updated>2024-02-18T09:21:30.094Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-nc/4.0/</cc:license>
            <content:encoded><![CDATA[<h3>Terraform Console — A Powerful, Often Overlooked, Tool</h3><h4>Many ignore that Terraform offers a powerful tool to deeply inspect your configuration, or test ideas you have, out-of-the-box.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-yprKHvbCGH8R1ak" /><figcaption>Photo by <a href="https://unsplash.com/@lobosnico?utm_source=medium&amp;utm_medium=referral">Nicolas Lobos</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Terraform is a powerful IaC tool, that lets you provision and manage your infrastructure in a declarative way. Many people and companies use Terraform on a daily basis to provision their infrastructure.</p><p>However powerful, one of the problems many face when using Terraform, is that sometimes it feels kind of a black box.<br>For example, you write loops, complex variable transformations, come across a cryptic error, and wonder how could you inspect what is going on <em>“under the hood”</em>.<br>Or maybe, you just want to <em>quickly test</em> some expression, <em>inspect</em> a local value to see if it is the same as you expect, check a module’s output.</p><p>Luckily, Terraform offers a powerful command out-of-the-box, named console, you can start by running terraform console.</p><h4>How to Run it</h4><p>Terraform Console can be run either outside of a Terraform project, or inside one.</p><p>If you run it outside a Terraform project, it is sufficient to have Terraform installed and run terraform console.</p><p>If you want to run it inside a project, then you must also ensure that the variables have a value. For example, if you use a file named my.tfvars to valorise your variables, run terraform console -var-file=my.tfvars.</p><p>To exit an open console, just hit Ctrl+C or run exit.</p><h4>What it Can And Cannot Do</h4><p>Terraform console can be used for a multitude of purposes, and it is a powerful tool to have in your toolbox.</p><p><strong>What you can do</strong></p><ul><li>Print the value of a variable or local value (in the main module), just by running var.&lt;MY_VARIABLE&gt; or local.&lt;MY_LOCAL&gt;</li><li>Evaluate any Terraform expression (including using for loops, ternary statements, and functions)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*qSH0AQgvXTIGtWXMRDWH5g.png" /><figcaption>Example of expression run in Terraform console.</figcaption></figure><ul><li>Print a module’s outputs (only if the output was already generated in a previous apply) with module.&lt;MODULE_NAME&gt;.&lt;OUTPUT_NAME&gt;. Note, it is not possible to print top-level module outputs, to do so there is the terraform output command</li><li>Helps you catch errors that do not show up with terraform validate</li><li>Lets you tinker with the HCL syntax.</li></ul><p>Although powerful, Terraform console comes with its own set of limitations, and often looks like a tool that was a bit rushed, and outside HashiCorp’s priorities.</p><p><strong>What you can’t do</strong>*</p><ul><li>Tab-autocomplete your sentences. It can be very annoying, especially with long variable names, but it <a href="https://github.com/hashicorp/terraform/issues/27370">seems</a> like it won’t come anytime soon. Anyway, you can use up/down arrows for history at least, which is something already</li><li>Define new local values to use in subsequent commands. Unfortunately, if you need it, you need to either add them in your code locally and then run the console again, or try to pack everything into a single line</li><li>Inspect submodules. The only way you have to inspect what is happening in a module, is to define an output for it, or run terraform console in the module itself, providing values for its variables, which is not always possible nor very handy.</li></ul><p>For more information about Terraform console, check the <a href="https://developer.hashicorp.com/terraform/cli/commands/console">docs</a>.</p><p>* as of Terraform v1.7.3</p><h4>Keep in Mind</h4><p>One of the most important things to keep in mind when using terraform console is that the command <em>acquires the state lock</em> for the whole time it is running.</p><p>So, beware that if you run it in one terminal, you cannot run a plan in another terminal instance at the same time. Moreover, other people in your team, or a Terraform pipeline or workflow automation framework using that same Terraform, will be locked from doing operations that require the lock (e.g. plan and apply) for the whole time you’re acquiring the lock.</p><p>If you just want to check if your syntax is correct, you may run terraform validate instead, which does not acquire the state lock. However, Terraform validate catches fewer errors than the ones caught by Terraform console.<br>In a sense, there is a sort of matryoshka: all errors caught by Terraform validate, and more, are caught by console, and all errors caught by console, and more, are caught by plan.</p><p>That’s it for this article, I hope you find it useful. Happy Terraform!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=562762d687ef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Load Environment on Directory Change]]></title>
            <link>https://medium.com/codex/load-environment-on-directory-change-38e1e0541903?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/38e1e0541903</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[bash]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Wed, 23 Aug 2023 08:45:43 GMT</pubDate>
            <atom:updated>2023-11-16T17:06:00.687Z</atom:updated>
            <content:encoded><![CDATA[<h4>Change gcloud context, az login, Kubernetes context, set environment variables, and more just changing directory</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UYVs0ce1WutvGYFN" /><figcaption>Photo by <a href="https://unsplash.com/@ilyapavlov?utm_source=medium&amp;utm_medium=referral">Ilya Pavlov</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>During a typical day at work, I often need to change between two, or even three different repos, each with its relative gcloud, or az, or whatever configuration, its Kubernetes context, and so on.</p><p>At the beginning, I found it really frustrating and time-consuming to backward-search in history the commands I needed to run to be able to perform my tasks.</p><p>Out of frustration, I decided that I needed to find a way to be able to automate this boring task, and I came up with a simple solution that both saves me a lot of time when I need to switch context, but also helps me to track the steps needed to reach a certain configuration, in case I need to replicate it on another project.</p><p>After some stack-overflowing, I created these very simple zsh functions (portable to bash with little to no effort), to load hidden scripts on directory change. In particular, the two functions are named cda (that stands for cd and activate, or whatever), and sayonara (that means <em>goodbye</em> in Japanese):</p><pre># sourcing hidden files on cd<br>function cda() {<br>        local -a PATHS; PATHS=(&#39;.&#39; &#39;.vscode&#39; &#39;.idea&#39; &#39;.terraform&#39;)<br>        builtin cd &quot;$@&quot; || return # return on error<br><br>        for p (&quot;$PATHS[@]&quot;); do<br>                local hidden=&quot;${p}/.hidden.env&quot;<br>                if [[ -f &quot;$hidden&quot; ]]<br>                then<br>                        source &quot;$hidden&quot;<br>                fi<br>        done<br>}<br><br># cleaning up when leaving, sourcing actions to do (like deactivate venv)<br>function sayonara() {<br>        local -a PATHS; PATHS=(&#39;.&#39; &#39;.vscode&#39; &#39;.idea&#39; &#39;.terraform&#39;)<br>        for p (&quot;$PATHS[@]&quot;); do<br>                local hidden=&quot;${p}/.hidden.out.env&quot;<br>                if [[ -f &quot;$hidden&quot; ]]<br>                then<br>                        source &quot;$hidden&quot;<br>                fi<br>        done<br>        if [[ &quot;$#&quot; -gt 0 ]] &amp;&amp; local dest=(&quot;$@&quot;) || local dest=&quot;..&quot;<br>        builtin cd $dest || return<br>}<br>alias sayo=&quot;sayonara&quot;</pre><p>The command sayonara was also aliased with sayo for the sake of brevity.</p><h4>Why not Just Overriding Cd?</h4><p>I <em>could</em> have named the cda function cd and I could have sourced hidden files completely transparently to the user (I am only taking into account the cda function now, as the other is used more rarely), and initially this was the case. In fact, the cda function is written in a way that supports exactly this — note that I do not use cd but builtin cd instead, otherwise an infinite loop would be caused — but this can cause you problems if you have multiple shells opened, and you visit different directories with a hidden env file.</p><p>Let’s suppose, for example, that you need to work in dir1 with the hidden environment env1 and you’re working on this, but you need to visit code in dir2 with hidden env env2. With the hidden env being sourced on cda, you cda in dir1, and “only” cd in dir2, and you have no problems. But, if instead you named the function cd, when you cd in dir1 env1 is sourced, but, when you move to dir2, env2 will be sourced, potentially overriding some configurations of env1 (like the Kubernetes context, or the gcloud configuration).</p><p>Thus, for this problem, I preferred to give up on the dream of writing cd and having the right configuration always ready to use, and <em>“settled”</em> for the still very good solution of cda-ing into a directory when I want its environment to be activated, and cd-ing into it when I just want to visit it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=38e1e0541903" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codex/load-environment-on-directory-change-38e1e0541903">Load Environment on Directory Change</a> was originally published in <a href="https://medium.com/codex">CodeX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Run Scripts on Your Laptop From The Smartphone Using Bluetooth LE]]></title>
            <link>https://medium.com/geekculture/run-scripts-on-your-laptop-from-the-smartphone-using-bluetooth-le-df68fcad9e8b?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/df68fcad9e8b</guid>
            <category><![CDATA[bluetooth]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Tue, 21 Mar 2023 13:22:24 GMT</pubDate>
            <atom:updated>2023-03-22T08:41:34.525Z</atom:updated>
            <content:encoded><![CDATA[<h4>Create a framework to allow commanding the execution of scripts on a BLE central from your smartphone</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kl1q9SKsvjzgFYbzgp2LLw.png" /><figcaption>Example of the Android peripheral application in action.</figcaption></figure><p>In this article, I will talk about how you can command the execution of scripts on your laptop (acting as a Bluetooth LE Central) from your smartphone (acting as the Peripheral).</p><blockquote>If you don’t know what BLE is, or what BLE central and peripheral are, or what GATT is, it is highly recommended that you read something about it first, and then come back to continue with the article.</blockquote><p>Before starting to describe the details of the project, let’s start with a brief high-level overview of the goal.</p><blockquote>Create a system that, from a smartphone, is able to broadcast-command the execution of scripts to a set of BLE centrals.</blockquote><p>The <strong>goal</strong> is to create a system that, from a smartphone, can broadcast-command the execution of scripts to a set of BLE centrals, possibly supporting many different Operating Systems.<br>This project started from a project work assignment I did in university, and the “use case scenario” was that of an operator going around a factory and commanding the execution of “something” (a script) to all the machines nearby.</p><p>So, more specifically, what you are going to achieve if you follow the article is a “framework” in which an <strong>Android application</strong> is able to command the execution of scripts in all nearby devices running a <strong>client</strong> that continuously scan for “certain” peripherals, and connects to one whenever it finds one.</p><h4>The Project’s Stack</h4><p>The project consists of two main parts:</p><ul><li>the Android applications, implementing a GATT server</li><li>the multi-platform client.</li></ul><p>For the development of the Android application, it was used Kotlin as the language, and a permission management library I wrote myself to make it easier to manage permissions in the app.</p><p>For the client, on the other side, two different versions were developed:</p><ul><li>one using Python, and the Bleak library for Bluetooth</li><li>the other using Rust, and the Btleplug library for Bluetooth.</li></ul><p>Both implementations support Linux, Windows, and macOS, but the Python one is far more complete than the Rust one, partly because the Python library is more mature, partly because I’m lazy…</p><h4>The Messages Exchanged by The Devices</h4><p>The <strong>Peripheral</strong> exposes a service with UUID 0000ffe0-0000-1000-8000-00805f9b34fb having two characteristics:</p><ul><li><em>one </em>that will be used to command the script execution, and to which centrals <em>subscribes</em> to for notifications</li><li><em>one</em> that will be used by the centrals to inform the peripheral about the execution status (whether it has started, and the exit code).</li></ul><p>The typical communication flow is shown in <em>Figure 1</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U3-zF0UfUWAuDTv7nYwNFA.png" /><figcaption><strong>Figure 1</strong> — Typical communication flow: 1 — the client subscribes to notifications for the script characteristic. 2a — the user chooses the script to execute. 2b — the notification is sent to the client. 3 — the client signals the server that it is starting the execution by making a read to the status characteristic. 4 — when the client finishes the execution, it writes the exit code to the status characteristic.</figcaption></figure><p>To make it more clear, the script’s characteristic value is a string indicating what script to execute with the relative arguments (e.g. sum_two_numbers.sh 1 3). When the central starts to execute the script, it makes a read to the status characteristic, to signal <em>“hey! I’m about to execute your request”</em> and, when it finishes, writes to the status characteristic the exit code of the script (the exit code, not the script’s output!).</p><figure><img alt="Schema of the service" src="https://cdn-images-1.medium.com/max/1024/1*Jicn4OAQOZVPTOlZesBVtA.png" /><figcaption><strong>Figure 2</strong> — Service schema. script_characteristic is the characteristic that contains the script to execute, status_characteristic is used to manage status updates. Both characteristics have a characteristic descriptor, which is something you do not care much about.</figcaption></figure><h4>The Roles</h4><p>The <em>Android</em> app takes the role of <strong><em>BLE Peripheral</em></strong>, and is the one that is “<em>producing</em>” information (i.e. the script to run). In the project I created, it works by accepting connections from <strong>all</strong> the devices that request it, and <strong>broadcasts</strong> any new script execution request (that is, for example, if you run a script that start a server in localhost, all connected devices will try to start said server).</p><p>The “desktop” app takes the role of <strong><em>BLE Central</em></strong>, and is the one “<em>gathering and consuming</em>” information. The app <em>continuously</em> search for new device to connect to (i.e. devices exposing the service), and connect to the first available one. Once connected, it remains connected “forever” or until the connection is lost.</p><h4>The Android Application (BLE Peripheral)</h4><p>The application’s designed is very simple:</p><ul><li>on top, the list of <em>connected devices</em>, and info about their subscription and execution statuses will be shown</li><li>in the lower part, a field to insert the script you want to execute and its parameters is present, as well as a “<em>send notification</em>” button, to command the script’s execution, and a “<em>start/stop server</em>” button, to start and stop exposing the service.</li></ul><figure><img alt="Screenshot of the Android app UI, showing 3 connected devices, two of which are executing an operation and the other one not yet" src="https://cdn-images-1.medium.com/max/1024/1*ealBxCb_DusICzGOgAifPw.png" /><figcaption><strong>Figure 3</strong> — In the image, a screenshot of the application is taken, where three devices are connected to the peripheral, and subscribed to notification (the green dots). The peripheral has commanded the execution of a script, that has already started in two out of three devices.</figcaption></figure><blockquote>The link to the application repo is available at the end of the article.</blockquote><p>The application uses a <a href="https://medium.com/geekculture/android-permissions-made-easy-d5e3e81ba2ac">permissions&#39; management library</a> that I created to ensure that the necessary permissions are granted by the user, and also it checks that the Bluetooth LE is available on the device and turned on.</p><p>The app has various components organized into packages:</p><ul><li>the ble.gattserver package contains a set of components useful to manage the GATT server and its advertisement, that tries to be as general as possible</li><li>the blescriptrunner package, on the other hand, contains components that are specific to the actual use of the server that is made</li><li>the fragments package contains the connected devices fragment.</li></ul><p>The two main Android components that are used to manage the Bluetooth LE GATT server are the BluetoothLeGattServer and the BluetoothLeAdvertiser.<br>In particular, the advertiser is moved to a <em>Service</em>, while the GATT Server is encapsulated in the GattServerManager class, that tries to provide a (as much as possible) general implementation of a GATT server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kl1q9SKsvjzgFYbzgp2LLw.png" /><figcaption><strong>Figure 4</strong> — Screenshots taken on the peripheral (i.e. the smartphone) showing an example of execution flow. From left to right, the first screenshot shows the three devices connected to the peripheral, with the notifications enabled; in the second screenshot is noticeable that a notification was sent and that two of the three devices has started the execution, while the third devices start executing the script only in the third screenshot; the fourth screenshot shows that all the three devices has successfully concluded the execution.</figcaption></figure><h4>The Python Central</h4><p>The Python application is very simple, and it is developed using the <em>Bleak</em> library to manage the Bluetooth LE.</p><p>The application continuously tries to discover and connect to devices exposing the script runner’s service <em>UUID</em> and, Once one of such devices is found, and the application connects to it, it enables the notification for the script characteristic, and waits to receive notifications.</p><p>Within the app, it is <em>not</em> possible to command the execution of <em>any</em> script, but each script that you want to be executable from the smartphone must be put in a <em>specific</em> directory.</p><p>Once a notification is received, the notified characteristic value is put into an <em>asynchronous queue</em>, that is consumed (also asynchronously) by the <em>coroutine </em>that has the task to try the script’s invocation. This task parses the characteristic value (in a POSIX compliant way), and then tries to execute the script obtained by prepending arg[0] with the path to the script directory. If a script with that name does not exist in the script directory, the error is saved in the log file, but the program continues to run without being blocked.</p><p>The application is able to run scripts on Linux, Windows, and macOS and, if you provide different implementations of the same script (one for each platform) it is able to automatically add and/or correct the script’s extension to use the one that is correct for the central’s Operating System.</p><h4>The Rust Central</h4><p>The Rust application was created after the Python one, and because of limitations in the used library — <em>Btleplug</em> — and in my knowledge of Rust, it offers fewer features than the Python counterpart.</p><p>In particular, the Rust application is not able to:</p><ul><li>handle disconnections</li><li>be configured via command line options</li><li>add or correct the script extension.</li></ul><p>Anyway, the core functionalities are still present, and a scenario with some centrals using the Python application and some others using the Rust one is perfectly tolerable.</p><h4>Performance Assessment</h4><p>In the tests conducted using the Python central app, the results when having a single laptop (MacBook Air M1, with macOS 12.6) connecting over 14 trials were:</p><ul><li>1.28s — mean discovery latency (with a standard deviation of 1.58s)</li><li>0.76 — mean connection latency (with a standard deviation of 0.47s).</li></ul><p>Another test was conducted, whose results are shown in <em>Figures 5</em> and <em>6</em>. The test was carried out using three different laptops trying to connect at the same time or one at a time. The three laptops were a MacBook Air M1, a laptop with Ubuntu installed (Linux1), and a laptop with Manjaro installed (Linux2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*fPzqAKKqgCkNjwsEEv2Vag.png" /><figcaption><strong>Figure 5</strong> — Analysis of mean discovery latencies measured when connecting the devices one at a time, and when connected simultaneously. The bars of the bar plot shows the average latency measured on each device, while the blue line shows the variation of the mean of the mean latencies of each individual device in the two scenarios.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*DUFKX_5--NyLhz1zxcQNDg.png" /><figcaption><strong>Figure 6</strong> — Analysis of mean connection latencies measured when connecting the devices one at a time, and when connected simultaneously. The bars of the bar plot shows the average latency measured on each device, while the blue line shows the variation of the mean of the mean latencies of each individual device in the two scenarios.</figcaption></figure><p>The results of the performance analysis carried out on different devices, shows that the main factor influencing the latencies is indeed the Operating System mounted by the device (or how good is the Bleak backend for that OS). Indeed, as noticeable in <em>Figures 5</em> and <em>6</em>, the two Linux machines showed similar performances (despite being onboarded with different Bluetooth chipset, and different versions of Bluetooth too), while the macOS laptop had totally different performances.<br>In particular, despite being onboarded with a newer chipset, the macOS laptop had a significantly higher discovery latency, partly compensated on the other hand by a lower connection latency, compared with the two Linux laptops.</p><p>Moreover, using having all three laptops trying to connect simultaneously, or using them one at a time, did not show significant performance differences (at least, not that would be noticeable by a human user).</p><h4>Conclusions</h4><p>To conclude, by completing this project, I was able to explore more in depth the strengths and weaknesses of Bluetooth LE which is, in my opinion, a huge step forward compared to Bluetooth classic.</p><p>One of the biggest obstacles I faced while carrying out this project, was to find resources showing how to implement a GATT server. I have the impression that most resources focuses on how to create a Bluetooth LE Central, while the documentation and resources on how to create a service is much more scarce.</p><p>One of the biggest strengths of BLE is that it provides a fairly good communication range (5 to 10 meters in my experience), and a decent transmission speed, with a very low battery consumption.</p><p>On the other hand, I feel that more effort should be conveyed toward making Bluetooth LE easier to work with for developers (e.g. more and better resources and documentation), which is exactly what I am trying to achieve (at least a bit) with this article.</p><p>Let me know in the comments if you liked it or not, or if you have doubts or questions, I will try my best to answer!</p><h4>Resources</h4><ol><li><a href="https://github.com/lorenzofelletti/ble_script_runner">Project’s repo</a></li><li><a href="https://github.com/lorenzofelletti/SimpleBleApp/">Android application repo</a></li><li><a href="https://github.com/lorenzofelletti/permissions">Permission Management library repo</a></li><li><a href="https://pypi.org/project/bleak/">Bleak library (Python BLE Multiplatform Client)</a></li><li><a href="https://crates.io/crates/btleplug/">Btleplug crate (Rust BLE Cross-Platform library)</a>.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df68fcad9e8b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/run-scripts-on-your-laptop-from-the-smartphone-using-bluetooth-le-df68fcad9e8b">Run Scripts on Your Laptop From The Smartphone Using Bluetooth LE</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Android Permissions Made Easy]]></title>
            <link>https://medium.com/geekculture/android-permissions-made-easy-d5e3e81ba2ac?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/d5e3e81ba2ac</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[mobile-app-development]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Tue, 06 Dec 2022 17:31:59 GMT</pubDate>
            <atom:updated>2023-03-22T07:00:57.540Z</atom:updated>
            <content:encoded><![CDATA[<h4>One library to rule them all</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ukCRc2OGgNx3Iaaf" /><figcaption>Photo by <a href="https://unsplash.com/@markkoenig?utm_source=medium&amp;utm_medium=referral">Mark König</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Handling permissions requests in Android is <strong>not</strong> as straightforward as I thought it should be, <em>especially</em> for beginners. Thus, I decided to write a small library to help in managing them.</p><p>The library is very simple to use:</p><ul><li>choose a request code for each group of permissions you are going to ask altogether, and define an array containing those permissions (for example, if the application has a button enabling Bluetooth, and a view showing the user location on a map, you would create a BLUETOOTH_REQUEST_CODE = 1 and a LOCATION_REQUEST_CODE = 2, and define two arrays BLUETOOTH_PERMISSIONS = arrayOf(...), and LOCATION_PERMISSIONS = arrayOf(...))</li><li>create in the Activity that you want to request the permissions a PermissionManager</li></ul><pre>private val permissionsManager = PermissionManager(this)</pre><ul><li>Using the provided DSL, build an appropriate dispatcher</li><li>where you want to check or grant the permissions, call the manager’s checkRequestAndDispatch method</li></ul><pre>permissionManager.checkRequestAndDispatch(BLUETOOTH_REQUEST_CODE)</pre><ul><li>override the Activity’s onRequestPermissionsResult and add to it the line</li></ul><pre>permissionManager.dispatchOnRequestPermissionsResult(requestCode, grantResults)</pre><ul><li>enjoy!</li></ul><h4>The Dispatcher’s DSL</h4><p>The DSL to build the dispatcher has a few functions that helps to create the dispatcher in a human-readable way. The following is an example:</p><pre>permissionManager.buildRequestResultsDispatcher {<br>    withRequestCode(BLUETOOTH_REQUEST_CODE) {<br>        checkPermissions(BLUETOOTH_PERMISSIONS)<br>        doOnGranted {<br>            Log.d(TAG, &quot;Bluetooth permission granted&quot;)<br>            // something ...<br>        }<br>        doOnDenied {<br>            Log.d(TAG, &quot;Bluetooth permission denied&quot;)<br>            // something else ...<br>        }<br>    }<br>    withRequestCode(LOCATION_REQUEST_CODE) {<br>        checkPermissions(LOCATION_PERMISSIONS)<br>        doOnGranted {<br>            Log.d(TAG, &quot;Location permission granted&quot;)<br>            // something ...<br>        }<br>        doOnDenied {<br>            Log.d(TAG, &quot;Location permission denied&quot;)<br>            // something else ...<br>        }<br>    }<br>}</pre><p>Inside buildRequestResultsDispatcher, you can define as many as you want withRequestCode(requestCode) blocks, each containing the permissions to check (checkPermissions) and the behavior to dispatch in case all the permissions are granted (doOnGranted), or some were not (doOnDenied).</p><h4>The checkRequestAndDispatch Method</h4><p>This method does two things:</p><ul><li>it checks if the permissions were already granted, and if so it dispatches the same action associated in the dispatcher to the doOnGranted for the given request code</li><li>otherwise, it asks for the permissions that are not granted; and it would be the overridden onRequestPermissionsResult to dispatch the appropriate action after the permission request.</li></ul><h4>How To Use The Library</h4><p>The library is deployed on <a href="https://jitpack.io/#lorenzofelletti/permissions">Jitpack</a>, and you can easily add it to your project by adding to your settings.gradle:</p><pre>dependencyResolutionManagement {<br>    ...<br>    repositories {<br>        ...<br>        maven { url &#39;https://jitpack.io&#39; }<br>    }<br>}</pre><p>And to your module level build.gradle:</p><pre>dependencies {<br>    ...<br>    implementation &#39;com.github.lorenzofelletti:permissions:0.4.2&#39;<br>}<br></pre><h4>The showRationaleDialog Function</h4><p>It is recommended that you also manage the shouldShowRequestPermissionRationale, so since version 0.3.0 it is possible to do it. Two DSL functions are defined to do this:</p><ul><li>rationale — suitable for the cases when you need the maximum configurability, but you should manage “manually” some things, such as calling permissionManager.checkRequestAndDispatch again if needed with the comingFromRationale flag set to true</li><li>showRationaleDialog — suitable in all the cases when what you want is to show a dialog with a message. In that case, everything is already set up to function properly, you don’t have to do anything but choosing the message.</li></ul><p>An example of the latter is:</p><pre>permissionManager.buildRequestResultsDispatcher {<br>    withRequestCode(LOCATION_REQUEST_CODE) {<br>        checkPermissions(LOCATION_PERMISSIONS)<br>        showRationaleDialog(&quot;Location permissions are needed for this feature.&quot;)<br>        doOnGranted {<br>            // something ...<br>        }<br>        doOnDenied {<br>            // something else ...<br>        }<br>    }<br>}</pre><p>The library is still very young and new features may be added in the future, as well as the existing features may be modified.<br>The code is available on <a href="https://github.com/lorenzofelletti/permissions">GitHub</a>.</p><p>Please let me know your opinion on this simple library and if you want any feature added to it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d5e3e81ba2ac" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/android-permissions-made-easy-d5e3e81ba2ac">Android Permissions Made Easy</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Create a Bluetooth LE Scanner for Android]]></title>
            <link>https://medium.com/geekculture/how-to-create-a-bluetooth-le-scanner-for-android-8d27f63d4de9?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d27f63d4de9</guid>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[bluetooth]]></category>
            <category><![CDATA[mobile-app-development]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Mon, 24 Oct 2022 22:32:15 GMT</pubDate>
            <atom:updated>2023-06-01T07:58:44.668Z</atom:updated>
            <content:encoded><![CDATA[<h4>A simple tutorial that will guide you through the process of creating a BLE scanner app</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*v6Iz9_JrysrLRtSt" /><figcaption>Photo by <a href="https://unsplash.com/@pathum_danthanarayana?utm_source=medium&amp;utm_medium=referral">Pathum Danthanarayana</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Creating a Bluetooth LE scanner for Android is not too complicated. What you need is some understanding of the Android architecture and of Java/Kotlin.</p><p>The most difficult part of building such an app would be…managing the permissions, or even finding what are the required permissions (at least for me).</p><h4>What You Need to Follow This Tutorial</h4><ul><li>Android Studio installed on your computer</li><li>A BLE enabled smartphone for testing running Android 12 (or later)</li><li>A basic understanding of Kotlin (or Java, at least)</li><li>A (very) basic understanding of how to create a UI in Android</li></ul><h4>Creating the Project</h4><p>I will assume that you already know how to create a project in Android Studio, so create a new project, choose <em>Empty Activity</em>, and name it whatever you like, I named mine <em>SimpleBleScanner</em>. Also, select <em>API 31</em> as the <em>Minimum SDK</em>.<br>I know that requiring Android 12 could be a bit too tight of a requirement, but this choice was made to be able to experiment with the latest features. <br>Anyway, if you have the necessity to use an earlier version of the SDK, I think you won’t have to change much of the code.</p><h4>Managing the Permissions</h4><p>First, we are going to declare in the Manifest (AndroidManifest.xml) all the permissions we will use in the application. In particular, for Bluetooth LE scanning, the minimum required permissions are the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cb45c607d3b7af16b701a5cf273d23d2/href">https://medium.com/media/cb45c607d3b7af16b701a5cf273d23d2/href</a></iframe><p>As weird as it might sound, access to coarse and fine location is required in order to use BLE.</p><p>Next, inside the package containing the MainActivity, I chose to create another package named permissions, in which to put all the permission utilities.</p><p>Inside this newly created package, create two Kotlin objects named PermissionsUtilities and BleScanRequiredPermissions. These two objects will help us in managing the permissions without writing too much code inside the MainActivity.</p><p>BleScanRequiredPermissions will contain only the array of the permissions we are going to require for our app.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/376bfe94bf7117089ef3483ad297f13f/href">https://medium.com/media/376bfe94bf7117089ef3483ad297f13f/href</a></iframe><p>PermissionsUtilities, on the other hand, is a more complex object, containing all the utility functions to manage permissions in our application.<br>Anyway, the focus of the article is not on permissions handling, so I will go fast on this part. Feel free to check the permissions package by yourself if interested.</p><p>The main functions of this module are:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c17685f6096696396c80b4fa889b554a/href">https://medium.com/media/c17685f6096696396c80b4fa889b554a/href</a></iframe><p>Otherwise, you could check this article about a permission management library I made, and try to use it instead…</p><p><a href="https://medium.com/geekculture/android-permissions-made-easy-d5e3e81ba2ac">Android Permissions Made Easy</a></p><h4>Scanning for devices</h4><p>To manage the scanning, we will create a new package named blescanner, containing two sub-packages namedmodel and adapter.<br>The model the classes modeling the BleDevice and the BleScanCallback, while the adapter will contain the BleDeviceAdapter, used as adapter for the RecyclerView showing the results of the scan.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fe5248a50d46b3f1663f4014df9c08ce/href">https://medium.com/media/fe5248a50d46b3f1663f4014df9c08ce/href</a></iframe><p>Other than these three classes, we will create a BleScanManager, that will manage the scanning.</p><p>The BleScanManager class contains various members to manage the scanning state and what to do before/after the scanning.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bce81a51e8418fd81c3362f81b1d9b72/href">https://medium.com/media/bce81a51e8418fd81c3362f81b1d9b72/href</a></iframe><p>The constructor will take as input a BluetoothManager, a scan period (set to a default value if omitted), and a list of actions to do before and after the scanning. The list of actions is nothing but a list of functions that will be invoked before/after the scanning start.</p><p>The Handler is used to schedule the stopping of the scan after the scan period time.</p><p>The companion object of the BleScanManager will contain the default value of the scan period, and a function to execute the list of actions.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3a62514bcb6fe42a87b8c1091cdb6c2f/href">https://medium.com/media/3a62514bcb6fe42a87b8c1091cdb6c2f/href</a></iframe><p>The last lines to complete the BleScanManager are the following</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ec15dd33b78125aa77af99a7f182407d/href">https://medium.com/media/ec15dd33b78125aa77af99a7f182407d/href</a></iframe><h4>The Main Activity</h4><p>The main activity will contain the code that “connects” all the modules we created so far:</p><ul><li>the module managing the permissions</li><li>the BLE scan manager</li><li>the adapter for the RecyclerView</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/80a319a06b20d48d72becdf319ef3c65/href">https://medium.com/media/80a319a06b20d48d72becdf319ef3c65/href</a></iframe><p>The code is pretty straightforward, the only two methods are the onCreate and the onRequestPermissionResult.</p><p>The latter is used to manage what to do “after” requesting the permissions to the user, while the former sets the ground for the application by setting the RecyclerView adapter, creating the BleScanManager, and setting the onClickListener for the button to start the scan.</p><p>Right after the BleScanManager is created, we add to it before/afterScanActions the code to enable/disable the scan button — because we want to disable the scan button while we’re performing it.</p><p>The onClickListener to start the scan checks first that the required permissions are granted, and, if so, it proceeds to start the scan, otherwise it starts permissions request flow.</p><h4>The GUI</h4><p>The GUI will be very (very) simple, just a Button to start the scanning, and a RecyclerView to show the scanning results.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e8f8fce1507b3c141b2155c86c7059b3/href">https://medium.com/media/e8f8fce1507b3c141b2155c86c7059b3/href</a></iframe><p>Other than the activity_main.xml we will need another layout file to define the layout of a RecyclerView row.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/42bc84c91d1bd37398e9dee423b0bd5a/href">https://medium.com/media/42bc84c91d1bd37398e9dee423b0bd5a/href</a></iframe><h4>Conclusions</h4><p>This is all to create a simple Bluetooth LE scanner app in Android.<br>The results will look something like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O_3WualBNIkvummQJpHhXw.png" /><figcaption>The application&#39;s UI while performing a scan</figcaption></figure><p>As you can see, the app is very simple on the UI side, this because I wanted to focus more on the functionality. If you want, feel free to fork the <a href="https://github.com/lorenzofelletti/SimpleBleScanner">repo</a> and improve the application’s UI, or extend its functionalities.</p><p>This is all for this article, I hope you enjoyed reading it and that you found it useful. Let me know with a comment!</p><p>Maybe you are interested in my new article about making it possible to command the execution of scripts from your smartphone to your laptop using Bluetooth LE.</p><p><a href="https://lorenzofelletti.medium.com/run-scripts-on-your-laptop-from-the-smartphone-using-bluetooth-le-df68fcad9e8b">Run Scripts on Your Laptop From The Smartphone Using Bluetooth LE</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d27f63d4de9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/how-to-create-a-bluetooth-le-scanner-for-android-8d27f63d4de9">How to Create a Bluetooth LE Scanner for Android</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Merge” Last Two Commits]]></title>
            <link>https://lorenzofelletti.medium.com/merge-last-two-commits-5bad30fa044b?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/5bad30fa044b</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[version-control]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[git]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Tue, 05 Apr 2022 21:15:54 GMT</pubDate>
            <atom:updated>2022-11-11T11:25:55.946Z</atom:updated>
            <content:encoded><![CDATA[<h4>Have you ever wanted to merge your last two commits, maybe because you forgot to commit something in the first one, but you don’t know where to start? Well, read this article!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mo44sOd8QKKohUZx" /><figcaption>Photo by <a href="https://unsplash.com/@yancymin?utm_source=medium&amp;utm_medium=referral">Yancy Min</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>We have all been there, you commit some super-important changes you made, but suddenly realize that you forgot to commit something crucial.</p><p>Now you’re worried that your history will not be as clean as it was before! If only there was a way to merge your two commits together…</p><p>Well, luckily there is such a way, and is also quite simple:</p><h4>How to merge the last two commits</h4><p>First, run <strong>git rebase --</strong>interactive<strong> HEAD~2</strong>.<strong><br></strong>Then, from the interactive interface that’s presented you (e.g. <em>nano</em>) type:</p><pre>pick b76d157 commit message <strong># penultimate commit hash</strong><br>squash a931ac7 other commit message <strong># last commit hash</strong><br># Rebase df23917..a931ac7 onto df23917<br>#<br># Commands:<br>#  p, pick = use commit<br>#  r, reword = use commit, but edit the commit message<br>#  e, edit = use commit, but stop for amending<br>#  s, squash = use commit, but meld into previous commit<br>#  f, fixup = like &quot;squash&quot;, but discard this commit&#39;s log message<br># # If you remove a line here THAT COMMIT WILL BE LOST.<br># However, if you remove everything, the rebase will be aborted.<br>#</pre><p>And now you will be prompted to another interactive interface in which the two commits messages are presented, and you’re free to edit them as you want (keep both, only the one you like the most, write a completely new one).</p><h4>What if the two commits were already pushed?</h4><p>In this case, just use <strong>git push --force</strong>,<strong> </strong>but remember that this could be a risky operation depending on the situation, so use it carefully.</p><h4>What if the commits I wanted to merge were three, or four, or more?</h4><p>Nothing easier, just replace <strong>HEAD~2</strong>, which means two commits behind HEAD, use <strong>HEAD~3 </strong>or whatever number of commits you want, and then squash all the commits you want.<br>Example with three commits:</p><pre>pick f7f3f6d Change my name a bit<br>squash 310154e Update README formatting and add blame<br>squash a5f4a0d Add cat-file</pre><p>Et voilà!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5bad30fa044b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Build a RegEx Engine in Python]]></title>
            <link>https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-860a3083d7eb?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/860a3083d7eb</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[regex]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Mon, 25 Oct 2021 04:48:02 GMT</pubDate>
            <atom:updated>2021-10-25T04:48:02.234Z</atom:updated>
            <content:encoded><![CDATA[<h4>Part 7: The Backtracking System</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bLkAnEIWRMkVv2fX" /><figcaption>Photo by <a href="https://unsplash.com/@holly_buildalifeyoulove?utm_source=medium&amp;utm_medium=referral">Holly Stratton</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In this article, we will complete our regex engine with the backtracking system.<br>This article is the seventh part of my series on how to build a regular expression engine, so if this is the first article you read, I advise you to read the previous ones and then come back to this one.</p><p>The backtracking feature was one of the most difficult to implement and is by far the most interesting feature of this project.</p><p>Before coming up with the final and (<em>hopefully</em>) correct version of the backtracking system, I implemented a couple of “partially correct” solutions, each of them not working in some edge cases.</p><p>To find out all these bugs testing was crucial, and mainly unit testing. If you don’t already use unit tests in your projects, I strongly suggest you start using them.</p><h4>When to Backtrack?</h4><p>To figure out when it is proper to backtrack, and when it is not, is not a trivial task, but it is the first thing you have to sort out when implementing backtracking.</p><p>First, you have to understand what your engine can or cannot already do, to not implement the same feature twice.</p><p>For example, our engine so far can already do <em>some</em> backtracking: in case of an or (OrNode), if the left branch fails it tries to match the right branch, so we don’t have to implement this in our backtracking system.</p><p>When trying to match a <em>regular expression </em>to a given string, let’s call it <em>test string</em>, and you have an engine requiring backtracking (as we saw in part 6) you have to operate these steps:</p><ul><li>Start from index 0 of the test string, and the leftmost leaf of the AST representing the <em>regex</em></li><li>Read the first character of the test string<em> </em>and the leftmost leaf of the AST</li><li>Do they match? If yes, move on (index 1 or the test string, next leaf in the AST), if not try to backtrack</li><li>If backtracking fails, then you have a complete fail: the test string does not match the regex.</li></ul><p><strong><em>Backtracking</em></strong><em> </em>is possible if:</p><ul><li>I matched an element more times than the minimum number of times it has to be matched(e.g. we have an A to be matched 1+ times, and we matched it more than once)</li><li>An element B is matched the minimum number of times, but a previous element A is matched more than the minimum number of times.</li></ul><h4>Groups Sweet Groups!</h4><p>Our match_group function walks the AST going down a level every time a non-leaf node is encountered, by calling itself recursively.<br>So, from a match_group function call point of view, “the world is flat” — what it does is just to iterate through its children, children that are all at the same depth in the AST.<br>The only difference is that a leaf node is encountered there isn’t another function call to get to the new state, while when a non-leaf is found, another match_group is called, and at the end of its execution, the upper level gets the new state.</p><p>For a single match_group there is no tree, just a list of elements to iterate through.<br>And we can exploit this to make our backtracking system also “flat”, meaning that it takes care of only one level.</p><p>So, we will define a “flat” backtracking system, and every “level” (i.e. match_group) will have its own.<br>In doing this, we greatly simplify the backtracking system responsibility, but without losing generality, because the call stack will take care of executing the operations in the right order. In a way, the “depth management” part of our system comes “for free”, thanks to the call stack.</p><h4>How to Backtrack?</h4><p>Thanks to the fact that our engine works with groups, we don’t have to create “global” backtracking, but just a system capable of handling backtracking inside a group.</p><p>To backtrack, we have to <strong><em>store </em></strong>somewhere the information about the <strong><em>explored path(s)</em></strong>, and we also need a <strong><em>strategy to explore</em></strong> all the possible paths.</p><p>The information to store is pretty simple:</p><ul><li>An array (that we will use as a stack — called backtrack_stack) containing one element for every visited node.</li></ul><p>Each element of the array, representing a visited node, must contain the following information:</p><ul><li>the visited node(you don’t even need the whole node, but only its index in the <em>children array </em>— the aforementioned array on which match_groupoperates)</li><li>how many times have you matched a node</li><li>the minimum amount of times it must be matched</li><li>an array of how many characters were consumed each time it was matched.</li></ul><p>That’s it, this is all the information we need to store in the <em>backtrack stack </em>every time we visit a node.</p><p>A <em>backtrack stack</em> will be created for each match_group invocation, tracking only the path explored for sibling nodes.</p><h4>Let’s Backtrack</h4><p>Now, with the backtrack stack so defined and the backtracking rules I described earlier, we can state that:</p><ol><li>When there is a request for backtracking and the <em>element</em> on top of the <em>stack</em> is matched more times than the minimum, the last consumption must be removed from its consumption list and the consumed characters of the test string updated as follows:<br>test_str_consumed_characters -= last_consumption</li><li>When there is a request for backtracking and the <em>element</em> on top of the <em>stack</em> is already matched the minimum number of times, the element should be removed and the consumed characters of the test string updated as follows:<br>foreach consumption in consumed_list_of_the_element: test_str_consumed_characters -= consumption</li><li>When there is a request for backtracking on an empty <em>backtrack stack</em>, all the possible paths have been explored and the request must cause the failure of the match (i.e. <em>regex</em> and <em>test string</em> doesn’t match).</li></ol><p>By applying these simple three rules in our code, and exploiting the recursion of match_group, we cover all the possible backtracking cases we can encounter, and we obtain the <em>full-fledged</em> RegEx Engine that was our goal to build.</p><h4>Conclusion</h4><p>With this article we finished building the RegEx Engine we wanted to build, thus this article concludes this series.</p><p>However, if one day I will want to deepen aspects of this project I didn’t so far, I cannot rule out that I will write other articles about this topic, but for now, the series is complete enough in my opinion.</p><p>Let me know if you liked this series of articles, or if you have any suggestions or doubts about the topics discussed here.</p><h4>Previous articles</h4><ul><li><a href="https://medium.com/geekculture/how-to-build-a-regex-engine-in-python-part-1-the-grammar-d3cc91cc1784">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-part-2-the-lexer-f22d0cfe961c">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-c06472bdb2f9">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-535bef84d1f7">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-d687de633270">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-6150341dfc17">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-55fde6bccffa">How to Build a RegEx Engine in Python</a></li></ul><p><em>Link to repo: </em><a href="https://github.com/lorenzofelletti/pyregex"><em>GitHub — lorenzofelletti/pyregex</em></a></p><p><em>More content at </em><a href="http://plainenglish.io/"><strong><em>plainenglish.io</em></strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=860a3083d7eb" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-860a3083d7eb">How to Build a RegEx Engine in Python</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Build a RegEx Engine in Python]]></title>
            <link>https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-55fde6bccffa?source=rss-94dc0e4baf1a------2</link>
            <guid isPermaLink="false">https://medium.com/p/55fde6bccffa</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[regular-expressions]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Lorenzo Felletti]]></dc:creator>
            <pubDate>Tue, 28 Sep 2021 09:03:04 GMT</pubDate>
            <atom:updated>2021-10-25T09:29:21.694Z</atom:updated>
            <content:encoded><![CDATA[<h4>Part 6: the engine</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7_hU1-xGtrVvpiHItiXTCA.png" /><figcaption>RegexEngine class.</figcaption></figure><p>It is finally time to write about the engine of the regex engine we’re building.</p><p>As we already saw, the parser produces an AST as a result of the parsing of a regular expression.<br>Now, the engine has to check whether a given string matches or not a regex. <br>To do so, the engine uses the AST representation of the regex, which <strong>extracts </strong>the fundamental features/elements of the regular expression, and tries to “find” if the test string has them.</p><h3>The Engine</h3><p>The engine is again a Python class, having a fundamental method match, which is a closure containing the engine and all the needed auxiliary methods.</p><p>Match signature:</p><pre>def match(self, re: str, string: str)</pre><h4>Match Semantic</h4><p>The semantic we will give to the match method is the following:</p><ul><li>it returns on the first occurrence of the regex in the string</li><li>if there is no match, it tries again shifting the input string one character to the right (that is, moving the string one character to the right) until it reaches the end of the string.</li></ul><p>That is, if the input string is abac and the regex is c, on the first attempt abac won’t match c, so match tries again, but shifting abac one character to the right this time, so tries to match bac with c, resulting in another fail, so it tries again with ac and c (failing), and, lastly, it tries c with c, resulting in a match, thus returning true.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WLmU2thR8DiblAczeVoQ_A.png" /><figcaption>The match function so far.</figcaption></figure><h4>How To Match</h4><p>match_group is the most important method of the engine.</p><p>It works by iterating through an AST node’s children, and by calling itself recursively when a GroupNode is found or, with some differences, when an OrNode is found.</p><p>If an OrNode is found, it checks if the left children (the one in position 0) matches, if it doesn’t, it tries the right one.</p><p>When a leaf node is found (ElementNode, …) it tries to match it.</p><p>After every match, the str_i (the index of the string) is increased of the matched characters.<br>The string index is set to position 0 (or the first position to consider more in general) at the beginning, and is kind of a “global variable”, but local to the match function.</p><p>The mechanism of the recursive calls makes it easy to manage the “point in the tree” and “point in the string” considered. It also allows you to think of each group as a separate entity, making it easier to follow mentally the code execution.</p><p>Obviously, this isn’t the whole engine, because as stated in the first article, we will need a backtracking system to assure to explore all the possible paths to match the string.</p><h4>A little problem</h4><p>In fact, the semantic of leaf nodes and groups is to match them to the most characters possible by default, and while this is convenient most of the time, there are some edge cases in which this won’t lead to a correct answer.</p><p>Let’s examine this example:</p><p>If you have a regex a+ab and a test string aab:</p><ul><li>the resulting AST will consist of a GroupNode, with three children, all leaves</li><li>a first a to be matched [1, +∞[ times</li><li>a second a to be matched exactly one time</li><li>a final b to be matched exactly one time.</li></ul><p>During the execution of our match function as presented, this is what will happen:</p><ul><li>match_group is called and start iterating on its children</li><li>the first a element is matched as much as possible, so two times aa</li><li>then the engine tries to match the second a element, but it fails as now the only left character is a b</li><li>false would be returned, but we all know the correct answer is true.</li></ul><p>This is because of the semantics we gave to our match function. What we would like the system to do is to “try another way”:</p><ul><li>since the first element could accept 1 or more occurrences of the letter a, and it matched two of them, we would like to be able to tell it “hey, please try by matching it one time less”</li><li>in this case, the first a would be matched one time, leaving enough room to fit the second a element, and then also the b element, our third and last</li><li>this process worked, and so the answer would be true, as it should be.</li></ul><p>This process is called backtracking and will be the topic of the next article, for now, let’s ignore the problem.</p><h3>Conclusion</h3><p>Overall, we covered pretty much everything we needed to cover to explain the behavior of the match method.</p><p>The skeleton of our code will look something like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RyROzKkxwQXDPzy2-eu7Og.png" /><figcaption>match function’s structure.</figcaption></figure><p>In the next article, we will cover the interesting topic of backtracking and will complete our regex engine.</p><h4>Following article</h4><p><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-860a3083d7eb">How to Build a RegEx Engine in Python</a></p><h4>Previous articles</h4><ul><li><a href="https://medium.com/geekculture/how-to-build-a-regex-engine-in-python-part-1-the-grammar-d3cc91cc1784">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-part-2-the-lexer-f22d0cfe961c">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-535bef84d1f7">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-d687de633270">How to Build a RegEx Engine in Python</a></li><li><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-6150341dfc17">How to Build a RegEx Engine in Python</a></li></ul><p><em>Link to the repo: </em><a href="https://github.com/lorenzofelletti/pyregex"><em>GitHub — lorenzofelletti/pyregex</em></a></p><p><em>More content at </em><a href="http://plainenglish.io/"><strong><em>plainenglish.io</em></strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=55fde6bccffa" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/how-to-build-a-regex-engine-in-python-55fde6bccffa">How to Build a RegEx Engine in Python</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>