<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Neal Davis on Medium]]></title>
        <description><![CDATA[Stories by Neal Davis on Medium]]></description>
        <link>https://medium.com/@neal.davis?source=rss-4c7308b488b9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*rYZLthlPeNWi5j3uD-F3RQ.jpeg</url>
            <title>Stories by Neal Davis on Medium</title>
            <link>https://medium.com/@neal.davis?source=rss-4c7308b488b9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 06:41:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@neal.davis/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Writing Robust Hoon — A Guide To Urbit Unit Testing]]></title>
            <link>https://medium.com/dcspark/writing-robust-hoon-a-guide-to-urbit-unit-testing-82b2631fe20a?source=rss-4c7308b488b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/82b2631fe20a</guid>
            <category><![CDATA[urbit]]></category>
            <category><![CDATA[gall]]></category>
            <category><![CDATA[unit-testing]]></category>
            <dc:creator><![CDATA[Neal Davis]]></dc:creator>
            <pubDate>Wed, 02 Feb 2022 16:08:28 GMT</pubDate>
            <atom:updated>2022-02-02T16:08:28.635Z</atom:updated>
            <content:encoded><![CDATA[<h3>Writing Robust Hoon — A Guide To Urbit Unit Testing</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*_Ie7iFFGKwKuGjnKFVdadQ.png" /></figure><p><a href="https://urbit.org/">Urbit</a> is a novel system which supports <a href="https://urbit.org/docs/userspace/dist/dist">peer-to-peer software distribution</a> using public desks (analogous to Git branches). This opens up a wide array of possibilities, however as a developer how can you have confidence in the performance and behavior of Gall agents you write?</p><p><em>Unit testing</em> is one of the available approaches which allows you to verify expected behavior as rigorously as you choose.</p><p>Since Urbit runs as a virtualized Nock machine, you don’t have to worry about memory management and other runtime binary concerns. Nor do you have to worry too much about external penetration testing (as long as you rely on Urbit system services to authenticate callers as necessary). However, you still need to make sure that library code and applications do what they are advertised to do.</p><p>As such in this post we will be diving into how to go about unit testing your Gall agents and thereby ensuring your end users will have a smooth experience.</p><h3>The Structure of a Unit Test</h3><p>A unit test measures program results against a known standard of behavior. Ideally a unit test examines the single orthogonal aspect of a function or feature. Discrete test composition makes it easier to narrow down misbehavior.</p><p>Hoon provides two built-in unit testing tools:</p><ol><li>-test is a thread which locates and runs unit tests at a specified path, by convention in /tests at the path of the corresponding file.</li><li>%test is a Gall agent that builds all of %generators, %agents, or %marks, confirming basic operability.</li></ol><h3>-test Thread</h3><p>-test unit tests are included in a |% barcen core and consist of chained <a href="https://urbit.org/docs/hoon/reference/stdlib/2q#tang">tang</a>s (formatted output structures) produced by ++expect-eq (equality of two values) and ++expect-fail (failure/crash). By convention, the expected value is listed first and the actual value second.</p><p>/tests/lib/fib/hoon:</p><pre>++  test-fibonacci<br>  ;:  weld<br>  %+  expect-eq<br>    !&gt;  ~[1 1 2 3 5 8]<br>    !&gt;  (fib 6)<br>  ==</pre><p>For instance, suppose one wished to test the (real) Hoon gate ++sub. The following tests would verify that integer subtraction works as expected. Many unit tests may appear trivial because they are spot checks that things work as expected.</p><p>/tests/lib/sub/hoon:</p><pre>/+  *test<br>|%<br>++  test-zero<br>  ;:  weld<br>  %+  expect-eq<br>    !&gt;  0<br>    !&gt;  (sub 0 0)<br>  %+  expect-eq<br>    !&gt;  1<br>    !&gt;  (sub 1 0)<br>  %+  expect-eq<br>    !&gt;  0<br>    !&gt;  (sub 20 20)<br>  %-  expect-fail<br>    |.  (sub 0 1)<br>  ==<br>++  test-one<br>  ;:  weld<br>  %+  expect-eq<br>    !&gt;  0<br>    !&gt;  (sub 1 1)<br>  %+  expect-eq<br>    !&gt;  1<br>    !&gt;  (sub 2 1)<br>  %+  expect-eq<br>    !&gt;  19<br>    !&gt;  (sub 20 1)<br>  %-  expect-fail<br>    |.  (sub 1 2)<br>  ==<br>--</pre><p>Unit tests are frequently used with a testing framework which provides a scaffolding for running many tests at a time. In this case, we use the -test thread. Save the foregoing code on a new %sandbox desk at /tests/lib/sub/hoon</p><pre>dojo&gt; |merge our %base %sandbox<br>:: copy the code in on the Unix side<br>dojo&gt; |commit %sandbox</pre><p>and run the tests with</p><pre>dojo&gt; -test /=sandbox=/tests/lib/sub/hoon ~ </pre><p>(Don’t miss the ~ at the end there.)</p><p>You should see a notification for each test in the core at that path:</p><pre>built   /tests/lib/sub/hoon<br>OK      /lib/sub/test-zero<br>OK      /lib/sub/test-one</pre><h3>%test Agent</h3><p>The %test agent is more macroscopic than -test, building all of %generators, %agents, or %marks. Using %test builds all of the candidate elements on the %base desk. (As of this January 2022 writing, cross-desk execution is in development by Tlon.) As desk affordances are built out, an equivalent operation should be introduced to check other desks.</p><pre>dojo&gt; |start %test<br>dojo&gt; :test %agents<br>&gt;=  <br>built   /app/acme  <br>built   /app/aqua  <br>built   /app/azimuth  <br>built   /app/azimuth-rpc  <br>built   /app/azimuth-tracker  <br>built   /app/claz  <br>built   /app/dbug  <br>built   /app/dns-collector  <br>built   /app/dojo  <br>built   /app/eth-sender  <br>built   /app/eth-watcher  <br>built   /app/gaze  <br>built   /app/herm  <br>built   /app/hood  <br>built   /app/language-server  <br>built   /app/lens  <br>built   /app/ping  <br>built   /app/roller  <br>built   /app/roller-rpc  <br>built   /app/shoe  <br>built   /app/spider  <br>built   /app/test  <br>built   /app/time  <br>%all-agents-built  <br>&gt; :test %agents  <br>&gt;=</pre><p>The %test Gall agent forms part of the system-wide integrity check for a distributed desk, but for the most part we will rely on the -test thread to validate program behavior.</p><p>Between the two methods, note any failed tests. The test and the code should both be examined to determine which is at fault and then corrected. Once a high degree of confidence has been attained in the unit tests, then these may be reliably used to identify failures in a repeatable and auditable way. You should construct a small matrix of common failures, even if trivial, then use your imagination to examine more liminal cases. In addition, any time a failure arises in practice, a unit test should be added to hedge against its occurrence in the future. Rigorous discipline in unit testing will reduce the number of errors in the codebase and occasionally unmask latent errors.</p><h3>Basic Debugging on Urbit</h3><p>What do we mean when we say something has gone wrong? It means that a result happened in contravention of our expectation. Some bugs yield subtle errors: they are transient, or embedded in the seventh decimal place. Others are dramatic, such as crashes or absurd results. We will call an <em>error</em> the actual source of a bug. Essentially any nontrivial program has bugs of varying severity. A <em>failure</em>, on the other hand, is an observable, the incorrect behavior of a program. A failure can reveal itself as an actual crash or an incorrect behavior or result.</p><p>Tests should find things that are supposed to go right and things that are supposed to go wrong (i.e. expected errors/crashes, perhaps due to bad input).</p><p>For all our rigor, Dijkstra reminds us that:</p><blockquote><em>Program testing can be used to show the presence of bugs, but never to show their absence.</em></blockquote><p>Due to the structure of the Urbit runtime, entire categories of errors are impossible to the pure Hoon virtual machine layer: no null pointer dereferencing, no deadlocks, no underflows. (This doesn’t prevent the possibility of memory leaks or array bound overwrites in the runtime or jets, of course, but we set those aside and assume the runtime works as advertised, generally a good assumption so far.)</p><p>Hoon encourages a “traditional” style of debugging, relying on debugging printfs (<a href="https://urbit.org/docs/hoon/reference/rune/sig#-sigpam">~&amp; sigpam</a> with logging degrees &gt; info, &gt;&gt; warning, and &gt;&gt;&gt; error) and stack trace hints (<a href="https://urbit.org/docs/hoon/reference/rune/zap#-zapcol">!: zapcol</a> to turn on stack trace debugging; <a href="https://urbit.org/docs/hoon/reference/rune/sig#-sigbar">~| sigbar</a> to add an annotation in case of crash). The imperative programming model of breakpoints doesn&#39;t map particularly well to functional programming language models and there&#39;s not a debugging-oriented step-by-step execution framework right now.</p><p>The easiest way to check for errors is simply to aggressively enforce type using the <a href="https://urbit.org/docs/hoon/reference/rune/ket#-kethep">^- kethep</a> and <a href="https://urbit.org/docs/hoon/reference/rune/ket#-ketlus">^+ ketlus</a> runes. The static compiler will handle type errors at build time, meaning these by and large do not affect your agent at runtime.</p><h3>The Structure of a Gall Agent</h3><p>Most of the time, Urbit developers will ship Gall agents as their primary product rather than generators or libraries. Recall that a Gall agent has ten arms. In many applications, several are left as default arms which pass-through or do nothing. Prefer to use the default library&#39;s specification for clarity (rather than rolling your own do-nothing arm).</p><pre>|_  =bowl:gall<br>+*  this      .<br>    default   ~(. (default-agent this %|) bowl)<br>++  on-init   on-init:default<br>++  on-save   on-save:default<br>++  on-load   on-load:default<br>++  on-peek   on-peek:default<br>++  on-poke   on-poke:default<br>++  on-watch  on-watch:default<br>++  on-leave  on-leave:default<br>++  on-agent  on-agent:default<br>++  on-arvo   on-arvo:default<br>++  on-fail   on-fail:default<br>--</pre><p>Unit tests for several of these arms, such as ++on-init and ++on-load, aren&#39;t really necessary since any mistake tends to be immediately manifest in development. (For instance, failure to adequately define a state transition in ++on-load will yield an error.) Others, however, in particular the interaction arms merit rigorous testing. While it has been common to spot-check behaviors interactively, we outline below an approach using automated unit testing which can be used to validate the behavior of each arm.</p><p>The +dbug generator which can wrap the Gall agent permits a direct view of the agent&#39;s internal state. This is crucial for your interactive testing, but insufficient for an automatic testing framework. We emphasize that any internal state variable should have a corresponding peek action that you can test automatically.</p><h3>An Approach to Testing Gall Agents</h3><h3>%hark-store</h3><p>The above approach is adequate for library code, but most end-user applications on Urbit are Gall agents.</p><p>Conventionally, Urbit instruments unit tests via the -test thread, which scans a path for arms beginning with &#39;test-&#39; and runs each of them. For instance, to run all the built-in library tests on the %base desk, you can use:</p><pre>dojo&gt; =dir %landscape<br>dojo&gt; -test %/tests/lib ~</pre><p>which will build the tests, run each test, and report:</p><pre>dojo&gt; -test %/tests/lib ~<br>built   /tests/lib/der/hoon<br>...<br>&gt;   test-veri-wrong-key: took 13805µs<br>OK      /lib/vere/dawn/test-veri-wrong-key<br>&gt;   test-veri-pawn-key-mismatch: took 19752µs<br>OK      /lib/vere/dawn/test-veri-pawn-key-mismatch<br>&gt;   test-veri-pawn-invalid-life: took 16129µs<br>OK      /lib/vere/dawn/test-veri-pawn-invalid-life<br>&gt;   test-veri-pawn-good: took 16359µs<br>OK      /lib/vere/dawn/test-veri-pawn-good<br>&gt;   test-veri-pawn-already-booted: took 15366µs<br>OK      /lib/vere/dawn/test-veri-pawn-already-booted<br>...</pre><p>For /app agents, the situation is a bit more complicated. For instance, the agent must be running before tests can be made against the agent. Accommodation must be made for the fact that agents run on different desks.</p><p>The only significant example of /app testing in the current (January 2022) Urbit codebase is %garden&#39;s %hark-store. This constructs a number of specific data structures, inserts them into a =^-pinned version of the %hark-store agent, and compares for expected results. For instance, here is the logic testing the “half-open” data structure in ++test-half-open-double, which manages the number of active notifications.</p><pre>++  test-half-open<br>  =|  run=@ud <br>  =^  mov1  agent  <br>     (~(on-poke agent (bowl run)) %hark-action !&gt;((add-note run)))<br>  =^  mova  agent<br>     (~(on-poke agent (bowl run)) %noun !&gt;(%sane))<br>  =.  run  +(run)<br>  =^  mov2  agent  <br>     (~(on-poke agent (bowl run)) %hark-action !&gt;(read-count))<br>  =^  mov3  agent<br>     (~(on-poke agent (bowl run)) %noun !&gt;(%sane))<br>  =/  expected-archive=notification:hark<br>    [(add *time (mul ~s1 0)) bin ~[(body 0)]] <br>  =+  !&lt;(=state on-save:agent)<br>  =/  actual-archive=notification:hark<br>    (~(got re archive.state) (add *time ~s1) bin)<br>  (expect-eq !&gt;(expected-archive) !&gt;(actual-archive))</pre><p>This logic does the following:</p><ol><li>Adds a note and checks state (%sane).</li><li>Sets the read count and checks state (%sane).</li><li>Constructs the reference case expected-archive.</li><li>Retrieves the actual case from %hark-store&#39;s state.</li><li>Asserts equality of the vases.</li></ol><p>Another test, ++test-half-open-capped, adopts a slightly different approach. Basically, this test looks at whether notifications are set and passed through properly.</p><pre>++ test-half-open-capped<br>  =|  run=@ud<br>  |-<br>  ?:  =(run 31)<br>    =+  !&lt;(=state on-save:agent)<br>    (expect-eq !&gt;(~) !&gt;(half-open.state))<br>  =^  movs  agent<br>    (~(on-poke agent (bowl run)) %hark-action !&gt;((add-note run)))<br>  =^  mavs  agent<br>    (~(on-poke agent (bowl run)) %hark-action !&gt;(read-count))<br>  $(run +(run))</pre><p>This logic does the following:</p><ol><li>Sets an uninitialized run counter, which counts the number of moves the state log has recorded.</li><li>Constructs the agent state with a note added.</li><li>Constructs the agent state with a read count for the test path.</li><li>(at the top of the trap) Checks the exit condition and returns the testing arm result.</li></ol><p>Running these tests yields the following:</p><pre>dojo&gt; -test %/tests/app/hark-store ~<br>built   /tests/app/hark-store/hoon  <br>[   by-place  <br> { [ p=[desk=%landscape path=/graph/~zod/test]  <br>     q={[lid=[%archive time=~2000.1.1..00.00.01] path=/]}  <br>   ]  <br> }  <br> ~  <br>]  <br>[   by-place  <br> { [ p=[desk=%landscape path=/graph/~zod/test]  <br>     q={[lid=[%archive time=~2000.1.1..00.00.01] path=/]}  <br>   ]  <br> }  <br> ~  <br>]  <br>&gt;   test-half-open-double: took 28744µs  <br>OK      /app/hark-store/test-half-open-double  <br>&gt;   test-half-open-capped: took 17636µs  <br>OK      /app/hark-store/test-half-open-capped  <br>[by-place={[p=[desk=%landscape path=/graph/~zod/test] q={[lid=[%unseen ~] path=/]}]} ~]  <br>[by-place={[p=[desk=%landscape path=/graph/~zod/test] q={[lid=[%unseen ~] path=/]}]} ~]  <br>[   by-place  <br> { [ p=[desk=%landscape path=/graph/~zod/test]  <br>     q={[lid=[%archive time=~2000.1.1..00.00.01] path=/]}  <br>   ]  <br> }  <br> ~  <br>]  <br>[   by-place  <br> { [ p=[desk=%landscape path=/graph/~zod/test]  <br>     q={[lid=[%archive time=~2000.1.1..00.00.01] path=/]}  <br>   ]  <br> }  <br> ~  <br>]  <br>&gt;   test-half-open: took 30218µs</pre><p>This constructivist approach to testing allows that the developer to control the created agent state sufficiently to verify all behaviors as needed. For instance, note the references in the foregoing testing arms to ++bowl. This creates an artificial bowl for the agent state:</p><pre>++ bowl<br>  |=  run=@ud<br>  ^-  bowl:gall<br>  :*  [~zod ~zod %hark-store]<br>    [~ ~]<br>    [run `@uvJ`(shax run) (add (mul run ~s1) *time) [~zod %garden ud+run]]<br>  ==</pre><p>As a result of using run, each subsequent event takes place one second subsequent to the previous event.</p><p>The entire testing framework for %hark-store can be seen in the Urbit repo on the %garden desk at <a href="https://github.com/urbit/urbit/blob/master/pkg/garden/tests/app/hark-store.hoon">/tests/app/hark-store.hoon</a>.</p><p>Many code features are automatically checked: in particular, problems with /sur and /mar files are typically caught up front by Hoon&#39;s static type checker. E.g., if a ?+ or ?- switch statement is missing a possible value, the compiler will raise an exception and refuse to build. This simplifies the state conditions that the developer needs to test in /tests.</p><h3>%pomodoro, A Toy Agent</h3><p><a href="https://github.com/dcSpark/hoon-snippets/tree/main/tutorials/unit-testing/src/base">We include a simple agent to illustrate some tests.</a> %pomodoro is a prototype <a href="https://lifehacker.com/productivity-101-a-primer-to-the-pomodoro-technique-1598992730">productivity</a> agent which mock-emits a signal every 25 seconds (in real Pomodoro terms, minutes) and then every 5 seconds (minutes) in alternating intervals to a list of watchers. (We avoid calling these “subscribers” since that has a special meaning in Urbit; it would be compatible with a slight tweak of this demo but requires a testnet or livenet.)</p><p>Urbit prefers a data-reactive operating model. Peeks can reveal the internal state of the program, pokes can alter or return state, and subscriptions manage the timer notifications. Some parts of this system are more straightforward to test than others; for instance, adding or removing a watcher from a list should be straightforward (and testable). Other kinds of events may produce follow-on effects or gifts much later (and asynchronously), and are more difficult to test with an automated -test thread.</p><h4>Testing %pomodoro</h4><p>To test the behavior of adding a watcher to the list, we need to build a reference state and compare the modified state. For a simple agent, we can do this manually, as below:</p><pre>++ test-add-client<br>  =| run=@ud<br>  =^ move agent (~(on-poke agent (bowl run)) %pomodoro-action !&gt;([%add-client ~zod]))<br>  =+ !&lt;(=state on-save:agent)<br>  %+ expect-eq<br>    !&gt; [%0 watchers={~zod} status=%break]   :: reference state<br>    !&gt; watchers.state                       :: actual state<br>  ==</pre><p>More moves can of course be chained into this (like %hark-store uses) and adding and removing agents can be tested.</p><p>After including this testing file, install %pomodoro for the first time on your development ship by following these steps:</p><ol><li>Clone the hoon-snippetsrepository from github.</li></ol><pre>$ git clone https://github.com/dcSpark/hoon-snippets.git</pre><p>2. Copy in the contents of the %pomodoro project/src directory to the %base desk. E.g.,</p><pre>$ cp -r hoon-snippets/tutorials/unit-testing/src/* zod/base</pre><p>3. Commit the contents.</p><pre>dojo&gt; |commit %base</pre><p>4. You can use the %test agent to make sure it builds properly. (Pending clarification and support for same-named agents across multiple desks.)</p><p>5. Start and poke the agent to test that it works properly.</p><pre>dojo&gt; |start %pomodoro dojo&gt; :pomodoro &amp;pomodoro-action [%start-timer ~]<br>dojo&gt; :pomodoro &amp;pomodoro-action [%add-client ~zod]</pre><p>At any point, you can also check the agent’s state with the %dbug wrapper:</p><pre>dojo&gt; :pomodoro +dbug [%0 watchers={~zod} status=%break]</pre><p>6. The test cases have been installed and are ready to go in /tests. Trigger them using the -test thread.</p><pre>dojo&gt; -test %/tests/app/pomodoro ~</pre><h3>Testing Agents Automatically with CI</h3><p>How do we make sure that our tests are run by the continuous integration system? The testing methods we used above can be automatically triggered from outside of a running ship using the Urbit herb utility.</p><p>The main Urbit repo builds and runs tests on the %base desk automatically. The basic testing method can be invoked:</p><pre>$ nix-build -A urbit-tests</pre><p>The tests are triggered in nix/lib/test-fake-ship.nix. This triggers all unit tests on the %base desk (default %).</p><pre>herb ./pier -d &#39;~&amp;  ~  ~&amp;  %test-unit-start  ~&#39;<br>herb ./pier -d &#39;####-test %/tests ~&#39;<br>herb ./pier -d &#39;~&amp;  ~  ~&amp;  %test-unit-end  ~&#39;</pre><p>For instance, the agent-based unit tests are triggered using herb and the %test Gall agent:</p><pre>herb ./pier -p hood -d &#39;+hood/start %test&#39;</pre><pre>herb ./pier -d &#39;~&amp;  ~  ~&amp;  %test-agents-start  ~&#39;<br>herb ./pier -p test -d &#39;%agents&#39;<br>herb ./pier -d &#39;~&amp;  ~  ~&amp;  %test-agents-end  ~&#39;</pre><p>herb is a Python interface which can carry out some very limited operations from outside your ship. In this case, it pokes the %test agent, which is instrumented for locating and running particular tests. (#### is a herb hack to prevent Bash from expanding the -test as a command-line argument.)</p><p>Urbit uses <a href="https://builtwithnix.org/">Nix</a> to manage its runtime build process. Nix uses a functional deployment model to produce reproducible dependencies throughout the build chain. Nix conceives of tests as artifacts of its process, so you may need to make trivial changes to files if you are attempting to see results having run nix-build -A urbit-tests previously.</p><p>If you are publishing your own software on an Urbit desk, you’ll need to set up CI on your own fork on GitHub. (See the .github folder in your Urbit repo for examples.)</p><h3>Going Further</h3><p>Other considerations in validating your program’s behavior include checking Eyre endpoints, which can use a conventional unit testing framework and Git hooks, for instance, or a continuous integration service.</p><p>We would also like to see future development enable developers to test:</p><ul><li>Anticipated system upgrades (e.g. %zuse version decrements on a desk)</li><li>Interactions of different versions of Gall agents</li></ul><p>As more developers build Urbit tools, including tools which interact with <a href="https://urbitvisor.com/">dcSpark’s Urbit Visor</a> flagship, we can use the existing Urbit testing infrastructure to build reliable agents and applications. Urbit makes it easy to use secure primitives and user state in building Web3 applications, while Urbit Visor makes it easy to bridge Web3 back into Web 2.0.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=82b2631fe20a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dcspark/writing-robust-hoon-a-guide-to-urbit-unit-testing-82b2631fe20a">Writing Robust Hoon — A Guide To Urbit Unit Testing</a> was originally published in <a href="https://medium.com/dcspark">dcSpark</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Authenticate Website Users Using Urbit ID]]></title>
            <link>https://medium.com/dcspark/authenticate-website-users-using-urbit-id-e6dc8c4cb4fa?source=rss-4c7308b488b9------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6dc8c4cb4fa</guid>
            <category><![CDATA[urbit]]></category>
            <category><![CDATA[identity]]></category>
            <category><![CDATA[login]]></category>
            <category><![CDATA[auth]]></category>
            <category><![CDATA[authentication]]></category>
            <dc:creator><![CDATA[Neal Davis]]></dc:creator>
            <pubDate>Fri, 12 Nov 2021 15:03:24 GMT</pubDate>
            <atom:updated>2021-11-12T17:08:25.913Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://urbit.org/">Urbit</a> provides a platform guaranteeing cryptographically secure identity and authentication. If a website can demonstrate that a user owns and operates the ship he claims to, then the website can use that fact of authentication as a direct login and additionally use the Urbit ship as a backend data store. How can a website verify Urbit user identity without having to check the blockchain? The same way many websites use email to identify a user: send a unique message to her inbox and check for a match.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/379/0*ppTd96QquvJhG58s" /></figure><p>The %authenticate-with-urbit-id <a href="https://urbit.org/docs/userspace/gall/gall">Gall</a> agent plays the role of messenger and guardian, ensuring that any user who requests authorization for a particular ship does in fact control that ship. Similar to email-based or SMS-based token authentication, %authenticate-with-urbit-id utilizes direct messages to demonstrate that a secret token passed to the user is also received from the client ship.</p><p>In this tutorial, we will demonstrate how a website can use the %authenticate-with-urbit-id agent to authenticate a user, and we will show how the agent itself is constructed. (The latter assumes some knowledge of Hoon, the Urbit programming language.)</p><h3>Website Authentication</h3><p>A traditional webserver runs applications which interact with a client-side user’s browser on the one hand and a database on the other. The website must authenticate the client browser session using a password, a token, or a cookie. Once this has taken place, future interactions are considered secure (modulo a variety of attacks) since the user is “known” to be who he or she claims to be.</p><p>Urbit acts as a personal server, complementary to a personal client (or web browser). Besides its more exotic features, such as an event log, Urbit affords the owner of a “ship” (or unique instance) a persistent database and a cryptographic identity. Given these facts, a website can use Urbit to both uniquely identify a user and to store user-side data. %authenticate-with-urbit-id facilitates the first of these.</p><p>Since Urbit’s branding as a “personal server” can create some confusion in terms, we need to distinguish the following elements of website authentication:</p><ol><li>Website (classically the server-side application).</li><li>User (classically the client-side application, typically browser-based).</li><li>Website ship (which runs the %authenticate-with-urbit-id agent and authenticates for the website).</li><li>User ship (which is run by the user and needs to be authenticated).</li></ol><p>A website server that wishes to authenticate via %authenticate-with-urbit-id needs to run Urbit and possess a unique Urbit ID of its own. Free transient IDs, called <a href="https://urbit.org/docs/glossary/comet">comets</a>, are available upon startup, but for most purposes a website should prefer to run a stable secure <a href="https://urbit.org/docs/glossary/planet">planet</a> or <a href="https://urbit.org/docs/glossary/moon">moon</a>. We assume that you are able to run a planet or moon ship, but <a href="https://urbit.org/getting-started#next-steps">reach out to the Urbit community</a> if you require assistance in this step. This is the “website ship.”</p><p>Once the website ship is running, the %authenticate-with-urbit-id agent should be installed and started. This exposes two public HTTP endpoints, /~initiateAuth and /~checkAuth. (That is, public to the server machine running the ship at localhost unless configured as a public-facing server.)</p><ul><li>/~initiateAuth accepts a JSON including the unique identity or @p of the user ship to be authenticated and returns a JSON including a generated token. The agent also sets up a subscription to the direct message inbox and watches for the token to be received from the user ship.</li><li>/~checkAuth accepts a JSON including the @p of the user ship and returns a JSON indicating whether the token has been received from the user ship yet. If the user ship has authenticated successfully, the agent also clears the status as a security measure.</li></ul><p>Using curl, one can submit a JSON to the ship and receive a token in reply:</p><pre>curl --header &quot;Content-Type: application/json&quot; \<br>     --request PUT \<br>     --data &#39;{&quot;ship&quot;:&quot;sampel-talled&quot;,&quot;json&quot;:&quot;sampel-palnet&quot;}&#39; \<br>     <a href="http://localhost:8080/~initiateAuth">http://localhost:8080/~initiateAuth</a></pre><p>With this token, the website should have the user send the token (and only the token text itself) to the website ship. (This may be done programmatically, as by Urbit Visor, or manually. This step is underspecified by the agent itself since there are many possible effective solutions.)</p><p>One can also query the authentication status with curl:</p><pre>curl --header &quot;Content-Type: application/json&quot; \<br>     --request PUT \<br>     --data &#39;{&quot;ship&quot;:&quot;sampel-talled&quot;,&quot;json&quot;:&quot;sampel-palnet&quot;}&#39; \<br>     <a href="http://localhost:8080/~checkAuth">http://localhost:8080/~checkAuth</a></pre><p>Any website can use this method to match a client session to authenticated ownership of an Urbit ship.</p><h3>The %authenticate-with-urbit-id Agent</h3><p>The Urbit hosted operating system consists of the core system loop or event log, surrounded by system services each having a characteristic structure. System services provide network events, instrument the filesystem, build software, etc. Gall runs user <em>agents</em>, which act like system daemons and play the role of applications. Every Gall agent has a similar structure which enables the Urbit OS to consistently route events and data between agents.</p><blockquote><em>If you are interested in understanding how Gall works to instrument Urbit’s userspace via agents, we recommend </em><a href="https://github.com/timlucmiptev/gall-guide"><em>~timluc-miptev&#39;s Gall Guide</em></a><em> for a deeper dive.</em></blockquote><p>%authenticate-with-urbit-id is a Gall agent. The source code is available on GitHub at <a href="https://github.com/dcSpark/authenticate-with-urbit-id">dcSpark/authenticate-with-urbit-id</a> under the MIT License. This section of the tutorial walks through the structure and logic of %authenticate-with-urbit-id.</p><p>Every Gall agent has ten arms, frequently with a “helper core” providing other necessary operations.</p><h3>Starting the Agent</h3><p>Upon startup, %authenticate-with-urbit-id registers the two HTTP endpoints and remains available to handle any requests sent via those paths. It also initializes the internal state, which consists of a token map (associative array) from ship name @p to token tape (string) and an authentication map from ship name @p to status as a true/false quantity.</p><pre>++  on-init<br>  ^-  (quip card _this)<br>  ~&amp;  &gt;  &#39;%authenticate-with-urbit-id initialized successfully&#39;<br>  =.  state  [%0 *(map @p tape) *(map @p ?(%.y %.n))]<br>  :_  this<br>  :~  [%pass /bind %arvo %e %connect [~ /&#39;~initiateAuth&#39;] %authenticate-with-urbit-id]<br>      [%pass /bind %arvo %e %connect [~ /&#39;~checkAuth&#39;] %authenticate-with-urbit-id]<br>  ==</pre><p>The HTTP endpoint registration %passes a message to the %eyre server vane connecting each endpoint to %authenticate-with-urbit-id. Any HTTP PUT or GET requests sent to those endpoints are redirected to this Gall agent.</p><h3>Handling Pokes</h3><p>A poke is a one-time command. Pokes are responsible to change agent state. A poke receives a mark (or data structure rule) and a vase (or value wrapped with its type). Here, a switch statement (<a href="https://urbit.org/docs/hoon/reference/rune/wut#-wutlus">?+ wutlus</a>) defaults to the on-poke failure response and otherwise handles only %handle-http-request values generated by Eyre from the website or curl JSON data.</p><pre>++  on-poke<br>  |=  [=mark =vase]<br>  ^-  (quip card _this)<br>  |^<br>  =^  cards  state<br>    ?+    mark  (on-poke:default mark vase)<br>        %handle-http-request<br>    :: ... endpoint handlers here ...<br>    ==<br>  [cards this]</pre><ol><li>If the requested URL matches /~initiateAuth, then the state must be modified to produce the tokens, subscribe to the %dm-inbox (the direct message path), and only then return the token (from ++handle-auth-request).</li></ol><pre>?:  =(url.request.inbound-request &#39;/~initiateAuth&#39;)<br>   ~&amp;  &gt;  &quot;%authenticate-with-urbit-id request hit /~initiateAuth&quot;<br>   =/  st  (get-source-target:main inbound-request)<br>   =/  source  -.st<br>   =/  target  +.st<br>   =^  cards  state  (produce-token:main source target)<br>   =^  cards  state  (subscribe-dms:main source target)<br>   :_  state   ^-  (list card)<br>   %+  weld  cards<br>     %+  give-simple-payload:app:server  id<br>     (handle-auth-request:main source target)</pre><p>A card enables each vane of the Urbit OS to communicate with each other. It is basically a discrete event consisting of destination and necessary data. We see a few cards in the %authenticate-with-urbit-id code, such as:</p><pre>[%give %fact ~[/tokens] [%atom !&gt;(tokens.state)]] [%give %fact ~[/status] [%atom !&gt;(status.state)]]</pre><p>which update the state tokens and status maps for the requested ship; and</p><pre>[%pass /graph-store %agent [our.bowl %graph-store] %watch /updates]</pre><p>which subscribes to the %dm-inbox to check for incoming messages. Urbit prefers a data-reactive model in which subscriptions are activated and events are only processed when updates are received.</p><p>These are ultimately returned by the standard ++on-XXX arms of the agent to the Urbit OS, which uses them to update the event log and thereby the system state.</p><p>A series of functions in the helper core produce a list of cards to be effected on the state. For instance, ++handle-auth-request is located in the helper core main:</p><pre>++  handle-auth-request<br>   |=  [source=<a href="http://twitter.com/p">@p</a> target=<a href="http://twitter.com/p">@p</a>]<br>   ^-  simple-payload:http<br>   =,  enjs:format<br>   =/  auth  (~(gut by status.state) target %.n)<br>   =/  token  (~(got by tokens.state) target)<br>   %-  json-response:gen:server<br>   %-  pairs<br>   :~<br>     [%source [%s (scot %p our.bowl)]]<br>     [%target [%s (scot %p target)]]<br>     [%token [%s (crip token)]]<br>   ==</pre><p>This gate wraps the target ship and the corresponding token in a JSON response which will be emitted after the poke concludes.</p><p>The target ship was obtained from a JSON PUT input of the form</p><pre>{&quot;ship&quot;:&quot;~sampel-talled&quot;, &quot;json&quot;:&quot;sampel-palnet&quot;}</pre><p>(We are intentionally being inconsistent with the tilde ~ here! We compensate for this below.)</p><p>JSON parsing in Hoon requires two stages: one step which yields a tagged data structure and another which extracts the particular values of interest. These take place in the helper arm ++get-source-target.</p><pre>++  get-source-target<br>   |=  [req=inbound-request:eyre]<br>   ^-  [<a href="http://twitter.com/p">@p</a> <a href="http://twitter.com/p">@p</a>]<br>   =/  payload  (de-json:html `<a href="http://twitter.com/t">@t</a>`+511:req)<br>   ?~  payload  !!<br>   =/  payload-array  u:+.payload<br>   =/  st  u:+:(req-parser-ot payload-array)<br>   =/  source-t  (trip -.st)<br>   =/  source<br>     ?:  =(&#39;~&#39; (snag 0 source-t))  u:+:`(unit <a href="http://twitter.com/p">@p</a>)`(slaw %p (crip source-t))<br>     u:+:`(unit <a href="http://twitter.com/p">@p</a>)`(slaw %p (crip (weld &quot;~&quot; source-t)))<br>   =/  target-t  (trip +.st)<br>   =/  target<br>     ?:  =(&#39;~&#39; (snag 0 target-t))  u:+:`(unit <a href="http://twitter.com/p">@p</a>)`(slaw %p (crip target-t))<br>     u:+:`(unit <a href="http://twitter.com/p">@p</a>)`(slaw %p (crip (weld &quot;~&quot; target-t)))<br>   [source target]</pre><p>Here, the parser yields payload from de-json:html. This is the entire JSON string itself converted into the Hoon internal JSON representation, which must be processed and checked to make sure that the result isn&#39;t ~ (failure to parse).</p><pre>=/  payload  (de-json:html `<a href="http://twitter.com/t">@t</a>`+511:req)<br>?~  payload  !!<br>=/  payload-array  u:+.payload</pre><p>The original JSON</p><pre>{&quot;ship&quot;:&quot;sampel-talled&quot;, &quot;json&quot;:&quot;sampel-palnet&quot;}</pre><p>is converted into Hoon representation</p><pre>[ ~<br>   [ %o<br>     p={ [<br>           p=&#39;ship&#39;<br>           q=[%s p=&#39;sampel-palnet&#39;]<br>         ]<br>         [<br>           p=&#39;json&#39;<br>           q=[%s p=&#39;~sampel-talled&#39;]<br>         ]<br>       }<br>   ]<br>]</pre><p>with type tags on each value indicating the source JSON type.</p><p>The Hoon data structure is then fed into a reparser, which must be constructed for the particular expected entries:</p><pre>++  req-parser-ot<br>   %-  ot:dejs-soft:format<br>   :~  [%ship so:dejs-soft:format]<br>       [%json so:dejs-soft:format]<br>   ==</pre><pre>=/  st  u:+:(req-parser-ot payload-array)</pre><p>In this case, the only information required by the arm is the user ship target and the website ship source. These are extracted from the Airlock-specified JSON structure.</p><p>However, the Urbit API is not completely consistent about specifying tildes in front of ship names! So we check for both possibilities (present and absent) and parse accordingly, using ++slaw to convert the result text into a @p identity.</p><pre>=/  source-t  (trip -.st)<br>=/  source<br>   ?:  =(&#39;~&#39; (snag 0 source-t))<br>   u:+:`(unit <a href="http://twitter.com/p">@p</a>)`(slaw %p (crip source-t))<br>   u:+:`(unit <a href="http://twitter.com/p">@p</a>)`(slaw %p (crip (weld &quot;~&quot; source-t)))</pre><p>The output of this gate passes through two other gates which ultimately yield a list of cards.</p><p>Back to the big picture: if the URL request does not match /~initiateAuth then it <em>must</em> match /~checkAuth (else an assertion error is raised by <a href="https://urbit.org/docs/hoon/reference/rune/wut#-wutgar">?&gt; wutgar</a>).</p><pre>?&gt;  =(url.request.inbound-request &#39;/~checkAuth&#39;)<br>   ~&amp;  &gt;  &quot;%authenticate-with-urbit-id request hit /~checkAuth&quot;<br>   =/  st  (get-source-target:main inbound-request)<br>   =/  source  -.st<br>   =/  target  +.st<br>   =/  status  (~(gut by status.state) target %.n)<br>   ~&amp;  &gt;  status<br>   =^  cards  state  (clear-auth:main source target)<br>   :_  state<br>   %+  weld  cards<br>     %+  give-simple-payload:app:server  id<br>     (handle-auth-check:main source target status)</pre><p>As before, we have two chains of events which need to resolve in order. We need to check the state and then clear it if it has been set. We also need to return the status of the check. First, the state must be checked and the reply JSON formed:</p><pre>++  handle-auth-check<br>   |=  [source=<a href="http://twitter.com/p">@p</a> target=<a href="http://twitter.com/p">@p</a> status=?(%.y %.n)]<br>   ^-  simple-payload:http<br>   =,  enjs:format<br>   %-  json-response:gen:server<br>   %-  pairs<br>   :~<br>     [%source [%s (scot %p our.bowl)]]<br>     [%target [%s (scot %p target)]]<br>     [%status [%s ?:(status &#39;true&#39; &#39;false&#39;)]]<br>   ==</pre><p>The token must then be cleared in sequence if it is active. (Notably, the relative isolation of Gall agent arms from each other here makes it necessary to extricate the information more than once from the active request.)</p><pre>++  clear-auth<br>   |=  [source=<a href="http://twitter.com/p">@p</a> target=<a href="http://twitter.com/p">@p</a>]<br>   ^-  (quip card _state)<br>   =/  auth-status  (~(gut by status.state) target %.n)<br>   ::  only clear the tokens if the status is %.y, else it&#39;s too soon<br>   =.  tokens.state  ?:(auth-status (~(del by tokens.state) target) tokens.state)<br>   =.  status.state  ?:(auth-status (~(del by status.state) target) status.state)<br>   :_  state<br>   :~  [%give %fact ~[/tokens] [%atom !&gt;(tokens.state)]]<br>       [%give %fact ~[/status] [%atom !&gt;(status.state)]]<br>   ==</pre><p>%authenticate-with-urbit-id is a straightforward agent which demonstrates internal state updates, API endpoint exposure, graph-store subscriptions, and card structure and order.</p><p>Since every DM is examined for content, we recommend that this agent run on a designated ship (such as a moon), rather than on a ship used for social purposes.</p><p>In closing, it is worth considering how one would construct an agent that can receive more than one fact from the input.</p><ol><li>One can wrap the JSON in the json entry of the request JSON. This may require escaping internal JSON elements and gets messy to construct.<br>&#39;{&quot;ship&quot;:&quot;sampel-talled&quot;, &quot;json&quot;:&quot;{\&quot;ship\&quot;:\&quot;sampel-palnet\&quot;, \&quot;foo\&quot;:\&quot;bar\&quot;}&quot;)&#39;</li><li>One can pass in more arguments to the request JSON.<br>&#39;{&quot;ship&quot;:&quot;sampel-talled&quot;, &quot;json&quot;:&quot;sampel-palnet&quot;, &quot;foo&quot;:&quot;bar&quot;}&#39;</li></ol><p>In either case, additional reparsing will need to be carried out to extract the target values.</p><p>— — — — <br>Checkout the code for Authenticate With Urbit ID on github: <a href="https://github.com/dcSpark/authenticate-with-urbit-id">https://github.com/dcSpark/authenticate-with-urbit-id</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6dc8c4cb4fa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dcspark/authenticate-website-users-using-urbit-id-e6dc8c4cb4fa">Authenticate Website Users Using Urbit ID</a> was originally published in <a href="https://medium.com/dcspark">dcSpark</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>