<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Snapdocs Product &amp; Engineering Blog - Medium]]></title>
        <description><![CDATA[Powering Homeownership - Medium]]></description>
        <link>https://medium.com/snapdocs-product-blog?source=rss----868a6161c805---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Snapdocs Product &amp;amp; Engineering Blog - Medium</title>
            <link>https://medium.com/snapdocs-product-blog?source=rss----868a6161c805---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 04:28:11 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/snapdocs-product-blog" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Sidekiq Scheduled Job Deduplication (Without Killing Redis)]]></title>
            <link>https://medium.com/snapdocs-product-blog/sidekiq-scheduled-job-deduplication-without-killing-redis-d3a6d3084032?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/d3a6d3084032</guid>
            <category><![CDATA[redis]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[sidekiq]]></category>
            <category><![CDATA[sidekiq-unique-jobs]]></category>
            <dc:creator><![CDATA[John Langewisch]]></dc:creator>
            <pubDate>Mon, 21 Sep 2026 17:57:29 GMT</pubDate>
            <atom:updated>2026-09-22T02:03:28.900Z</atom:updated>
            <content:encoded><![CDATA[<p>At Snapdocs, many of our services are Rails applications running Sidekiq to handle asynchronous jobs. As our platform has scaled, our primary service’s Sidekiq instance has grown to process millions of jobs per day.</p><p>Often, in the early stages of a Rails product, it can be easy to use Sidekiq as a platform for scheduling long-term jobs. Whether or not doing so is a good idea or not is a separate question for a different article (I’m not even sure we’ve landed on a definitive answer there), but at the time of this writing, our Sidekiq instance had evolved to have a steady state of up to ~80,000 scheduled jobs at any given time.</p><p>We were discovering that our Redis instance was consistently operating at a higher CPU utilization than expected for a supposedly fast memory store, and this was causing system performance and stability issues. Since Redis is single threaded, adding more cores and memory wouldn’t solve our problem. And since Sidekiq depends on immediately consistent responses, adding additional Redis instances is not supported.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TEuLBZ_pZRX0LszirKMBPQ.png" /><figcaption>Redis CPU Utilization</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*z9NtfxUNqjzzmyOTh9nycg.png" /><figcaption>Redis Errors</figcaption></figure><p>Now, a Redis-backed Sidekiq is normally known to be a fairly performant architecture as Redis can process the type of data Sidekiq uses very quickly. Sidekiq’s hot path only ever asks Redis for things Redis can answer in constant or logarithmic time:</p><ul><li>Enqueue is LPUSH: O(1)</li><li>Dequeue is BRPOP: O(1)</li><li>Scheduling a future job is ZADD: O(log N)</li><li>Promoting a due job is a small Lua script using ZRANGE then ZREM: O(log N)</li></ul><p>That is why a large number of scheduled jobs is not, by itself, a problem. Sidekiq typically never reads the ScheduledSet as a whole.</p><p>The trouble starts when you ask Redis a question Sidekiq’s data model has no index for. In particular, the way Sidekiq stores the data in its ScheduledSet results in very slow searches for jobs using ZSCAN-type methods in Redis. An item in the ScheduledSet consists of the entire serialized job payload, scored by the time when it will fire. There is no indexing on class or JID, so any question of the form “is there a job for class X with args Y?” can only be answered by walking every member and substring-matching the payload.</p><p>Our code had several instances where we were trying to remove future jobs for one reason or another (a signing appointment changed, adjust the time reminder emails are sent), and the way we were accomplishing this was by scanning the ScheduledSet for a particular job id or class of job with certain parameters, and then deleting it. This pattern resulted in a LOT of ZSCANs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LaBNysqfI8dHq-PmpTMPCQ.png" /><figcaption>Redis Commands: Many ZSCANs</figcaption></figure><p>The observant reader will notice the almost exact correlation between the number of ZSCAN operations and the CPU utilization of the Redis instance. There were a handful of primary culprits for this behavior:</p><ul><li><strong>Sidekiq::ScheduledSet#scan </strong><br>This was being used to find a job by class/params, resulting in an O(N) search before deleting with an O(log N) ZREM.</li><li><strong>Sidekiq::ScheduledSet#find_job(jid)</strong><br>Similarly, this was being used to find a job by JID and was O(N) (although one might expect it to run in constant time, it sadly does not, and the maintainers even warn against its usage).</li><li><strong>sidekiq-unique-jobs</strong><br>This is a popular gem that offers Sidekiq job deduplication. Unfortunately, some of the conflict strategies use a similar scanning behavior (technically a paged ZRANGE loop in a lua script) that does not scale well with tens of thousands of jobs in the ScheduledSet.<br>In addition, the gem configures an “Orphan Reaper” that runs every 10 minutes by default, checking if the job lock digests the gem creates are still in use. To do so, it ZSCANs the ScheduledSet and RetrySet for each digest. It only has 10 seconds to do so by default, and stops if it exceeds that time. For our ScheduledSet size, the Orphan Reaper was never getting through the set in time, and was essentially doing nothing… and using a LOT of CPU to do it…</li></ul><p>Okay, ZSCANs eat up a lot of CPU when we have a lot of scheduled jobs. What’s the fix?</p><p>Like I’d said earlier, Redis and Sidekiq are very good at executing a LOT of jobs. What if, when we scheduled a job, rather than deleting duplicate jobs we no longer wanted to run, we simply left them there, but ensured they did nothing?</p><h3>Enter “LastScheduledWins”</h3><p>LastScheduledWins is a small module that deduplicates execution of a job for a particular job class and params (i.e. the last job scheduled runs). The main idea is:</p><ol><li>When you schedule a job, track the JID of the job for that job’s class and params in Redis</li><li>When you run a job for a particular class and params, check if the current job’s JID is the tracked JID. If not, don’t do anything.</li></ol><p>I’m going to cover the highlights of it here, and then include the full source for reference and to include all the nuanced nitty gritty details.</p><p>The perform_at method:</p><pre>def perform_at(scheduled_at, job_class, *args)<br>  jid = SecureRandom.hex(12)<br><br>  begin<br>    store_jid(redis_key(job_class, args), jid, ttl_seconds: ttl_for(job_class, scheduled_at))<br>  rescue Redis::BaseError, RedisClient::Error, ConnectionPool::TimeoutError =&gt; e<br>    Rails.logger.warn(&quot;LastScheduledWins: store_jid failed&quot;)<br>  rescue StandardError =&gt; e<br>    Rails.logger.error(&quot;LastScheduledWins: store_jid unexpected error&quot;)<br>  end<br><br>  job_class.set(jid: jid).perform_at(scheduled_at, *args)<br><br>  jid<br>end</pre><p>This is the entry point for scheduling a Sidekiq job that is deduplicated by LastScheduledWins. Note that it precomputes and assigns a JID (the same way Sidekiq does) to avoid race conditions storing the JID and scheduling the job. We also opted to fail open on error (running the job if something fails storing the tracking JID). The method is called like:</p><pre>LastScheduledWins.perform_at(<br>  5.minutes.from_now,<br>  MyJob,<br>  param1, param2, etc.<br>)</pre><p>The redis_key (the “key” that will point to the job’s JID “value” that we chose to store in Redis) is computed like:</p><pre>def redis_key(job_class, args)<br>  key_args = if job_class.respond_to?(:last_scheduled_wins_key_args)<br>    job_class.last_scheduled_wins_key_args(args)<br>  else<br>    args<br>  end<br><br>  args_key = Digest::SHA1.hexdigest(canonicalize(key_args).to_json)<br><br>  &quot;#{KEY_PREFIX}:#{job_class}:#{args_key}&quot;<br>end</pre><p>The last_scheduled_wins_key_args allows the job to define a method that determines which params make the job execution unique for deduplication.</p><p>The last_scheduled_wins_retry_window_seconds used by ttl_for allows the job to customize the tracking JID TTL to ensure it outlasts the job’s retry schedule. If the tracking JID expires before a retry happens, the retry fails open and runs, which could result in duplicate executions.</p><pre>def ttl_for(job_class, scheduled_at)<br>  seconds_until_fire = [scheduled_at.to_i - Time.current.to_i, 0].max<br><br>  retry_window =<br>    if job_class.respond_to?(:last_scheduled_wins_retry_window_seconds)<br>      job_class.last_scheduled_wins_retry_window_seconds.to_i<br>    else<br>      DEFAULT_RETRY_WINDOW_SECONDS<br>    end<br><br>  seconds_until_fire + retry_window<br>end</pre><p>Next up, the store_jid method:</p><pre>def store_jid(key, jid, ttl_seconds:)<br>  # Atomic TTL-read + SET to prevent concurrent writers from overwriting each other&#39;s TTLs<br>  lua = &lt;&lt;~LUA<br>    local existing_ttl = redis.call(&#39;TTL&#39;, KEYS[1])<br>    local ttl = math.max(tonumber(ARGV[2]), existing_ttl, 1)<br>    redis.call(&#39;SET&#39;, KEYS[1], ARGV[1], &#39;EX&#39;, ttl)<br>  LUA<br>  <br>  attempts = 0<br>  begin<br>    attempts += 1<br>    Sidekiq.redis do |redis|<br>      redis.call(&#39;EVAL&#39;, lua, &#39;1&#39;, key, jid, ttl_seconds.to_s)<br>    end<br>  rescue Redis::BaseError, RedisClient::Error, ConnectionPool::TimeoutError<br>    raise if attempts &gt;= STORE_JID_MAX_ATTEMPTS<br>    Kernel.sleep((STORE_JID_RETRY_BACKOFF_MS * attempts) / 1000.0)<br>    retry<br>  end<br>end</pre><p>This is probably the most interesting bit since it does some fancy (although still relatively simple and performant) lua code to ensure writing a tracking JID is an atomic redis operation.</p><p>Finally, the superseded? method. This is what the job will use to determine whether or not its JID still “owns” the execution for that class and params:</p><pre>def superseded?(job_class, jid, *args)<br>  return false if jid.blank?<br><br>  stored = Sidekiq.redis { |redis| redis.get(redis_key(job_class, args)) }<br>  return false if stored.blank?<br><br>  stored != jid<br>end</pre><p>The most important note here is that we decided to fail open (i.e. when a tracking JID does not exist, run the worker rather than requiring the JID to match). This made the migration easier in our specific case. However, that decision did have implications on how we handled TTLs of the tracking JIDs.</p><p>Now, let’s take a look at what the Sidekiq job needs to do to implement this:</p><pre>class MyJob<br>  include Sidekiq::Job<br><br>  def perform(*args)<br>    return if SidekiqHelper::LastScheduledWins.superseded?(self.class, jid, *args)<br>    # the rest of the job …<br>  end<br>end</pre><p>That’s it! Now, when a job is scheduled with LastScheduledWins, its JID is stored to show it “owns” the execution. If any other job with the same class and params is scheduled, it overwrites with its own JID. Then, the very first thing the job does is check if its JID still “owns” the execution, and if not, it does nothing.</p><p>Note that some of the implementation details are specific to our use case:</p><ul><li>We could tolerate a long TTL because we had more than enough memory to store a large number of tracking JIDs to allow them to outlive the jobs they would supersede. Having a long TTL (vs. trying to clean up tracking JIDs after job execution) simplified some things.</li><li>We opted for an explicit API for this module to avoid changing signatures of Sidekiq jobs that would require a more difficult migration. I also prefer seeing “weird” things front and center rather than hiding them behind some magic or a class hierarchy. An alternative implementation of this could use a more declarative “magical” configuration/API if desired.</li><li>Migrating a job sometimes meant we needed to backfill the tracking JID for the previously scheduled jobs before switching it to LastScheduledWins.</li><li>Since we used Flipper to roll out this migration, we needed to “untrack” the jobs in the “flipper off” path in case of a rollback.</li><li>Sidekiq-unique-jobs provides other conflict strategies where the LastScheduledWins model didn’t apply. In those other cases we used variations of job idempotency as well as the fantastic <a href="https://github.com/leandromoreira/redlock-rb">redlock</a> gem to lock jobs during execution. We specifically wanted to move completely off of sidekiq-unique-jobs since its Orphan Reaper and other patterns weren’t scaling with the number of scheduled jobs we had, but your use case might differ.</li></ul><h3>The most interesting part: Our results!</h3><p>We completely eliminated all ZSCANs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D-TDY_FOxVhrpzvY5FK8mQ.png" /><figcaption>Redis Commands During LastScheduledWinsMigration: Far Fewer ZSCANs</figcaption></figure><p>Our Redis CPU utilization went from spiking into the 90% range, to hovering in the low single digit percentages, which, in my years of experience optimizing various things, is a pretty wildly good optimization result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2XoyCzuqMrgUZ_vVU5Mwuw.png" /><figcaption>Redis CPU Utilization During LastScheduledWins Migration</figcaption></figure><p>Our Redis instance is much happier for it, throwing fewer than 25 errors since the migration was complete:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZlV3jyOqGARr51GKeEuPSg.png" /><figcaption>Redis Errors During LastScheduledWins Migration</figcaption></figure><p>We’ve shown the idea of “no-op rather than delete scheduled jobs” is wildly more performant for Redis. While we’re providing our implementation, this could be done in a number of different ways, with various tradeoffs on reliability, visibility, error handling, etc.</p><p>Most importantly, our overall system has seen drastically increased stability, especially at the end of the month, which I’m sure is keeping our customers and on-call engineers happy as well.</p><p>Thanks for reading! Keep making things better!</p><p>Here is a link to our LastScheduledWins code (note it is slightly different than what was discussed in the interest of keeping things more generically applicable):</p><p><a href="https://github.com/snapdocs/last-scheduled-wins">https://github.com/snapdocs/last-scheduled-wins</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d3a6d3084032" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/sidekiq-scheduled-job-deduplication-without-killing-redis-d3a6d3084032">Sidekiq Scheduled Job Deduplication (Without Killing Redis)</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[2025: A Year of AI Coding]]></title>
            <link>https://medium.com/snapdocs-product-blog/2025-a-year-of-ai-coding-6332bd92067a?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/6332bd92067a</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-coding-assistant]]></category>
            <category><![CDATA[ai-coding]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Steve Gore]]></dc:creator>
            <pubDate>Mon, 15 Dec 2025 18:01:27 GMT</pubDate>
            <atom:updated>2025-12-15T18:01:25.994Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OX_ILdZ84U0hff9am5av5A.png" /><figcaption>And of AI image generation</figcaption></figure><p>It is hard to believe it has been less than a year since I was first introduced to autonomous coding agents.</p><p>On January 22, 2025, during an engineering offsite in San Francisco, Patrick Mee (our VP of Engineering) demoed Cline. He didn’t just use it to autocomplete a line of code; he showed it writing an implementation, generating tests, running those tests, failing, and then iterating to get them passing — all without manual intervention.</p><p>I was instantly hooked.</p><p>Two days later, we signed up with <a href="https://openrouter.ai/">OpenRouter</a> to provide every engineer at Snapdocs with access to the best models available. When <a href="https://code.claude.com/docs/en/overview">Claude Code</a> launched on February 24th, we encouraged the team to explore. We didn’t mandate specific tools; we simply asked engineers to find workflows that improved their productivity and happiness.</p><p>The tooling landscape changed constantly. I personally experimented with VS Code extensions like <a href="https://cline.bot/">Cline</a>, <a href="https://kilo.ai/">Kilo</a>, and <a href="https://code.visualstudio.com/docs/copilot/overview">Copilot</a>. While many tools were impressive, I found I was getting the most consistent results with <a href="https://roocode.com/">Roo</a>, which has become my daily driver for IDE-based work. For non-IDE tasks, like the majority of our engineering team, I rely heavily on Claude Code, with some great results coming from <a href="https://www.warp.dev/">Warp</a> as well.</p><h3>The Evolution of Usage</h3><p>More interesting than the specific tools, however, is how our interaction with them has matured. Looking back at the last ten months, I see a distinct evolution in how we collaborate with AI:</p><ol><li><strong>Non-Core Activities:</strong> We started safely. The initial use cases were high-tedium, low-risk tasks: generating boilerplate unit tests, updating stale documentation, or converting file formats. It was about offloading the “chores” of engineering.</li><li><strong>Clearly Scoped Implementation:</strong> As trust grew, we moved to feature work. We found that if a task was clearly defined — think a single user story with acceptance criteria — the agent could handle the implementation and testing end-to-end.</li><li><strong>Designs and Refactoring:</strong> The usage then moved upstream. We started using agents to write Technical Design Docs and implementation plans <em>before</em> writing code. We also found them effective at providing ideas for refactoring mature components of our code base.</li><li><strong>Automated Code Reviews:</strong> We began using agents as a “first pass” reviewer. They aren’t a replacement for human insight, but they provide immediate feedback, catching logical errors and edge cases so the human reviewer can focus on architecture and maintainability.</li></ol><h3>Current Experiments</h3><p>This leads to where we are today: moving from established workflows into new, experimental territory where the tooling is just starting to emerge:</p><ol><li><strong>Handling Ambiguity:</strong> We are beginning to test how agents can handle larger, more ambiguous features — working at the Epic level rather than the Story level. We are seeing frameworks appear to support this (like <a href="https://github.com/github/spec-kit">spec-kit</a> or <a href="https://github.com/GWUDCAP/cc-sessions">cc-sessions</a>), and we’re actively experimenting with how to effectively structure these larger requests without getting lost.</li><li><strong>Parallelization:</strong> We’re also starting to explore the concept of “agent orchestration” — launching simultaneous tasks across local and cloud agents. The tooling here is still new, but I think the future is a shift from watching a single agent work to managing a fleet of them.</li></ol><h3>The Future: Operations and Maintenance</h3><p>What I am interested in next is applying this intelligence to the operational side of software. When a system issue occurs, I want an agent that can safely analyze our logs, infrastructure, and code to triage what happened. It needs to spot patterns across distributed services and piece together incomplete information. However, any operational agent must work strictly within our privacy and compliance requirements, ensuring customer data never leaves our secure perimeter.</p><h3>A Partner, Not a Replacement</h3><p>This journey hasn’t been without frustration. I’ve watched agents stubbornly delete valid tests just to make a build pass. I’ve seen them hallucinate new libraries rather than using our existing shared code.</p><p>These moments reinforce my belief that for enterprise-level software, these are tools to support professional engineers, not a force to replace them. They amplify our ability to solve problems, but they still require an experienced hand on the wheel.</p><p>And when it works, it brings us back to why we write code in the first place. By stripping away the tedium, these agents allow us to focus on the fun parts of engineering: solving hard problems and iterating at the speed of thought. Giving us the ability to instantly visualize five different implementations to weigh trade-offs. And the power to go from a vague idea to a working prototype in a single day.</p><p>Ultimately, these agents haven’t replaced the joy of building software — they’ve unlocked it. I am excited to see how these workflows evolve over the next month, let alone the next twelve. If 2025 was the year we adopted the tools, 2026 will be the year we master them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6332bd92067a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/2025-a-year-of-ai-coding-6332bd92067a">2025: A Year of AI Coding</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Tool Sprawl to Standardized Labeling: Boosting Data Science Efficiency with Turkle]]></title>
            <link>https://medium.com/snapdocs-product-blog/from-tool-sprawl-to-standardized-labeling-boosting-data-science-efficiency-with-turkle-edd6cffa4705?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/edd6cffa4705</guid>
            <category><![CDATA[data-labeling]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[mechanical-turk]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Steve Gore]]></dc:creator>
            <pubDate>Mon, 01 Dec 2025 15:28:38 GMT</pubDate>
            <atom:updated>2025-12-01T15:28:38.075Z</atom:updated>
            <content:encoded><![CDATA[<p>For over a decade, Snapdocs has been solving customer problems by leveraging different data science techniques to understand and model mortgage documents. This is an exponential problem, as documents vary significantly by loan program, lender, title company, and state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h1_LfjzKWDZxuZ9vgM8G1A.png" /><figcaption>A data science labeling pipeline</figcaption></figure><p>To power our document intelligence we need a pipeline through which we can label a large set of documents, quickly. Labeling is the process of having highly skilled human reviewers tag specific pieces of information — like identifying a ‘Total Loan Amount’ field or classifying a ‘Promissory Note’ — to create our critical “ground truth.” The ground truth produced by this pipeline is used to build new models as customers onboard and maintain state of the art capacity in our already deployed models as documents change.</p><h3>The Challenge of Custom Tooling</h3><p>Our traditional approach has been engaging our software engineering team to build custom tooling for each labeling use case. This tooling was custom-designed for our exact workflow and enabled our success for many years.</p><p>However, this approach had some significant downsides. As we uncovered new use cases (Funding Quality Control, Title Quality Control, or CD Balancing), we built new custom tools.</p><ul><li>These tools took time to build. While they were being built, our team relied on slow, cumbersome, manual processes to label data for our emerging products.</li><li>Maintaining and updating the custom tools required continual investment from our engineering team, competing with user-facing features to improve our product experiences.</li><li>We were creating tool sprawl with similar, but not-quite-consistent, user experiences and functionality.</li></ul><h3>A New Goal and a Tipping Point</h3><p>At the beginning of 2025, we set a goal of improving the efficiency of our data science team while maintaining the quality our customers rely on. A significant piece of that was around our labeling process and came to a head when two separate teams proposed building separate labeling tools for the same document: a Closing Disclosure.</p><p>As we discussed this goal, a few key ideas emerged:</p><ol><li>We wanted a solution that could allow our data science team to launch new workflows and make updates to existing workflows themselves.</li><li>We wanted a solution that allowed less experienced team members to easily participate in the labeling process.</li><li>Our approach of labeling all of the data relevant to a use case for a full page at the same time was not scalable. We needed to shift to a standardized framework that would allow labeling a single piece of data for any use case for any page.</li></ol><p>We also discussed leveraging third-party services. The most obvious is Amazon’s Mechanical Turk. For those unfamiliar, Mechanical Turk is a crowdsourcing marketplace that allows businesses to farm out micro-tasks — like labeling an image or transcribing a short audio clip — to a large, distributed workforce.</p><p>While we ruled it out due to our strict security and compliance requirements, it led to our key insight: what if we launched an internal-only Snapdocs Mechanical Turk?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FWoUueZa51qlSc-DZTgg7w.png" /><figcaption>A simple labeling task implemented in Turkle</figcaption></figure><h3>Finding and Implementing Turkle</h3><p>We evaluated a few open-source, Mechanical Turk-like services and landed on <a href="https://github.com/hltcoe/turkle">Turkle</a>.</p><p>Turkle allows users to create interfaces for labeling data using a small amount of HTML, CSS, and JavaScript with simple templating. Data is uploaded in a CSV, the interface with the data is reviewed by a human, and answers are submitted. We added a small extension to allow Turkle to securely access images of our documents, and we began to roll it out.</p><p>Our successes came quickly.</p><h3>Success 1: Rapid Prototyping</h3><p>Our first success came when data scientists needed labeling for new experiments and proofs of concept. Instead of dealing with multiple spreadsheets, numerous manual steps, and lengthy reviews, we could create a new labeling interface in an hour and have results by the end of the day. It’s worth noting that AI coding tools are very effective at creating the simple HTML, CSS, and JavaScript pages Turkle requires.</p><h3>Success 2: A New Production Workflow</h3><p>Our next success came when we launched the labeling tool for a new production use case in Turkle instead of building a custom internal tool. This proved the end-to-end process of automatically ingesting production data, providing a high-quality labeling tool, and integrating our analytics platform — at scale and with customer SLAs. Even for the first time doing this, it took roughly 20% of the time and effort compared to building a custom reviewer.</p><h3>Success 3: Migrating and Retiring Code</h3><p>Our most recent success has been successfully migrating from a custom internal tool to a workflow built in Turkle. Changing existing processes and tools while in production and at scale has been more complicated, but it demonstrates the value of this approach. Our engineering team has been able to remove over 4,000 lines of code that supported this internal tool. Our data science team is now able to split work between expert (classifying documents, labeling missing data) and non-expert tasks (labeling extracted data) further increasing our efficiency.</p><h3>The Future</h3><p>While implementing Turkle has certainly improved our efficiency, the biggest value is in increasing our velocity. We see a tremendous amount of opportunity in future use cases and technologies, and we’re now better equipped to rapidly explore and iterate on ideas.</p><p>As we continue to expand our usage of Large Language Models (LLMs) alongside our traditional models, the need for high-quality, targeted datasets only grows. This platform is perfectly suited for generating the fine-tuning data, validation sets, and human-preference feedback that modern AI systems demand. Instead of waiting weeks for a new labeling tool, our data science team can now spin up an experiment in an hour to test a new model or a creative new prompt.</p><p>This creates a powerful flywheel for innovation. Faster labeling leads to faster model iteration, which leads to better products and new solutions for our customers. It allows our data scientists to ask more “what if” questions and get answers quickly. At the same time, it frees our engineering team to focus on building robust, customer-facing product experiences rather than maintaining a suite of internal tools.</p><p>We’re excited to continue leveraging Turkle and this flexible framework to accelerate our R&amp;D cycle, supporting new and innovative solutions for the mortgage industry.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=edd6cffa4705" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/from-tool-sprawl-to-standardized-labeling-boosting-data-science-efficiency-with-turkle-edd6cffa4705">From Tool Sprawl to Standardized Labeling: Boosting Data Science Efficiency with Turkle</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Taking the Leap… in a Pandemic?]]></title>
            <link>https://medium.com/snapdocs-product-blog/taking-the-leap-in-a-pandemic-f792b3c670c7?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/f792b3c670c7</guid>
            <category><![CDATA[onboarding]]></category>
            <category><![CDATA[pandemic]]></category>
            <category><![CDATA[culture]]></category>
            <category><![CDATA[job-search]]></category>
            <category><![CDATA[remote-working]]></category>
            <dc:creator><![CDATA[ndkerby]]></dc:creator>
            <pubDate>Tue, 21 Dec 2021 17:25:40 GMT</pubDate>
            <atom:updated>2021-12-21T17:25:40.478Z</atom:updated>
            <content:encoded><![CDATA[<p>Like a scene in a Hollywood movie, I’d historically worked in an office. I’m talking about those giant buildings with elevators, conference rooms, and the endless rows of cubicles. That place you race to in bumper to bumper traffic as you are outpaced by a guy with a walker. Once you finally arrive, you take a trip to the 5th floor because you know that is where the good coffee is kept. This place may even have social circles that resemble high school and that one coworker with a red Swingline stapler under lock and key. After finding your niche, there are some fun advantages to an otherwise bland routine — like going out to lunch and bonding over the recent episode of “What did leadership decide now?” All jokes aside, there can be some big benefits to being in an office, such as being more easily seen by leadership. You are also able to more easily build relationships with your team, walk up to a white board to collaborate, and maybe even benefit from the occasional free donut. Then, it all changed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gMYDfJk_6j_pRjhX" /><figcaption>Photo by <a href="https://unsplash.com/@kate_sade?utm_source=medium&amp;utm_medium=referral">kate.sade</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>But then… COVID</h3><p>Overnight, we hunkered down for the pandemic and I traded in my driving commute for one where I could walk. My office now sits next to my home kitchen and my officemates lack all forms of office etiquette — but their floppy ears and wagging tails make up for it. Like many of my coworkers, I had occasionally worked from home before, but this was different. Our transition to a full remote culture was not a smooth one. There was no option to find a whiteboard to draw this out together or decide to chat about it when we’re back in the office tomorrow. Luckily, we had worked together for a while so we were able to find solutions to our hiccups — but that couldn’t be said for everything. Beyond the team, our regularly scheduled program of “What did leadership decide now?” went on hiatus and we eagerly waited to hear if there would be a season renewal. By the time we finally heard the “good news”, my worry was replaced with a new one. This sinking feeling there might be an “Office Space” like efficiency consulting team with their own pair of Bob’s starting to ask “So what would you say you do here?” before showing me the door….my own door? Whether the worry was real or not I had decided it was time for a change.</p><h3>Time for a Change</h3><p>Many more doors had opened at this point as “remote” had become the destination for all job postings… but would there be an eventual return to office? Would I want to be conveniently located near one of their offices already just in case? Keeping this in mind, I brushed up my resume and searched for roles that both interested me and would challenge me too. Just like that I was off to the races.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*N5qlLLD0_-ovySDg" /><figcaption>Photo by <a href="https://unsplash.com/@alschim?utm_source=medium&amp;utm_medium=referral">Alexander Schimmeck</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>As my interviews went on, Zoom fatigue had taken on a whole new meaning with the “virtual onsite.” At first, they didn’t seem like they would be that bad, it would be just like the regular onsite interview… right? That couldn’t have been farther from the truth as I learned it was a totally different ball game. Sure, the concept was similar to an onsite interview but there is a different level of exhaustion after spending the majority of the day in the same chair on the same zoom call. The whole time, hoping that not only am I making a good impression, but also trying to learn as much as I can about what it would be like to work there.</p><p>I learned a lot about different companies throughout my journey and how companies, big and small, were all impacted by the pandemic. Some had confidence in their roadmaps, while others had their products under a total rehaul because of COVID. Some were waiting on budget changes and approvals while others were full steam ahead. Ultimately, there was one company that caught my eye, Snapdocs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pVmawqaIKam8dEc5" /><figcaption>Photo by <a href="https://unsplash.com/@homajob?utm_source=medium&amp;utm_medium=referral">Scott Graham</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Making it Official</h3><p>I had officially signed on the dotted line to join Snapdocs. I was very excited to make this transition as Snapdocs checked a lot of boxes for me; a product I believed in, an intriguing industry, transparent leadership, and most importantly, a collaborative culture I could learn and grow in. This excitement wasn’t without its own anxieties. The concept of starting a new job remained the same, but I had never started a new job remotely. What would it be like joining a company having never met anyone in person? Would it be hard getting to know my colleagues outside of “shop talk”? What would the team dynamic look like virtually? Would it be hard learning the lay of the code land from my kitchen? What would setting up my equipment and environment look like over Slack?</p><p>My start date with Snapdocs was approaching fast and I was eager to experience what a 100% virtual onboarding would look like. I wouldn’t be driving or flying anywhere for my first day, no handshakes, office tours, lunch with my onboarding buddy, or anything I had experienced before. My computer and accessories were set to be delivered to my door. Then it would be on me to at least get logged in and online… no pressure right? Being pleasantly surprised, this was the smoothest equipment setup I have had in a long time. All my accounts were ready, passwords worked and I was able to jump into getting my development environment running.</p><p>My first day flew by and before I knew it, two weeks had gone by. There were still some of the usual events like meeting your team, learning more about the company’s goals and product, and learning about all the different Slack channels. There were also new events, as Snapdocs fostered connections between new and existing employees both within their departments, and across the company.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cybwQNnx8OqeSewQ" /><figcaption>Photo by <a href="https://unsplash.com/@cwmonty?utm_source=medium&amp;utm_medium=referral">Chris Montgomery</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Connecting with the virtual team</h3><p>I, like every other new hire, was paired with a buddy whom I could ask questions and use as a resource to get up to speed. For me, the buddy system was perfect as it allowed me to get some tips on Ruby, insight into some of the processes around design and architecture, and learn some of the fun history in the department. I officially met with my buddy on a weekly basis to chat about — almost everything. We sometimes used the time to talk about our service architecture and the automation we had. Other times we would chat about what was going on outside of the virtual world, hear about the debate on what paint color to paint the living room, the latest episode of “The Expanse”, and different hobbies we have. We would also chat outside of the weekly sync but much more story and feature focused.</p><p>As I got up to speed, I started learning more about the different services and which team owned what. In a virtual setting, I found it to be just like following a breadcrumb trail that eventually led me to the right person, team, or slack room. Certainly not as adventurous as walking around the floor to find the team that is set up with the nerf guns or has a duck balloon in their row. Breadcrumbs got the job done — and left a better first impression! Gone were the days of awkwardly standing at their desk interrupting their thoughts. It wasn’t long before everything became second nature and it felt like I had always been a member of the team.</p><h3>Connecting at the virtual water cooler</h3><p>Snapdocs also has some more game focused events like our monthly virtual trivia night. At that event, we form random teams and attempt to gain the most points. The theme changes monthly and sometimes even correlates with a holiday or awareness like women’s history month, or even some that are company related — like “Interns Trivia” in July. I’ve never been great at trivia but I still have a fun time trying!</p><p>We have even found ways to host events virtually that were traditionally in person. For instance, our “Women at Snapdocs” ERG recently hosted a Wine and Paint night. The paint supplies were all mailed to us prior to the event and the drinks were optional. We had a host from the paint company join us on zoom and teach us what we needed to know. They walked us through how to make the different colors and tips on the shapes to draw. All of our pieces came out differently and we celebrated what made our pieces unique. It was so fun to see what others had come up with.</p><p>Have you ever done an escape room? I’ve haven’t done one in person but our Engineering department’s virtual escape room was a blast. Similar to the Wine and Paint night, we had a host from the Escape Room join us on zoom. They had all the clues and props ready to go and set us off on our way to find the Art thief’s next heist. We broke off into our teams and worked together to solve the clues as fast as we could. The faster your team was, the higher score you received at the end.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tWilGxvzgLP9aQ9Z" /><figcaption>Photo by <a href="https://unsplash.com/@countchris?utm_source=medium&amp;utm_medium=referral">Count Chris</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>It is a little crazy to think that as I reach my 1 year anniversary with Snapdocs, the concept of everything still being 100% virtual because of the pandemic is very real. I know I am glad I didn’t wait for the storm to pass at my old gig and am especially proud to have joined Snapdocs. They have cultivated an environment and culture where it feels so similar to being in person that you often forget you’ve never actually met anyone in real life.</p><p>It might not always be rainbows and butterflies but “The Great Resignation” isn’t the only solution either. There are many companies out there that are finding solutions with a remote-first culture. There are companies, like Snapdocs, that value both their employees and culture. If you’re looking to make a change, I’d say go for it! It worked great for me.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f792b3c670c7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/taking-the-leap-in-a-pandemic-f792b3c670c7">Taking the Leap… in a Pandemic?</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pioneering Go in a Ruby World]]></title>
            <link>https://medium.com/snapdocs-product-blog/pioneering-go-in-a-ruby-world-917d5a7f796?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/917d5a7f796</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Hannah Tran]]></dc:creator>
            <pubDate>Mon, 13 Sep 2021 13:48:40 GMT</pubDate>
            <atom:updated>2021-09-23T15:18:22.543Z</atom:updated>
            <content:encoded><![CDATA[<p>We’ve all been there — the late-night shooting the breeze over video chat or a barstool — the brag about your incredible time with punch cards and Pascal — the grumbling over new-fangled languages these days — no one really wins the “coding language body count” game.</p><p>Snapdocs has had a fairly small coding language footprint, especially if you compare them to the Tech Giants or your Uncle Hal’s tall tales. Before 2021, our tech stack was primarily Ruby on Rails, with React on the frontend, along with a little bit of sprinkling of Python and some Bash (let’s be honest, who hasn’t sprinkled in a little bit of Bash when the time calls for it!).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/198/1*Ly4iE1CwIdIVfC4JcoMa1A.png" /><figcaption><a href="https://commons.wikimedia.org/wiki/File:Ruby_logo.svg">Official Ruby logo</a></figcaption></figure><p>I’d like to walk you through our process of changing that. First, our conversations around if we should find another language, second — the choice to select one (spoiler alert — we added Go), and finally, the guidelines we followed to start coding with it. We don’t know everything about this — this was our first time, honestly — but we want to share the journey so when the time is right, you can walk your own.</p><h3>Ask When, Resolve Why</h3><p>At some point, as developers continue to specialize in new languages the popular and trending rankings of programming language choice will continue to change and morph. That reflects not only people looking for new challenges but the push for frameworks and functionality to meet the changing application and developer needs of what we want to build and imagine for our future. Companies must adapt to this changing landscape.</p><p>At Snapdocs, it was easy to be a “Ruby shop” when there were only seven employees. As our frontend, API, services, and scaling needs continued to grow exponentially, however, we started to look around to evaluate if we wanted to continue as that, asking if it still really made sense for us. Beyond spiking on Typescript, Snapdocs took on the mantle to look for a secondary backend language.</p><p>The next hurdle to overcome was to resolve “The Question” — which one?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RE_QSblICkXhX_t8MbADgw.jpeg" /><figcaption><a href="http://snapdocs.com/">How to Choose?</a></figcaption></figure><h3>Evaluate for Your Needs</h3><p>We evaluated new languages with a simple list of requirements.</p><h3>Gap</h3><p>Where is our current codebase not shining? Where is there space for “better”? We didn’t want a parallel language that would bring us no extra value. This eliminated several popular languages as we felt they had the same drawbacks and not a lot of additional advantages compared against what our current stack offered.</p><h3>Support</h3><p>There were a number of factors to consider — external, internal, and the long-term path.</p><p>Externally, using a framework or package that has very little community support or quality documentation is rough and painful. We didn’t want to introduce something that would cause frustration for a simple web search, and there were sure to be a lot of those.</p><p>Internally, we needed people who were interested, passionate and with a background in the new language to support, train and guide engineers who were more familiar with another stack.</p><p>The future, of course, is impossible to predict. The creators of Algol <a href="http://groups.umd.umich.edu/cis/course.des/cis400/algol/algol.html">The ALGOL Programming Language</a> and its fans probably never expected the decline in usage and recognition of the language in a few years time. But as an engineer, you must acknowledge as well — no one has time machine yet, you just have to make your choice and move forward. There’s always space to adjust if needed.</p><h3>Hiring</h3><p>Retention and attracting new talent hinges on people staying interested in the technology, excited by the challenge, and invested in the future. We wanted to continue to keep the interest and vibe for everyday work, and went looking for a language that could provide that for us.</p><h3>Decision</h3><p>Go met our needs for all of the above — for both now and the future state we wanted to be in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/247/1*gEWAehvNzmgl4bh3-DiA5Q.jpeg" /><figcaption><a href="https://golang.org/doc/gopher/">An image of the Go gopher, the mascot of the programming language Go</a></figcaption></figure><p>It met a need and resolved a gap, especially compared to other popular languages like Python. Go offered a language syntactically closer to Ruby with several of the advantages of C. We wanted to experiment with types and compiled languages while seamlessly integrating with our existing architecture.</p><p>Go had a community of support — anything you could search for with one language, you could do the same search in Go!</p><p>Here are a few of my recent searches for things I could do easily in Ruby:</p><ul><li>“turn string to byte array Golang”</li><li>“Golang “.env” files for local systems”</li></ul><p>Each of these effortlessly gave me the desired results.</p><p>Additionally, it had internal proponents — there were some excited engineers who were very invested in what Go could bring the organization. It created a buzz of excitement among our current employees and incoming hires. We have already had people join with more C-like or Go backgrounds, and they were excited to see our implementation and come in on the ground floor. It also gave our engineers space to innovate in their personal skills — we have a number of services now making their way out to production.</p><p>Our Tech council wrote the following around their decision to use Go:</p><blockquote><em>The language we’ve seen the most positive support for within Snapdocs Engineering is Go. Please keep in mind that we’re not necessarily looking for ideal here (that’s likely impossible), so whether or not Go supports a specific feature (like generics) does not necessarily make it the wrong choice. We were evaluating for why Go would not be suitable here. This is strongly in line with how we discuss ideas within the Tech Council to mitigate bike-shedding </em><a href="https://en.wikipedia.org/wiki/Law_of_triviality"><em>https://en.wikipedia.org/wiki/Law_of_triviality</em></a><em>.</em></blockquote><p>So, we had a decision and a starting place. Time to move forward — our next step was to trial it responsibly and roll it out.</p><h3>Pioneer Sensibly</h3><p>When you pioneer, whether you’re crossing the Oregon Trail or planting your first tree, there’s always preparation beforehand and learnings along the way. We used four guidelines to ensure we kept our experiments — the “Golang Trail” if you will — moving forwards.</p><h3>“The Right Way” is subjective</h3><p>One of the hardest present and future balancing acts we face is how to introduce something new. Do you rip the old out by the roots? Do you force everything going forward to adapt? Do you do changes on a one-off basis?</p><p>The answer to this question is heavily dependent on the situation and rarely straightforward. We thought through the technology, our knowledge base, and company needs. We have a culture of engineers changing teams and also changing parts of another team’s codebase. We needed to ensure that our support and maintenance process would remain fluid.</p><p>Snapdocs chose an iterative support approach to start off — how many people can we get trained up, how much foundation can we get under us — so we could eliminate any blockers that could affect the success of Go long term in the organization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XWuvnmUYpQlf_GGmDfNApg.jpeg" /><figcaption><a href="http://snapdocs.com/">Have an Open Mind</a></figcaption></figure><h3>Have an open mind</h3><p>Let’s face it — change is hard. Developers don’t have any special powers to adjust to change any better than other humans. How do you balance pragmatism and strong feelings with a real need to adapt and grow?</p><p>In our case, we did have some teammates who said “I can’t do this exact thing in Go, so we shouldn’t use it” or “I don’t like how Go does this” or “No one needs to use anything but Ruby, Ruby is great”.</p><p>Our most important duty was redirecting these comments. We didn’t get into arguments of if Ruby is great, if Rails is a good framework, if tabs are better than spaces. Instead, we responded with “Go does X well. Go handles Y in a really simple way. Here’s an example of Z with tests written in Go”. We stayed away from conversations of replacing Ruby and instead looked for ways Go could support Ruby in our tech stack and also shine on its own, identifying and calling out where it had strong value.</p><p>I personally have opinions on the language, especially given my Ruby heavy recent background, so for me, it was an active task of evaluating and seeing: where does Go make sense? And that’s how we started building it out and where we started using it. From there it spread to several other implementations and continues to grow in usage across teams.</p><h3>Build a solid foundation</h3><p>Even though we weren’t trying to mirror our Ruby process, architecture or implementation, we knew there were some core things every team would need to resolve.</p><h4>Patterns</h4><p>We needed solid patterns to ensure everyone would develop along the same lines. Especially coming from such an opinionated framework as Rails and Ruby, figuring out consistent patterns for Go implementations was important to both document and write examples for.</p><p>One such outcome — our Developer Efficiency team devised a Go template that any new Go service could start off with. People could just create a new repo for a service and have all the boilerplate code ready at their fingertips.</p><p>We also invested heavily in testing — testing to 100% test coverage isn’t a natural thing with Go, and felt much smoother with languages like Ruby and Javascript when we first started out. We found a way to get to our needs — automatic linting, thorough testing, and maintaining the output of each. Then we rolled out several examples and ensured everyone had access to help and training.</p><h4>Migrations</h4><p>Next, we faced down a challenge we had not anticipated — standardizing our database migrations and flow. We wanted to continue to ensure engineers could easily understand the database for a service, and be able to switch between our Rails and Go implementations, while not needlessly hamstringing the power and flexibility of Go.</p><p>Rails provides an ORM for its migrations, and we built our existing migrations and flow around ActiveRecord’s strong opinions. Pure SQL and ORM implementations exist for Go, but there is no de-facto standard.</p><p>Our journey began with asking an engineer to actually go through, investigate, and share examples for the top options. This helped teams make informed decisions and, hopefully, for us to choose a process that will make sense for us. We’re not sure if we will narrow down to one pattern for migrations yet — maybe that won’t make sense for where we want to be — so for now, we’re trying them out and comparing across services.</p><h4>Deploys</h4><p>Also still in process, we wanted a really clear way to not only deploy our go services but also manage services, gems, and other processes and setup we had ironed out for Ruby.</p><p>With two languages, it was especially important to ensure our architecture and structure was held to a standard that would allow continual easy communication without maintaining parallel implementations across different services. Frameworks like gRPC allowed us to not change that interface too much, and we were able to simply adapt our existing tooling to ensure Go and Ruby gems could interface seamlessly.</p><p>With the support of our devops team, our service communication we had practiced locally deployed fairly easy. We built tooling and worked to attain easy, painless “boring” deployments with the outcome to ensure Go needed no special attention during our release cycle.</p><h3>EVANGELIZE like crazy</h3><p>Even with the most incredible, well meaning team, fear and uncertainty around brand new things can impede progress. Change can be an uphill battle and we wanted to level the playing field as much as we possibly could.</p><p>Don’t let your Golang Trail end up like this!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GaruNWrVTFw1_QwsTHZe5A.png" /><figcaption><a href="https://freesvg.org/8-bit-dysentery">8bit dysentery on a trail to Oregon</a></figcaption></figure><p>We strove to constantly and continually ensure everyone had a chance to learn and be as productive, comfortable and happy as possible with the new languages as they were in their current stack.</p><p>We wanted to make sure all the current networks we offered would help support people in this way. One such tool — our messaging service! We made a dedicated channel for Go (called #gophers) so people had a space to fearlessly ask questions, muse, share current findings, and announce fun learnings.</p><p>Here are some examples of posts on the #gophers channel among the early adopters:</p><blockquote><em>Hey BTW we got a mix of .config and env vars using </em><em>viper.EnvKeyReplacer(strings.NewReplacer(&quot;-&quot;, &quot;&quot;, &quot;.&quot;, &quot;&quot;))</em></blockquote><blockquote><em>Hello all, have you done scheduled jobs for any of the Go services?</em></blockquote><p>We also monitored other channels for Go comments and concerns and made sure we could do quick resolutions for each of them.</p><p>Other teams picked up the language and process as well, and we’ve had a number of context-sharing calls in our current flow of meetings, and additional calls ad hoc as needs arose. Our team even had an engineer from another pod spend dedicated time on our team’s stories to work in Go to ensure he could learn the language. Other teams took up the mantle as well and began to write new services in Go, sharing their struggles and learnings along the way. We now have a number of Go-specific services deployed all the way out to production.</p><p>And of course — it’s a journey. Pioneers aren’t done when they stop walking — they’ve got to live and adjust and maybe even walk some more depending on what they discover!</p><h3>Get Moving</h3><p>Where to go from here? How should you or your team approach the adoption of a new language or framework?</p><p>Follow the same steps — or blaze your own trail! Again, the “right way” to do this is subjective to you and your organization. Figure out what you’re really solving for, choose the one that works well for your needs, and implement it sensibly. Most of all, just try new things and adjust when needed!</p><p>We’ll keep you updated with our process along the way. See you on the Trail!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=917d5a7f796" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/pioneering-go-in-a-ruby-world-917d5a7f796">Pioneering Go in a Ruby World</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Snapdocs’ Product and Engineering Teams are building a new foundation for homeownership]]></title>
            <link>https://medium.com/snapdocs-product-blog/snapdocs-is-building-a-new-foundation-for-home-ownership-166c2600e2c6?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/166c2600e2c6</guid>
            <category><![CDATA[product-design]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[engineering-culture]]></category>
            <category><![CDATA[mortgage-services]]></category>
            <dc:creator><![CDATA[Job Portraits]]></dc:creator>
            <pubDate>Tue, 13 Nov 2018 22:28:33 GMT</pubDate>
            <atom:updated>2018-11-16T20:37:37.693Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*crri4z3CpyzBKN7LFzd81Q.jpeg" /></figure><p><em>More than 5 million families bought a home in the U.S. last year. Nearly all of them started their search online, yet on closing day, the vast majority sat down to sign a thick stack of papers they likely didn’t understand — and that likely contained errors. Now, thanks to Snapdocs, that’s beginning to change. Below, Briana Whelan (Director of Product), Devin Powell (Product Designer), Leslie Stetz (Frontend Engineer), and Greg Krathwohl (Software Engineer) explain the technical challenges and company culture that first drew them to Snapdocs and share what’s exciting about the road ahead.</em></p><p><em>Interested in joining the team? Check out </em><a href="http://bit.ly/2Raf8tD"><em>open positions</em></a><em> or email Head of Talent Greg Russell at greg@snapdocs.com.</em></p><h3>What problem does Snapdocs solve?</h3><p><em>Briana: </em>On the surface, the real estate industry is different today than it was in 2008, when the financial crisis hit, but it’s still operating on the same infrastructure that led to the crisis in the first place. A lot of the technology is a decade behind. Most mortgage closings still happen on paper, and more than half of them have errors. Closings are also extremely expensive because there are lots of stakeholders, and all those costs get dumped in the buyer’s lap. As a homebuyer it’s not like you can say no; you have no control over closing costs. So there’s not much accountability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QQUNCJqQR7krCDO77BTDFw.jpeg" /><figcaption>Left to right: Leslie Stetz (Frontend Engineer), Devin Powell (Product Designer), Briana Whelan (Director of Product), and Greg Krathwohl (Software Engineer).</figcaption></figure><p>The insight Aaron had — Aaron is our founder and CEO — was that there’s nothing connecting the dozen sub-industries that are involved in the process. That’s what we’re building at Snapdocs. We think of the platform as the connective tissue that’s going to unite this entire pillar of the U.S. economy. Lenders and title companies use it to collaborate on mortgage closings with everyone else involved, which makes the process not only more affordable, but faster, more transparent, and a better experience for home buyers. We’re focused on closings right now, and there’s a ton more to do down the road.</p><p><em>Devin: </em>Mortgages are still collapsing at the last moment because a name is misspelled. People are still using fax machines. So there’s room for improvement. I’m confident in saying there’s no version of the future where closing on a home will still cost thousands of dollars and take two months.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yeVtGD2NyEFxT8f_RlkkmQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9gsSzJz7HaXNugeQWFPmEg.jpeg" /><figcaption>Snapdocs provides a morning treat on a regular basis. Sometimes it’s breakfast, sometimes it’s a barista, today it’s omelets. Left: Cheyenne greets the chefs who make it happen. Right, Wendy and Eduardo mix breakfast with a bit of work.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZD6_H3oDATcBBKz76mKAWw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zExCNSC5LpJLd9GYMV1B2w.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZVCU-y9XLpRiYVJQfvmjmg.jpeg" /><figcaption>Left: Xavier contemplates the secret ingredient — of the pasta sauce or life, hard to say. Center: Tristin spices things up. Right: Though thirsty, Neel is never far from the breakfast chat.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jQqBAm7GPl1F7d5C_ghwcw.jpeg" /><figcaption>As breakfast ends, Leslie catches up with Chelsey on their walk back to their desks.</figcaption></figure><h3>Why did you decide to join the team?</h3><p><em>Briana: </em>I wanted to join a team where I could help build not only the product, but also the company. And I was so impressed with the people. Aaron has a strong vision, but he’s open to letting others help execute. As a product person, that balance was really important to me.</p><p>Building a product in the real estate space was also exciting to me, because it’s one of the only major industries where technology hasn’t had a big impact. And just like retail, it’s moving toward more consumer control. Pre-approval has already been brought online, so buyers are getting a pleasant, smooth experience at the beginning of the process. They want the same thing at the end, and they have access to more options than ever before. So digital closing has become a competitive advantage for lenders.</p><p><em>Devin:</em> I joined for a lot of the reasons people choose a small company — ownership, impact. There’s an incredible opportunity to make a difference in the mortgage industry. I met Aaron right after Snapdocs finished Y Combinator, and I remember asking why he wasn’t going crazy with marketing like the other companies. He told me he’d rather focus on executing. He said, “My dream is to build a great product and deliver so much value to the industry that folks look up one day and realize Snapdocs is the standard.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8OR6vSy-hKJdFQ7CoN5aWw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bPxEarG8PPqlbTaV-NAjZQ.jpeg" /><figcaption>Left: Sandy asks the tough questions. Right: Christian listens.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SMJ24vdRWOOSballPXWbZg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uRkluu5x8dRhHcJE_39QiQ.jpeg" /><figcaption>Left: Evan (CTO) wonders aloud to Greg. Right: Focus time.</figcaption></figure><p><em>Leslie:</em> I joined for the technical opportunity. Snapdocs was running a Ruby on Rails monolith and wanted to move to microservices, and I wanted help lead that transition. We’re building the frontend in React, Redux, and modern JavaScript, and we’ve been able to move quickly — in two months we went from deciding to work in React to actually having React in production. Speed comes with pros and cons, but I like experiencing firsthand what does and doesn’t work. And the team is very open to new ideas, so it’s been a great place to grow.</p><p><em>Greg: </em>The tools we use were part of the reason I joined, too. In addition to the frontend work Leslie mentioned, there’s a lot of interesting stuff happening in the backend and on the data science side. We’re using machine learning for things like classifying documents and adding annotations. And our DevOps team is doing cool stuff, too. We’re using Kubernetes in all of our staging environments and plan to roll it out to our production environment. There are lots of interesting problems to solve and lots of ways to learn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OQ0iDdPKjw3-pH6eDIRrrw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c7EhFJml-RP46BHMFtAjOg.jpeg" /><figcaption>During the monthly all-hands, Aaron (CEO) gets his team’s attention with quick wins for the next month and long-term goals for 2019 and beyond.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xRoIPK5j6MvjBc9QV_rxfA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BNS2MCypXI63QtvF7_lHyw.jpeg" /><figcaption>Left: At the end of the all-hands, Kayleigh congratulates Ashley on her 1 year anniversary with the company. Right: Aaron chimes in with a hearty thank you.</figcaption></figure><h3>How would you describe the work environment and company culture at Snapdocs?</h3><p><em>Greg: </em>Day to day, engineers, designers, and product people are all working together. We have a daily standup where we talk about our current projects, and we use Jira and Slack to keep up to date. We’re also in close communication with Support whenever we’re rolling out something new, or if there’s an issue we need to address.</p><p><em>Leslie:</em> It’s easy to collaborate, because everyone is super self-aware and down-to-earth. They really listen, and we can have open, honest conversations. I’m always trying to find ways to make it easier for people to build on the frontend even if they’re not frontend experts, and sometimes, my solutions end up interfering with something. But it’s never like, “Oh, Leslie, you messed up.” I just ask questions, learn from the experience, and try a different path.</p><p><em>Devin:</em> Yeah, there’s a real sense of pragmatism. When we first rolled out a new product for lenders, for example, we knew the onboarding flow we built wasn’t perfect. But that’s okay. We solved the immediate problem, and delivered an early win for our customers. When the data wasn’t super clean a few months down the line, we were in a position to solve that, too. Before we build for scale, sometimes we just need to answer a basic question, and we can do that because there’s space here to iterate and make incremental improvements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8EB0OV724ptaQFyGdTUuLg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NSsmZYACPRniToxS_JT78A.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DE-I-7DxWMGjSSVbjL8uDg.jpeg" /><figcaption>A few slices of engineering life.</figcaption></figure><p><em>Briana: </em>Transparency is big in our culture, too, and it’s something we look for in job candidates. We ask them to tell us about a time they failed, but it’s not really about the failure. It’s about whether they can be open and honest, and whether they applied what they learned.</p><p>We also celebrate a lot — happy hours and team outings, but little things, too. For example, there was this annoying blue bubble that was popping up on some page, and the other day, someone added an X so you can make it go away. One of our teammates sent a Slack message that said, “Whoever fixed the blue bubble, you just made half the company the happiest people on Earth!”</p><p><em>Greg: </em>And then a bunch of other people were like, “Confirmed! I am now the happiest person on Earth!”</p><p><em>Briana: </em>We share messages from users, too. One of our team goals is to make escrow officers happy, and someone recently shared a quote from an escrow officer that read, in all caps, “I LOVE SNAPDOCS!” We were like, “We did it!”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g0X5wYteurBK_eNHa3oLHA.jpeg" /><figcaption>In a daily “pump zone” session, team members are out of their chairs and into their best planks.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jr__pppGPdW4YO7cJ6E7ow.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mludfFnh8V13EpY3ArPQ3A.jpeg" /><figcaption>Left: Tristin catches some heads down time. Right: Never underestimate the power of a post-it. Or a dozen of them.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l7jKGNL4oy-XEf1qZdHfug.jpeg" /><figcaption>Cheyenne and Michele enjoy a quiet moment, sponsored by comfy couches and plants more ambitious than their pots.</figcaption></figure><h3>Tell us more about your team’s approach to challenges.</h3><p><em>Devin: </em>We focus on the problems that are immediately in front of us, where we know we can make a difference, and I think the team is very empathetic to the people who work in this space. A lot of companies just want to disrupt things and blow everything up. For us, it’s more about understanding the challenges people are facing and how we can help them do better, faster work and make fewer errors.</p><p><em>Leslie:</em> And one of the cool things about our product is that part of our team uses it every day, so they understand the problems firsthand. We can test things on ourselves and get great, quick feedback.</p><p>We also have a lot of support to really try things out. When we release something, we live with it for a certain amount of time, but we don’t hesitate to analyse it and make improvements. As we’ve been building this new frontend with React and Redux over the last six months or so, we’ve been figuring out which patterns work and which ones don’t. Now we understand the challenges, and we’ve been able to come together as a team and design an informed, holistic road map for the application of our dreams.</p><p><em>Greg:</em> “Application of our dreams.” I like that. To me the big challenge is balancing that long-term vision with the practical short-term approach. We start by satisfying the needs of one individual company, and then another, and another. Sometimes, that can lead to a mess, but as we explore those individual solutions, we get closer to a solution that will work for everyone.</p><p><em>Briana:</em> Right — you have hypotheses about which features will be successful, but you can’t know for sure. In consumer software, you might use A/B or fake door testing. But in enterprise, because our paying customers have high expectations, we have to get creative if we want to test a new idea.</p><p>Often, we’ll try something manually at first. If we want e-signatures on every page of a loan document, for example, we can’t write a rule and throw a template on there, because every page is different. But to truly solve that problem, we’d have to take on a ton of machine learning and algorithm work. So instead, we’ll offer a service manually to one customer and see if it’s something they’re willing to pay for. Sometimes, we try things people don’t want, and we shut them down. But other features turn out to have huge demand, and then we invest in building the technology behind them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1SICQ9KfGkQKG8zDF_c5_Q.png" /><figcaption>As the day winds down, Ashley (center, Customer Success Manager), grabs Aaron (left, CEO) for a quick conversation about an upcoming product update.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i2tqLOMpcQG86bH5P8vpVQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y8S3ydt82UtYpD5PXt7qOQ.jpeg" /><figcaption>….But before folks go home, on the rooftop, a few team members find some beers and an evening almost as perfect as the hedges.</figcaption></figure><h3>Looking ahead to the future, what are you most excited about?</h3><p><em>Devin: </em>We’ve already provided a lot of value from an operations standpoint, behind the scenes. And that’s great — title companies and lenders see that value and they’re willing to pay for it. But a homebuyer might not even realize Snapdocs is involved. Ultimately, we want to help consumers feel more comfortable and confident about their mortgage, and we’re exploring ways to do that more directly.</p><p><em>Briana: </em>Because of the base we’ve built, now we have the opportunity expand our impact, and decide how much of the process we want to be involved in. It just feels like a really exciting time. People talk about rocket ships, but I’ve never felt like I was on one at other companies, even when we were growing 40 percent every year. Snapdocs feels different. We’ve had some great early success, and I think if we continue to execute, there’s a huge opportunity right around the corner.</p><p><em>Interested in joining the team? Check out </em><a href="http://bit.ly/2Raf8tD"><em>open positions</em></a><em> or email Head of Talent Greg Russell (pictured below!) at greg@snapdocs.com.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1-zz6yMsNqDUMd7o-2aNlQ.jpeg" /><figcaption>Your moment of Zen: Evan (CTO) and Greg (Head of Talent) take a moment between meetings and do not regret it.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=166c2600e2c6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/snapdocs-is-building-a-new-foundation-for-home-ownership-166c2600e2c6">How Snapdocs’ Product and Engineering Teams are building a new foundation for homeownership</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Convolutional Neural Nets for Rapid Recognition of Mortgage Docs]]></title>
            <link>https://medium.com/snapdocs-product-blog/convolutional-neural-nets-for-rapid-recognition-of-mortgage-docs-2e8c01dc06fc?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/2e8c01dc06fc</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[saas]]></category>
            <category><![CDATA[mortgage]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Jon Tang]]></dc:creator>
            <pubDate>Fri, 21 Sep 2018 21:49:59 GMT</pubDate>
            <atom:updated>2019-01-08T21:43:08.136Z</atom:updated>
            <content:encoded><![CDATA[<p>At <a href="http://www.snapdocs.com/">Snapdocs</a>, we help mortgage professionals close over 50,000 mortgages a month all across the U.S. The vast majority of these <a href="https://www.consumerfinance.gov/ask-cfpb/what-is-a-mortgage-closing-what-happens-at-the-closing-en-176/">mortgage closings</a> go smoothly, but sometimes problems come up at the signing table. From our analysis, we found that a large percentage of signing-table disputes happen because of confusion over the loan terms. For example, the cash-to-close or interest rates are not what the borrower expects. At Snapdocs, we want every closing to be perfect and error-free, so we started looking into ways to prevent this problem.</p><p>In every closing package (full set of mortgage documents), one of the most important pieces of paperwork is the <a href="https://www.consumerfinance.gov/owning-a-home/closing-disclosure/">Closing Disclosure</a>, a five page form that provides the final details about the mortgage loan. A sample of the first page of a Closing Disclosure is shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*rsPpMl5bcZ3zwzhrHuANcw.png" /><figcaption>A sample page 1 of the 5-page Closing Disclosure document. Source: <a href="https://www.consumerfinance.gov/owning-a-home/closing-disclosure/">https://www.consumerfinance.gov/owning-a-home/closing-disclosure/</a> .</figcaption></figure><p>As you can see from this sample page, the Closing Disclosure holds a lot of good summary points about the mortgage. However, this form is buried somewhere within the closing package, which is often a 200+ page PDF document. We imagined how quickly identifying the Closing Disclosure page and displaying this information would resolve some of these signing-table disputes.</p><h3><strong>Challenge 1: Data Pre-processing Speed</strong></h3><p>We first attempted to build a simple text-based machine learning model to identify the first, second, and fifth pages of the Closing Disclosures, which contain the most relevant data. A text-based classifier seemed like an obvious choice because these pages of the Closing Disclosure contain very distinct keywords.</p><p>We quickly realized, however, that the required pre-processing steps for generating the data in production took too long. You first have to convert all pages of a closing document from PDF into high-resolution images. Then the images containing text need to be converted into machine-encoded text using <a href="https://en.wikipedia.org/wiki/Optical_character_recognition">Optical Character Recognition</a> (OCR). The entire process could take several minutes for a 200+ page document, which is longer than we’d like to power some features that are time-sensitive. Additionally, several minutes per package wouldn’t be scalable for our current volume.</p><p>Another downside is that many of our closing documents are scanned documents, rather than electronically generated, and therefore are often marred by ink spots and visual noise. Performing OCR on these scans often results in unreliable text for accurate classification.</p><p>We stumbled upon an interesting idea after skimming through preview images of closing package pages. As you can see below, Closing Disclosure pages make use of distinctly formatted tables that make it visually distinguishable from non-Closing Disclosure pages. Very small, low-resolution images are sufficient for a human to be able to categorize them quickly at a glance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dxnuNi2LqA65enEjHRWHvQ.png" /><figcaption>Examples of low resolution images of first, second, and fifth pages of the Closing Disclosure, and other non-Closing Disclosure pages.</figcaption></figure><p><strong>Our Approach and Results<br></strong>This motivated us to try using an image-based approach. One advantage is that the only required pre-processing step is the conversion of the PDF into JPEGs. Additionally, we don’t need to generate the images at the high level of resolution (300 dpi recommended) required for accurate OCR. We found that 70 dpi resolution images were good enough, reducing the total pre-processing time from 2 seconds to 20 milliseconds per page.</p><p><strong><em>That’s a drop in total pre-processing time of nearly 107-fold!</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RzpLIXl2kVNUjlIpc-0gCA.png" /><figcaption>A comparison of the data pre-processing steps for using a text-based classifier versus and image-based classifier.</figcaption></figure><h3><strong>Challenge 2: Lack of Labeled Data</strong></h3><p>When we started this project, we had identified less than 300 Closing Disclosures from our entire database of closing packages. We knew that to create any reliable classifier from such a small dataset would be extremely challenging.</p><p><strong>Our Approach and Results</strong></p><p>Rather than go through our collection of closing packages and manually label more Closing Disclosure pages, we thought this problem could benefit from a technique called <a href="https://medium.com/nanonets/how-to-use-deep-learning-when-you-have-limited-data-part-2-data-augmentation-c26971dc8ced">data augmentation</a>. Data augmentation works by taking your existing dataset and making minor alterations to create additional labeled data. In our case, as shown below, we used each page in our original dataset as a template to create additional example images that contained slight rotations and translations that we would often find in some of our scanned images of closing pages. Because pages are often scanned in upside-down, we also included examples that were full 180 rotations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VXQNqRsl2F2-UQEE8cRnjQ.png" /><figcaption>We augmented our data by taking each page in our original dataset (in yellow) to create additional example images that contained slight rotations and translations (in blue) that we would often find in some of our scanned images of closing pages.</figcaption></figure><p>Using this technique, we were able to create a dataset consisting of over 250,000 images of closing package pages with more than 10,000 of them being the first, second, or last pages of Closing Disclosures. Additionally, this gave our dataset broader coverage of the types of scanned mortgage documents we see in the real-world.</p><h3>Machine Learning Model</h3><p>We tested several different machine learning models and hyper-parameter sets in <a href="https://www.tensorflow.org/">TensorFlow</a> until we settled on the following <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network">convolutional neural network</a> architecture. In production, the model has a micro-average <a href="https://en.wikipedia.org/wiki/F1_score">f1 score</a> of 99% and takes a couple of seconds on average to identify our Closing Disclosure pages from an entire closing document.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1zzFi3lfQkuy8DwuSyytGw.png" /><figcaption>Our final convolutional neural network model architecture after testing other machine learning algorithms and hyperparameter tuning.</figcaption></figure><h3><strong>Summary</strong></h3><p>While a text-based classifier seems like the most obvious choice for a document classification problem, our unique constraints challenged us to find a more radical approach. In our case where we required near instantaneous document classification, we chose to use an image-based classifier because the data pre-processing steps were more than 100x quicker in production.</p><p>Machine learning, <a href="https://en.wikipedia.org/wiki/Computer_vision">computer vision</a>, and automation are continuing to play a large role at Snapdocs as we strive towards our company goal of bringing greater efficiency, accuracy, and joy to mortgage closings. Aside from this Closing Disclosure classification model, we have developed other models to help support fast hybrid signings. For example, we use machine learning models to quickly and accurately identify pages from mortgage documents for e-signing, wet-signing, or previewing. We use computer vision to automate the task of identifying signature lines on pages that require digital signatures from the consumer. In the future, we also anticipate features such as creating a ‘Table of Contents’ for consumers to see each of their docs page-by-page. This will also allow lenders and settlement agents to swap out individual documents, instead of entire packages when there’s an error.</p><p><strong>Come Join Our Team. We’re Hiring!</strong></p><p>Find our work interesting? Want to help us tackle the massive mortgage closing space? We’re hiring. <a href="https://jobs.lever.co/snapdocs">Come join our team</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e8c01dc06fc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/convolutional-neural-nets-for-rapid-recognition-of-mortgage-docs-2e8c01dc06fc">Convolutional Neural Nets for Rapid Recognition of Mortgage Docs</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Model Services: for the cloud, from the cloud (Or: Cloud9 + Lambda + Gateway = Data in use)]]></title>
            <link>https://medium.com/snapdocs-product-blog/model-services-for-the-cloud-from-the-cloud-or-cloud9-lambda-gateway-data-in-use-d161de65a9cf?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/d161de65a9cf</guid>
            <category><![CDATA[cloud9]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-infrastructure]]></category>
            <category><![CDATA[aws-api-gateway]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Scott Breudecheck]]></dc:creator>
            <pubDate>Fri, 07 Sep 2018 17:45:28 GMT</pubDate>
            <atom:updated>2018-09-07T17:45:28.372Z</atom:updated>
            <content:encoded><![CDATA[<p>Data science is only useful when it is actionable. If no one reads an analyst’s report or a recommendation engine sits untouched on an engineer’s laptop, all that was achieved was a high-cost training exercise. At a startup lacking the defined specialists teams and/or established processes, I’ve found that the responsibility to realize impact from data science lies first with myself, the data scientist. So in that light, I have been eager to see how AWS’s Lambda could be used to bring data projects into the real world.</p><h3>Motivation. Or: keep going till you’re no longer blocked</h3><p>A couple months ago I came across a fantastic tutorial on using AWS Lambda to create a callable model. <a href="https://towardsdatascience.com/data-science-for-startups-model-services-2facf2dde81d">Seriously, it’s worth a read.</a></p><p>But…I couldn’t get it to work. Sure, I could get the ‘hello, world’ version. But I couldn’t deploy the model as an API. Something was wrong with my environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3gX5V68sliJIaR7SfdfX6w.png" /><figcaption>Error message #291 I came across</figcaption></figure><p>Every change required zipping a folder, uploading to s3, and repointing the lambda to pick it up. It would work fine locally, in new environments, on a brand-new ec2, hell I even bit the bullet and learned docker to get a perfect environment, but every zip -&gt; s3 upload -&gt; rebuild Lambda resulted in an error, rarely the same. I was stuck in a frustrating loop that required spending more time clicking through multiple services than actually debugging.</p><p>Just when I had given up, I heard about AWS Cloud 9. <a href="https://aws.amazon.com/blogs/aws/aws-cloud9-cloud-developer-environments/">A cloud-based IDE</a> with more features than I’m aware of, my main draw was it could be used to build and test local versions of lambda.</p><h3>Basic tutorial. Or: “hello, world”</h3><p>Let’s start with a simple example.</p><h4>Here’s what we’ll be doing</h4><ol><li>Create a Cloud9 environment</li><li>Create a Lambda function</li><li>Test the Lambda function in Cloud9</li><li>Deploy the Lambda function</li><li>Set up API Gateway to call Lambda</li></ol><h4>Let’s get to it</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wunaah52lUEQ7CiqZSbx1w.png" /><figcaption>Creating an environment from cloud9 console</figcaption></figure><p>Navigate to “Cloud9” in console, and click “Create environment”. Create an environment named “anotherLambdaTest”, leave all the settings on default.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bCjdRHMR7VCoz8XuJsy5jA.png" /><figcaption>Hello, cloud9</figcaption></figure><p>On the right tab, go “AWS Resource” &gt; “Local Functions” &gt; ƛ⁺(“Create a new Lambda function”). Let’s call the function “simpleTutorial” in the application “anotherLambdaTest”. And set it up as empty-python, with no trigger.</p><p>This should pop up lambda_function.py in the IDE with some barebones function. At this point, we just copy over the <a href="https://towardsdatascience.com/data-science-for-startups-model-services-2facf2dde81d">simple tutorial</a> script:</p><pre>def lambda_handler(event, context):<br>    print(event)    <br>    result = &#39;Hello from &#39; + event[&#39;queryStringParameters&#39;][&#39;msg&#39;]<br>    return {  &quot;body&quot;: result }</pre><p>Save it! <strong><em>There’s no autosave!</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/243/1*bQMcPP6BmUuutyH1S-AtlQ.png" /></figure><p>Ok, so now let’s test it. No need to zip and upload and deploy. Just…click test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*Lz7DPC85HwIBns8Xw3vqYQ.png" /><figcaption>Wow that was painless</figcaption></figure><p>A payload window pops up to create a custom event. It runs from the IDE, and you get results and any print messages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/245/1*LqJUi6Hlidz0rSLI32FYqQ.png" /></figure><p>Ready to make this an API? Click “Deploy the selected Lambda function”.</p><p><em>et voila!</em>, your lambda function exists.</p><p>Head over to the console for Lambda, select your newly deployed lambda (it’ll be prefixed “cloud9…” and have last been modified &lt;1 minute ago). From here, it’s pretty much the same: Add an API Gateway trigger and configure it (create new API, make it open, and save the lambda). Then throw a test event against it via the API Gateway test</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*mXbYe-wNvjVT3XENwX6zVg.png" /><figcaption>Naming things has never been AWS’s strong suit</figcaption></figure><p>Finally, hit the actual api by first deploying (“Actions” &gt; “Deploy” &gt; “default”) to generate your API url. You can copy that url, append your Lambda’s name and append the url params. In my case <a href="https://atkgc42ggg.execute-api.us-west-2.amazonaws.com/default//cloud9-anotherLambdaTest-simpleTutorial-X4VIU30USH5C?msg=TheWeb">https://atkgc42ggg.execute-api.us-west-2.amazonaws.com/default/cloud9-anotherLambdaTest-simpleTutorial-X4VIU30USH5C?msg=TheWeb</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/1*MxVh2ZCMSFdAKle6o-VL2Q.png" /></figure><h3>Deploying a model. Or: adding data “science”</h3><h4>Here are the steps (this should look familiar)</h4><ol><li>Create a model in Cloud9</li><li>Create another Lambda function that uses that model</li><li>Test the Lambda function in Cloud9</li><li>Deploy the Lambda function</li><li>Set up API Gateway to call Lambda</li></ol><h4>Let’s get to it</h4><p>Instead of building a new environment from scratch, we can just add on.</p><p>Back in Cloud9, add another Lambda function and application. Let’s call both “logitTutorial”.</p><p>Our model will need some additional modules. This is not quite as straight forward as might be liked, but relative to the normal virtualenv, pip, zip, s3, deploy, it’s a cake walk. In the application folder, open a terminal and navigate to the function’s folder ( ~/environment/logitTutorial:).</p><p>In this newly opened terminal you can import by specifying the target.</p><p><em>there’s a space and double hyphen before “target”. Medium auto-formating has issues.</em></p><blockquote><em>python3 -m pip install — target=./ numpy pandas scipy sklearn</em></blockquote><p>Now, we need to create the model file (‘logit.pkl’). To do that, I’m just going to create a new file in the environment (anotherLambdaTest/logitTutorial/build_logit_pkl.py), copy over the code <a href="https://medium.com/u/a80e1f69e782">Ben Weber</a> provided in his tutorial, and run that from Cloud9’s terminal. This will drop logit.pkl right in the same folder.</p><pre>import pandas as pd<br>from sklearn.externals import joblib<br>from sklearn.linear_model import LogisticRegression</pre><pre>df = pd.read_csv(<br> &quot;<a href="https://github.com/bgweber/Twitch/raw/master/Recommendations/games-expand.csv">https://github.com/bgweber/Twitch/raw/master/Recommendations/games-expand.csv</a>&quot;)</pre><pre>y_train = df[&#39;label&#39;]<br>x_train = df.drop([&#39;label&#39;], axis=1)</pre><pre>model = LogisticRegression()<br>model.fit(x_train, y_train)</pre><pre>joblib.dump(model, &#39;logit.pkl&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mCCqhv5UZ0YXDolT-0JPAA.png" /></figure><p>The last python-ic step is to update lambda_function.pywith code to call the model (again, see <a href="https://medium.com/u/a80e1f69e782">Ben Weber</a>’s tutorial for full code).</p><pre>from sklearn.externals import joblib<br>import pandas as pd</pre><pre>model = joblib.load(&#39;logit.pkl&#39;)</pre><pre>def lambda_handler(event, context):    <br>    p = event[&#39;queryStringParameters&#39;]<br>    print(&quot;Event params: &quot; + str(p))<br>    x = pd.DataFrame.from_dict(p, orient=&#39;index&#39;).transpose()<br>    pred = model.predict_proba(x)[0][1]</pre><pre>result = &#39;Prediction &#39; + str(pred)<br>    return {  &quot;body&quot;: result }</pre><p>Then we’re ready for testing. Note: unlike the tutorial you must use double quotes when testing here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NDsq3HHFl2pGwQtRi-Pegw.png" /></figure><p>From here, we can deploy as before, then head over to Lambda console to set up this new Lambda with its API Gateway just like before.</p><pre><a href="https://kd32u9hkzg.execute-api.us-west-2.amazonaws.com/default/cloud9-logitTutorial-logitTutorial-JW9VGD8H7A4W?G1=1&amp;G2=1&amp;G3=1&amp;G4=1&amp;G5=1&amp;G6=1&amp;G7=1&amp;G8=1&amp;G9=1&amp;G10=1">https://kd32u9hkzg.execute-api.us-west-2.amazonaws.com/default/cloud9-logitTutorial-logitTutorial-JW9VGD8H7A4W?G1=1&amp;G2=1&amp;G3=1&amp;G4=1&amp;G5=1&amp;G6=1&amp;G7=1&amp;G8=1&amp;G9=1&amp;G10=1</a></pre><p>==&gt; “Prediction 0.39762327867823355”</p><h3>Conclusion. Or: let engineers work on engineering</h3><p>While not entirely without frustrations, cloud-based IDE development and employment is a powerful combination. Far less time spent on setting up deploys + a tighter turnaround to testing = more time on actual data science.</p><p>Combining this with other AWS services: eg <a href="https://aws.amazon.com/machine-learning/">AWS Machine Learning</a> and <a href="https://aws.amazon.com/sagemaker/">SageMaker</a>, make it manageable to create an endpoint that bridges a production app (built by engineers/product) and predictive analytics (built by data team).</p><p>With no data engineers and a swamped product engineering team, being able to self-serve our work product to the rest of the team opens up huge opportunities!</p><p>To that end, we are now working on abstracting the data science features from our product. Instead of Engineers coding up Ruby functions based on our models, they can simply call a versioned endpoint. Any changes to the model can simply be swapped in behind the endpoint, with our air-traffic-control lambda function re-pointing to the latest model and feeding it with the latest relevant features. This frees up our engineers to focus on design and app infra work, while reducing our wait time for new models to be adopted.</p><p>It also allows wins from one project to be immediately available for research on another project. <a href="https://medium.com/snapdocs-product-blog/our-mission-at-snapdocs-is-to-make-mortgage-less-complicated-1d4f7f1eca29">At Snapdocs we modernize mortgage closings</a>, and recently <a href="https://www.snapdocs.com/resources/new-progress-on-the-digital-closing-front">we built automatic document classification to make finding e-signable pages fast</a>. Using this approach, we could expand that application to finding Closing Disclosures and the mortgage details therein, surfacing relevant details for our customers and their clients even when doing a traditional signing. Improvements in the underlying model immediately flow to both projects with no major code changes required!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d161de65a9cf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/model-services-for-the-cloud-from-the-cloud-or-cloud9-lambda-gateway-data-in-use-d161de65a9cf">Model Services: for the cloud, from the cloud (Or: Cloud9 + Lambda + Gateway = Data in use)</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Our User Pain Points with Text Data Mining]]></title>
            <link>https://medium.com/snapdocs-product-blog/understanding-our-user-pain-points-with-text-data-mining-dfc7822bc241?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/dfc7822bc241</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[real-estate]]></category>
            <category><![CDATA[mortgage]]></category>
            <dc:creator><![CDATA[Jon Tang]]></dc:creator>
            <pubDate>Mon, 09 Oct 2017 17:18:56 GMT</pubDate>
            <atom:updated>2017-10-11T05:14:55.463Z</atom:updated>
            <content:encoded><![CDATA[<h4>We used natural language processing (NLP) and analytics to learn what are the most common reasons that signing agents failed to meet expectations during signings.</h4><p>At Snapdocs, we’re trying to simplify and bring transparency to mortgage closings. Check out our <a href="https://medium.com/snapdocs-product-blog/our-mission-at-snapdocs-is-to-make-mortgage-less-complicated-1d4f7f1eca29">previous post</a> on how our online platform provides a centralized gathering point for all parties involved in the closing.</p><p><strong>The Frustrations of Mortgage Closings</strong></p><p>The final signing on a property is one of the last steps in purchasing real estate and should be an exciting event for a home buyer. However, there are numerous opportunities for complication that can throw the final signing off track. From deal-breaking issues like failing to sign critical disclosure forms to minor irritations such as having a signing agent (a notary) show up late to a signing, many things can lead to bad experiences and frustrations for Escrow/Title Officers and their customers, the home buyers.</p><p>At Snapdocs, we strive to make mortgage closings run as smoothly as possible. One way that we do this is by surfacing the best qualified notaries for a signing. We are constantly collecting feedback on notary performance. Just like Uber wants to know if your driver had a clean car or drove aggressively, we want to know if your agent dressed professionally or was late to the signing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*j2pD53eF2_0Qj-ty75ymaQ.png" /><figcaption>Screenshot of Uber’s interface for collecting feedback on their drivers.</figcaption></figure><p>As data scientists at Snapdocs, we wanted to leverage this feedback data to gain insight into how well our notaries have met the expectations of Escrow/Title officers. In this blog post, I’ll first show you how we analyzed our data to determine how often mortgage professionals have unsatisfactory experiences with notaries during signings. I’ll then show you an approach we took using natural language processing (NLP) and analytics to learn what are the most common reasons that notaries failed to meet expectations. Finally, I’ll conclude with how we’re incorporating this information back into our product development lifecycle to improve the overall user experience.</p><p><strong>Gathering Data on Notary Performance</strong></p><p>On the Snapdocs platform, we enable companies to easily search a database of 65,000+ notaries, see statistics on a particular notary’s past performance, and choose the top ranked, available notary for a signing. After a signing has completed, an Escrow/Title Officer can rate a notary’s performance as positive, neutral, or negative and then include an optional detailed comment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/526/1*Rtu87pSYOvbZQnbinUbVoA.png" /><figcaption>Screenshot of the Snapdocs interface for collecting feedback on notaries after a signing.</figcaption></figure><p>I looked at over 220K orders from the period of March 2017 to mid-August 2017 and found that an overwhelming 99% of signings received positive feedback ratings, 0.2% received neutral feedback ratings, and only 0.8% received negative feedback ratings! Our notaries are doing exceptionally well at meeting expectations.*</p><p>*<em>We also found that about 40% of the signing events that received a negative rating were carried out by notaries that received a negative rating on at least one other signing. Perhaps these notaries are repeating their mistakes. In any case, our hope is that we may be able to prevent bad experiences like these in the future.</em></p><p><strong>Topic Modeling of Negative Feedback Text</strong></p><p>While negative feedback on the Snapdocs platform is rare, I decided to delve more deeply into the reasons why our customers were having these unsatisfactory experiences. Looking specifically at the text data from negative feedbacks, I sought to identify common themes using NLP techniques. I used one of the simplest approaches for topic modeling, <a href="https://www.youtube.com/watch?v=hDmNF9JG3lo">K-means analysis</a>, to group feedback into clusters based on the similarity of words used.</p><p>To do this, I took a standard approach for pre-processing text data. I first used the NLTK package to remove non-alphanumeric characters and tokenized the feedback text, breaking them down into their constituent words. I then filtered out stop words, which are commonly used words with presumably little meaning (e.g. ‘the’, ‘is’, ‘are’). The remaining words were then stemmed using the Porter Stemmer to reduce them to their basic form (e.g. “update”, “updates”, “updated”, and “updating” all reduce to “updat”) to facilitate identifying overlapping words. The collection of these processed words were then converted to a <a href="https://www.youtube.com/watch?v=PhunzHqhKoQ">TF-IDF</a> vector of weighted term frequencies so that each feedback text could be represented as a mutually comparable vector for the k-<br>means algorithm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/1*3qNiRuGVMERv4gPieGDbFA.png" /><figcaption>The steps we took for data pre-processing and conversion of feedback text data into TF-IDF vectors for K-means analysis.</figcaption></figure><p>For k-means analysis, a number of clusters or topics, k, must be specified. Choosing the best value of k can be more of an art than science, often with heuristics being used. I decided to begin with a large and diverse set of topics to work with that I could then later prune, so I chose an initial value of 35 clusters. After analyzing the results and unifying smaller similar clusters together, I settled on 30 final clusters. Each cluster was then manually labeled after taking into account the most common words in the cluster and reading example feedback content from the cluster. For example, from the word cloud (showing the most common words) and the example feedbacks for the first cluster shown below, we concluded that this cluster of feedbacks is about the notary being late to the signing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*Zdw89-HyNy040GbkpY-0_w.png" /><figcaption>Word cloud showing the most common words from the first cluster identified in our analysis.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*PTJi0QQXGQ6rS1SbP8Vx6Q.png" /><figcaption>Example feedback text from the first cluster identified in our analysis.</figcaption></figure><p><strong>Not All Negative Feedbacks Are Equal</strong></p><p>On our platform, if a negative rating is given to a notary after a signing, the Escrow/Title Officer also has the option to deactivate a notary so that they are no longer considered in their future notary searches. To understand which clusters of negative feedbacks were inconvenient versus those that were deal-destroying, we looked at which feedbacks led to notary deactivations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*K7WxvRbH9L3EMMNa." /><figcaption>We identified 3 major collections of topics: (1) Errors, (2) Quality and promptness of documents, faxes, and scans sent back to the Escrow/Title Officer after the signing event, and (3) Notary personal and customer facing attributes. The number of negative feedbacks that did and did not result in deactivation of the notary are shown for each of the 30 final topics identified in our analysis.</figcaption></figure><p>As you can see from the first collection of topics (in pink) in the chart above, making a mistake during the signing (an “Error”) is usually not enough to deactivate a notary. While minor errors make up the biggest group of negative feedbacks, only 18% result in deactivation of the notary. Talking to some of our customers in user interviews, we learned that general errors are usually correctable: you can eSign an addendum or even have the notary go back with the single page needing a correction.</p><p>We found a second collection of topics (in orange) that related to quality and promptness of documents, fax, and scans being sent back to the Escrow/Title Officer after the signing event. These were generally more rare, but had relatively higher deactivation rates (up to 39% deactivation rate for having documents missing, “Doc/Fax/Scan — Missing”). Loan closings have a very tight timeline to be completed, so even a day delay by a slow notary can put a deal at risk.</p><p>Most interestingly, the last collection of topics (in green) have deactivation rates as high as 71% and appear to include feedbacks from the worst signing experiences. Some topics in this group include notaries that either were poorly presented (“Rude”, “Unprofessional”) or put the signing in jeopardy (“No Show”, “Changed Time”, “Late Cancellation”).</p><blockquote>Our main takeaway from this analysis is that mistakes happen and are forgivable, but people seem to have very low tolerance for jerky behavior.</blockquote><p><strong>Impacting Product Development</strong></p><p>This sort of analysis helps Snapdocs in two ways. First, we can share these learnings and specific feedback with notaries directly. We’re evaluating the best place to incorporate these results in notary training, early warnings to notaries via text/email, and category-specific scoring on our platform. We want notaries to understand what their customers care about and how they can best be successful. We also want notaries to know when they are underperforming, so that we can guide them to take the necessary steps to continue doing business on the Snapdocs platform.</p><p>Second, now we have some real data representing problems for us to solve and ideas for features we could build. We’ve started building a feature we’re calling the “Signing Checklist” which should improve the likelihood that notaries walk away from the signing event confident they did a good job. We’ll be building better scheduling and communication tools so everyone –notary, escrow agent, buyer — is on the same page and there are no surprises.</p><p>While the engineers embark on the development process to incorporate our learnings from this analysis of negative feedbacks, we’re going to tackle the next data challenge: positive feedbacks! What sort of things do notaries get praised for? Which categories are highly associated with becoming a “favorite” of a company? What does “above and beyond” mean with respect to notary performance at the signing table?</p><p><strong>Come Join Our Team. We’re Hiring!</strong></p><p>Find our work interesting? Think you might have an alternative approach? We’re hiring. <a href="https://jobs.lever.co/snapdocs">Come join our team</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dfc7822bc241" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/understanding-our-user-pain-points-with-text-data-mining-dfc7822bc241">Understanding Our User Pain Points with Text Data Mining</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Our nationwide hunt to find notaries]]></title>
            <link>https://medium.com/snapdocs-product-blog/our-nationwide-hunt-to-find-notaries-ccbe7360a1?source=rss----868a6161c805---4</link>
            <guid isPermaLink="false">https://medium.com/p/ccbe7360a1</guid>
            <category><![CDATA[real-estate]]></category>
            <category><![CDATA[saas]]></category>
            <category><![CDATA[mortgage]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[tech]]></category>
            <dc:creator><![CDATA[Scott Breudecheck]]></dc:creator>
            <pubDate>Wed, 23 Aug 2017 17:10:36 GMT</pubDate>
            <atom:updated>2017-06-14T21:52:17.338Z</atom:updated>
            <content:encoded><![CDATA[<p>Our mission at Snapdocs is to make mortgage less complicated. In an age where you can book travel in a few minutes on your phone, we think it’s time buying a home catches up. <a href="https://medium.com/snapdocs-product-blog/our-mission-at-snapdocs-is-to-make-mortgage-less-complicated-1d4f7f1eca29">Check out this post for a fuller explanation of Snapdocs</a>.</p><h3>Manhattan, KS vs Manhattan, NY: Not all Manhattans are created equal</h3><p>We operate across the US, whether you’re buying a bungalow on Kauai, refinancing your condo in Seattle or setting up a line of credit in Charleston, our workflow tools help people work with notaries to close loans. To guarantee nationwide coverage we need access to a deep pool of notary experts everywhere. <strong>Our goal is to start our client with 100 notaries for every single search</strong>. After all, we know nothing makes you switch to Lyft faster than when your flight lands, you pull up Uber and get this….</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*nUPzayUZpyIVy1Ik0CPnIw.png" /><figcaption>Stranded in upper Idaho…</figcaption></figure><p>As our Data Scientist, one of my projects was to identify and prioritize regions where registered notaries were spread thin. I thought about going on a massive road trip, but I get car sick easily.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/468/0*bpksEVbT6-R8pSPc." /><figcaption><em>Screen grab from </em><a href="https://www.youtube.com/watch?v=OPwPo-IAQ-E"><em>https://www.youtube.com/watch?v=OPwPo-IAQ-E</em></a></figcaption></figure><p>Every month close to 1,000 notaries register on Snapdocs so connect with our clients. We go through an extensive verification process to validate the quality: bond verification, NNA certification, criminal background checks, etc. This thoroughness means that in some regions we fall short of our 100 notary goal per search.</p><p>In these cases, when a customer is looking to find a notary our search will expand from a 10-mile search radius out to a 50-mile radius. At 50-miles, you can cover huge swathes of territory. Notaries that lose a whole day driving to your signing will want a higher fee! Travel delays also mean the signing can start late.</p><p>The whole experience is suboptimal for our <em>customer</em> (higher fee), the <em>notary</em> (more time driving than working) and the <em>signer</em> (sitting around waiting). You can imagine the disappointment of searching Tinder for San Francisco and only getting results in Sacramento.</p><p>So we set a goal for ourselves: <strong>reduce the number of 50-mile searches!</strong></p><h3>Educating vs Recruitment</h3><p>In some cases finding 100 notaries isn’t possible because there are too few humans (I’m looking at you Nome, Alaska).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*p4rF0Vtu4jc0We4Q." /><figcaption>By ra64 from nome, usa (words to live by) [CC BY-SA 2.0 (http://creativecommons.org/licenses/by-sa/2.0)], via Wikimedia Commons</figcaption></figure><p>But lack of notaries isn’t the only reason for a 50-mile search. Since Snapdocs facilitates a variety of loan closings, each search can have different criteria. A <a href="http://www.mortgagecalculator.org/helpful-advice/what-is-a-refinancing.php">refinance</a> is pretty basic and quickly handled by a professional; but a two-party <a href="https://reverse.org/what-is-a-reverse-mortgage/">reverse mortgage</a> can take hours and requires in-depth knowledge of real estate contracts.</p><p>Some companies are refi-shops, putting together relatively simple refinance packages; while others specialize with the more tricky reverse mortgages. The more stringent a company is, the more likely the search has to expand to 50-miles.</p><p>To serve the former, we could focus wholly on notary <strong><em>recruitment</em></strong>: state-certified notaries are competent and it’s just a matter of having enough of them. But to serve our more tricky signings we focus on <strong><em>educating</em></strong>: nudging existing notaries to obtain and upload certifications.</p><p>This led me to wonder: <em>do some regions have a mix of 10- and 50-mile orders simply because different companies operate there?</em> After all, St Louis requires both easy refinance deals and complicated two-party sales, so it’d make sense that some stringent searches would be co-located with easier ones.</p><p>If the bulk of 50-mile searches took place alongside 10-mile searches then we would need to focus our efforts on <em>education</em>. But if 50-mile searches clustered together in specific geographies with no 10-mile searches, then we’d need to ramp up general <em>recruitment</em>.</p><p>To examine this, I turned to our search history which stores the distance, results and ZIP code for every search. I plotted each ZIP code against two factors: how many total notaries worked in that area (<em>x-axis, volume</em>) and what fraction of searches returned all of them (<em>y-axis, strictness</em>).</p><p>Because there are thousands of ZIP codes, I used a <a href="https://en.wikipedia.org/wiki/Multivariate_kernel_density_estimation">kernel density</a> (fancy words for “contour”) graph to see the distribution of searches. Darker regions denote more ZIP codes in that area.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*q9_uisUxSishrmPbbTQuKg.png" /><figcaption>Every search over the past two months landed on this graph (most in the upper left)</figcaption></figure><p>If this were an<em> education</em> issue, I’d expect scattered points around the middle of the graph. This would be because some searches would be pretty strict and only consider a few notaries, and thus &lt; 100% of searches would look return all the notaries in the area.</p><p>But instead, we see that the vast majority of ZIPs sit in the upper left: 100% of searches consider 100% of the local notaries! This tells me that (in most places) notary education is fine, instead we need to <strong><em>focus on getting more notaries</em></strong>!</p><h3>Where should we spend our time?</h3><p>While we want to improve everywhere, we’ve got to start somewhere. We could just look for gaps in notary coverage (like those Verizon ads), but I’m ok if we’re short on notaries in areas we don’t do a lot of business.</p><p>To incorporate the business impact, I factored in search volumes. For every ZIP code I mapped a dot, the size is proportional to the amount of business we do in that ZIP code while the shade is relative to how many searches were done at 50-mile radius.</p><p>Blue dots are great: no 50-mile searches! Big, red dots are problems: we do a lot of business there and all of it at 50-miles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VO7j0ZEOTI6cJg-D." /></figure><p>Using this map, virtually flew across the country identifying clusters of big, bad ZIP codes. Some trends started to emerge:</p><ul><li>Appalachia is a hotbed of activity that needs more notaries</li><li>The notary community in the Midatlantic is huge and more than able to handle all the business</li><li>Texas is a paradox: the large cities are amply covered, but there’s a sharp dropoff in available notaries in the smaller cities</li></ul><h3>Now to go get them!</h3><p>With these findings, I paired up with <a href="https://medium.com/u/83d8621816e9">Felicia</a> , our Notary Community Specialist, to brainstorm what tactics we can use to identify, engage and ultimately recruit notaries. We decided to focus on Texas because it seems like a state that just needs a little nudge to have a big impact. We do a lot of business across the state, we have evidence we can recruit there, and the problem areas are relatively large communities (100k+ population).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/0*EuT0F53hrPhH_WPV." /><figcaption>Live in Texas and looking for a new career? snapdocs.com/notaries</figcaption></figure><p>We’ve picked a dozen smaller cities in Texas to try a few different ideas, and hopefully at least one will resonate and we can roll it out nationwide! If you live in El Paso, Corpus Christi, Wichita Falls or a handful of other locales, we’d love to hear from you.</p><h3>Come work here section</h3><p>Find this post interesting? Have a better approach? <a href="https://jobs.lever.co/snapdocs">We are hiring!</a> Data analysts, engineers, customer support, implementation managers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ccbe7360a1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/snapdocs-product-blog/our-nationwide-hunt-to-find-notaries-ccbe7360a1">Our nationwide hunt to find notaries</a> was originally published in <a href="https://medium.com/snapdocs-product-blog">Snapdocs Product &amp; Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>