<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jonas on Medium]]></title>
        <description><![CDATA[Stories by Jonas on Medium]]></description>
        <link>https://medium.com/@kathrynjonas?source=rss-c73376c8754a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*2QfNM7_kuwLH4nflA4n1FA.png</url>
            <title>Stories by Jonas on Medium</title>
            <link>https://medium.com/@kathrynjonas?source=rss-c73376c8754a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 15:38:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kathrynjonas/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Elixir Job Processing with Verk]]></title>
            <link>https://medium.com/@kathrynjonas/elixir-job-processing-with-verk-b7e0e916b88d?source=rss-c73376c8754a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7e0e916b88d</guid>
            <category><![CDATA[queue]]></category>
            <category><![CDATA[task-queues]]></category>
            <category><![CDATA[elixir]]></category>
            <dc:creator><![CDATA[Jonas]]></dc:creator>
            <pubDate>Tue, 20 Nov 2018 21:25:40 GMT</pubDate>
            <atom:updated>2018-11-20T21:25:40.163Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*owgqNGGWoxG5Fxt3PHpXrg.png" /><figcaption>From <a href="https://github.com/edgurgel/verk">https://github.com/edgurgel/verk</a></figcaption></figure><h3>An overview of the Elixir Verk Package</h3><p><a href="https://github.com/edgurgel/verk">Verk</a> is an Elixir job processing system backed by Redis. Verk provides isolation for executing jobs by running each queue in its own supervision tree. The Verk supervision tree manages a pool of queue workers, enqueuing and dequeuing jobs from Redis, and executing queue jobs. Verk uses GenStage for job event notifications. Redis is used for storing queue jobs and is the source of truth for jobs and their status. Verk has one connection to Redis for each queue and a general connection for tasks such as deleting a job.</p><p><strong>Implementing Verk</strong></p><p>Verk can be setup by first adding Verk to your mix.exs dependencies, then adding the Verk.Supervisor to you supervision tree. As per the <a href="https://github.com/edgurgel/verk/blob/master/README.md">Verk Installation</a> guide:</p><pre><strong>defmodule</strong> <strong>Example</strong>.App <strong>do</strong><br> <strong>use</strong> Application</pre><pre><strong>def</strong> <strong>start</strong>(_type, _args) <strong>do</strong><br> import Supervisor.Spec<br> tree = [supervisor(Verk.Supervisor, [])]<br> opts = [name: Simple.Sup, strategy: :one_for_one]<br> Supervisor.start_link(tree, opts)<br> <strong>end</strong><br><strong>end</strong></pre><p><strong>Working with Queues</strong></p><p>Queues are created in your application’s config.ex file:</p><pre>config :verk, queues: [“queue:task”: 10]</pre><p>The queues argument accepts two values, the queue name and the concurrency for job processing. The following configuration can be set universally across all queues:</p><ul><li>max_retry_count (Default: 25)</li><li>poll_interval (Default: 5000)</li><li>start_job_log_level (Default: info)</li><li>done_job_log_level (Default: info)</li><li>fail_job_log_level (Default: info)</li><li>node_id (Default: 1)</li><li>redis_url: The URL for the Redis instance used for storing jobs.</li></ul><p>Queues can also be dynamically created and removed.</p><pre>Verk.add_queue(:new, 10) # Adds a queue named `new` with 10 workers<br>Verk.remove_queue(:new) # Terminate and delete the queue named `new`</pre><p>When queues are initialized, they are stored in an ETS table with the status :running. This ETS table is used by the Supervisor to check on queue status before enqueue and dequeue operations. When operations are performed on a queue that result in a status change, the status in the ETS table is updated. The queue is removed from the ETS table when the queue is deleted.</p><p>The Verk supervision tree is based on the defined queues. Individual queues are supervised with a :one_for_one strategy, ensuring that failures on one queue process does not affect the others.</p><p>Every queue has a Queue Manager that interacts with Redis to dequeue jobs from the specified queue. The Queue Manager also provides methods that can be executed against queues:</p><ul><li>Clear: Clears jobs from the queue.</li><li>Count: Counts how many jobs are enqueued in a queue.</li><li>Delete: Deletes a specific job from a queue.</li><li>Range: Lists enqueued jobs in the queue.</li></ul><p>Verk collects information about each Queue and stores it in an ETS table. These stats are part of what powers the <a href="https://github.com/edgurgel/verk_web">Verk Web</a> interface.</p><ul><li>Queue Status</li><li>Current amount of running jobs</li><li>Amount of finished jobs</li><li>Amount of failed jobs</li></ul><p><strong>Working with Jobs</strong></p><p>The Worker Manager interacts with Redis to dequeue jobs and execute them. Workers communicate with the Queue Manager when a job is done and then shuts down. The Worker Manager provides methods that affect job execution.</p><ul><li>Pause: Pauses dequeuing of jobs in the queue. Enqueueing of jobs will still continue.</li><li>Resume: Resumes the dequeuing of jobs in the queue.</li><li>Add: Create a new queue with the supplied number of workers, signal supervisor to start a new child process for the queue.</li><li>Remove: Delete a queue, signal to the supervisor to stop child processes for that queue.</li></ul><p>Jobs can be enqueued by calling the enqueue function and providing the queue, the class the job will execute, and any arguments for the execution. Optionally, a max_retry_count can also be supplied.</p><pre>Verk.enqueue(%Verk.Job{queue: :default, class: “ExampleWorker”, args: [1,2]})</pre><p>When a job is enqueued, it’s added to a Redis sorted set using the LPUSH command, which puts the job at the head of the list. When jobs are enqueued, the current time is added to the job in the field enqueued_at. This piece of metadata is further updated as a job is retried.</p><p>Jobs can also be scheduled to run at a future point. The schedule function implements the Redis ZADD command, which adds an element to a sorted set with a score. The score is set with a DateTime formatted time argument given to the schedule function. Job times are converted to Unix when stored in Redis.</p><pre>perform_at = Timex.shift(Timex.now, seconds: 30)<br>Verk.schedule(%Verk.Job{queue: :default, class: “ExampleWorker”, args: [1,2]}, perform_at)</pre><p>Verk jobs are Elixir structs with the follow attributes:</p><ul><li>Error message</li><li>Failed at</li><li>Retry count</li><li>Queue</li><li>Class</li><li>Jid</li><li>Finished at</li><li>Retried at</li><li>Error backtrace</li></ul><p>The Job Queue and Class are provided when a Job is enqueued. The other values are evaluated by Verk as jobs are executed.</p><p><strong>Job Execution</strong></p><p>Jobs are executed by Workers defined in your application. The Verk Event Producer is a <a href="https://hexdocs.pm/gen_stage/GenStage.html">GenStage</a> producer that broadcasts events to subscribed consumers. Verk takes advantage of the GenStage backfill design to ensure that consumers are not overwhelmed by the number of enqueued jobs. GenStage consumers only receive as many events as they can process. The Worker subscribes to the Verk producer to execute jobs, and the number of Workers will be based on the concurrency limit defined in the queue. The following events are broadcasted by the Event Producer:</p><ul><li>JobFinished</li><li>JobStarted</li><li>JobFailed</li><li>QueueRunning</li><li>QueuePausing</li><li>QueuePaused</li></ul><p>The event status enables the consumer to filter events, such as for error handling:</p><pre><strong>defmodule</strong> <strong>Consumer</strong>.JobEventHandler <strong>do<br></strong> <strong>use</strong> GenStage<br> require Logger</pre><pre><strong>def</strong> <strong>start_link</strong>() <strong>do</strong><br> GenStage.start_link(__MODULE__, :ok)<br> <strong>end</strong></pre><pre><strong>def</strong> <strong>init</strong>(_) <strong>do</strong><br> {:consumer, :state, subscribe_to: [Verk.EventProducer]}<br> <strong>end</strong></pre><pre><strong>defp</strong> <strong>handle_event</strong>(%Verk.Events.JobFailed{job: job, stacktrace: stacktrace}) <strong>do</strong><br> Logger.error(<br> “Job Failure reported via handler, Stacktrace: #   {inspect(stacktrace)}, Job data: #{inspect(job)}”<br> )<br> <strong>end</strong></pre><p>Failed Jobs will be retried based on the max retry count (default: 25). When a job fails, the current retry count is incremented and stored in the job’s retry_count field. The job is then placed in a Retry queue to be retried until the max retry count is met or the job succeeds. Jobs are added to the Redis Retry queue using the ZADD operation and weighted by a calculated retry time. If a custom retry_at method has not been defined in the consumer, the retry time is based on Verk’s default exponential backoff logic.</p><pre><strong>defp</strong> <strong>default_calculate_retry_at</strong>(failed_at, retry_count) <strong>do</strong><br> delay = :math.pow(retry_count, 4) + 15 + (:rand.uniform(30) * (retry_count + 1))<br> failed_at + delay<br> <strong>end</strong></pre><p>When a job is ready to be retried, the Schedule Manager places it back in the original queue to be retried. Additionally, jobs in the retry queue can be immediately executed by calling requeue_job.</p><p>When a job has failed and the retry count has been met, the job it put into the Dead queue, ordered by the timestamp of when the job last failed. Dead jobs remain in the queue for one week and the limit on the dead queue is 100 jobs.</p><p><strong>Verk Strengths &amp; Limitations</strong></p><p>Verk as a solution for job queueing and batch processing in Elixir has several key strengths and limitations to consider when reviewing your application requirements.</p><p><strong>Strengths</strong></p><ul><li>Clean API that is simple to implement.</li><li>Backing of Redis and use of Lua to implement <a href="https://redis.io/commands/rpoplpush">RPOPLPUSH</a> enables reliable job processing, ensuring all jobs will be processed at least once.</li><li>Use of GenStage for producing and consuming jobs provides built in back pressure for job consumers. Jobs are dequeued based on the consumer demand.</li><li>Built in exponential retry with method to customize.</li><li>Clean process for error handling and tracking.</li><li>Verk Web is great additional tool for metrics and easy monitoring of queue status.</li></ul><p><strong>Limitations</strong></p><ul><li>There is no broadcast event for dead jobs. This has to be implemented in the consumer by checking if the current retries matches the limit.</li><li>Since there is only a single dead job queue, additional dead job filters are required to implement queue-specific dead job handling.</li><li>The max queue size for the dead queue is not configurable and the 100 job limit can result in job loss when a spike in dead jobs occurs.</li><li>There is no ability to limit queue size. In practice this means queue size will be based on Redis memory limits. In certain failure scenarios or traffic spikes, this can cause Redis and the queueing system to fail.</li><li>Related to the limitation above, Verk does not provide a method to rate limit or back-pressure enqueueing of jobs and thus prevent the queue size from exhausting Redis memory limits. This must to be implemented in the job producers.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7e0e916b88d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cultural Polyglots]]></title>
            <link>https://medium.com/@kathrynjonas/cultural-polyglots-a207c7b0a405?source=rss-c73376c8754a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a207c7b0a405</guid>
            <category><![CDATA[cross-cultural]]></category>
            <category><![CDATA[engineering-team]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[leadership]]></category>
            <category><![CDATA[diversity-in-tech]]></category>
            <dc:creator><![CDATA[Jonas]]></dc:creator>
            <pubDate>Mon, 12 Feb 2018 18:29:11 GMT</pubDate>
            <atom:updated>2018-02-12T18:29:11.079Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building Effective Cross-Cultural Engineering Teams</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sEIigbETlNgzdSwGdq1LIg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/OhJmwB4XWLE?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Bruno Martins</a> on <a href="https://unsplash.com/search/photos/language?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>A meeting invite with your boss and the head of HR is never a good sign. My mind flicked through all of my technology projects, analyzing potential issues for reprimand. Sitting down, my boss asked me how things were going with my team, specifically my database admin and my IT systems admin. I was unsure how to respond. Their different personalities had caused conflict in the past, but I had worked through it with them and believed we were a harmonious team. My Human Resources director then informed me that they hadn’t been speaking to each other for weeks.</p><p>I was completely stunned. How had I been unaware that my two team members had such a falling out? As a new manager, this news crushed my confidence in my ability to lead. While it was hard to accept at the time, this failure was a learning opportunity, not just in how to be a better manager, but also how to work more effectively in a cross-culture team.</p><p>I was an American working in Beijing, managing the IT department of a Chinese company. While I encountered the obvious language blunders and pop culture confusion, there were deeper, more critical differences in communication, working, and management styles that I failed to recognize.</p><p>Since then, I’ve worked for several international, cross-cultural organizations and managed teams with members spread across the globe. As an American, it can seem easy to work internationally with English as the primary language and American culture so global, but there are hidden icebergs that can sink a team. Here are practices and mindsets I’ve found to overcome these challenges and make teams more effective, collaborative, and fun.</p><h4><strong>Culture Harmony</strong></h4><p><strong>Don’t be afraid of failure</strong></p><p>You’ll make cultural faux pas. Too often this fear prevents engagement. People are so nervous about making mistakes that they don’t attempt to connect. Communicating in another language only exacerbates the problem. The key is to change mindsets and see failure as learning. In programming, we write code with errors, applications that crash, and we learn from the mistakes to build better systems. Working in new cultures and with diverse groups is no different. Admit when you’ve made a mistake, and share your learning with others. In Beijing, I was open with other managers about the mistakes I made with my team and not only gained invaluable insight, but also created space for all of us to share where we had stumbled and reflect on ways to improve.</p><p><strong>Use Your Teacher Voice</strong></p><p>Several organizations I worked for were education focused, so I’ve frequently collaborated with teachers. When I say “Use Your Teacher Voice”, I’m not talking about speaking clearly and concisely (though that’s very important!) Teachers ask students effective questions that check for understanding. A teacher that introduces a new topic and asks her students “Do you understand?” will probably get a classroom of blank stares or subtle nods at best. Instead, teachers ask students to explain concepts back, apply them to other situations, or encourage students to teach peers<strong>. </strong>When working in a cross-cultural team, embody your best teacher and apply techniques that build understanding.</p><p>Practices such as asking clarifying questions and learning through collaboration are effective ways to ensure important information isn’t lost in translation. When leading a data visualization training for an international group, I spent very little time presenting and walking through examples. Instead I introduced an approach, quickly highlighted an example, then had small groups work together to recreate the example and apply the technique to a new data set. Each group then presented back their work. As a manager, I often prompt team members to explain new features, hold a session on a new tool, or take the lead on writing user stories or technical specs. These practices promote cohesion and comprehension in a team.</p><p><strong>Do Your Homework</strong></p><p>Learning about each other’s cultures is an extremely rewarding experience and transforms a group of engineers into a team. Joining a team with a diverse background is intimidating, there aren’t as many shared topics, jokes, or cultural norms to connect on. See the game last night? What about that hilarious meme? Technology gives us a baseline, but being so closely linked to our day-to-day work it limits the growth of play, trust, and collaboration. Take the initiative to make connections outside of tech. Some ways I’ve done this include:</p><ul><li>Sharing local holidays and how they’re celebrated</li><li>Organizing informal events, such as hackathons, that allow for more casual conversation</li><li>Holding monthly potlucks and cooking competitions</li><li>Creating a Spotify playlist and getting contributions from team members</li></ul><p>Another valuable approach is finding a mentor, especially when working in an environment where the predominant culture isn’t your own. A mentor supports career growth generally, and in a cross-cultural context, informs how to adapt management styles. My mentor provided a sounding board for questions and a space for sharing the frustrations of navigating an unfamiliar environment.</p><p>These approaches all require you to do your homework. You don’t learn a new technology just by sitting in conferences and hearing people talk about it, you have to get your hands dirty. When working in cross-cultural teams you need to take initiative and find the resources that will help you navigate new surroundings.</p><h4><strong>Hit the Ground Running</strong></h4><p>These approaches lay a foundation for successful cross-cultural technology teams. With that foundation in place, several best practices help teams work together more effectively.</p><p><strong>Learn what form of communication works best</strong></p><p>When working with non-native speakers, some are more comfortable with verbal communication, others with written. Learn where the strengths in your team lie and be sure that you cater training, Agile rituals, and management to them. Encourage team members with different language abilities to work together and provide clarification.</p><p>I recommend all important communications be shared verbally and written and translate if possible. At a recent organizational meeting I attended, a group of Spanish speakers had a friend who quietly helped translate. As the meeting progressed, someone injected “Can you please stop whispering?!” and was quickly abashed upon realizing it was a translator, not a inconsiderate disruptor. So be sure to give space for translation and transparency on everyone’s communication needs. Furthermore, I’ve seen huge returns in engagement from investing a small amount in translation.</p><p><strong>But did you document it?</strong></p><p>Technical documentation is especially critical in cross-cultural teams, it creates a shared language for a team’s technical approach. Software practices vary across countries and cultures. Documentation establishes the standards for the team. This also includes clearly defined tools for communications, meeting management, Agile tracking, and other company norms. When I joined an organization using Skype and email integration, I assumed Skype was the primary tool for chat. After a few lonely, quiet days, I learned my team was chatting away on instant messenger. Documentation is also especially helpful if your team is spread across time zones. It ensures important information doesn’t get lost between Slack channels and email chains.</p><p><strong>Watch your language</strong></p><p>One of my previous managers used peanut butter and jelly sandwiches as an analogy for a business strategy and quickly lost the non-US audience. When I taught English as a Second Language, I realized how heavily slang, historical, and cultural references influence language. As a teacher, I learned to proofread my verbal communication and use more culturally neutral language, but there will always be things I miss. After telling a Slovakian coworker I needed to “grab some water”, I was informed about a Slovak folk tradition of dousing women in water on Easter, and that my comment implied I was about to join in. Practice mental editing and paying special attention to cultural references in your communication. You won’t be able to catch everything, but the goal is to ensure that your language doesn’t exclude anyone due to lack of cultural context.</p><p><strong>Less is not More</strong></p><p>Engineers love brevity and simple, elegant solutions. However, when working across cultures and languages, less is not more. Don’t be afraid to over explain as this will air out any hidden assumptions. Always assume good intentions from your team and ask for more clarification or follow up. A response that may seem rude or blunt is often simple communication differences. Comment your code regularly and include your reasoning in code reviews and pull requests.</p><p>When in the UK for a job training, the combination of jetlag, learning new tech, and an instructor with a strong accent left me constantly confused. I apologized and regularly asked for clarification and code examples, which my other trainee, a non-native English speaker, thanked me for later. Over communication is a much easier problem to manage than the confusion, mistrust, and lost time due to under communication.</p><p><strong>Be Curious</strong></p><p>One of my favorite things about technology is the diverse people and perspectives. While there’s room to improve engineering diversity, it attracts individuals from across the world to solve a range challenges. Stay curious. Continue to learn from the challenges, inevitable failures, and new insights gained from distributed, cross-cultural colleagues.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a207c7b0a405" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing the Trust Protocol]]></title>
            <link>https://medium.com/@kathrynjonas/implementing-the-trust-protocol-9ec1a2785ae6?source=rss-c73376c8754a------2</link>
            <guid isPermaLink="false">https://medium.com/p/9ec1a2785ae6</guid>
            <category><![CDATA[schema]]></category>
            <category><![CDATA[project-trust]]></category>
            <category><![CDATA[how-to]]></category>
            <category><![CDATA[seo]]></category>
            <dc:creator><![CDATA[Jonas]]></dc:creator>
            <pubDate>Wed, 22 Nov 2017 16:33:25 GMT</pubDate>
            <atom:updated>2017-11-22T16:39:32.808Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ECyO7cQQKteLnEMo_Luw1A.png" /><figcaption>Image Credit <a href="https://thetrustproject.org/">https://thetrustproject.org/</a></figcaption></figure><p>In the rising tide of “fake news” and overall decline of trust in media organizations, the Trust Project emerged as an effort to build transparency and confidence in quality journalism. Beginning with user studies and interviews, the Trust Project sought to understand what helps readers evaluate the trustworthiness of a story. These insights were brought to several development and design hackathons and evolved into a set of<a href="https://thetrustproject.org/faq/#indicator"> Trust Indicators</a>. The Trust Indicators are a protocol providing insight and understanding of a story, the publisher, and the journalistic process.</p><p><strong>The Trust Protocol</strong></p><p>The Trust Protocol was designed to inform readers and machine platforms such as Facebook, Google, and Bing. These platforms have agreed to collaborate with the Trust Project in surfacing the indicators to help readers determine the reliability of journalism on their platforms.</p><p><em>The Economist</em> was a partner in developing the Trust Protocol and implementing the MVP phase alongside<a href="https://www.scu.edu/news-and-events/press-releases/2017/nov-2017/the-trust-project-helps-readers-identify-reliable-news.html"> 14 other leading news organizations</a>. The Protocol relies on<a href="http://schema.org/"> Schema.org</a> standards and the Project has worked closely with Schema.org to add Trust Indicators to the relevant schemas. This structured data layer can be validated against <a href="https://search.google.com/structured-data/testing-tool/u/0/">Google’s structure data validator tool</a> and exposed to external platforms in a website’s HTML markup.</p><p>There are three primary data structures required to fulfill the Trust Protocol: <a href="http://pending.schema.org/NewsMediaOrganization">NewsMediaOrganization</a>, <a href="https://schema.org/NewsArticle">NewsArticle</a>, and <a href="https://schema.org/Person">Person</a>. Below is an overview of implementing these schemas at the Economist, which can serve as a guide for getting started with the Protocol in your own organization. Additional samples can be found at <a href="https://github.com/TheTrustProjectCode/Trust-Project-Schema.org-Markup-Examples-2017">The Trust Project Github</a>.</p><p><strong>NewsMediaOrganization</strong></p><p>The NewsMediaOrganization schema contains the primary information about the publisher.</p><ul><li><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.i8j7winq0jry">Ethics Policy</a> — <a href="http://schema.org/ethicsPolicy">URL: ethicsPolicy</a></li><li><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.dvwpf7htnol3">Diverse Voices Statement</a> — <a href="http://schema.org/diversityPolicy">URL: diversityPolicy</a></li><li><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.4vs4tcepfqh0">Diversity Staffing Report</a> — URL</li><li><a href="http://schema.org/correctionsPolicy">Corrections</a><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.hai5sayd2k0k">Ownership/Funding</a> — <a href="http://schema.org/correctionsPolicy">URL: correctionsPolicy</a></li><li><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.5tlxqug5082o">Founding date</a> — <a href="http://schema.org/foundingDate">ISO 8601 Date: foundingDate</a></li><li><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.cwwcg11rz2f1">Masthead</a> — <a href="http://schema.org/masthead">URL: masthead</a></li><li><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.kzqnubcebaug">Mission/Coverage Priorities</a> — <a href="http://schema.org/missionCoveragePrioritiesPolicy">URL: missionCoveragePrioritiesPolicy</a></li><li><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.9968vs7d9vd8">Verification/Fact-checking</a> — <a href="http://schema.org/verificationFactCheckingPolicy">URL: verificationFactCheckingPolicy</a></li><li><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.hn4otnmhf7g9">Unnamed Sources Policy</a> — <a href="http://schema.org/unnamedSourcesPolicy">URL: unamedSourcesPolicy</a></li><li><a href="https://docs.google.com/document/d/1jdt4V92XtveciID3TBl79aiwQcYs5uGSDVdN72PGcpw/edit#bookmark=id.960o5m11tka">Actionable Feedback</a> — <a href="http://schema.org/actionableFeedbackPolicy">URL: actionableFeedbackPolicy</a></li></ul><p>At <em>The Economist</em>, content is modeled in a JSON structure aligned with Schema.org standards and publisher information is stored alongside our internally managed Economist Group Wikidata entry.</p><pre>{<br> &quot;actionableFeedbackPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#contact-us">https://www.economist.com/about-the-  economist#contact-us</a>&quot;,<br> &quot;contactPoint&quot;: [{<br>   &quot;<a href="http://twitter.com/type">@type</a>&quot;: &quot;ContactPoint&quot;,<br>   &quot;contactType&quot;: &quot;Newsroom Contact&quot;,<br>   &quot;telephone&quot;: &quot;+44 (0) 20 7830 7000&quot;,<br>   &quot;email&quot;: &quot;<a href="/service/mailto:letters@economist.com">letters@economist.com</a>&quot;,<br>   &quot;url&quot;: &quot;<a href="https://stage.economist.com/about-the-economist/#contact-us">https://stage.economist.com/about-the-economist/#contact-us</a>&quot;<br>  },<br>  {<br>   &quot;<a href="http://twitter.com/type">@type</a>&quot;: &quot;ContactPoint&quot;,<br>   &quot;contactType&quot;: &quot;Public Engagement&quot;,<br>   &quot;email&quot;: &quot;<a href="/service/mailto:letters@economist.com">letters@economist.com</a>&quot;,<br>   &quot;url&quot;: &quot;<a href="https://stage.economist.com/about-the-economist/#audience-engagement">https://stage.economist.com/about-the-economist/#audience-engagement</a>&quot;<br>  }<br> ],<br> &quot;correctionsPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#corrections">https://www.economist.com/about-the-economist#corrections</a>&quot;,<br> &quot;description&quot;: &quot;Media company headquartered in London, United Kingdom&quot;,<br> &quot;diversityPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#diversity-policy">https://www.economist.com/about-the-economist#diversity-policy</a>&quot;,<br> &quot;diversityStaffingReport&quot;: &quot;<a href="https://www.economist.com/about-the-economist#staff-diversity-data">https://www.economist.com/about-the-economist#staff-diversity-data</a>&quot;,<br> &quot;ethicsPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#ethics">https://www.economist.com/about-the-economist#ethics</a>&quot;,<br> &quot;foundingDate&quot;: &quot;1843-02-09T00:00:00Z&quot;,<br> &quot;headline&quot;: &quot;Economist Group&quot;,<br> &quot;id&quot;: &quot;<a href="http://www.wikidata.org/entity/Q474153">http://www.wikidata.org/entity/Q474153</a>&quot;,<br> &quot;inLanguage&quot;: &quot;en&quot;,<br> &quot;keyword&quot;: [<br>  &quot;The Economist Group&quot;,<br>  &quot;The Economist Newspaper Limited&quot;<br> ],<br> &quot;license&quot;: &quot;<a href="https://creativecommons.org/publicdomain/zero/1.0/">https://creativecommons.org/publicdomain/zero/1.0/</a>&quot;,<br> &quot;ownershipFundingInfo&quot;: &quot;<a href="https://www.economist.com/about-the-economist#ownership">https://www.economist.com/about-the-economist#ownership</a>&quot;,<br> &quot;masthead&quot;: &quot;<a href="https://www.economist.com/about-the-economist#governance">https://www.economist.com/about-the-economist#governance</a>&quot;,<br> &quot;missionCoveragePrioritiesPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#editorial-philosophy">https://www.economist.com/about-the-economist#editorial-philosophy</a>&quot;,<br> &quot;type&quot;: [<br>  &quot;NewsMediaOrganization&quot;<br> ],<br> &quot;verificationFactCheckingPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#fact-checking-standards">https://www.economist.com/about-the-economist#fact-checking-standards</a>&quot;,<br> &quot;unnamedSourcesPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#unnamed-sources">https://www.economist.com/about-the-economist#unnamed-sources</a>&quot;,<br> &quot;url&quot;: {<br>  &quot;canonical&quot;: &quot;<a href="http://www.economistgroup.com/">http://www.economistgroup.com/</a>&quot;<br> }<br>}</pre><p>For machine consumption, The NewsMediaOrganization data is required to be present on a website’s landing page for the best practices. At <em>The Economist</em>, the data is included on our<a href="https://www.economist.com/about-the-economist"> About Us</a> page, which includes our Trust Project Best Practices. The data is exposed as a JSON object in the page’s HTML markup, inside the HEAD element.</p><pre>&lt; script type = &quot;application/ld+json&quot; &gt; {<br> &quot;ethicsPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#ethics">https://www.economist.com/about-the-economist#ethics</a>&quot;,<br> &quot;diversityPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#diversity-policy">https://www.economist.com/about-the-economist#diversity-policy</a>&quot;,<br> &quot;diversityStaffingReport&quot;: &quot;<a href="https://www.economist.com/about-the-economist#staff-diversity-data">https://www.economist.com/about-the-economist#staff-diversity-data</a>&quot;,<br> &quot;correctionsPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#corrections">https://www.economist.com/about-the-economist#corrections</a>&quot;,<br> &quot;ownershipFundingInfo&quot;: &quot;<a href="https://www.economist.com/about-the-economist#ownership">https://www.economist.com/about-the-economist#ownership</a>&quot;,<br> &quot;masthead&quot;: &quot;<a href="https://www.economist.com/about-the-economist#governance">https://www.economist.com/about-the-economist#governance</a>&quot;,<br> &quot;missionCoveragePrioritiesPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#editorial-philosophy">https://www.economist.com/about-the-economist#editorial-philosophy</a>&quot;,<br> &quot;verificationFactCheckingPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#fact-checking-standards">https://www.economist.com/about-the-economist#fact-checking-standards</a>&quot;,<br> &quot;unnamedSourcesPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#unnamed-sources">https://www.economist.com/about-the-economist#unnamed-sources</a>&quot;,<br> &quot;actionableFeedbackPolicy&quot;: &quot;<a href="https://www.economist.com/about-the-economist#contact-us">https://www.economist.com/about-the-economist#contact-us</a>&quot;,<br> &quot;foundingDate&quot;: &quot;1843-02-09T00:00:00Z&quot;,<br> &quot;<a href="http://twitter.com/context">@context</a>&quot;: &quot;<a href="http://schema.org/">http://schema.org</a>&quot;,<br> &quot;<a href="http://twitter.com/type">@type</a>&quot;: &quot;NewsMediaOrganization&quot;,<br> &quot;name&quot;: &quot;The Economist&quot;<br>} &lt; /script&gt;</pre><p>To implement this schema in your own organization, assuming you’ve already developed the content, designate a page for the primary location of Best Practices. Collect the links required for the Best Practice Indicators and ensure they are organized in the NewsMediaOrganization schema in your data layer. Then expose these in the HTML markup of your Best Practices landing page.</p><p><strong>NewsArticle</strong></p><p>The NewsArticle schema holds data specific to a particular piece of content.</p><ul><li><a href="https://docs.google.com/document/d/1pLUM-JvGDl5NMUNMi2Hl74MnsoBGw_B3cYIAQ8NyO_M/edit#bookmark=id.ejwf0ass672w">News</a> — <a href="http://schema.org/ReportageNewsArticle">Text label: ReportageNewsArticle</a></li><li><a href="https://docs.google.com/document/d/10QQuXvle8bT3hqRiGCaato6qhO_UwaM3-bsUdTsO2j0/edit#bookmark=id.wvh5nm1dukn6">Internal Original Documents and Data</a> — <a href="http://schema.org/citation">[URLs]: citation</a></li><li><a href="https://docs.google.com/document/d/10QQuXvle8bT3hqRiGCaato6qhO_UwaM3-bsUdTsO2j0/edit#bookmark=id.w3k8dc4i7mzi">External Original Documents and Data</a> — <a href="http://schema.org/citation">[URLs]: citation</a></li><li><a href="https://docs.google.com/document/d/10QQuXvle8bT3hqRiGCaato6qhO_UwaM3-bsUdTsO2j0/edit#bookmark=id.y3gtc9plm0yp">Secondary News Sources</a> — [URLs]</li></ul><p><em>The Economist</em> content structure is based on the NewsArticle schema so the protocol implementation required adding the Project specific fields such as publishingPrinciples and citations.</p><pre>{<br> &quot;id&quot;: &quot;21730808&quot;,<br> &quot;type&quot;: [<br>  &quot;NewsArticle&quot;,<br>  &quot;AnalysisNewsArticle&quot;<br> ],<br> &quot;datePublished&quot;: &quot;2017-10-31T00:01:19Z&quot;,<br> &quot;title&quot;: &quot;The political economy of witchcraft&quot;,<br> &quot;url&quot;: {<br>  &quot;canonical&quot;:   &quot;<a href="http://www.economist.com/blogs/graphicdetail/2017/10/daily-chart-21">http://www.economist.com/blogs/graphicdetail/2017/10/daily-chart-21</a>&quot;,<br>  &quot;alias&quot;: [<br>   &quot;<a href="http://www.economist.com/node/21730808">http://www.economist.com/node/21730808</a>&quot;<br>  ]<br> },<br> &quot;citations&quot;: [<br>  &quot;<a href="http://old.absentis.org/ergotism/oster_weater_witches.pdf">http://old.absentis.org/ergotism/oster_weater_witches.pdf</a>&quot;,<br>  &quot;<a href="https://sites.google.com/site/corneliuseconomics/home">https://sites.google.com/site/corneliuseconomics/home</a>&quot;,<br>  &quot;<a href="http://www.peterleeson.com/">http://www.peterleeson.com/</a>&quot;<br> ]<br>}</pre><p>While <em>The Economist</em> has an entire team dedicated to research and fact checking, the process is conducted outside of our primary editing tool. As a result, our citation and reference data is limited to what is directly linked in the article body. Future work will ensure references are associated directly with content, but to fulfill the Trust Protocol a Golang parser was built to extract links from body text and associate them to the citations field.</p><pre>func (a *Article) processCitations() {<br> body := strings.NewReader(a.Text)<br> z := html.NewTokenizer(body)<br> for {<br>  tt := z.Next()<br>  switch {<br>  case tt == html.ErrorToken:<br>   return<br>  case tt == html.StartTagToken:<br>   t := z.Token()<br>   if t.Data == &quot;a&quot; {<br>    for _, v := range t.Attr {<br>     if v.Key == &quot;href&quot; {<br>      a.Citations = append(a.Citations, v.Val)<br>      break<br>     }<br>    }<br>   }<br>  }<br> }<br>}</pre><p>The NewsArticle data is also exposed in an article’s HEAD element as a JSON object.</p><pre>&lt; script type = &quot;application/ld+json&quot; &gt; {<br> &quot;<a href="http://twitter.com/context">@context</a>&quot;: &quot;<a href="http://schema.org/">http://schema.org</a>&quot;,<br> &quot;<a href="http://twitter.com/type">@type</a>&quot;: [&quot;NewsArticle&quot;, &quot;AnalysisNewsArticle&quot;],<br> &quot;url&quot;: &quot;<a href="https://www.economist.com/blogs/graphicdetail/2017/10/daily-chart-21">https://www.economist.com/blogs/graphicdetail/2017/10/daily-chart-21</a>&quot;,<br> &quot;publisher&quot;: {<br>  &quot;<a href="http://twitter.com/type">@type</a>&quot;: &quot;NewsMediaOrganization&quot;,<br>  &quot;name&quot;: &quot;The Economist&quot;,<br>  &quot;publishingPrinciples&quot;: &quot;<a href="https://www.economist.com/about-the-economist">https://www.economist.com/about-the-economist</a>&quot;<br> },<br> &quot;headline&quot;: &quot;The political economy of witchcraft&quot;,<br> &quot;mainEntityOfPage&quot;: &quot;<a href="https://www.economist.com/blogs/graphicdetail/2017/10/daily-chart-21">https://www.economist.com/blogs/graphicdetail/2017/10/daily-chart-21</a>&quot;,<br> &quot;citation&quot;: [&quot;<a href="http://old.absentis.org/ergotism/oster_weater_witches.pdf">http://old.absentis.org/ergotism/oster_weater_witches.pdf</a>&quot;, &quot;<a href="https://sites.google.com/site/corneliuseconomics/home">https://sites.google.com/site/corneliuseconomics/home</a>&quot;, &quot;<a href="http://www.peterleeson.com/">http://www.peterleeson.com/</a>&quot;]<br>} &lt; /script&gt;</pre><p>At <em>The Economist</em> we struct a balance between autogenerated data, such as citations, an editorial curated information, such as article types. Each organization should evaluate what their content editing systems and editorial workflows will provide and how they can supplement it with calculated values in order to fulfill the Protocol requirements.</p><p><strong>Person</strong></p><p>Articles with a byline use the Person schema to provide information about the author.</p><ul><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.g91grarfkn23">Name (byline)</a> — <a href="http://schema.org/name">Text: name</a></li><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.84bnr7pk5oz7">Location</a> — <a href="http://schema.org/workLocation">{place,state,country} and/or latlong: workLocation</a></li><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.g2ezhixg8ot7">Languages Spoken</a> — <a href="http://pending.webschemas.org/knowsLanguage">BCP-47 codes, e.g. “en-UK”: knowsLanguage</a></li><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.qire21ghf1dl">Areas of Expertise</a> — <a href="http://pending.webschemas.org/knowsAbout">Text: knowsAbout</a></li><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.2yag5awtmajr">Biography</a> — <a href="http://schema.org/description">Text: description</a></li><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.nvp84u9dtt2j">Contact Details</a> — <a href="http://schema.org/contactPoint">Email,phone,URL: contactPoint</a></li><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.nu5mtd6g11v8">Persistent ID</a> — Social handle URL, sameAs</li><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.k0kpzeioj9ya">Affiliation with Publisher</a> — <a href="http://schema.org/jobTitle">Text: jobTitle</a></li><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.z7pzpy6lejk8">Archive</a> — <a href="http://schema.org/sameAs">URL: sameAs</a></li><li><a href="https://docs.google.com/document/d/1lK_F_6r__seoE90ze6Kk6geONR3C6OcPmfw7WRJ3GYQ/edit#bookmark=id.552gcwp5xhx">Author/Reporter Page Link</a> — <a href="http://schema.org/sameAs">URL: sameAs</a></li></ul><p>The Person data is mapped to the author field on the Article and included in the HTML markup.</p><pre>&lt;script type=”application/ld+json”&gt;<br> {“<a href="http://twitter.com/context">@context</a>”:”http://schema.org”,<br> “<a href="http://twitter.com/type">@type</a>”:”ReportageNewsArticle”,<br> “author”:{“<a href="http://twitter.com/type">@type</a>”:”Person”,”name”:”Jane Doe”,”sameAs”:[“https://twitter.com/janedoe”, “http://www.economist.com/janedoe&quot;]}}<br> &lt;/script&gt;</pre><p>For organizations without a byline, such as <em>The Economist</em>, a<a href="https://www.scu.edu/ethics/focus-areas/journalism-ethics/programs/the-trust-project/collaborator-materials/trust-protocol-phase-i-mvp-/"> byline exception policy</a> is still being developed and will be implemented in the near future. It is our hope that including an explanation on why we don’t use bylines and information on our editorial staff will provide enough transparency for reader to evaluate how the article was written and the expertise supporting it. Including links to specific <a href="https://docs.google.com/document/d/1Oy1Gbo9MFf4GpeTVSxD-c2tpf6m6odaYNYY1zmL8UGs/edit?usp=sharing">methodology</a> for an article is another avenue we will explore for providing more context on how an article is researched and developed.</p><p><strong>Beyond MVP</strong></p><p>The Trust Protocol indicators will continue to evolve as user feedback is collected and news organizations share their perspectives. <em>The Economist</em> is currently working with UX and Analytics to design trust specific research questions and analytics data points in order to analyze the impact of these indicators, especially in Social Media channels.</p><p>Critical to the success of the Trust Project is additional news organizations taking the initiative to apply the Protocol. The more readers recognize the Trust Logo and seek out the Trust Indicators, the more we can elevate quality journalism. I encourage writers and engineers both to check out the <a href="https://thetrustproject.org/">The Trust Project</a> site for further information and join the movement to restore trust in journalism.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ec1a2785ae6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transitioning Logging and Monitoring Systems at The Economist]]></title>
            <link>https://medium.com/hackernoon/transitioning-logging-and-monitoring-systems-at-the-economist-3c6116ba30a8?source=rss-c73376c8754a------2</link>
            <guid isPermaLink="false">https://medium.com/p/3c6116ba30a8</guid>
            <category><![CDATA[economist]]></category>
            <category><![CDATA[logging]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[engineering]]></category>
            <dc:creator><![CDATA[Jonas]]></dc:creator>
            <pubDate>Thu, 16 Nov 2017 21:21:48 GMT</pubDate>
            <atom:updated>2019-05-23T09:09:29.145Z</atom:updated>
            <content:encoded><![CDATA[<p>At The Economist, transitioning from a monolithic legacy system to a distributed, microservice-based architecture required a fundamental shift in the way we worked as a digital team. While fully engaged in the exciting and challenging process of transforming our technology, our logging and monitoring approach wasn’t able to keep up. We quickly hit the pain points of not translating logging practices from a single, integrated system to a distributed system:</p><ul><li>Logs from different applications have differing schema or no schema at all</li><li>Containerization creates multiple applications running on the same server</li><li>Dynamic host IPs as instances scale or are regenerated</li><li>HTTP requests not easily traceable through distributed applications and services</li></ul><p>Stuck in the monolithic approach of simply sending out system logs to an aggregation tool, the usability of our visibility tools quickly degraded. As a result, Economist digital teams were often left unable to determine why a system had failed and thus response and resolution time to business critical issues suffered. The overall effect was a lack of transparency into our systems and diminished confidence in its ability to perform at scale.</p><p><strong>The Solution</strong></p><p>After going cross-eyed staring at jumbled logs, it was time for a change. A small, grassroots initiative began in the development groups, working with architects and product owners to understand each team’s current logging situation and biggest pain points. These discussions inspired the creation of a logging standard and format guidelines that would ensure all applications used structure logging. Key requirements include:</p><ul><li>All logs in a structure JSON format with key-value pairs</li><li>Use of Log Levels to control what is sent to aggregators and filter in query tools</li><li>Implementation of a Trace ID, generated at the start of a system event and passed to all subsequent calls and subroutines</li><li>Required fields with aligned naming such as service, msec, host, and message</li></ul><p>Additional required fields were documented based on the application type, such as request method and status for client applications. Specific recommendations for implementing the standards in a logging library included:</p><ul><li>Review of existing language specific libraries</li><li>Middleware for log initialization and Trace ID generation</li><li>SHA1/base 32 encoding method for generating a unique Trace ID</li><li>Use of contexts for passing Trace IDs to subroutines and requests</li></ul><p>These standards were circulated around the teams until a consensus was reached, at which point, we had a beautiful document.</p><p><strong>The Actual Solution</strong></p><p>While agreeing on a standard is no insignificant milestone, once it’s been acheived the time comes to dig in and do the work. As this work touched every team in Digital Solutions and was not specific to any one product or project, it was important to coordinate implementation across all <a href="https://hackernoon.com/tagged/engineering">engineering</a> teams. An offhand suggestion to put developers in a room with coffee and sugar for a few days quickly evolved into a proposal for a hackathon focused on logging and monitoring.</p><p>To expedite the process of applying logging standards and ensure alignment, The Economist hosted a Logging and Monitoring Hackathon with participants from each digital team. The goal was to implement the new logging standards and develop dashboards and alerts using the logging format. The hackathon would ensure each team had a participant who could learn the new standards and share them back with their teams.</p><p><strong>Day 1: Kindling</strong></p><p>The first day of the hackathon prioritized implementing the logging standards across our platforms. The morning kicked off with an overview on the new standards and recommendations for how logging libraries should be designed. Each team was then prompted to spend roughly an hour on logging design for their system, examining where specific challenges, such as mobile and third party applications, would require more custom solutions. As teams shared their initial design, we identified key collaboration areas, such as data encryption and security and held feisty discussions around centralized vs decentralized formatting. As the day went on, teams focused on their specific implementation challenges including:</p><ul><li>What library implementation would be best provide ubiquitous logging and ease of use for developers</li><li>Abstracting log formatting to be reusable by different applications</li><li>Lightweight solutions for serverless frameworks</li></ul><p>As the sun set and the coffee pots emptied, PRs were issued and the seeds of standardized logging were planted.</p><p><strong>Day 2: Timber!</strong></p><p>With the beginning of standardized logs in place, Day 2 strategized monitoring requirements. The goal was to define key methods for understanding system latency, traffic, errors, and saturation, as per the <a href="https://landing.google.com/sre/book/chapters/monitoring-distributed-systems.html">Google SRE recommendations</a>. Members of the infrastructure team joined to provide feedback and advice on how to configure different types of monitoring tools. Bringing developers and infrastructure together to focus on the big picture was an opportunity for the team to hash out new ideas and debate the merits of different alerts and dashboards. The age old questions of “Will anyone actually look at this dashboard a month from now?” and “Will I get so many email alerts I start to ignore them all?” arose as we discussed methods for understanding overall system health. There were no obvious answers to these questions. Rather, developers and <a href="https://hackernoon.com/tagged/devops">devops</a> worked together on building solutions that balanced visualizations, dashboards, and alerts that aligned to the most critical elements of system stability. For example, an email alert was built to respond to sudden spikes in HTTP error responses while a simple green-red dashboard was recommended for quick visibility of deployment statues.</p><p>As the day came to an end it was time for the teams to present their progress. Participants organized a brief presentation on their work and plans for moving the work forward. Pizza and beer were ordered, and the rest of the office was invited to watch and participate with questions or suggestions. By decree of the loudest round of applause, a winner was chosen and crowned in The Economist T-shirts as the Logging and Monitoring monarch.</p><p><strong>Key Takeaways</strong></p><p>The work continues. Logging and Monitoring are iterative processes and teams should incorporate logging and monitoring reviews as part regular maintenance and performance work. This is critical for ensuring that the relevant information is recorded and analyzed. At the end of the hackathon, each team left with a strategy for their logging and monitoring and stories in their backlog or defined next steps to move the work forward. Participants also scheduled times to present their work back to their teams. So while the work is never done, the hackathon laid the groundwork and built a shared vision for where our logging and monitoring can grow. More importantly, it brought developers together to share ideas, learn from one another, and maybe, just maybe, fall in love with standardized logging.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fdde502%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fhackernoon%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3c6116ba30a8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/transitioning-logging-and-monitoring-systems-at-the-economist-3c6116ba30a8">Transitioning Logging and Monitoring Systems at The Economist</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>