<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Simon KP on Medium]]></title>
        <description><![CDATA[Stories by Simon KP on Medium]]></description>
        <link>https://medium.com/@simekp?source=rss-1e5f28a3d3dd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Vl_3HB82eJ49IiXkvwfvtw.png</url>
            <title>Stories by Simon KP on Medium</title>
            <link>https://medium.com/@simekp?source=rss-1e5f28a3d3dd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 18:56:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@simekp/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to buy cryptocurrency — first steps]]></title>
            <link>https://medium.com/@simekp/how-to-buy-cryptocurrency-first-steps-ff311e921ab3?source=rss-1e5f28a3d3dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff311e921ab3</guid>
            <category><![CDATA[exchange]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[bitcoin]]></category>
            <dc:creator><![CDATA[Simon KP]]></dc:creator>
            <pubDate>Sat, 05 May 2018 07:43:20 GMT</pubDate>
            <atom:updated>2018-05-05T08:00:11.812Z</atom:updated>
            <content:encoded><![CDATA[<p>There is a lot of theoretical and speculative material out there regarding cryptocurrency trading, blockchain technologies and their possible use cases. However when it comes time to purchase your first <em>ether</em> or <em>bitcoin</em>, it is time to roll up your sleeves and open your wallet. Note: I’m not talking about investing, technical analysis of the graphs etc. This is just basic getting your feet wet purchasing a little bit of this crypto stuff.</p><p>To purchase cryptocurrency with your fiat money you’ll need to go to an exchange. The most popular one, with the newbies like me, seems to be Coinbase. I first created my account with Coinbase awhile back but didn’t end up going through with all the verification steps needed to actually Buy and Sell. To do that you’d need to verify your identity by providing an image of your photo ID and link either a credit card or bank account to transfer the funds.</p><p>After that is done, one other verification that Coinbase does is create a pending transaction of somewhere between $1 and $2 into your linked account for you to verify the decimal digit in that transaction. The transaction itself will expire after 10 days and your account will not be charged. However I had some trouble with this as my ID and credit card I was using were from different countries so the numbers displayed on my statement were in the currency different to what was needed for confirmation. So make sure the country in your Coinbase profile matches that of your bank.</p><p>While I was having these issues I decided to check out some of the alternative exchanges that are out there. The other two that I came across that sounded trustworthy are GDAX and CEX. Coinbase and GDAX are actually two products by the same company and so you will be able to re-use your Coinbase login to authenticate to GDAX, however document verifications will need to be done again. CEX is a separate exchange from Great Britain with document verifications similar to Coinbase which I didn’t manage to get through at this time.</p><p>The difference between Coinbase and GDAX is that Coinbase functions basically as a hosted <a href="https://hackernoon.com/crypto-wallet-vs-address-54f7fb980bd3">wallet</a> and lowers the barrier of entry into cryptocurrency trading world for newcomers, albeit at a higher fees than GDAX (around 1.5% compared to .25%). GDAX on the other hand, is a full blown exchange for <em>professional traders. </em>The overview panel it presents and some of the terms used might be overwhelming for someone who’s new to the game. Where is Coinbase has a very nice intuitive and responsive interface together with fairly easy to interpret cryptocurrency’s overview of price graphs. One thing to note is that transfers are free between Coinbase and GDAX.</p><p>After the purchase is complete it is a good idea to spread the cryptocurrency across a number of wallets rather than keeping the lot on one exchange.</p><p>For example you could transfer some of the freshly bought currency into GDAX and trade for another currency.</p><p>Then transfer some to other, <a href="https://coincenter.org/entry/what-is-multi-sig-and-what-can-it-do">more secure wallets</a>. Amongst other wallet security guidelines, having secure access and storage of the <em>private key</em> associated with your wallet is paramount to the security of your currency.</p><p>The steps above should give you a chance to get your head around how transfers work, get familiar with any fees associated with them and most importantly the management of wallets and associated keys.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff311e921ab3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Practical Services Monitoring with Prometheus and Docker]]></title>
            <link>https://medium.com/airtame-engineering/practical-services-monitoring-with-prometheus-and-docker-30abd3cf9603?source=rss-1e5f28a3d3dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/30abd3cf9603</guid>
            <category><![CDATA[prometheus]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[services]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Simon KP]]></dc:creator>
            <pubDate>Tue, 28 Mar 2017 12:31:00 GMT</pubDate>
            <atom:updated>2017-03-29T21:22:25.639Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://airtame.zendesk.com/hc/en-us/articles/208889129-Airtame-Cloud">Airtame Cloud</a> is a web service which enables organisations to remotely monitor and administer large deployments of <a href="https://airtame.com/is">Airtame</a> devices. We take availability and monitoring very seriously to keep helping our customers accomplish their goals and needs. The following introduces our monitoring configuration setup that helps us avoid failure and remain resilient.</p><p>The cloud web services being monitored are deployed as containers running inside a <a href="https://docs.docker.com/engine/swarm/">Docker Swarm</a> cluster on AWS infrastructure. Also the monitoring system itself is deployed as Docker containers, using the official Docker images. Configuration management and provisioning is done with <a href="https://www.ansible.com/">Ansible</a>. We use <a href="https://prometheus.io/">Prometheus</a> as a monitoring system which itself has a very modular and fairly straight forward deployment model. The main components being: Prometheus server itself, the <a href="https://prometheus.io/docs/alerting/alertmanager/">Alertmanager</a> which manages alerts sent to it by Prometheus server(s), and lastly, various <em>exporters</em> which Prometheus scrapes for metrics. Prometheus was chosen over a myriad of other monitoring solutions for it’s very active and open development, seamless integration with other software tools, clear documentation, and of course, its functionality, which we’ll explore below. Let’s dive in.</p><h4><strong>Types of Prometheus exporters</strong></h4><p>Prometheus scrapes these for various <a href="https://prometheus.io/docs/concepts/metric_types/">types of metrics</a>. All of these but <em>Node Exporter</em> are deployed as Docker containers to query various services and expose <em>&lt;IP address&gt;/metrics</em> endpoint for Prometheus to scrape. <em>Node Exporter</em> runs outside Docker as a system daemon to be able to access the host’s low level system metrics.</p><ul><li><strong><em>Node Exporter — </em></strong>Runs on each EC2 instance as a daemon and exposes system metrics like I/O, memory and CPU.</li><li><strong><em>Cloud Metrics Exporter — </em></strong>Custom exporter, written in-house, that shows us some important metrics by querying production databases.</li><li><strong><em>MySQLd Exporter — </em></strong>One of these per MySQL instance. Queries each environment’s database instances.</li><li><strong><em>Blackbox Exporter — </em></strong>Blackbox monitoring can be seen as “monitoring from outside”. It simply cares whether the instance is up or down.</li><li><strong><em>cAdvisor — </em></strong>Exposes resource usage data and performance characteristics of running containers.</li></ul><h4>Target Service Discovery</h4><p>We use 2 Prometheus servers in what’s called <a href="https://prometheus.io/docs/operating/federation/">Federation</a>. Federation allows a Prometheus server to scrape selected metrics from another Prometheus server. So we have one Prometheus server running as a Docker Swarm service on the same virtual network as the cloud services to be able to scrape them. The sole purpose of this Prometheus server is to scrape these services. To avoid confusing the two, let’s call this server “<em>services-prom”</em>. The second Prometheus server scrapes <em>services-prom</em>, along with having the most of the monitoring configuration for our infrastructure. Let’s call this server “<em>main-prom”</em>. Running Prometheus server on the same virtual Swarm network as the services allows us to use Prometheus DNS service discovery to find out which endpoints to scrape:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d801eeb7be19c098419ba650f51948e/href">https://medium.com/media/6d801eeb7be19c098419ba650f51948e/href</a></iframe><p>Here we have a job configured with DNS service discovery inside the <em>services-prom </em>configuration file. With services being created and destroyed all the time, we can’t just hardcode their addresses for Prometheus to scrape. Not only this would be a bad configuration management practice but we also need a more flexible setup for this system’s architecture. In Docker Swarm, while inside the created virtual network, we can run a special DNS query with <em>tasks.myservice</em> and get back all the IP addresses for the running Swarm tasks with that name. This can be tested with nslookup command from inside a container running on the virtual network:</p><pre># nslookup tasks.myservice<br>Server:    127.0.0.11<br>Address 1: 127.0.0.11</pre><pre>Name:      tasks.myservice<br>Address 1: 10.0.0.13 myservice.1.ygx1zy6cuay9w3ownyplzux4w.proxy<br>Address 2: 10.0.0.15 myservice.2.iutlkj12kdu90s0dldkdk4dnl.proxy</pre><p>The addresses returned and the port specified are then used to build the list of endpoints for Prometheus to scrape. The port number is the one that we expose with Docker which has our application listening inside the container.</p><p>When creating the Prometheus server Docker service (<em>services-prom),</em> we use Docker service constraint to only run <em>services-prom</em> on a specified node so we always know its IP address and port. This information is then used by the second (<em>main-prom)</em> server when scraping for the specified metrics. Federation job is then defined in <em>main-prom </em>configuration as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/57dc7bac0cafb313a2aac5cd5dc1ced3/href">https://medium.com/media/57dc7bac0cafb313a2aac5cd5dc1ced3/href</a></iframe><h4><strong>Host, Container and Service monitoring</strong></h4><p>When thinking about services monitoring in the cloud, one can break it up into individual components that are equally important. The main components are <em>system metrics, </em>which we get from EC2 instances and Docker containers. These metrics assure us that the infrastructure is ready to accept the load of the services. The second component is customer facing<em> service state</em>. We need to know that the software we are running actually provides value. Once we have these metrics we use <em>dashboards</em> to graph and easily understand the view of the world. The last component is useful <em>alerting</em> that will let us know when the key metrics have crossed their threshold before they cause any outages.</p><h4><strong>Host System Metrics</strong></h4><p>Prometheus’ <em>Node Exporter</em>, which is deployed with Ansible, runs on each monitored host in EC2 as a system daemon and exposes system metrics at <em>&lt;IP address&gt;:9100/metrics</em> endpoint. Of course it’s not practical to keep a list of IP addresses to monitor so the monitored hosts are automatically discovered by Prometheus EC2 service discovery. This is configured in <em>main-prom</em> configuration as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/45e3b0a48675edbb887210ea1eaecb3a/href">https://medium.com/media/45e3b0a48675edbb887210ea1eaecb3a/href</a></iframe><p><em>Note</em>: <em>acccess_key</em> and <em>secret_key</em> contain the keys for a user created in IAM with AWS managed <em>AmazonEC2ReadOnlyAccess </em>policy.</p><p>By default, Prometheus will label instances with the private IP addresses. We use the following to relabel the targets to the <em>instance</em> tag:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/364bcc2e77f64aa920dae6942e193d79/href">https://medium.com/media/364bcc2e77f64aa920dae6942e193d79/href</a></iframe><p><em>Note:</em> instance’s unique EC2 Name tag is set in <a href="https://www.terraform.io/">Terraform</a> where the instance is defined.</p><p>Once we have these system metrics in Prometheus, we use <a href="https://grafana.com/">Grafana</a> to query Prometheus and build dashboards with the data available. We display hosts’ and services’ metrics and their health, also some application-specific numbers from our custom exporter. The dashboards are then accessible to everyone in the company with each team being able to build a dashboard that is the most appropriate to them. We have <em>a lot</em> of TVs hanging on the walls around the office so we use Airtame’s <a href="https://airtame.com/display-dashboard">dashboard display</a> function to get some more use out of them.</p><h4><strong>Container System Metrics</strong></h4><p>In a very similar fashion to Host monitoring above, we configure Prometheus to scrape the container’s system metrics exported by <a href="https://github.com/google/cadvisor">cAdvisor</a> on port <em>8080</em>. cAdvisor is deployed with Ansible and runs inside a Docker container on every monitored host. This time, we only want to monitor those hosts that have containers running on them. Similarly to the <em>ec2_instances</em> job configuration above, we configure a job called <em>ec2_containers </em>to only scrape instances that match the regex specified. The relevant snippet is below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/012cda468a356d5c1e3cf4c46d28d61a/href">https://medium.com/media/012cda468a356d5c1e3cf4c46d28d61a/href</a></iframe><h4><strong>Service Health and Instrumentation</strong></h4><p>Not only do we need to know the health of our hosts and containers by monitoring various system metrics but we also need to know the health of the public facing services. The idea is to be able to identify which service components are responsible for the issues we’re seeing. For example, is it the service itself at fault or any of it’s down streams, like a database?</p><p>Each service/container exposes its metrics at <em>&lt;IP address&gt;:&lt;port&gt;/metrics</em> endpoint, which are then scraped by <em>services-prom</em> server periodically as explained above.</p><p>First and foremost, we need to know the state of a service, its availability. For this, we utilise Prometheus <em>up{job=”myservice”}</em> default time series which is populated after each scrape. The value of 1 is returned when the scrape succeeded and 0 if the scrape failed. Of course we can alert on this, as we’ll see below.</p><p>Other service specific metrics we track are <em>requests/second</em> per service function and also endpoint <em>response times</em>. This is the area that needs to be paid more love and attention. More visibility is required into service’s key metrics such as <em>latency</em>, <em>number of clients,</em> and/or <em>response success</em> and <em>errors</em>. Our services are written and instrumented in <a href="https://golang.org/">Golang</a> but there are a number of Prometheus <a href="https://prometheus.io/docs/instrumenting/clientlibs/">client libraries</a> available that can be used to add code instrumentation depending on the language of the application.</p><h4><strong>Alerting</strong></h4><p>Automated alerts are crucial to any monitoring system setup so Alertmanager is deployed beside <em>main-prom</em> server. It can actually handle alerts sent to it from multiple Prometheus servers and dispatch notifications with automatic de-duplication and grouping. We have standard system alerts configured in Prometheus like high CPU, low disk space etc. But also some alerts specific to <a href="https://airtame.cloud/">Airtame Cloud</a> like the total number of online devices, which should not be below a certain number. If it is, then there is probably a problem. An alert definition in Prometheus can look like this:</p><pre>ALERT InstanceHighCpu<br>  IF 100 - (avg by (instance) (rate(node_cpu{mode=&quot;idle&quot;}[5m])) * 100) &gt; 90<br>  FOR 20m<br>  ANNOTATIONS {<br>    summary = &quot;High CPU Usage on {{ $labels.instance }}&quot;,<br>    description = &quot;CPU usage exceeds threshold (currently {{ $value|humanize }}% in use)&quot;,<br>  }</pre><p>Here, we alert on instance’s CPU being over 90% for 20 minutes. Notice the test definition is in Prometheus’ very powerful <a href="https://prometheus.io/docs/querying/basics/">query language</a> (PromQL). The same query is used when defining alerts as well as in Prometheus’ web interface. The web UI can be useful while coming up with an alert condition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0tHZFtiyCrODzb-UEv3s8g.png" /><figcaption>Prometheus UI with PromQL query</figcaption></figure><p>If an alert has been triggered, Prometheus will dispatch a message to Alertmanager which in turn sends alerts to the configured <em>receivers</em>. An example alert payload may look something like the following. It’s a JSON message that can be sent as a test to Alertmanager receiver’s API with this script using curl command:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8f08c4193994fa28691df4d1eee06df4/href">https://medium.com/media/8f08c4193994fa28691df4d1eee06df4/href</a></iframe><p>Notice here we set labels manually, where, in practice, Prometheus will gather metrics as metadata from the service discovery method configured with the job. The concept of labels is very powerful as we already saw when we were filtering targets to scrape with a regex.</p><p><em>Slack</em> and <em>Pushover</em> are the two of many different receivers that can be configured in Alertmanager. <a href="https://pushover.net/">Pushover</a> is setup on the mobile phones of the engineers who are on-call. Slack channel includes all the engineers, with notifications enabled, who may be interested in the status of the cloud infrastructure. Over are the days of email alert notifications.</p><h4><strong>Formatting alert messages</strong></h4><p>The Slack channel receiving alerts includes engineers from different teams, some of which are not cloud infrastructure related. We need to make alerts as clear as possible to make sense to everyone. This is what default Slack alert looks like, not very informative to a person seeing one of these for the first time:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*Sr1HsvZZhPQ9PMHj." /></figure><p>We can see, in the square brackets, that the alert is in the <em>firing</em> state and the number of alerts of that type. Followed by the name of the alert and a bunch of alert labels in brackets. With the knowledge of Prometheus’ labels we can make it more descriptive to an engineer responding to this alert.</p><p>This is a snippet from Alertmanager’s <em>alertmanager.yml</em> configuration file showing one of the <em>receivers</em> configuration, in this case Slack:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8f25a2f1c5b19fbddacf7a5af135bca1/href">https://medium.com/media/8f25a2f1c5b19fbddacf7a5af135bca1/href</a></iframe><p>We customise the message that gets sent to Slack in case of an alert by templating the title and text attributes. Taking advantage of the <a href="https://prometheus.io/docs/visualization/template_reference/">templates</a> we can make the alert more readable with relevant information and also point an operator to the documentation associated with the alert. The link to the alert docs, containing instructions and tips, will come in very handy to an engineer that has to deal with this alert at 3am. So with a little tweaking we end up with this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*ImiMV-WlBjy-ccgZk-usbA.png" /></figure><h4><strong>This is only the beginning</strong></h4><p>We’ve seen how to get up and running with a monitoring system that supports dynamic cloud environments. The complex systems that we deal with today have a large number of moving components interacting with each other at will. So the end to end visibility is crucial to operating a resilient web service. Some things to keep in mind going forward:</p><ul><li>The ability to deal with long lived metrics. This is actively being worked on by Prometheus developers. There is a need for data retention collected for a year or more to have insight into what happened in the past. This would help establish a baseline — to know what “normal” is.</li><li>With so many metrics coming in by the second it might be tempting to go out and build all the dashboards. One thing to remember is that the data displayed needs to be actionable.</li><li>Instrumentation is hard and needs to be done better, to have as much visibility into service operations as possible. As well as the metrics mentioned above, we need to be able to see the impact on the infrastructure after each service deploy.</li><li>Not all alerts are equal. Different alerts have different degree of urgency. Alertmanager allows us to configure severity so we know when human intervention is needed and when it’s ok to wait ‘til the morning.</li><li>Currently, our monitoring system sits very close to the infrastructure it monitors which is not very nice for monitoring redundancy. So we would also like to have another Prometheus server either in a different AWS region or even with a different cloud provider for redundancy.</li></ul><p><em>If you have any thoughts on this post please do let us know! And if it sounds like something you’d like to work with, we’re always on the lookout for talented engineers. Check our our </em><a href="https://airtame.com/jobs"><em>open positions</em></a><em> and get in touch.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=30abd3cf9603" width="1" height="1" alt=""><hr><p><a href="https://medium.com/airtame-engineering/practical-services-monitoring-with-prometheus-and-docker-30abd3cf9603">Practical Services Monitoring with Prometheus and Docker</a> was originally published in <a href="https://medium.com/airtame-engineering">Airtame Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>