<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[ITA Labs - Medium]]></title>
        <description><![CDATA[High load identification and management systems.  Architecture design and custom development - Medium]]></description>
        <link>https://medium.com/italabs?source=rss----c114ea447f28---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>ITA Labs - Medium</title>
            <link>https://medium.com/italabs?source=rss----c114ea447f28---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 09:12:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/italabs" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Token and digital certificates life-cycle management system]]></title>
            <link>https://medium.com/italabs/token-and-digital-certificates-life-cycle-management-system-3c8689fa4e09?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/3c8689fa4e09</guid>
            <category><![CDATA[projects]]></category>
            <category><![CDATA[x509]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[smart-cards]]></category>
            <dc:creator><![CDATA[Alexander Solovey]]></dc:creator>
            <pubDate>Wed, 19 Aug 2020 10:43:23 GMT</pubDate>
            <atom:updated>2020-08-19T10:43:23.577Z</atom:updated>
            <content:encoded><![CDATA[<p>Development of server side components of the token and digital certificates life-cycle management system for use in a corporate environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aibEo9tz1i3qM2pgBhPLmg.jpeg" /></figure><h3>Highlights</h3><p>· Development of server components of a specialized high-load fault-tolerant software system</p><p>· Scales up to support 100,000 concurrent users</p><p>· Expandable functionality (10+ extension points for various plug-ins development)</p><p>· Fast deployment and configuration</p><p>· Countrywide integration</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4N6hLdbkf9URsJfRJq88ZA.jpeg" /></figure><h3>The challenge</h3><p>The customer had a need to automate the accounting and maintenance of tokens (and X.509 certificates on these tokens), which are in use by employees, in order to reduce costs and reduce employee downtime. Moreover, the number of simultaneously served users can reach 100,000.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3VWbpXQPjv3CtHvc8-z_Hg.jpeg" /></figure><h3>The solution</h3><p>The contractor implemented server components of the corporate application platform which became a basement for automatic tokens and certificates accounting and life cycle management system (to whom it belongs, who issued it and when it was revoked, etc.), with reference to users and the organization hierarchy and with the possibility of automatic certificates issuance for these tokens through various types of CAs. The system is fault tolerant and supports horizontal scaling. The system is very flexible — it is expanded by plugins and connectors due to the large number of expansion points.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C-0JyGlF5xe3lJDKk5yhpw.jpeg" /></figure><h3>The results</h3><p>The customer got a solution that saves a lot of time and resources on the maintenance of tokens and certificates, as well as build new corporate applications that are seamlessly integrated with each other.</p><p>An ordinary user of the organization has no downtime due to waiting for certificate renewals, there is no need to track the life of certificates on tokens and manually compose applications for certificate renewals, and make calls to IS administrators. The profile mechanism ensures uniformity of certificate issuance policies applied to users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*khPir1FkcTt27YGEa_cNuQ.jpeg" /></figure><p><a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=portfolio_certmgmnt"><em>ITA Labs</em></a><em>, High load identification and management systems. Architecture design and custom development.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3c8689fa4e09" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/token-and-digital-certificates-life-cycle-management-system-3c8689fa4e09">Token and digital certificates life-cycle management system</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Web-based system for managing incidents and internal investigations in the corporate environment]]></title>
            <link>https://medium.com/italabs/web-based-system-for-managing-incidents-and-internal-investigations-in-the-corporate-environment-694e5729aa87?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/694e5729aa87</guid>
            <category><![CDATA[portal]]></category>
            <category><![CDATA[italabs]]></category>
            <category><![CDATA[projects]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Alexander Solovey]]></dc:creator>
            <pubDate>Thu, 13 Aug 2020 19:48:53 GMT</pubDate>
            <atom:updated>2020-08-13T19:48:52.977Z</atom:updated>
            <content:encoded><![CDATA[<p>The development of a software system for managing incidents and internal investigations in a corporate environment and designed to process and analyze information about violations of corporate ethics, fraud, and the facts of theft.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uLaBjpMibZdgiKHZL1sx6Q.jpeg" /></figure><h3>Highlights</h3><ul><li>Development of a specialized software package</li><li>Incident and Internal Investigation Management System</li><li>SMS and E-mail notifications</li><li>Implementation and integration at the company group level</li><li>SaaS deployment</li><li>Document flow</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mypMNlMVWKHM3F4NFHFeLw.jpeg" /></figure><h3>The challenge</h3><p>The customer required a specialized software suite for incident management and internal investigations in a corporate environment. The system has to be designed to process and analyze information on violations of corporate ethics, fraud, facts of theft, and was supposed to inform participants throughout the entire cycle of internal investigation, provide analytics on incidents and completed investigations.</p><p>The system’s capabilities were supposed to include managing the progress of the investigation, storing accompanying documents, notifying the user by E-mail and SMS about major events based on the role-based security model, creating custom reports that can be viewed both online and sent out according to a preset schedule. The system was supposed to carry out full-text semantic search in documents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WaV5MNF0Raf4bokgzbIAyw.jpeg" /></figure><h3>The solution</h3><p>The contractor studied in detail the customer’s non-trivial business processes, defined the rules for state transitions of system entities and the relationship between processes. An optimal solution was proposed and agreed with the customer, which exactly met his expectations.</p><p>The contractor, anticipating the use of the complex in various fields of activity and for different customers, has designed a solution in accordance with the requirements, which can be deployed and used as a service by the customer’s external client.</p><p>A flexible role model was designed with the possibility of subsequent expansion with new access rights to parts of the system. The notification system can be easily extended with a new types.</p><p>The product consists of several main modules:</p><ul><li>Database</li><li>Request management system</li><li>Modules for sending SMS and E-mail notifications</li><li>Web-portal for managing incoming messages from subscribers</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y4AcTgy_5Gbi-7RXyRutkA.jpeg" /></figure><h3>The results</h3><p>Despite the short timeframe, contractor has developed the software system that fully satisfied the customer’s requirements and solved the assigned tasks.</p><p>The product successfully passed bench tests in the customer’s infrastructure and was deployed in the product environment. At the moment, the complex is being actively used and modernized.</p><p>By introducing the software package, the customer automated the process of registering messages from subscribers, their processing and passing through various authorities, thereby reducing the cost of document circulation and increasing the speed of decision-making on incoming requests.</p><p>The product turned out to be highly customizable and extensible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9HpAiUAGQv0frHsEwILPaw.jpeg" /></figure><p><a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=portfolio_hotline"><em>ITA Labs</em></a><em>, High load identification and management systems. Architecture design and custom development.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=694e5729aa87" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/web-based-system-for-managing-incidents-and-internal-investigations-in-the-corporate-environment-694e5729aa87">Web-based system for managing incidents and internal investigations in the corporate environment</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scalable distributed biometric video monitoring and face tracking system]]></title>
            <link>https://medium.com/italabs/scalable-distributed-biometric-video-monitoring-and-face-tracking-system-1c1b618c977f?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/1c1b618c977f</guid>
            <category><![CDATA[highload]]></category>
            <category><![CDATA[video-surveillance]]></category>
            <category><![CDATA[projects]]></category>
            <category><![CDATA[biometrics]]></category>
            <category><![CDATA[italabs]]></category>
            <dc:creator><![CDATA[Alexander Solovey]]></dc:creator>
            <pubDate>Fri, 26 Jun 2020 13:54:42 GMT</pubDate>
            <atom:updated>2020-06-26T14:20:34.542Z</atom:updated>
            <content:encoded><![CDATA[<p>Face Tracking System is a scalable distributed biometric video monitoring software. It is designed to detect, track and identify people by faces in live video streams, generates events in real time, accumulates historical data, provides access to them through the API and web portal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MhWdg1Dg_ybx65z_FxceWw.jpeg" /></figure><h3>Highlights</h3><p>· Distributed video monitoring and tracking system</p><p>· Hybrid scalable architecture: on-prem and cloud</p><p>· IP-cameras connectivity and configuration</p><p>· Biometry/Faces providers integration</p><p>· Real time faces detection up to 20 in the single frame</p><p>· Fast face search in real time</p><p>· Event consumers (including access control systems) integration</p><p>· Web portal and on-prem agent for the system monitoring and management</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0V1uwpk_Acvh4hoWa5MTlA.jpeg" /></figure><h3>The challenge</h3><p>The customer needed to develop a video monitoring system for use in solving automated biometric access control tasks.</p><p>On the one hand, as an automated video monitoring system, it had to be able to work with IP cameras, track people on their video streams in real time, and identify them by database. On the other hand, it had to have two external integration loops: one for using the existing biometric database, the other for communicating with the access control system, transmitting events to it.</p><p>In addition to this, there were requirements for collecting a historical audit, providing it to both users on the web portal and external integration systems via the API.</p><p>Moreover, the system was supposed to be distributed (including geo), scalable and multi-tenant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aPhrlWhMjv0cL9468OTwgQ.jpeg" /></figure><h3>The solution</h3><p>The contractor, taking into account the uncertainties of requirements for various parameters, as well as seeing the possibilities for generalizing the solution, designed the most flexible system, without relying on its application directly only in the field of access control.</p><p>The system consists of several modules divided into 2 groups:</p><p>• On-prem group, modules that are deployed directly to the consumer, include a video server (Video Server), processing video streams from surveillance cameras, its database (Database) and management agent (Management Agent).</p><p>• Cloud group, the modules that are deployed in the cloud include a cloud server (Cloud Server) for managing on-prem groups, its database (Database) and web portal (Web Portal).</p><p>Integrations with existing systems (providers of events, event consumers, consumers of historical data) are maximally generalized, and do not know anything about the specifics of the application (access control systems, or anything else).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TUT7ktLdA7FNRqdkW2aOQA.jpeg" /></figure><h3>The results</h3><p>The system was successfully implemented at the end customer and allowed to execute ACS scenarios in a contactless mode in a pandemic 2020.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ch9JE-Y47nzgW2IjPpdI1Q.jpeg" /></figure><p><a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=portfolio_facetracker"><em>ITA Labs</em></a><em>, High load identification and management systems. Architecture design and custom development.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c1b618c977f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/scalable-distributed-biometric-video-monitoring-and-face-tracking-system-1c1b618c977f">Scalable distributed biometric video monitoring and face tracking system</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deadlocks in SQL Server — part1: What the locks are?]]></title>
            <link>https://medium.com/italabs/sql-server-deadlocks-part1-b9926aec779a?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/b9926aec779a</guid>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[italabs]]></category>
            <category><![CDATA[deadlock]]></category>
            <category><![CDATA[stories]]></category>
            <dc:creator><![CDATA[Alexander Solovey]]></dc:creator>
            <pubDate>Mon, 04 Feb 2019 11:01:01 GMT</pubDate>
            <atom:updated>2019-03-06T20:38:56.237Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deadlocks in SQL Server — part1: What the locks are?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K9VX8Oq6GqbgiNaVo0cXAw.jpeg" /></figure><h3>Introduction</h3><p>Complex, distributed, high-load systems very often experience the concurrent data access issue which triggers various kinds of locks in database queries. Besides, deadlocks may also occur, and they need to be dealt with.</p><p>We have started a series of articles dedicated to locks in databases. In these articles, we’ll share our experience accumulated in <a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=deadlocks1_en">our company</a> on the following issues:</p><p>- What types of locks we used while developing our products;</p><p>- What diagnostic tools and means we used to reveal deadlocks;</p><p>- How we dealt with deadlocks.</p><p>In this article series, we’ll talk about locks we came across when developing software products.</p><p>Before we proceed, let’s briefly recollect why we need locks in the first place.</p><h3>Why locks are necessary</h3><p>In distributed systems used as database storage, various side effects with parallel data access may emerge:</p><p>- The <strong>lost update</strong> issue happens when two transactions simultaneously UPDATE the same row, and modifications introduced by one transaction are erased by another one;</p><p>- <strong>Dirty read</strong> is such a read when data from some other transaction, which is subsequently rolled back, may be read, added, or modified;</p><p>- <strong>Non-repeatable read</strong> manifests itself when the previously read data is modified during the rereading operation in a single transaction;</p><p>- <strong>Phantom reads</strong> can be witnessed when one transaction is executed selecting multiple rows several times according to the same criteria. At the same time, in the intervals between these selections, another transaction adds or deletes rows or modifies the columns of some rows, which are used in the selection criteria for the previous transaction, and ends successfully. As a result, the same selections in the previous transaction return different sets of rows.</p><p>Modern DBM Systems are capable of solving these issues, using built-in mechanisms. For example, MSSQL Server executes this operation, using different levels of transaction isolation:</p><p>- Read uncommitted;</p><p>- Read committed;</p><p>- Repeatable read;</p><p>- Serializable;</p><p>- Snapshot.</p><p>In turn, the locking mechanism is used to implement the transaction isolation requirement. Consequently, the locking mechanism is designed to solve concurrency problems. It’s also worth noting that locks can lead to performance degradation, deadlocks, and data loss if used improperly.</p><h3>Locks</h3><p>MS SQL Server includes many different types of locks, and many good articles and technical documentation have been written on this topic. In this article, we’ll cover only a part of them. They’re the locks we experienced in practice while developing our products.</p><h3>The task of data extraction from the queue</h3><p>The classic task when locking is used is the item extraction from the queue with which a limited number of clients can simultaneously work. Only one queuing client can gain access to such an element at a certain time.</p><p>In several of our products, we use a queue, stored in a database, access to which has to be synchronized among clients. Let’s see, how we solved this issue in our products.</p><p>Suppose we have a queue of tasks for clients, which is stored in the database. Each task can be simultaneously accessed by a maximum of N clients. Therefore, we have a connection of the task with its results on our hands — it’s a 1-N connection. This aspect brings in additional complexity since the data extracted is stored in two tables linked by a foreign key. The tables have a structure like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/433/1*Wx5RBB-_7BfsxLwNgUjX7Q.png" /><figcaption>Fig. 1</figcaption></figure><p>Task — the table with tasks</p><p>TaskResults — the table with task results</p><p>To solve this issue, we used the update lock (UPDLOCK) together with the ROWLOCK and READPAST hints applied to the Task root table entries.</p><p>For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/747/1*JjssnX8FxNN0Q1Ze4jlEGw.png" /></figure><p>Using the UPDLOCK operator guaranteed that other clients were unable to access the task. Using the ROWLOCK hint, we recommended MS SQL Server to use the lock at the row level, and the READPAST hint chose only those entries which weren’t blocked by other transactions. Besides data extraction, we also included the UPDLOCK and ROWLOCK hints in all data update operations for both root and child tables.</p><p>That’s the way we solved the issue of client concurrent access to tasks.</p><h3>Extracting data modified simultaneously</h3><p>In high-load systems developed by ITA Labs, it’s often necessary to monitor data modified by parallel transactions. If data condition only in a particular point in time matters for us, there’s no need to set any lock at all. In that case, there is a risk of reading inconsistent data at a particular moment, but it’s eventually updated after some time. In addition, we increase the displayed data reading speed instead of waiting for locks release. This is very important for the timely software response which increases the UI user-friendliness.</p><p>In one of the developed products, we had to display the history of the tasks performed by a specific operator, as well as select or filter instructions conforming to certain criteria. Because a number of clients with update operations accessed these data simultaneously, the table was locked, and there was no possibility to read data during the required time. Therefore, a timeout occurred. We solved this issue by reading the data without locking but using the NOLOCK hint. Though, we have to admit now that this method is not suitable if it’s necessary to take any decision based on the data read. For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*d0yMFTw006fzIGfn0alxiQ.png" /></figure><p>The NOLOCK hint instructs SQL Server to skip using locks when executing this transaction, and as a result, data is read quickly. However one particular point is worth noting here — one type of lock will be actually set even when the NOLOCK hint is used. What we are talking about is a schema stability lock (Sch-S). Nevertheless, it’s actually a rare occasion when a database schema change is required during business logic execution affecting the work with the database, and therefore, this lock won’t have any effect. However, in the systems where the database schema is changed simultaneously with access to the data stored in the database, we should take this circumstance into account and use other locking mechanisms.</p><h3>The task of synchronizing access to the data modified</h3><p>In distributed systems, we often come across the task of synchronizing access to data and its subsequent modification. There are similar cases in our products as well. In these instances, we have to use the UPDLOCK hint in the data extraction operator to prevent updates. This way we ensure that the data, retrieved for subsequent modification, is locked until the transaction is complete.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*dQ03kBb-0ifYA4APKf8vAg.png" /><figcaption>Fig. 2</figcaption></figure><p>Let’s consider the example shown in Fig. 2. Suppose we have two concurrently executed transactions selecting the second row from the Table for later modification. To prevent simultaneous access to this row for its modification, we applied the UPDLOCK hint on the SELECT operator. Under these circumstances, the transaction that used the update lock first gains access to the second row. The second transaction will wait until the applied lock is released.</p><h3>Lock escalation management</h3><p>Lock escalation is a process aimed at optimizing server operation, which allows replacing multiple low-level locks with one or more locks of the higher level.</p><p>For example, if we applied many row-level locks, and all the rows belong to one page, the server can replace them all with a single page lock and then with a table lock to save resources (see Fig. 3).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/951/1*9Ng9hbhceCx3hHURuRYFkw.png" /><figcaption>Fig. 3</figcaption></figure><p>In our products, we used lock escalation option management where it was necessary as well. In one of the projects, we came across a lock escalation option going from the row level to upper levels like pages or extents. The escalations occurred when the load on several related tables became high. Transactions locked each other and were executed by mistake. To solve this issue, we disabled lock escalation for some tables referring to many other tables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*g_bhbvFud65eYnkAgaRAXw.png" /></figure><p>After disabling lock escalation, the issue with deadlocks was solved. The overall system throughput also increased because the total number of locks significantly decreased.</p><p>We also have to note that it would be a poor decision to abuse this option and disable escalation in the environment where it operates properly. Unreasonably applied lock escalation can adversely affect the SQL Server performance and reduce the speed at which SQL instructions are executed.</p><h3>Deadlocks</h3><p>A deadlock is a situation where one process requires the resource occupied by another process to continue its execution, and the latter process needs access to the resource used by the former process. In this situation, both processes are in a locked condition and cannot continue their execution.</p><p>Let’s consider the simplest example. Suppose we have a table listing some students and consisting of 2 rows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/247/1*Wc4VqsJFswz4SZ-LI3SljQ.png" /><figcaption>Fig. 4</figcaption></figure><p>Two parallel transactions are launched:</p><ul><li>The 1st transaction</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*L338FI7_ffr_I44U7LuCPw.png" /></figure><ul><li>The 2nd transaction</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*wxSl8xbRSa0ic2Vsh6hIwA.png" /></figure><p>The update operator gets a lock on both transactions. So, what’s going on here? The SELECT operator in both transactions applies the update lock to the second and first rows respectively. Then, executing the UPDATE operation in one transaction, the process tries to apply an exclusive lock to the first row but cannot do this because it’s locked by the other transaction and continues to wait until it’s released. The same way, executing the UPDATE operation, the latter transaction tries to apply an exclusive lock to the second row but it’s already locked by the former transaction and the process holds and waits until the lock is released. This case is a classic example of a deadlock. Both transactions wait for the resources locked by each other until they are released. However, these processes don’t actually freeze. Using the built-in lock manager, MS SQL Server detects deadlocks and resolves them the simple way — it sacrifices one of the transactions. The server just rolls it back and returns an error. The other transaction will continue. MS SQL Server itself selects the transaction to sacrifice, based on performance considerations or using the DEADLOCK_PRIORITY statement.</p><h3>Summary</h3><p>In this article, we’ve told you about locks and why we need them. In short, locks are a dedicated mechanism used to implement requirements set by different transaction isolation levels.</p><p>We’ve given consideration to the issues which had to be practically solved using locks in a multi-user concurrent environment:</p><p>- it’s the task of data extraction from the queue when only one client has access to data at a given time;</p><p>- it’s the task of modifiable data extraction with concurrent access when clients simultaneously access shared and often modified data but only for reading;</p><p>- it’s the task of modifiable data access synchronization when it’s necessary to access data with subsequent modification.</p><p>We’ve also mentioned the lock escalation issue when lower level locks are raised to the level of pages and tables upon exceeding the critical value.</p><p>And, to top it off, we’ve briefed you on deadlocks — the situations in which two processes are waiting for the resources locked by them from each other. This topic deserves special attention. So, in our following articles, we’ll discuss how to diagnose and deal with deadlocks.</p><p><em>Author: Nikolay Ivanov, Senior Developer, </em><a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=deadlocks1_en"><em>ITA Labs</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b9926aec779a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/sql-server-deadlocks-part1-b9926aec779a">Deadlocks in SQL Server — part1: What the locks are?</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Half a billion requests to IoT devices per day]]></title>
            <link>https://medium.com/italabs/security-sensors-monitoring-a7ae1b38a5c0?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/a7ae1b38a5c0</guid>
            <category><![CDATA[highload]]></category>
            <category><![CDATA[italabs]]></category>
            <category><![CDATA[projects]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Alexander Solovey]]></dc:creator>
            <pubDate>Fri, 09 Nov 2018 14:00:07 GMT</pubDate>
            <atom:updated>2019-02-01T09:18:28.686Z</atom:updated>
            <content:encoded><![CDATA[<p>Development of a high-performance IoT server designed to control and monitor security sensors geographically distributed over country region</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k2z1dmIlK8ljz7Kj_g1o6Q.jpeg" /></figure><h3>Highlights</h3><ul><li>Development of a high-performance IoT server</li><li>Management and monitoring of 800,000 security sensors</li><li>Simultaneous control of over 50,000 IoT devices</li><li>Processing 500 million requests per day</li><li>Integration with external systems</li><li>Implementation and integration at the regional level of the country</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8egp4O_d-zy7r0Ddda_7cw.jpeg" /></figure><h3>The challenge</h3><p>The customer required a specialized high load productive software complex for controlling and receiving / processing signals from security sensors for the technological support of thousands of objects.</p><p>The software complex was supposed to serve the devices that are in operation, both for ordinary individuals and in large industrial facilities.</p><p>The software complex had to receive information in a timely manner and send signals to control panels, as well as allow remote control of devices, including reconfiguration and flashing.</p><p>There were also requirements for scalability (vertical), fault tolerance and high system reliability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*y4F7xB2218s4a4n7I-UNig.jpeg" /></figure><h3>The solution</h3><p>The executor, anticipating the growth of customer needs, designed a modular and expandable solution, initially incorporating the means in the product to support the future various families of customer devices, monitor performance, and increase the number of supported devices.</p><p>The product consists of several main modules:<br>- database,<br>- communication module for networking with devices via a secure channel,<br>- device / command monitors,<br>- Event Manager for registering events and sending to external systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W5dLVM3zxWXqC7iao_mEuQ.jpeg" /></figure><h3>The results</h3><p>As a result, the development turned out to be a lightweight and high-performance software package that exceeded customer expectations. The reserve of processor time reached 50 percent with the maximum number of simultaneously connected devices.</p><p>The product has successfully passed bench tests in the infrastructure of the customer with a flow of up to 500,000,000 requests per day.</p><p>The software complex supports more than 50,000 devices on a single node, each aggregating up to 16 security sensors, which gives the total simultaneous control and monitoring of more than 800,000 security sensors.</p><p>A high load server maintains simultaneous connections with all 50,000 or more devices, performs real-time continuous monitoring of their availability and status using a secure protocol, transmits pick-up signals to private security panels, and provides specialists with remote control of devices, including programming their settings and firmware updates.</p><p>Having implemented the software package, the customer reduced the costs of maintaining and maintaining their servers and devices, since The developed product replaced several old ones in terms of performance, thereby reducing the costs of both hardware and software.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/912/1*RAU5LsoB8t1h9Qt3XKlEVQ.png" /></figure><p><a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=portfolio_sdms"><em>ITA Labs</em></a><em>, High load identification and management systems. Architecture design and custom development.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a7ae1b38a5c0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/security-sensors-monitoring-a7ae1b38a5c0">Half a billion requests to IoT devices per day</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TIGeR on guard for performance — Part 3]]></title>
            <link>https://medium.com/italabs/tiger-performance-part3-en-bb3e468b296e?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/bb3e468b296e</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[italabs]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[stories]]></category>
            <dc:creator><![CDATA[Lisa Botty]]></dc:creator>
            <pubDate>Wed, 31 Oct 2018 15:01:01 GMT</pubDate>
            <atom:updated>2019-02-01T09:20:44.202Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>TIGeR on guard for performance — Part 3</strong></h3><h4>Introduction</h4><p>In the <a href="https://medium.com/italabs/tiger-performance-part2-en-140008c250e2">previous</a> (second) part of the article, we went into more details about the TIGeR components architecture and told you how to install and set them up in their basic configuration.</p><p>In this (third) part, we will describe how to set up Telegraf for reading the specific Windows counters (PerfCounters) of our product, apply load to the tested server (we will feed the TIGeR), show how to display values from these counters in Grafana, and also consider some Grafana’s fine-tuning issues related to visualization, variables, and notifications (start “TIGeR training”).</p><h4>Part 3. Taming the TIGeR</h4><h4>Feeding the TIGeR (applying load)</h4><p>First of all, let us remind that InfluxDB and Grafana’s server part should be launched and running by now (for example, they are allocated to a separate server where they are always running).</p><p>For our test product, we use a load client, which we created specifically for this purpose. You will probably have your own methods to apply load (dedicated tools), but in any case, we will record metrics from the Windows counters:</p><p>· The counters which our product creates during the installation process (all important code places are covered with them)</p><p>· System counters (CPU, Memory, Disk, etc.) — optional, depending on objectives.</p><p>At this point, we have to choose counters to take metrics from. In our configuration, we have the basic, system counters (let us leave them there), but we will add our own ones. To do so, insert the following lines at the very end of the Telegraf configuration file:</p><p><em>[[inputs.win_perf_counters.object]]</em></p><p><em># Your product name</em></p><p><em>ObjectName = “</em><strong><em>Test Engine — Client Manager</em></strong><em>”</em></p><p><em>Instances = [“</em><strong><em>default</em></strong><em>”]</em></p><p><em>Counters = [</em></p><p><em>“</em><strong><em>Counter name 1</em></strong><em>”,</em></p><p><em>“</em><strong><em>Counter name 1</em></strong><em>”,</em></p><p><em>“</em><strong><em>Counter name …</em></strong><em>”</em></p><p><em>]</em></p><p><em>Measurement = “</em><strong><em>Test_CM</em></strong><em>”</em></p><p><em>FailOnMissing = </em><strong><em>true</em></strong></p><p>The text in bold type shows the places you will have to replace with your options:</p><p>· <strong><em>Test Engine — ClientManager and Counter name …</em></strong> — category and counters names: you can check them when you add them to the Windows System Monitor snap-in:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*bDFoVirCsUjx_UvPpn5tIg.png" /></figure><p>· <strong><em>default</em></strong> — the selected object (counter) instance name, an example is shown in the screenshot above</p><p>· <strong><em>Test_CM</em></strong> — the readings retrieval or run name; in fact, it is the table where the data will be recorded (we were reviewing it, using the InfluxDB chart in the second part of the article)</p><p>· <em>FailOnMissing = </em><strong><em>true</em></strong> — the debugging parameter (it can be deleted or set to <strong><em>false</em></strong> afterward); it interrupts the Telegraf startup if there are issues with finding any counters when it is run. Otherwise, the agent will start but will not retrieve readings from such “bad” counters. Consequently, nothing will be displayed in Grafana, and that may prove confusing — there is no load, the counter does not work, or there is an issue with Telegraf.</p><p>Now, to simplify understanding, we are showing an example with one load scenario for one run of autotests. In real testing, we can have more scenarios during one run (for example, “Rough working morning”, “Calm day off”, etc.). That is why we have to somehow split the data stream for each scenario in such cases so that <strong>during one run, we could distinguish among the tested scenarios displayed in Grafana afterward (as for us, this is a very important point).</strong></p><p>For example, we can do it by:</p><p>· adding specific tags to the Telegraf agent’s configuration section [global_tags] and using them to have a possibility to group data on the charts:</p><p><em>[global_tags]</em></p><p><em>scenario = “morning”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/1*vPHNRBkkJbeUJTOAJnU8_A.png" /></figure><p>· changing the Measurement (table) name in the same place — in the agent configuration — and then filtering data by these parameters when plotting the chart:</p><p><em>Measurement = “Test_CM_Morning”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*CpvR6BO1oW6bLrNoLRKMNQ.png" /></figure><p>We will touch on this topic in more detail in the next parts of the article when we start automating data collection, taking the product build numbers into account.</p><p>Now, it is time to apply the load (we run the load client on our side: we mentioned it at the very beginning of this part of the article) and start the Telegraf agent after that. If you do it the other way round, the agent will return zero data for a while, and that may “spoil” the final chart appearance. On the other hand, you may need a scenario where it will be important to “capture” the moment when the load starts. So, both start options may prove useful.</p><p>In general, the TIGeR components start charts look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*DUvNErLoimI6VPZ295IAHQ.png" /></figure><p>While the load is being applied and data is being obtained, it is time to slightly fine-tune our test dashboard in Grafana:</p><p>· add a new Graph-type panel and get in its settings</p><p>· select Data Source (it is <em>Perf_DB</em> with us, we created it above)</p><p>· specify <em>FROM default Test_CM SELECT field</em> (counter name) below in the query builder</p><p>· we can similarly add other counters in the same builder (the Add Query button)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fe2SQDp-oCJjRa3RH4DyQA.png" /></figure><p>We return to the dashboard after that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*syeuwK-oFTf1igXj3uLacw.png" /></figure><p>Marginally, this is already enough to see the big picture (see the screenshots above — the example of two load test runs with two charts on two counters).</p><p>However, we commonly have to adjust something for our specific objectives as well (in the first part of the article, we set them on our own example), and now, we will start making improvements.</p><h4>Training the TIGeR (visualization, variables, and alerts)</h4><p>Let us start with alerts. They are warnings sent via the pre-configured channel (like in Grafana) generated by a specific trigger. Channels are more than enough:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/477/1*_b2M1VHBcWgySwEo0yNGFA.png" /></figure><p>We will not consider everything — mail, instant messengers, webhook, etc. now. Everything is simple there, but in our test sample, notifications are set to be sent to mail. At the same time, we want to note that the SMTP parameters are set up in the Grafana configuration file — <em>defaults.ini</em></p><p>Let us have a look at one example: send a letter if the server processor utilization exceeds 50% in a certain period of time while testing the product.</p><p>Select the “CPU Usage” property on our test dashboard, then, select the Alert tab and create a rule:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NZpPsysUE6ChVfqfJAd1cw.png" /></figure><p>The arrows indicate what default values were altered:</p><p>· threshold value of <em>50</em> (%, as in the chart vertical scale) was set</p><p>· in the <em>query</em> box <em>(A, 5m, now)</em> we altered <em>5m</em> for <em>1m</em>,</p><p>here A is the query sequence number from the builder (it is displayed in the builder, on the “Metrics” tab next to the queries)</p><p><em>5m (1m)</em> and <em>now</em> — view the period from the current time to 5 minutes (1 minute) back.</p><p>As a result, according to our settings, we will receive alerts to the mail specified if in each such time interval (between the current time and one minute back) the average value of all “A” query results (in this panel, there is one query for the CPU) will exceed 50%.</p><p>Put it simply: the load was applied and one minute passed, and then, if the average CPU utilization level during this minute was higher than 50%, a letter will come. And every minute it will happen like this — the alert will come or will not. Certainly, this is just an example. Actually, with such settings, the alert letters may “bury” us (if the CPU is highly utilized as often happens during the tests). Therefore, you have to make settings based on experience and on your notification requirements.</p><p>All the other alert parameters are not difficult to comprehend. So, we will not describe them.</p><p>Practically, it happens like this: we apply the load to the workstation’s processor (where the Telegraf agent is installed) to periodically exceed the 50% utilization level, and soon we will receive an alert mail titled “[Alerting] CPU Usage alert”, which has a chart snapshot attached to the time interval when this alert triggered:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/1*Z1tMXtucsUc616qfX1wWCw.png" /></figure><p>A vertical red broken line shows the place the alert triggered. A horizontal one shows an alert threshold value (in our example — 50%).</p><p><strong>A crucial point:</strong> no variables (like <em>/^server$/ or $server</em>) should be selected in the metric query designer:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/569/1*P-4n6Y6mp4fOlfVt-NUW7A.png" /></figure><p>Otherwise, the alerts will not work because variables in metrics are not supported in the query subsystem for alerts:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*Y7e8aMJ-EPTw0A5-FOHkTw.png" /></figure><p>The official bug tracker contains lots of complaints about this issue or flaw (for example, <a href="https://github.com/grafana/grafana/issues/6557">here</a>), but the developers have not yet fixed or implemented it (this limitation is officially commented <a href="http://docs.grafana.org/alerting/rules/">here</a>).</p><p>This issue makes it difficult to configure alerts in many scenarios. You will either have to give up on them in this metric collection chart or skip using variables. For example, if we want to handily filter data and charts on the dashboard according to the build numbers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YnpaEyVhj3cVZBEtiuqN_w.png" /></figure><p>We will have to use a variable, for example, <em>“Build”:</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*YXKNnnSBhFUabF7isHKYEQ.png" /></figure><p>Now, if we want to add an alert to this chart, Grafana will return a message “Template variables are not supported in alert queries” (see the example in the screenshot above, Fig 11).</p><p>Here are some facts about variables in Grafana. They are meant to customize the display of panels and other dashboard visualization units according to predetermined filters, for example, data source, station, disk, network adapter name, product build number, and more. They are not limited by their dashboard only. Find out more details about the variables in the official <a href="http://docs.grafana.org/reference/templating/">documentation</a>.</p><p>Variable controls are in the dashboard settings:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3OQPPf7o7hg340EkN4kV3g.png" /></figure><p>Here we can add variables of different types:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/1*aDV6jj6ptmh6m56nq6S22Q.png" /></figure><p>In the example shown, from the query results tags, we get the server name, drive letter, adapter name, and product build number. Next, we can use these variables in the metric query designer and when filtering the dashboard data (see several screenshots presented above with the variable “<em>Build</em>” as an example).</p><p>At the end of this part of the article, we would like to recapitulate Grafana’s panels, tables, and other data display units available to be added to the dashboard:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*lzK1plY1WS8eoXATgVm9OA.png" /></figure><p>Actually, there are many more of them than shown in the screenshot because this Grafana feature is expanded with plug-ins, the complete list of which is <a href="https://grafana.com/plugins?type=panel">here</a>.</p><p>Each panel can be flexibly configured for different data display logic. We will get back to this feature when we show how to calculate the number of supported “virtual clients” for our test product in the following parts of the article about automation and load autotests.</p><p>As your homework, try to “play around” with the built-in panels: add and customize them for your objectives and preferences. You can also see many samples of different types of dashboards and panels. You can also “play around” with their settings, on the Grafana <a href="https://play.grafana.org/">demo portal</a>.</p><h4>Summary</h4><p>In this third part of the article, we described how to configure Telegraf to read specific counters tracking our product, applied a load to the test server using a load client, showed how values picked up by these counters are displayed in Grafana, and saw some of Grafana’s configuration issues regarding visualization, variables, and notifications.</p><p>Next, in the fourth part, we will start automating the TIGeR components operation by connecting the test automation server (“teach TIGeR to act by itself”), as well as configure Telegraf and Grafana to display our test product performance dynamics both in test run scenarios and for each build (“observe TIGeR success”).</p><h4>Links</h4><p><a href="https://medium.com/italabs/tiger-performance-part1-en-a3c56972c355">TIGeR on guard for performance — Part 1</a></p><p><a href="https://medium.com/italabs/tiger-performance-part2-en-140008c250e2">TIGeR on guard for performance — Part 2</a></p><p><em>Author: Eugene Klimov, System Engineer, </em><a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=tigr_part3_en"><em>ITA Labs</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb3e468b296e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/tiger-performance-part3-en-bb3e468b296e">TIGeR on guard for performance — Part 3</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TIGeR on guard for performance — Part 2]]></title>
            <link>https://medium.com/italabs/tiger-performance-part2-en-140008c250e2?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/140008c250e2</guid>
            <category><![CDATA[italabs]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[stories]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[grafana]]></category>
            <dc:creator><![CDATA[Lisa Botty]]></dc:creator>
            <pubDate>Wed, 24 Oct 2018 15:01:03 GMT</pubDate>
            <atom:updated>2019-02-01T09:21:30.472Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>TIGeR on guard for performance — Part 2</strong></h3><h4>Introduction</h4><p>In the previous part of this series “<a href="https://medium.com/italabs/tiger-performance-part1-en-a3c56972c355">TIGeR on guard for performance — Part 1</a>”, we sorted out why we need to test performance, why we need to test it automatically, and shaped up the original objective.</p><p>In this (second) part, we will take a closer look at the selected solution components architecture (“the TIGeR anatomy”), and we will also describe installation processes and ways to set the components up in their basic configuration (“cage the TIGeR”).</p><h4>Part 2. Studying the TIGeR</h4><h4>More details on the “TIGeR anatomy” (component architecture)</h4><p>Let us briefly recollect the components which the TIGeR bundle consists of, namely Telegraf + InfluxDB + Grafana:</p><p>· <strong>Telegraf</strong> is an agent aimed at collecting all sorts of the system metrics.</p><p>· <strong>InfluxDB</strong> is a time series database specializing in time series storage.</p><p>· <strong>Grafana</strong> is a web server designed to visualize data.</p><h4>Telegraf</h4><p>It is an agent designed to collect system metrics from stand-alone stations. In our case, we will install it on the server side of the product tested. As you can see on the chart, the agent will transfer the collected metrics into the InfluxDB database in the format understandable for both of them (Telegraf and InfluxDB):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*fSVFv3uBK7WRE8GGV2jICw.png" /></figure><p>Telegraf consists of an executable file and a configuration file. Therefore, no special installation is required.</p><p>Plug-ins are among the main advantages of this agent: there are a lot of them (more than 100 for now). However, annoyances sometimes do happen — for example, it contains no plug-in for MSSQL (as a database, and therefore, no possibility to store metrics there).</p><p>As for the capabilities that the Telegraf already has without plug-ins, we can highlight the following:</p><p>· <strong>Outgoing</strong> data formats, namely, metrics transfer to various databases or just like communication lines: InfluxDB, JSON, Graphite, ElasticSearch, common text file, Prometheus, etc.</p><p>· <strong>Incoming</strong> data formats, namely, metrics sources: Win_perf_counters, MSSQL, RabbitMQ, Memcached, MongoDB, MySQL, simple HTTP, JSON, Filestat (different file statistics), Docker, Cassandra, SMART, DNS, etc.</p><p>Check a more detailed list at <a href="https://docs.influxdata.com/telegraf/">https://docs.influxdata.com/telegraf/</a> and <a href="https://github.com/influxdata/telegraf/tree/master/plugins">https://github.com/influxdata/telegraf/tree/master/plugins</a></p><p>Plug-ins, not built into Telegraf, can only be downloaded as source files, and that is both an advantage and a disadvantage. On the one hand, you can flexibly fine-tune the plug-in, and on the other hand, you will have to spend extra time to integrate it.</p><p>One more thing: unfortunately, we could not compile them into a separate library file and then insert it into the folder with the executable file. It is necessary to include source files of such plug-ins in the Telegraf source files and re-compile the entire agent completely. There are more details on this process on github (see the link above), but now, we will not dwell on this peculiarity, as Telegraf has enough built-in capabilities to meet our objectives.</p><h4>InfluxDB</h4><p>Let us have a look at a simplified structural chart of this database:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/1*Wkk9L36eSZGCrsTHvrFROw.png" /></figure><p>· Database available;</p><p>· Inside it, there are tables — Measurements, each has its own name and retention;</p><p>· The tables include a Time column — the main and compulsory columns where timestamps are stored;</p><p>· There are Fields columns (not indexed) — the very metrics data is stored here;</p><p>· Tags columns (indexed) usually store group records, for example, a metric from some server, developed by some user, for some process, etc.</p><p>To query data from the database, an SQL-like language is used.</p><p>Main <strong>advantages</strong>:</p><p>· Quick installation (just unpack the archive, executable files, and configuration files);</p><p>· High performance and low resource utilization;</p><p>· Record and read data over HTTP (access data through a browser, curl, etc.);</p><p>· Support for various data sources except for Telegraf and similar agents (see the list in the previous part in the chapter “What tools, designed for performance testing, exist?”), it is possible to work using the ready libraries, including .NET C#.</p><p>What we can highlight among <strong>disadvantages</strong> is the lack of a graphical user interface (GUI) dedicated to working with the database (interestingly, it was available in previous builds but was removed in the latest versions), meaning that all services should be carried out using Command-Line Interface (CLI).</p><p>That is, actually all. Simple, isn’t it?</p><h4>Grafana</h4><p>It consists of a client (usually a browser) and a server used as a proxy to the database (there is also a possibility to connect directly, using the client — we will talk about this mode in a little more detail below), has a simple role model, various data sources and a set of panels grouped by dashboards (henceforth — dashboard).</p><p>The entire rendering is performed on the client side. For example, here is a sample of one of those dashboards (however, all the possibilities to set up beautiful charts in Grafana are a topic for a separate article):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-xrzY6Ar4uUL7UASju8_EQ.png" /></figure><p>In addition to InfluxDB, we can use various databases as data sources (almost all most widespread ones, except MSSQL) and common text files. Initially, there are few supported data sources, but InfluxDB is available and that will do for us.</p><p>We can extend Grafana’s capabilities installing plug-ins (unlike with Telegraf, it is much easier to do so — all you have to do is to run the installation command of the required plug-in from the command line). Besides additional data sources, there are plug-ins adding various panels to get an even more flexible and beautiful visualization than already available.</p><p>Now, we will briefly tell you about the data request modes, mentioned above:</p><p>· <strong>Proxy</strong> — the client opens Grafana, and it independently executes all requests in InfluxDB;</p><p>· <strong>Direct </strong>— the request goes directly from the browser to InfluxDB (for example, when Grafana does not have access to InfluxDB, but the client has it):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*m51lvev7u-CZg9EfH3lApw.png" /></figure><p>Grafana’s <strong>advantages</strong>:</p><p>· Easy to install (just unpack the archive);</p><p>· Very convenient, flexible, and rich dashboard customization;</p><p>· Simple (relatively) and flexible data sources query builder;</p><p>· Alert system;</p><p>· Snapshots — these are very detailed reports on the conducted test with a link entry point:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gJLC8uzv5mdDKIfwUGoe-w.png" /></figure><p>Despite all the advantages, very many people choose it primarily for its visualization beauty:</p><p>As for the <strong>disadvantages</strong>, we can only highlight the complicated configuration system. There are lots of instructions, articles, tips, and discussions on this topic, but in our opinion, this shortcoming is by far compensated with advantages.</p><h4>“Caging” the TIGeR (installation and basic configuration)</h4><p>From the previous chapter, it has become clear that the installation process of all three components consists of:</p><p>· Downloading archives</p><p>o Telegraf and InfluxDB — <a href="https://portal.influxdata.com/downloads">https://portal.influxdata.com/downloads</a></p><p>o Grafana — <a href="https://grafana.com/grafana/download?platform=windows">https://grafana.com/grafana/download?platform=windows</a></p><p>· Unpacking into any directory;</p><p>· Setting up in configuration files (for Grafana dashboards as well).</p><p>The general chart of interaction with our test product will look as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*iLQ7tMM1h3SzhvSboJlPpw.png" /></figure><p>· A loading client (or several) loads the tested product, with which the Telegraf agent is installed</p><p>· Telegraf collects predefined metrics and records them to InfluxDB</p><p>· Observers or operators address Grafana, using the browser</p><p>· Grafana makes a query to the InfluxDB database, receives, and visualizes data</p><p>In the example above, InfluxDB and Grafana are on the same machine, but actually, they can be on different ones.</p><h4>InfluxDB</h4><p>Everything is really simple in this case: at the minimum — you do not have to change anything even in the configuration file:</p><p><em>influxd.exe -config influxdb.conf</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/977/1*t_n262VdId1lLRoa_u5-_Q.png" /></figure><p><strong>Important remark:</strong> initially, all parts of TIGeR cannot work as Windows services — we have to use some third-party solutions, for example, to “wrap” executable files into services (for example, NSSM — <a href="http://nssm.cc/">Non-Sucking Service Manager</a>). We will not dwell on this issue in our article anymore, you will decide on your own whether to leave it as it is or to configure the launch as services. For example, in our <a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campain=tigr_part2_en">company</a>, we have set up InfluxDB and Grafana to be launched as services, and Telegraf as a common application.</p><p>We will not expatiate upon the InfluxDB configuration parameters either — there are lots of them, but most of them will not be of use for us right now. Later on, we can change the port, authentication, logging, etc.</p><p>The executable file influx.exe is the command interface to InfluxDB, with which you can manage various databases and their data. We do not need it for the time being, but we will return to it if necessary.</p><h4>Telegraf</h4><p>Almost everything with Telegraf is just the same as it is with InfluxDB, but we will have to tweak the default settings in the configuration file <em>telegraf.conf</em> a little bit:</p><p>· <em>logfile</em> = “path to the folder and the file name for the log” — better to enter your own values;</p><p>· In the section <em>[outputs.influxdb]</em>:</p><p><em>urls</em> (correct to the real server address where InfluxDB is installed);</p><p><em>database</em> — database name (specify any, but if it is not set, a database with a default name is created automatically);</p><p>Next, the file contains the list of Windows Performance Counters — you can leave the values as they are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C-xVjwP14LSuByiW65TLwA.png" /></figure><p>Run:</p><p><em>telegraf.exe -config telegraf.conf</em> — and let it collect the metrics for the time being (Windows Performance Counters), but we will go on.</p><h4>Grafana</h4><p>In this case, it is even simpler and more difficult at the same time — we can right away run:</p><p><em>grafana-server.exe</em></p><p>and follow the link <a href="http://localhost:3000/"><em>http://localhost:3000</em></a> (login: password — admin: admin, the host name should be replaced with the real one if you log in with Grafana not locally).</p><p>The further dashboards configuration and metrics display are all that is difficult here. In this part of the article, we will just run a ready dashboard to comprehend how it works. So, there is nothing difficult for now.</p><p>After we log in to Grafana, the first thing to do is to create a data source:</p><p>· Source name (any), type — InfluxDB, URL in our example — <a href="http://localhost/"><em>http://localhost:</em></a><em> 8086</em> (should be replaced with your own if InfluxDB is not installed locally but on some other port);</p><p>· Access type — commonly Proxy (we have talked about these types above);</p><p>· If authentication is set up in InfluxDB (initially it is not), you also have to specify the required user credentials;</p><p>· Database name (see configuration file for Telegraf), time interval — 10 seconds.</p><p>That will do for now:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iXcZuy3ACQjJVMAuFm7GPw.png" /></figure><p>Next, you have to download the ready dashboard template (you will have time to create your own one later on, but for now, it is better to use an example) — <a href="https://grafana.com/dashboards/1902">https://grafana.com/dashboards/1902</a></p><p>Dashboards come as files with the json extension. Hence, they contain the usual json data.</p><p>The package (on the dashboard page) also contains a configuration file for Telegraf: you can compare it with your own and correct points that differ or replace your file with the one from the package (do not forget to correct the parameters mentioned above in the chapter about Telegraf launching).</p><p>After that, we have to import the downloaded json file into Grafana, specify the dashboard name, and select the data source (we have already created it above):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kn9LaBQr_aBVBCrVunQ_HA.png" /></figure><p>Unfortunately, not all panels from this template will work at once — some of them will return the “No data points” message:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EfqGaIgfCPgxjU2ytGQ83A.png" /></figure><p>We will have to get in the settings of the problem panels and change the host to <em>/^server$/</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*51aFD6Zx-9S8vvdECeR4OQ.png" /></figure><p>Then, everything will operate as it should:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LXya89Ld2R4QZIIUZR-qkQ.png" /></figure><p>In the upper right corner, we can flexibly adjust the display time and the data refresh rate.</p><p>Now, we can observe how the charts are drawn and do something on the server where Telegraf is running, just for a change. To get a better feel of Grafana, we can “play” with various settings and parameters right on the test dashboard.</p><h4>Summary</h4><p>In the second part of our article, we had a closer look at the TIGeR components architecture and also explained how to install and set them up in their basic configuration.</p><p>Next, in the third part of the article, we will set up Telegraf to read specific Windows Perf Counters for our product, apply the load to the tested server (i.e. “feed the TIGeR”), learn how to display the values returned by the counters in Grafana, and also consider some Grafana’s fine tuning-issues relating to visualization and notifications (first “TIGeR training”).</p><h4>Links</h4><p><a href="https://medium.com/italabs/tiger-performance-part1-en-a3c56972c355">TIGeR on guard for performance — Part 1</a></p><p><em>Author: Eugene Klimov, System Engineer, </em><a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=tigr_part2_en"><em>ITA Labs</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=140008c250e2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/tiger-performance-part2-en-140008c250e2">TIGeR on guard for performance — Part 2</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TIGeR on guard for performance — Part 1]]></title>
            <link>https://medium.com/italabs/tiger-performance-part1-en-a3c56972c355?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/a3c56972c355</guid>
            <category><![CDATA[italabs]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[stories]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Lisa Botty]]></dc:creator>
            <pubDate>Wed, 17 Oct 2018 16:01:02 GMT</pubDate>
            <atom:updated>2019-02-01T09:22:12.120Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>TIGeR on guard for performance — Part 1</strong></h3><h4>Introduction</h4><p>In this round of publications consisting of several parts, we will tell you about how we looked for, researched, and implemented the solution helping us keep ourselves up to date with the current situation regarding our products performance.</p><p>Part after part, step by step, from theory to practice — we will show you what issues we have encountered and how we have solved them.</p><p>Having read all the parts of this publication, you will apprehend why we actually need it, why we have chosen this method, and, having studied this practical example, you will be able to easily use it for your products.</p><h4><strong>Part 1. How we have traced that fairy-tale beast</strong></h4><h4>What kinds of performance testing exist</h4><p>To start with, let us recall all the existing methods used to test performance:</p><p>1. <strong>Load testing </strong>is the main type of testing used to measure various server parameters when supplying a load from a loading tool mimicking the client work.</p><p>2. <strong>Stress testing</strong> loads the system using its maximum resources and removes the load afterwards to find out whether the tested system is capable of recovering.</p><p>3. <strong>Testing with a large bulk of data </strong>presumes the preliminary or simultaneous generation of as much data as system resources can handle and the subsequent metrics recording. To some extent, it is a subtype of tests mentioned above but they are usually subdivided.</p><p>4. <strong>Stability testing</strong> is the longest one aiming at identifying various resource leaks and performance degradation at moderate loads but for a longer period of time.</p><p>5. <strong>Scalability testing</strong> is necessary to find out whether the productivity will increase with a larger number of resources, and if it will, what the proportion will be.</p><p>6. <strong>Configuration testing</strong> is checking the product functionality on various equipment configurations, test rigs, etc.</p><p>A visual representation of all the above-mentioned types is shown in this diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*_U71MyuqzmkDKy4PNXyWnQ.png" /></figure><p>In this article, we will only delve into the type I (load) testing, but you can equally apply this publication material and the methods described for other types of testing.</p><p>Load testing also presumes two methods of conducting it:</p><p>1. <strong>Manual testing</strong> is usually long, sometimes even expensive, but it provides more or less absolute numbers of load metrics.</p><p>2. <strong>Automatic testing</strong> is aimed at identifying the dynamic pattern in selected metrics after some modifications in the product logic were made. What we mean is we only have to compare the current figures with the figures obtained when previous product versions were tested.</p><h4>Why it is necessary to test performance</h4><p>Our company <a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campain=tigr_part1_en">ITA Labs</a> is committed to this wording:</p><p>“You have to test performance to comprehend to what extent a specific product meets expectations (both customers’ and ours) and identify possible issues associated with it.”</p><p>Performance testing often deals with potential issues not just in the product code but also in software configuration, hardware, and other problems.</p><h4>Why we need it</h4><p>First of all, we have to determine the components of which the initial data consists to set the objective:</p><p>1. We have a client-server product (hereinafter referred to as the “product”), and we have to test how the server end handles the client load.</p><p>2. We have clients for this product who keep asking a reasonable question, “How many clients (users) will it be able to service simultaneously within a single node of a high-availability cluster? We need a clear and preferably some standardized report on performance tests — HPLoadRunner, for example…”</p><p>3. Our company <a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campain=tigr_part1_en">ITA Labs</a> has set a task to periodically test our components and check if any performance degradation takes place when we introduce this or that feature, or on the contrary, how much the performance increases after the specific optimization.</p><p>It usually takes testers a lot of time to conduct such tests, the reports look complicated and messy, and consequently, they are incomprehensible to customers. Moreover, it will not be right to skip such tests because we can get absolute figures (i.e., deliver the maximum load on powerful stands) and then show them to our clients.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*jRF7pVr_pAitAIUIKlLRDA.png" /></figure><p>Now, we have a long-pending issue on our hands: how can we avoid occupying testers for too long but see the actual components performance rates more often. Obviously, we cannot get absolute figures when the objectives are set like this. Therefore, we started looking for a compromise.</p><h4>Stating the original objective</h4><p>So, we found a compromise. We were reasoning like this: since we cannot get actual and absolute performance figures on a regular basis, we will take our product of the specific build as a reference (reference point) for which these absolute figures are known, and then, we will simply compare all subsequent builds with it.</p><p>Actually, as a result, we will observe the product performance dynamic pattern according to the selected metrics — compare figures regardless of their absolute value. Thus, it (the dynamics) will show us at once which metrics go beyond acceptable limits and in which build. The data will be applicable for both performance losses and gains (when, for example, the code was optimized).</p><p>Moreover, to achieve this, we will use regular automatic performance test runs for each build, and see charts like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*UXGlxrgvvMk0D89Jc2R9LQ.png" /></figure><p>The chart itself represents one of the metrics or its derivative.</p><p>Across– the product build number, and up– the performance index. In our example –what maximum number of clients one server end of the product supports.</p><h4>Specifying the original objective</h4><p>You have probably noticed that one thing is missing in the objective statement –analysis and notifications automation. We will see charts, but it is inconvenient to retrieve them “manually” for each build.</p><p>That is why, let showing charts be a minimum objective, but we have to specify the maximum one:</p><p>1. A short performance testing according to predefined scenarios should automatically run for each product build.</p><p>2. The indicators change in components performance should be shown clearly according to the predetermined metrics.</p><p>3. If any performance losses or sharp gains take place, they should be automatically analyzed, and detailed warning messages should be displayed.</p><p>All this is done to either quickly fix the issue in the code, in technical specs, or somewhere else or define the solution which is currently optimal.</p><p>Our chart will now look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*920sZWKmW9FTSQLHCy69WQ.png" /></figure><p>We added warning thresholds — all values beyond the lower and upper thresholds will warn that we experience a drastic change in a certain performance metric.</p><p>Sounds tempting, but what is in practice?</p><h4>Why HPLoadRunner was not good for us</h4><p>First, we earnestly took up <a href="http://www8.hp.com/sg/en/ad/load-runner/load-runner.html">HPLoadRunner</a> because its reports are considered to be an essential standard in testing serious products produced by serious companies.</p><p>We installed HPLoadRunner, configured its basic functionality, made several test runs for our tested website, and got a couple of reports — everything worked OK, so, the beginning was not bad.</p><p>After that, we were trying to achieve the main objective– emulate the load client. Our components are coded using .NET, and HPLoadRunner has a so-called “.NET Vuser” integrated, which, as it seemed to us, could “pretend” to be a real client if it was prepared for it beforehand, specifically, its operation script was recorded.</p><p>As we recorded this script, it turned out that our client-server exchange protocol or more precisely — the moment of client authentication, could not be emulated using “.NET Vuser”. The matter became even worse when we found out that the HP Company itself had no detailed documentation on that component.</p><p>When we finally realized that HPLoadRunner could not help us, we took a decision to have a look at other products designed to conduct performance testing, and what solutions other companies use for this purpose.</p><p>We will provide more details about our agony of choosing, but actually, we chose the simplest and, at the same time, most flexible and reliable solution approved by other testers (to be more precise, it was the basis for the solution because our products features and infrastructure imposed certain difficulties which we had to solve). What other tools, designed for performance testing, exist?</p><p>Now, let us talk about the promised agony of choosing in respect of tools and solutions.</p><p><a href="http://www8.hp.com/sg/en/ad/load-runner/load-runner.html">HPLoadRunner</a> — it was the first app we tested from among major professional industry solutions. In addition, the product is, in fact, the standard, as many clients know it and trust it. What can we say: reliable, complicated, and expensive.</p><p>The product is interesting, and perhaps, we will publish a separate article on it in future.</p><p><a href="https://jmeter.apache.org/">JMeter</a> — if we could compare HPLoadRunner with Windows in the world of Operating Systems, JMeter would be Linux in this case: powerful, free, flexible, but often needs various modifications (both in the JMeter itself and in the tested products).</p><p>JMeter currently almost keeps up with HPLoadRunner in its functionality, but we experienced the same problem with it –we could not emulate our client due to the client-server protocol specifics. JMeter has a support aimed at their own plug-ins development, but for our specific purpose, such a plug-in seemed to be complicated to implement.</p><p>Nevertheless, the product is interesting as well and deserves a dedicated article too.</p><p><a href="https://gatling.io/#/">Gatling</a> is a serious competitor to JMeter but without a GUI. However, it is considered to be one of the most powerful instruments. In addition, its test description language is among the standardized ones. With other features equal to JMeter, it provides a more productive socket module.</p><p>If we have an article on JMeter, it will necessarily contain a comparison to Gatling. However, as part of this article and for our product load testing purposes, it is not suitable for us because of the same reasons as both solutions mentioned above.</p><p><a href="https://gettaurus.org/">Taurus</a>, <a href="https://tech.yandex.ru/tank/">Yandex.Tank</a> are other similar solutions meant exclusively for website testing (but our components are not exactly websites, or they are not websites at all to be more precise). However, in their niche, these solutions celebrate a deservedly good reputation. Yandex uses “Tank” for its websites inclusive–admit it, it is a good advertising trick even as it is. Unfortunately, we have not been able to “fit” these solutions into our products yet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-d9WaNXIxZoTpJ4Q80V8lA.png" /></figure><p>Then, we started looking towards the solutions, of which we learned from developers and testers at various forums, conferences, and just in casual publications.</p><p><a href="https://www.influxdata.com/">InfluxData</a> or <strong>TICK </strong>Stack (Telegraf, InfluxDB, Chronograf, and Kapacitor) was not originally meant for performance testing, but many developers quickly understood its advantage and utilize it for this purpose as well. It was primarily conceived as a solution used to monitor metrics deriving from various sources.</p><p>Note that it was the very TICK Stack which seriously influenced our final choice. Moreover, right at this stage of our research, we understood we had to choose the solution with the most complete support for Windows counters because our products actively use them.</p><p>There are also different kinds of agent bundles (Carbon, NetData, Node_Exporter, CollectD, Zabbix, Telegraf), databases (MySQL, ElasticSearch, Prometheus, InfluxDB), and visualization interfaces (the choice is smaller in this category — Chronograf, Kibana, Graphite, and Grafana which is actually over Graphite’s add-on). In most respects, the choice here depends on personal preferences because these agents actually compete with each other.</p><p>We also tried to understand what solutions other software companies use for automated testing. However, now, no one is eager to share their experience with others for some reason: either they are to that extent secretor there is nothing to share.</p><p>As a result, having researched and tried several different options and combinations of the solutions described above, we shaped up a certain preference that influenced our final choice.</p><h4>Why we chose TIGeR and what kind of beast it is</h4><p>As many of you have already guessed reading the article title, as well as descriptions of existing solutions, we have chosen “Tiger”, to be more specific: Telegraf + InfluxDB + Grafana = TIG(e)R</p><p>Let us briefly view the separate components included in this solution:</p><p>1. <a href="https://grafana.com/">Grafana</a> is a standalone web server designed to visualize data and all sorts of cyclic metrics.</p><p>It does not support its own database, but it is able to use various data sources (there is a built-in support for several sources which can be expanded using plug-ins). It is usually called simply “dashboard” because of the specific concept of allocating objects.</p><p>All the advantages can be simply described as follows: nice, flexible, simple, http-API, many plug-ins, and a great recognition among system administrators and testers. This product has recently shown the tendency of being used it in other different areas.</p><p>The entire scope of capabilities inherent to the platform is definitely the topic for a separate article or, possibly, for several ones.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tERRyvkUOD5t4zrgJIVW2A.png" /></figure><p>2. <a href="https://docs.influxdata.com/influxdb">InfluxDB</a> is the so-called “timeseries” database, meaning it specializes in storing time series and in handling them.</p><p>It ideally fitted in our chosen bundle: it is installed simply, an easy and fast solution which seamlessly integrates with the rest of the TIGeR bundle components, widely available and documented, and has http-API as well.</p><p>Therefore, our choice of this particular database was obvious and not long in coming.</p><p>3. <a href="https://docs.influxdata.com/telegraf">Telegraf</a> is an agent designed to collect various metrics from the system (in which it is installed), Windows in particular. It is implemented as a regular executable file with a separate configuration file responsible for tweaking all the logic of this agent’s operation.</p><p>The agent is simple and easy to use with abuilt-in support for multiple input metrics, including Windows counters (the most important feature for us as we marked above) and output data formats.It is expanded with plug-ins, fully supports InfluxDB as a metrics transfer database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/414/1*TVQCurMUlU_v-cjuz84TQQ.png" /></figure><p>All the three mentioned components are coded using the Go (Golang) programming language. This language is developed by the Google team, it is predicted to replace C (++), and it is specifically optimized for multithreading. You can find more details about it and about its other interesting features on its official <a href="https://golang.org/">website</a>.</p><p>All this is compatible with Windows, which is significant for our infrastructure, as we:</p><p>· are official Microsoft partners</p><p>· mainly work on the Microsoft technology stack</p><p>Nevertheless, you will probably never find someone who utilized the entire TIGeR bundle in the Windows environment, except the Telegraf agent perhaps. After all, the product is initially orientated for Linux, and that affects the way it is utilized.</p><p>All the three components are open source products (MIT license for Telegraf and InfluxDB, Apache license for Grafana).</p><p>Let us have a look at the TIGeR’s operation chart (here is the chart from the official <a href="https://www.influxdata.com/">InfluxData</a> website but redesigned to reflect our objective):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*BCtdNviOuHVa0D082dS0ew.png" /></figure><p>InfluxDB functions as the main link receiving data from Telegraf and transferring it to Grafana in a format comprehensible to all solution participants.</p><h4>Summary</h4><p>In the first part of the article, we analyzed why we need performance testing and why we need to test it automatically, and we also set the original objective.</p><p>Then, we showed the solution search process, and having analyzed the whole variety of products, we selected a specific bundle Telegraf + InfluxDB + Grafana = TIGeR</p><p>In the second part of the article, we will consider the bundle components architecture in more detail (“the TIGeR anatomy”), and we will also describe installation processes and ways to set the components up in their basic configuration (“cage the TIGeR”).</p><blockquote>Author: Eugene Klimov, System Engineer, <a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=tigr_part1_en">ITA Labs</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a3c56972c355" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/tiger-performance-part1-en-a3c56972c355">TIGeR on guard for performance — Part 1</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Trillions of biometric transactions]]></title>
            <link>https://medium.com/italabs/afis-95ad983a3cd0?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/95ad983a3cd0</guid>
            <category><![CDATA[highload]]></category>
            <category><![CDATA[projects]]></category>
            <category><![CDATA[biometrics]]></category>
            <dc:creator><![CDATA[Alexander Solovey]]></dc:creator>
            <pubDate>Thu, 11 Oct 2018 12:32:41 GMT</pubDate>
            <atom:updated>2019-02-01T09:19:01.940Z</atom:updated>
            <content:encoded><![CDATA[<p>Development of a high-performance and scalable biometric fingerprint user identification system</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RUnW6xR8uQ6dFDmn9CjwUw.jpeg" /></figure><h4>Highlights</h4><ul><li>Development of the specialized high-load failover software system</li><li>Cluster of dozens of nodes running 24/7</li><li>Specialized failover architecture</li><li>Tens of millions of biometric identifiers</li><li>Trillions of comparison operations</li><li>Integration at the level of national government</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hecdqh3HQFSc-VJDekAJVQ.jpeg" /></figure><h4>The challenge</h4><p>The customer had a need for a high-performance automated fingerprint identification system (AFIS), designed for tens of millions of fingerprints, designed for 24/7 fault-tolerant operation.</p><p>The main target audience for the customer’s product is the use in projects of the Civil ID scale class from a large corporation to the level of a federal state system.</p><p>AFIS should have been integrated into a system of any scale as a ready biometric back-end using their APIs. The system had to have several software interfaces at once to ensure integration with third-party software packages operating in completely different software environments and platforms using different protocols and standards.</p><p>There were also requirements for scalability (both vertical and horizontal), fault tolerance and high reliability (both of the system as a whole and the data passing through it).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/1*-c5kTQGwlqvDJFbRJ1ewQQ.jpeg" /></figure><h4>The solution</h4><p>The executor, anticipating the growth of customer needs and taking into account the uncertainties of requirements for various parameters, designed a modular and extensible solution, initially incorporating fault tolerance, performance monitoring and expansion interfaces into the product.</p><p>The product consists of several main modules: a database, a cluster manager, a computer and a gateway. This set of “building blocks” allows you to deploy a fault-tolerant and highly scalable system for almost any need.</p><p>Interface variability (client SDKs, web services, message queues) provide flexibility for integration, and also (when using message queues) ensure data reliability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e9u90_6Jr9MkXQRxWowSNA.png" /></figure><h4>The results</h4><p>The development and evolution of the product has resulted in a flexible, reliable and simple system that has successfully proven its viability in productive high-loaded integrations around the world, from industry-specific security systems to systems that process data of millions of voters at the level of the whole country as part of the election of the head of state.</p><p>The product has been successfully integrated into federal systems in three continents of the world. Trillions of search operations performed without a single failure.</p><p>Using this product, the customer entered a new business segment, having the opportunity to participate in a completely new and more marginal class of projects.</p><p>Given that the product is deployed in a matter of minutes, product maintenance is also very simple, and therefore does not require highly specialized knowledge from specialists, the product has a significantly lower TCO than competing products and solutions.</p><p>Also a nice bonus for the customer was that the pre-designed product architecture allowed him to become a platform for a whole class of products and solutions built on its basis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QsYAbn8cJKIVuACTdU-5AA.jpeg" /></figure><p><a href="https://www.ita-labs.com/?utm_source=Medium&amp;utm_medium=Blogs&amp;utm_campaign=portfolio_afis"><em>ITA Labs</em></a><em>, High load identification and management systems. Architecture design and custom development.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=95ad983a3cd0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/afis-95ad983a3cd0">Trillions of biometric transactions</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[We have finally completed a long procedure of re-styling]]></title>
            <link>https://medium.com/italabs/we-have-finally-completed-a-long-procedure-of-re-styling-419bf0dc9578?source=rss----c114ea447f28---4</link>
            <guid isPermaLink="false">https://medium.com/p/419bf0dc9578</guid>
            <category><![CDATA[design]]></category>
            <dc:creator><![CDATA[Alexander Solovey]]></dc:creator>
            <pubDate>Thu, 30 Aug 2018 19:16:30 GMT</pubDate>
            <atom:updated>2018-08-30T19:16:30.214Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fa8dprc_BxGj925jHOEdmg.jpeg" /></figure><p>As it turned out, the development of a new corporate style, this is an extremely time-consuming process, consisting of a huge amount of small things and details. It turned out to be much more complicated and longer than we could ever expect.</p><p>But the main difficulty was that all participants in the process had their own opinion, quite often opposite, about design and style. And the time needed to negotiate the concept and elements of the corporate identity, colors and other small things grows exponentially, in a proportion to the number of participants in the process.</p><p>But that’s not all. After the approval of the new corporate style, it is necessary to develop a lot of practical elements on its basis for new style implementation.</p><p>These are logos and banners in many formats, sizes, resolutions and color profiles.</p><p>It is necessary to create new signatures for the e-mail client, which is a separate quest, because all e-mail clients have their own interpretation of HTML and CSS standards. Needless to say that there are really a lot of e-mail clients to support.</p><p>Also, new templates of documents and presentations are needed with a bunch of features depending on what they are for (printing or electronic use).</p><p>It was necessary to solve issues with colors, as CMYK option when being printed on physical media does not look the way it was on the screen in RGB palette during design negotiation and approval.</p><p>And, finally, we need to replace the elements of corporate identity in a huge number of documents, services, web sites and keep an eye on them to ensure that all the changes are in effect.</p><p>Of course, we did not expect such a volume of work and difficulties when were in planning and starting of this internal project.</p><p>Nevertheless, all this is successfully completed, the new style is already online and offline. Meet the updated ITA Labs!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=419bf0dc9578" width="1" height="1" alt=""><hr><p><a href="https://medium.com/italabs/we-have-finally-completed-a-long-procedure-of-re-styling-419bf0dc9578">We have finally completed a long procedure of re-styling</a> was originally published in <a href="https://medium.com/italabs">ITA Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>