<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[clay-one - Medium]]></title>
        <description><![CDATA[Clay One, sharing the first steps in building scale-out reliable distributed software systems - Medium]]></description>
        <link>https://medium.com/clay-one?source=rss----a4be5740a48f---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>clay-one - Medium</title>
            <link>https://medium.com/clay-one?source=rss----a4be5740a48f---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 15:02:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/clay-one" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Kafka Connect Cluster: An Introduction]]></title>
            <link>https://medium.com/clay-one/kafka-connect-cluster-an-introduction-26522e72a9af?source=rss----a4be5740a48f---4</link>
            <guid isPermaLink="false">https://medium.com/p/26522e72a9af</guid>
            <category><![CDATA[sql-server]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[apache-kafka]]></category>
            <dc:creator><![CDATA[Seyed Morteza Mousavi]]></dc:creator>
            <pubDate>Sun, 23 Dec 2018 07:47:04 GMT</pubDate>
            <atom:updated>2018-12-19T16:46:50.692Z</atom:updated>
            <content:encoded><![CDATA[<p>Apache Kafka is a streaming platform that allows developers to process streams of data easily and in real time. These data come from a variety of sources. In the simplest form, you have a producer application that generates a message and publishes it to Kafka cluster and in other hands, you have another application that consumes this message. This is typical for some scenarios, but sometimes you need to process streams of data that is not in your Kafka cluster. These data may be located in a SQL database like SQL Server, MySQL or a simple CSV file. In order to process those data, you have to move them from your database to the Kafka cluster. To this end, you have some options and two of them are:</p><ol><li>Create an application that reads data from your source storage system and produces them to Kafka cluster.</li><li>Or use Kafka Connect to move your data easily from source storage system to your Kafka cluster</li></ol><p>If you choose the first option you need to write codes that move your data to the Kafka cluster. Your code must deal with the failure of your application (for example it must store the offset of the last record of tables that are moved to Kafka, so it can continue to copy the records that were not inserted into Kafka), scalability, polling and….</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/0*u_4hrYcSatqushvV" /></figure><p>But if you choose the second option you can move data without writing any code. In fact, the Kafka Connect does the same job as the first option but in a scalable and fault-tolerant way. The process of copying data from a storage system and move it to Kafka Cluster is so common that Kafka Connect tool is created to address this problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/0*glCy7DNhEOjfmM7K" /></figure><p>Notice that transferring direction can be vice versa. That means you can move data from Kafka cluster to database system using Kafka Connect. In this article, we focus on copying data from external storage to Kafka. I share my understanding of some of the concepts of Kafka Connect cluster and after that, I show you how I created a Kafka Connect cluster.</p><h3>Kafka Connect Concepts</h3><p>Kafka Connect uses <strong>workers </strong>for moving data. Workers are just simple Linux (or any other OS) processes. Kafka Connect can create a cluster of workers to make the copying data process scalable and fault tolerant. Workers need to store some information about their status, their progress in reading data from external storage and so on. To store those data, they use Kafka as their storage. Note that Kafka Connect cluster (which is a cluster of workers) is completely different from the Kafka cluster (which is a cluster of Kafka brokers).</p><p>More workers mean that your copying process is more fault tolerant. Consider the scenario that you want to move data from your SQL Server database to Kafka. You can have a cluster of single worker (single process) and your data will be copied until when your worker process or machine restarted (for example for maintenance reason) or crashed. Your data will not be transferred unless the server becomes alive. But if you have 2 workers, when one of the workers stopped, your data copying process will be continued by the second worker. This way you achieve high availability.</p><p>Here you can see that 4 workers inside 3 physical machine have made a Kafka Connect cluster of 4 workers that each one of them is copying data from a database in parallel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*F08rOM5Zfjuq_pzH" /></figure><p>In the above image, if we have 12 tables inside the SQL Server database, each worker might read from 3 tables of the database. When one worker becomes unavailable other 3 workers take the responsibility of the unavailable worker:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XPd_0mV9ZYmVih13" /></figure><p>So probably 3 live workers now must do worker 1 responsibility so that each worker now should read 1 additional table (each one of them now reads from 4 tables).</p><p>If you just need to run your Kafka Connect in a single worker you can use <strong>standalone mode</strong> of the worker that store its data on disk. Actually, there is 2 type of workers and what we have discussed until now was <strong>distributed mode</strong> of worker that allows you to create a cluster of workers. I don’t cover standalone mode today.</p><p>Workers are a physical concept. They are actually processes that run inside JVM. Your job in Kafka Connect concepts is called a <strong>connector</strong>. It is something like this:</p><ul><li>Copy records from table ‘accounts’ of my MySQL to Kafka topic ‘accounts’. It is called a <strong>source connector </strong>because you move data from external storage to Kafka.</li><li>Copy each message from Kafka topic ‘product-events’ to a CSV file ‘myfile.csv’. It is called <strong>sink connector </strong>because you move data from Kafka to external storage.</li></ul><p>It does not define which worker is responsible for doing the job. Workers can cooperate with each other to do this job. Each connector has a set of one or more tasks that these tasks can copy data in parallel. Having multiple workers allow you to distribute these tasks in several machines.</p><p>Because there are many storage systems, Kafka Connect must know somehow, how to copy data from one source to Kafka or vice versa. For this purpose when you want to submit a connector to Kafka Connect cluster, you must specify a <strong>connector plugin</strong>.<strong> </strong>Connector plugin typically is one or more jar file that knows how to copy data from a specific storage system to Kafka or vice versa. For example, when you want to load data from SQL database like (SQL server, MySQL and …) you need a <a href="https://www.confluent.io/connector/kafka-connect-jdbc/">JDBC connector plugin</a>.</p><p>JDBC connector is included by default in Confluent installation package but if you need a connector that is not installed by default you first need to <a href="https://docs.confluent.io/current/connect/managing/confluent-hub/client.html">install Confluent Hub client</a> and using this client you can install any connector that is available inside <a href="https://www.confluent.io/hub/">Confluent Hub</a>. In the next section, I show you how you can view a list of already installed connector plugins. If you didn’t find your connector plugin inside Confluent Hub, <a href="https://docs.confluent.io/current/connect/devguide.html">you can implement your own connector plugin</a>.</p><p>To run worker you must provide a configuration file (which is a <a href="https://en.wikipedia.org/wiki/.properties">.properties file</a>) like this:</p><p>connect-distributed worker.properties</p><p>Here worker.properties contains a list of worker configurations. We will describe some of the main configurations in the next section. But for now, you must know that you can set plugins directory path in <em>plugin.path</em> configuration. Confluent ships with sample worker configuration file that you can look at them to find your installed plugin path. For example looking at <em>/etc/kafka/connect-distributed.properties</em> sample:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d977473e3277dbe64ecdae706589633b/href">https://medium.com/media/d977473e3277dbe64ecdae706589633b/href</a></iframe><p>Shows that my plugin path is <em>/usr/share/java</em>.</p><p>Now we are ready to create a Kafka Connect cluster.</p><h3>Setup Machines</h3><p>First, we need to prepare our machines. I have 3 Virtual Machine named:</p><ul><li>Kafka: The Kafka standalone broker that I want to move my data and also keep states of my Connector cluster instance and has 192.168.80.30:9092 URL. An instance of Zookeeper is also running in this machine.</li><li>Worker1: First worker instance</li><li>Worker2: Second worker instance</li></ul><p>All of them <a href="https://docs.confluent.io/current/installation/installing_cp/deb-ubuntu.html#systemd-ubuntu-debian-install">installed in Ubuntu</a>. And my SQL Server is located in 192.168.80.1 in my Windows 10.</p><h3>Configure Workers</h3><p>In both Worker1 and Worker2, I did the following steps. Create a directory named worker in the home directory:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a093f887c34e855ccbe0b6cf6b0fdbbe/href">https://medium.com/media/a093f887c34e855ccbe0b6cf6b0fdbbe/href</a></iframe><p>Now I want to create the worker configuration file.<em> connect-distributed.properties </em>is good sample file for worker configuration. Copy <em>etc/kafka/connect-distributed.properties</em> to <em>~/worker/myworker.properties</em>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4521547a052f85daec33414ad9b8d614/href">https://medium.com/media/4521547a052f85daec33414ad9b8d614/href</a></iframe><p>And open it in your editor. I change the following properties:</p><ol><li>Change <em>bootstrap.servers=localhost:9092</em> to your Kafka servers in my case it is <em>bootstrap.servers=192.168.80.30:9092</em></li><li>Change <em>group.id=connect-cluster</em> to <em>group.id=my-example-connect-cluster</em>. Not that workers with the same group.id is joined to the same cluster. So you have to ensure that all your workers have the same group.id value.</li></ol><p>Now you need to place your JDBC driver inside JDBC plugin directory. In my case, because I want to move data from SQL Server, I downloaded SQL Server JDBC jar file from <a href="https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc">Maven repository</a>.</p><p>After downloading JDBC driver you must move it to JDBC connector plugin directory. For example, because my JRE version is 10 I downloaded <a href="http://central.maven.org/maven2/com/microsoft/sqlserver/mssql-jdbc/7.0.0.jre10/mssql-jdbc-7.0.0.jre10.jar">mssql-jdbc-7.0.0.jre10.jar</a> file and copied it to the plugin folder.</p><p>So copy SQL Server JDBC driver to JDBC plugin folder:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/87681b5aa19513c8d6e2c238c2353bfa/href">https://medium.com/media/87681b5aa19513c8d6e2c238c2353bfa/href</a></iframe><p>This is contents of my JDBC connector plugin directory (/usr/share/java/kafka-connect-jdbc) now:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3604180abe46a216c556325d8edd9cd2/href">https://medium.com/media/3604180abe46a216c556325d8edd9cd2/href</a></iframe><p>If you don’t place this driver file inside plugin directory you will receive: <a href="https://stackoverflow.com/questions/49947810/sqlexception-no-suitable-driver-found-for-jdbc-sql-server/49990129?noredirect=1#comment94416444_49990129">SQLException No suitable driver found for jdbc sql server</a> error. Note that JDBC driver for PostgreSQL and SQLite databases already exists in the directory. For other databases like MySQL, you have to copy their drivers. Also, the driver must be installed in all workers, otherwise you might receive another error when you are trying to submit your connector to Kafka Connect cluster:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/150cd069ba5651ee97c941d8383e8653/href">https://medium.com/media/150cd069ba5651ee97c941d8383e8653/href</a></iframe><p>Now you are ready to join your workers to the Kafka Connect cluster by running following command (In both Worker1 and Worker2):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fab2c858c1f751b0554eb8ef110459ab/href">https://medium.com/media/fab2c858c1f751b0554eb8ef110459ab/href</a></iframe><p>Some logs appear on the screen and in the last line you will see:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/da2538828c6e728086d46a9a3d816a9e/href">https://medium.com/media/da2538828c6e728086d46a9a3d816a9e/href</a></iframe><p>This indicates that your workers are working successfully.</p><h3>Move Your Data From SQL Server to Kafka Cluster</h3><p>Now you have Kafka Connect cluster. You can view your worker information by running following command (in each worker) that call <a href="https://docs.confluent.io/current/connect/references/restapi.html#post--connectors">worker REST API</a>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/72a09450004e1436e6f3320186ac8e70/href">https://medium.com/media/72a09450004e1436e6f3320186ac8e70/href</a></iframe><p>You can also view the list of supported plugins using this REST API:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/49dd79530559e3145d9a34b901f0b7d3/href">https://medium.com/media/49dd79530559e3145d9a34b901f0b7d3/href</a></iframe><p>Now let’s create a simple database and table in SQL Server that we want to copy it’s data to Kafka. I create a database called <em>ConnectSampleDb</em> which has a single table <em>ConnectSampleTable </em>as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/342/0*t42Lwf8DgIV_9BWM" /></figure><p>And fill ConnectSampleTable like the picture</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*7c6XmiVMgJ58Xieb" /></figure><p>Now I create a file named sample-sql-server-jdbc-connector.json inside worker folder of Woker1 with the following content:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a7d7e93e3e0f8df685c46a89435c2fae/href">https://medium.com/media/a7d7e93e3e0f8df685c46a89435c2fae/href</a></iframe><p>This file contains the body of the HTTP request that we want to send to Kafka Connect REST API to create a source connector job. You must run the following command to create a connector (you see the JSON response in the line after that):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/79b60b304601e7b060e0f3e144de2ec2/href">https://medium.com/media/79b60b304601e7b060e0f3e144de2ec2/href</a></iframe><p>The above request and response show that our connector is submitted to Kafka Connect cluster successfully. You can also view the list of connectors using following curl command:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d03864280143bfdab2bee84104208271/href">https://medium.com/media/d03864280143bfdab2bee84104208271/href</a></iframe><p>To get status of a specific connector you can run the following command:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/daee159b82539c2d902f9ef25d330117/href">https://medium.com/media/daee159b82539c2d902f9ef25d330117/href</a></iframe><p>This indicates that your connector is working. You can use status API to monitor your connector. You can call all REST API from all workers (Worker1, Worker2).</p><p>So let’s view the list of topics:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7c0df91d1307ead577e6c8e1230cf75d/href">https://medium.com/media/7c0df91d1307ead577e6c8e1230cf75d/href</a></iframe><p>As you see test-sql-server-jdbc-ConnectSampleTable is the topic that contains data from ConnectSampleTable. We can view moved data using Kafka console consumer:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f5c417b214ae0549f8f616d4a60a72e9/href">https://medium.com/media/f5c417b214ae0549f8f616d4a60a72e9/href</a></iframe><p>As you can see the result contains both schema and payload. If you want just payload you can change <em>myworker.properties</em> before running worker:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1387f62bafb91130db9988ba9851e448/href">https://medium.com/media/1387f62bafb91130db9988ba9851e448/href</a></iframe><p>The result then should be like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4c687d6273705e15b91a0fde46c5a98/href">https://medium.com/media/e4c687d6273705e15b91a0fde46c5a98/href</a></iframe><p>I added another record with name ‘Harry’ to ConnectSampleTable and the result added to the topic successfully:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/37d0062ef13441efe2895b3e48834431/href">https://medium.com/media/37d0062ef13441efe2895b3e48834431/href</a></iframe><p>Let’s continue and create another table inside ConnectSampleDb database. I create a table named ConnectSecondTable:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/0*FEFennzyAP5lWx5_" /></figure><p>Insert one item into the table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/341/0*NHkEA6mrLqFOX7rO" /></figure><p>View topics list:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c952381cb8853d04976f5dccb135494b/href">https://medium.com/media/c952381cb8853d04976f5dccb135494b/href</a></iframe><p>As you see test-sql-server-jdbc-ConnectSecondTable topic is created for new table. Viewing new topic shows it successfully copied our ConnectSecondTable records.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3772dd1859b8b58038ec48a435c20148/href">https://medium.com/media/3772dd1859b8b58038ec48a435c20148/href</a></iframe><p>As you saw the JDBC connector plugin by default watch any changes in the database including newly create tables. If you want to move data from one or a small number of tables you can use <em>table.whitelist</em> configuration in <em>sample-sql-server-jdbc-connector.json</em> file before submitting your connector as <a href="https://docs.confluent.io/current/connect/kafka-connect-jdbc/source-connector/source_config_options.html">described with other configurations in the official document</a>. Note that you can update your connector configuration by the <a href="https://docs.confluent.io/current/connect/references/restapi.html#put--connectors-(string-name)-config">REST API</a>.</p><p>To test that our Kafka Connect cluster is fault tolerant, I killed Worker 1 process and insert another record inside ConnectSecondTable:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/0*m4NO_NyX3Y5FgqeI" /></figure><p>And viewing topic data shows that new record is inserted:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2a73e8bcb93a5f99128a7b0fae527907/href">https://medium.com/media/2a73e8bcb93a5f99128a7b0fae527907/href</a></iframe><p>It is interesting that you can also view data of connect-offsets topic:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/79f0192a972efffa6c08218bbae3b8f1/href">https://medium.com/media/79f0192a972efffa6c08218bbae3b8f1/href</a></iframe><p>But to get better information you need to print key (separated by ‘-’):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2dd6f2ab10c52e70b6856353dcd60afe/href">https://medium.com/media/2dd6f2ab10c52e70b6856353dcd60afe/href</a></iframe><p>As you see there is much more information in keys:</p><ul><li>Name of connector</li><li>Name of database and table.</li></ul><p>This way Kafka Connect can manage different connector offsets.</p><h3>Conclusion</h3><p>In this article, we introduced the Kafka Connect Cluster and some of its concepts like workers, connectors and connector plugins. We learned how to configure a set of workers that can coordinate connector tasks inside the cluster. We noticed that by default JDBC connector watches any changes in the database and move them to Kafka. I hope this article might be helpful for you. If you feel that you can add more information to this article or some of the information is not correct, please feel free to comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=26522e72a9af" width="1" height="1" alt=""><hr><p><a href="https://medium.com/clay-one/kafka-connect-cluster-an-introduction-26522e72a9af">Kafka Connect Cluster: An Introduction</a> was originally published in <a href="https://medium.com/clay-one">clay-one</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Birth of a new star — The Scope]]></title>
            <link>https://medium.com/clay-one/birth-of-a-new-star-the-scope-7f737b00cb48?source=rss----a4be5740a48f---4</link>
            <guid isPermaLink="false">https://medium.com/p/7f737b00cb48</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[software-design]]></category>
            <dc:creator><![CDATA[Kia Raad]]></dc:creator>
            <pubDate>Wed, 29 Aug 2018 08:46:01 GMT</pubDate>
            <atom:updated>2018-08-30T20:30:42.700Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*j6xIy2gosAooeB28DTPBhg.jpeg" /></figure><p>How does a platform take form? Where does the idea come from? How do people materialize those ideas into software? What is the process in which a software is designed?</p><p>If you think a platform is a large piece of software (more precisely a bundle of software), think of a service, like a message broker — that’s simple enough, right? Now lets see how do we try to systematically think through designing these kinds of systems, and what challenges does our way of thinking help solving.</p><p>Before going any further, lets be clear about what kind of software we are talking about. What we focus on is the kind of software that facilitates business application development, like identification/authorization service, credit calculation, message brokers, subscription engines, comprehensive distributed logs, historical data store, etc. What these software all have in common, are the ability to be distributed, scaled, reliable, and — as you might have guessed — easy to maintain and monitor. So the following content might not be suitable for designing every and all kinds of software. And it’s definitely not a one-rule-fits-all thing.</p><p>Every practical and useful system starts with hope to find an answer to a problem, and depending on how big the problem is, the answer may vary. One thing is common among all the possible answers though, they should all target a specific <strong>scope</strong>.<strong> </strong>Whether your particular problem is about managing a large immutable data store or a message broker or an identity verification service, you have to define a scope of what your solution <strong>MUST</strong> provide, and more importantly, what it <strong>MUST NOT</strong>. There are however some areas which do not quiet fall into the MUST and MUST-NOT categories, at least at the begging of the scope definition process. That’s why we, at clay-one, define a third “MAYBE” column and put those areas of ambiguity in that. most of the scope definition time is then spent brain-storming new scope aspects, and presenting proof for either the MUST-NOT column tenants, or the MUST column.</p><p>The MUST column is pretty simple to prove. Well, they are essentially the critical topics of your targeted problem, right? However the MUST-NOT and the MAYBE columns are trickier. You — as a software developer/architect — have to refrain from creating a god-system! (refer to the <a href="https://en.wikipedia.org/wiki/God_object">God object anti-pattern</a> in Object Oriented paradigm)</p><p>During a number of brain storming sessions — usually not more than three or four — a service scope definition document is drafted. The document comprises of 4 major sections which are roughly defined as follows:</p><h3>General Definition</h3><p>This section tries to provide a definition of the service and a 1000-mile overview of its purposes.</p><h3>Responsibility Matrix</h3><p>The categorization of the “MUST”s, “MAYBE”s and “MUST-NOT”s. Usually you start with a more populated “MAYBE” column and categorize its contents during the discussions. <a href="https://github.com/clay-one/nursery/wiki/BlackWhole-Service-Scope-Definition#responsibility-matrix">Here</a> is a good example of what a matrix would look like mid-way through the design.</p><h3>Instances / Variations</h3><p>A rough estimation of the final deployed product, how many instances it might have and whether or not it has variations of instances. For instance is it a monolith or a collection of micro-services. Are those services deployed on one machine? is it a single node deployment? These kinds of questions are addressed in this section.</p><h3>Open Issues / Problems</h3><p>This sections contains the problems that the current design has not addressed; Or they have not been talked about thoroughly enough. This is merely for keeping track of the challenges and/or issues the product might face during it’s design/development life-cycle.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f737b00cb48" width="1" height="1" alt=""><hr><p><a href="https://medium.com/clay-one/birth-of-a-new-star-the-scope-7f737b00cb48">Birth of a new star — The Scope</a> was originally published in <a href="https://medium.com/clay-one">clay-one</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Genesis]]></title>
            <link>https://medium.com/clay-one/genesis-9174bfb95da6?source=rss----a4be5740a48f---4</link>
            <guid isPermaLink="false">https://medium.com/p/9174bfb95da6</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[hello-world]]></category>
            <dc:creator><![CDATA[Kia Raad]]></dc:creator>
            <pubDate>Sat, 04 Aug 2018 20:00:13 GMT</pubDate>
            <atom:updated>2018-08-04T20:00:12.422Z</atom:updated>
            <content:encoded><![CDATA[<p>In the beginning, there was a bunch of code; Unstructured and unwieldy. On first day “<em>The dude</em>” said: “Let there be services”; and so there were a bunch.</p><h3>Who are we?</h3><p>We are a group of software freaks, eager to explore, create and share new ideas; Be that a piece of code, a fully functioning online service or even a small Gist. We are constantly facing new problems; Mostly revolving around high-performance, multi-user, under heavy load, <strong>simple </strong>— well, kinda — tasks that are cumbersome for business application developers to attack and solve in an engineered, tested, <strong>scalable </strong>way.</p><p>We do that :)</p><p>Most of our work is born from the actual problems we have faced (and in some cases are still facing) in our day-to-day programming tasks and for that reason, we believe that the solutions we come up with — with the help of others, oh yeah, we are open source! — might be helpful for others as well, hence sharing them. on <a href="https://github.com/clay-one">Github</a>. Period.</p><p>Feel free to take a look at the newly born <a href="https://github.com/clay-one">Github</a> page or even check our <a href="https://twitter.com/clay_one_code">Twitter</a> every once in a while.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9174bfb95da6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/clay-one/genesis-9174bfb95da6">Genesis</a> was originally published in <a href="https://medium.com/clay-one">clay-one</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>