<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by András Fincza on Medium]]></title>
        <description><![CDATA[Stories by András Fincza on Medium]]></description>
        <link>https://medium.com/@drawain?source=rss-8cb28c926fb9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*KFq3ZjjEG6nB9IxZ.jpg</url>
            <title>Stories by András Fincza on Medium</title>
            <link>https://medium.com/@drawain?source=rss-8cb28c926fb9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 14:49:18 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@drawain/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[First steps to a service architecture]]></title>
            <link>https://medium.com/emarsys-craftlab/first-steps-to-a-service-architecture-dafc576e73f6?source=rss-8cb28c926fb9------2</link>
            <guid isPermaLink="false">https://medium.com/p/dafc576e73f6</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[emarsys-dev]]></category>
            <category><![CDATA[greenplum]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[András Fincza]]></dc:creator>
            <pubDate>Tue, 20 Sep 2016 13:01:42 GMT</pubDate>
            <atom:updated>2017-02-01T12:36:45.735Z</atom:updated>
            <content:encoded><![CDATA[<p>Breaking the chains of the Monolith is never an easy task, especially if you have such a huge product and codebase as we had.</p><blockquote><em>We chose the path of </em><strong><em>careful evolution</em></strong><em> with trying out new things in separation with a distinct, narrow scope.</em></blockquote><p>At this time we already had experience with rolling out new features in small steps and progressively replacing the old ones. We have incrementally rewritten the email sending for example — one of the most important parts of our application. While we were developing the new sender functionality we’ve always tested and compared the emails sent out with both the new and the old email generator — but at the end of the day everything was sent with the original sender. This testing method is called <a href="http://tech.co/the-dark-launch-how-googlefacebook-release-new-features-2016-04">dark launch</a> and the main benefit is that you could try out your new codes without effecting the actual behaviour. We could enable parts of the new sender method one by one after the related dark launch was successful. To enable a new functionality we could easily turn on its <a href="http://martinfowler.com/articles/feature-toggles.html">feature toggle</a> — we just call it “<em>flipper</em>”.</p><h3>Apply feature toggles on Services</h3><p>These experiences led us to introduce services in a similar way. If we wanted to replace an old functionality we should be able to do it incrementally with the possibility to disable the new method and use the old one by turning off its flipper. The new features are also built in this concept — the related pages and the links pointing to them are hidden until the corresponding flippers get turned on. We had success with these practices and we are still following them today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-rnu2XhG0d0TPodYQoNZFQ.jpeg" /><figcaption>Flipper settings on the administration page</figcaption></figure><p>After we’ve designed and applied these concepts to our architecture we were prepared to iteratively introduce new functionalities using services and get the benefits of autonomous teams, separate deployment, isolated scaling and so on.</p><h3>Finding the path</h3><p>The first two services we made were the <em>Historical Data Service (HDS)</em> and the <em>Smart Insight (SI)</em>, both written in Ruby. Besides the timing and the technical similarities their creation had very different motivation. The former was made to cover an earlier problematic part of the Monolith — to store and provide behaviour data of our contacts, e.g. an event like opening or clicking in an email. It means a really massive amount of data which — with our old implementation — exceeded the MySQL databases in many ways, e.g. we’ve reached its record and column count limitations.</p><blockquote><em>It was even impossible to run a simple </em><strong><em>SHOW TABLES</em></strong><em> command!</em></blockquote><p>To avoid this problem we’ve decided to use a <a href="http://greenplum.org/">GreenPlum</a>-based “Big Data” warehouse for the new service.</p><p>The <em>Smart Insight</em> service is slightly different because it was a completely new feature and a new experiment as well. While it communicates with the Suite behind the scenes using a classic REST interface, it provides it’s own UI and session handling — so it’s a little application within our main application. To integrate it with the UI, we simply used iframes as the easiest way of integration. Because of the sensitive data they store and the fact that we had the infrastructure and human resources to manage these services, we kept them in-house. They are being continuously developed, so it can be surely stated that these are the largest services we have today.</p><blockquote><strong><em>Does it mean</em></strong><em> that we have created </em><strong><em>new monoliths</em></strong><em> in our system? </em><strong><em>Yes,</em></strong><em> </em><strong><em>maybe</em></strong><em> — </em><strong><em>but</em></strong><em> from the perspective of maintainability, testability and scaling </em><strong><em>they are performing fairly well right now.</em></strong></blockquote><p>They were developed with the mindset of splitting — so if it’s needed somehow (eg. to speed up the development or solve operational issues) we can easily break them down. Using the <a href="http://martinfowler.com/bliki/Yagni.html">YAGNI</a> principle is just as important in the architecture as it is in the codebase. Every service adds a perceptible complexity to your system so it’s good to avoid splitting services arbitrarily.</p><p>After the success of this architecture and armed with the experiences we got from these two services we’ve decided to move on. We stated that new features and bigger modifications in the Suite code should be implemented in separate services. Although it seemed to be a good idea, we were afraid that if everybody started creating their own service then it would be impossible to operate them in-house. There were no existing automations and interfaces to create a service machine from scratch and fully pass their operation to the development team.</p><h3>Failed project establishes the future</h3><p>Two years ago we made a funny little experiment: my team just got a small task — to record specific user behaviour throughout the UI — and it seemed that NodeJS and <a href="https://www.compose.com/">MongoDB</a> could be a very good choice to do it. Because there was no experience of operating these technologies in-house and we wanted to move fast, the team just decided to widen the boundaries of the team autonomy: we pioneered a new service integration from the cloud into our main application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r919b2054wkt8y9QJ3-oGg.png" /><figcaption>Infrastructure snapshot 9 months after the experiment</figcaption></figure><p>Although this project was never completed and it didn’t work as expected — so we were a little depressed — from the aspect of the integration it was a complete success. We’ve raised the stakes by choosing very high abstractions for everything: <a href="https://codeship.com/">Codeship</a> as <a href="https://en.wikipedia.org/wiki/Continuous_integration">CI</a> tool and <a href="https://www.heroku.com/">Heroku</a> as an <a href="https://en.wikipedia.org/wiki/Software_as_a_service">application platform</a>. Back then we didn’t know that the cloud and these providers will become the base for our new architecture.</p><h3>What’s next?</h3><p>In the next article I will write about how did we manage to integrate cloud services into our existing system architecture, where are we know, which problems have we solved and what are the actual difficulties.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dafc576e73f6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/emarsys-craftlab/first-steps-to-a-service-architecture-dafc576e73f6">First steps to a service architecture</a> was originally published in <a href="https://medium.com/emarsys-craftlab">Emarsys Craftlab</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The story of breaking down a Monolith]]></title>
            <link>https://medium.com/emarsys-craftlab/the-story-of-breaking-down-a-monolith-bb753a68a37e?source=rss-8cb28c926fb9------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb753a68a37e</guid>
            <category><![CDATA[monolith]]></category>
            <category><![CDATA[emarsys-dev]]></category>
            <category><![CDATA[emarsys]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[András Fincza]]></dc:creator>
            <pubDate>Tue, 20 Sep 2016 12:53:32 GMT</pubDate>
            <atom:updated>2017-02-01T12:35:37.060Z</atom:updated>
            <content:encoded><![CDATA[<p>Millions of lines of codes without a single automated test, stressful development and growing business needs. It’s not a new problem for the software industry, it has been around for the last decade.</p><blockquote><em>Yes, we had our </em><strong><em>own Death Star</em></strong><em> — but we tried to </em><strong><em>learn from</em></strong><em> the </em><strong><em>others</em></strong><em> and </em><strong><em>manage </em></strong><em>the situation</em><strong><em> by the actual best practices</em></strong><em>.</em></blockquote><p>Going through the path of stabilising then de-structuring a monolith and moving towards a system based on services taught us valuable lessons. I’m very pleased to be part of the evolution of a huge terrifying software which became a maintainable and scalable application. I’d like to share our story and motivate you to fight with your legacy, too.</p><h3>The age of the Monolith</h3><p>When I said we had a Death Star I meant a <a href="https://www.emarsys.com/">software</a> which sends out billions of marketing campaigns through multiple channels like <a href="https://www.emarsys.com/en-uk/products/email/">email</a> or <a href="https://www.emarsys.com/en-uk/products/social/">social platforms</a>, <a href="https://www.emarsys.com/en-uk/products/data-science/">analyses</a> the behaviour of the recipients, make predictions and <a href="https://www.emarsys.com/en-uk/products/marketing-automation/">automate</a> the next campaigns. It’s pretty much one piece of software, but in the web applications of the early 2000s it was a common pattern. Not surprisingly our monolith — called <em>Suite</em> — was completely written in PHP using MySQL and the file system to store data, assets and logs. Over the years all of the new features were developed straight into this single application codebase without any distribution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cBZJCW_lvZQn14tPiIe1Pw.jpeg" /><figcaption>Some server rack serving the Monolithic application</figcaption></figure><p>The always growing business needs and the importance of the new customers forced our <em>Operations Team</em> to create distinct environments of the same application. These were completely separated server clusters with their own database, running the same codebase — but serving different customers.</p><blockquote><em>At this time </em><strong><em>copying everything</em></strong><em> to distinct servers </em><strong><em>seemed to be the easiest path</em></strong><em> to solving the problem </em><strong><em>of scaling</em></strong><em>.</em></blockquote><p>However, cloning everything to new environments wasn’t a sustainable solution and managing them caused us massive operation issues.</p><h3>The time of awakening</h3><p>After almost ten years of developing Suite — in 2010 — our stakeholders realised that the slowing releases and the raising amount of bugs wasn’t a good perspective for the future. The company had almost all of the classic problems which can be caused by a huge monolith. Besides, the software’s scaling issues we had problems with the maintainability, the reliability and the testability.</p><blockquote><em>We could only </em><strong><em>release once per year</em></strong><em> due to the </em><strong><em>insanely hard work of merging</em></strong><em> </em><strong><em>and</em></strong><em> comprehensively </em><strong><em>testing</em></strong><em> both the new and the old features.</em></blockquote><p>The company agreed on that we need a completely new approach for the development.</p><p>From the beginning it was clear to us that getting rid of our legacy is not the best way to follow. We wanted to iteratively improve our application so we could keep the values we had instead of starting everything over with no guarantees and the possibility of over-designing the new application. We’ve agreed that in every aspect of the development process we have to use a surgical knife, to cut and repair small pieces. To improve an application with short but meaningful steps has its cost, however, we accepted it.</p><h3>Rebooting the Development</h3><p>First of all, we made reforms on our processes by restructuring the development department. Instead of big teams we created small, independent ones — with the mind- and skill-set of the development, testing and operation processes. We’ve adopted and propagated <a href="http://www.goodreads.com/book/show/67833.Extreme_Programming_Explained">eXtreme Programming techniques</a> in order to reach these goals and to be more flexible.</p><blockquote><em>In technical perspective we decided to </em><strong><em>stabilise</em></strong><em> the application </em><strong><em>by writing unit, integration and system tests</em></strong><em> and run them on a Jenkins-based CI pipeline.</em></blockquote><p>It’s funny but at very first the latter was IRC-based and a build could be started by asking a bot to do it — long before the <a href="https://www.pagerduty.com/blog/what-is-chatops/">ChatOps movement</a> has shown up.</p><p>We’ve agreed on following <a href="http://www.goodreads.com/book/show/3735293-clean-code">Clean Code</a> disciplines — and the rule of refactoring the context of the legacy codebase on the spot when you want to add a new functionality. It was a long and hard process, but slowly we gained faith in our codebase.</p><blockquote><em>Thanks to these improvements we’ve managed to </em><strong><em>decrease the deploy time</em></strong><em> from several months </em><strong><em>to two deployments per day</em></strong><em>. We’ve successfully minimised the number of released bugs from one deploy to another.</em></blockquote><h3>Pioneers of the future</h3><p>We didn’t stop improving our main application. In order to get more flexible and quicker release and operation processes which aren’t bound to the problems of the Monolith, a decision was made three years ago to move to a <a href="http://martinfowler.com/articles/microservices.html">service-based infrastructure</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*hOPgTtnp956eX-wSyOo2iw.png" /><figcaption>Emarsys Operational Dashboard cca. 2013</figcaption></figure><p>With this step we hoped that a service deploy shouldn’t wait for the relatively slow deploy of the <em>Suite</em>. It could use its own attached resources while improving the autonomy of the teams. Finally a single service can serve all of the former environments. Moving from the customer-based architecture distribution to splitting our application by features was one of the most effective step we’ve made — it almost completely solved our scaling issues.</p><p>In my next post I will get into more details about the additional challenges, so stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb753a68a37e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/emarsys-craftlab/the-story-of-breaking-down-a-monolith-bb753a68a37e">The story of breaking down a Monolith</a> was originally published in <a href="https://medium.com/emarsys-craftlab">Emarsys Craftlab</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>