Site reliability Engineering (SRE) today has become an important pillar in narrowing the gap between developers and IT operations. It represents the solution between what actually happens in the software and what we want to happen to guarantee an excellent user experience. It is a discipline that covers a set of principles and practices to solve infrastructure and operational problems.

The concept was created to handle all complex operations to be solved by technically reasonable solutions. SRE at Google centers on securing, sharing and creating programs and frameworks for all of Google’s open services, which is additionally a confirmation to high accessibility, more reliability and better performance of their services.

Ben Treynor, who is the founder, defines SRE in this interview as:

“Fundamentally, it’s what happens when you ask a software engineer to design an operations function…So SRE is fundamentally doing work that has historically been doe by an operations team, but using engineers with software expertise, and banking on the fact that these engineers are inherently both predisposed to, and have the ability to, substitute automation for human labor.”

SRE Goals are:

• Monitore and improve the reliability of systems
• Anticipe failures and Solve them
• Automate Operation tasks
• Speed and Performance

History Background :

According to SRE’s book, before SRE google used a system administrator approach to ran its operations. System Administrators worked on the operations aspects, whereas engineers worked on the development side. However, due to the fact that both groups had different backgrounds, skills and different perspectives, this approach caused a division and a conflict between the two. Developers were only concerned about creating new features while operation members were concerned about the efficiency and reliability of softwares and this is how SRE was Born.

The story of SRE started in 2003 when Benjamin Treynor, the founder of the term, was in control of running a small group of 7 engineers. The group was designed to make sure that google websites/services were accessible, dependable, and highly reliable.

As Benjamin was a Software engineer, he managed the team in a way that developers and IT operations were part of one team. He did this by letting the team spend a part of their time on operations tasks so that they have a better understanding on how things work at their end and to see the process of production. As he said:

SRE is “what happens when a software engineer is tasked with what used to be called operations.”

As a result this team became the first SRE team and SRE is now considered as the bridge between developers and IT operations.

When they started, google was much smaller and SRE was a group of few people. Today there are more than 2500 employees in the world.

SRE team was created to align with the grow of developments, as Andrew Widdenson ( SRE @ google ) said :

Our work is like being a part of the world’s most intense pit crew. We change the tires of a race car as it’s going 100mph

Importance of SRE :

Monitoring and improving the reliability of systems:

Monitoring is the core of the development of a business. It helps you save money and time. That’s why it is so important to choose your software wisely. SRE teams’job is to anticipate and alert on situations that requires more attention. Their ultimate goal is to build up a tool to help solve the diagnosed weaknesses.

This is a bit different of what DevOps does, which is basically a responsive approach to infrastructure to enable faster development of new products. In other words and to distinguish any confusion between the DevOps and SRE, they complement each other. DevOps is about the theory and philosophy while SRE is a perspective way of achieving that philosophy.

Troubleshooting escalation issues :

SRE teams are responsible to provide a good customer service. They are also excepted to be dealing with technical incidents, taking care of escalations cases and cooperating directly with clients. Yet, the goal here of a SRE team is to reduce critical incidents and manual work. This solution will give IT support and developments teams a push to care less about escalations and concentrate on implementing new services and building new features.

Case Study: Adopting SRE Principles at StackOverflow

In this section we will cover the experience of StackOverlow when adopting SRE principles at StackExchange.com/StackOverflow.com. An experience shared by Tom Limoncelli a former SRE member at Google who now works in Stack Exchange (home of ServerFault.com and StackOverflow.com).

For those who do not know what Stack Echange is, it is a website like Quora that covers many Questions/Answers. The difference between the two is that Quora is one website while Stack Exchange is a series of many websites, one field of knowledge each. It is considered to be the biggest, most trusted online community for engineers/developers to build their knowledge, share their information, and construct their careers.

In 2012 and rather than aiming to be the biggest software, Stack Exchange set out a SRE team to make their website more speedy and to guarantee a better user experience. Now the question How they did that?

The SRE team’s work was divided into two categories and as Mr Limoncelli called them easy and hard ones. We will first tackle the easy one and then the hard ones.

As they were a small team with a low budget, the firm considered the Developers and SRE’s members of one group, there was no separation of budget, they had a common staffing pool for SRE and developers.

When handling Outages/Incidents: they aimed for a maximum of two events per oncall shift. The purpose here is to well understand the origin and source of an incident to anticipate and prevent any other issues. Every outage is an opportunity of learning and improving the system.

They succeed on this by holding a post-mortem after every incident and by sharing the results with the member of the team and with Devs as well. This was very important for the team as Tom Limoncelli puts it :

Now we will cover the hard category:

One of the most hard challenges they encountered is SLA-Driven Operations and Monitorings. Defining SLAs was difficult as:

• They are Complex.
• Requires communication with many departments: Engineers, Product management…

They managed monitoring by building they own monitoring tool called www.bosun.org. The first monitoring system with an IDE ( Integrated development environment ) for designing and developing complex alerts and that can create formulas for these laters. Additionally to that, the system uses OpenTSDB for storage, rewrote agent in Go and works with Linux+Win, and can imply Graphite as a back end.

This tool helped in many aspects :

• IDE made it easier for Developers to make their own alerts about things they didn’t understand in performance and which eventually SRE members will find a solution for it.
• And easy with the library for developers to know what DATA must be collected

SLAs made it easier to know the goals behind an action/issue and to know what the gaps are, what to prioritize and when to stop. As Tom Limoncelli puts it :

It isn’t a service if it isn’t monitored against an SLA. If there is no SLA-Based monitoring then you are just running software.

This step is important and indispensable, SLAs are considered to be the basics of managing a service.

The last point was Controlling Operations Overload, in this stage we will discover how Stack exchange handled requests. They managed to handle the flow of requests by:

1. Running systems that can handle transactions to gain time. For example, they didn’t create monitoring rules for devs, they created a system that drives devs to be self-sufficient. Eventually, devs can develop they own monitoring rules and can select what needs to be collected by SREs.
2. Hiring only coders, Cap SRE operational load at 50% and share 5% Ops work with Devs teams.

Finally, thanks to this strategy of SRE, they speeded the process and the efficiency of the websites and eventually were able to delete the ticket system. Instead they pointed one SRE member to each development team as a sort of representative. Moreover, the deployment of this steps was a success and it created more time for SRE team to focus on service development, optimization and on other projects.

SRE Teams :

SRE teams are made of software engineers who uses tools (Git, Jira, Maven…) to improve the reliability of their systems. Being a SRE today is demandable but not easy as it requires the following hard and soft skills:

• Passion for technology and an excellent background in software developments and IT operations
• Excellent verbal and written communication skills
• The ability to direct and solve problems
• Believe in Team work and mutual trust
• Capacity of self-teaching

For those who are considering becoming an SRE member, here is a SRE Compatibility Quiz:

1. Do you like communicating and sharing what you know with others?
2. Do you enjoy doing analysis, thinking outside the box and eventually performing an in-depth analysis of the possible risks that a service may encounter?
3. Do you like watching services under production and to analyze their performance?
4. Do you manage to work under pressure and to stay calm when needed?
5. Do you dislike manual work and enjoy automating everything?
6. Can you spend time rotating on on-call responsibilities ?
7. Can you manage incidents by using monitoring and metrics?
8. Are you passionate about engineering, softwares and eager to know anything new?
9. Do you enjoy team work?
10. Are you someone who usually thinks about improving his knowledge?

If you are a software engineer of thinking of becoming an SRE, then I encourage you to pursue it. SRE is indeed a new discipline and is actively evolving. It is the ideal combination of skills for tightening the relationship between IT and developers, resulting in shorter feedback loops, improved communication, and more dependable applications.

Conclusion:

Site reliability Engineering is today the core to successful business and it is believed that in the upcoming years it will become one of the most popular software teams used to maintain the efficiency and reliability of systems.

It has grown to be a worldwide community, and the future as Benjamin Treyor the founder of SRE said is unpredictable

“ I have a lot of Ideas but the truth is I don’t really know for certain. Now that SRE had grown to be a global community and not just a google thing, the future of the profession is determined by the progress and innovations of everyone in SRE. And, if you had asked me in 2004 what would have become of the SRE team, I would not have predicted this. I thought it was going to be a very small niche part of computer science. So what I do predict is that the community of smart and motivated people who now make up the global SRE will continue to come up with new ideas, they will continue to innovate and to advance the state of the art and advance what is possible “

References Links:

https://authority-tech.website/why-sre-is-important/#more-133 http://adservio.fr/performance.html

http://adservio.fr/doc/PERFORMANCE.pdf

https://worksplatform.com/en/what-is-no-code/

Why SRE is important was originally published in ADSERVIO on Medium, where people are continuing the conversation by highlighting and responding to this story.

To illustrate the subject of our article, we used the famous quotation of the great Cardinal de Richelieu (Les maximes d’Etat 1623):

«The great flames are born of small sparks».

To tie up the subject, this article will be divided into 5 parts:

1/ What is Microbenchmarking?

2/ How is it useful?

3/ Microbenchmarking in BigData

4/ What problems could we encounter?

5/ Microbenchmark vs Macrobenchmark.

6/ How to set your reference scale?

1/ What is Microbenchmarking?

When dealing with a research, survey, study, etc. the most manageable reference volume is the one of small size. It is by assembling the small elements that we achieve a larger scale.

As a general rule, the Microbenchmark is associated with the measurement of the execution time of an isolated unit of code. It is therefore used in the short term to perform tests of software sub-programs.

For Big Data, microbenchmarking is considered the first step in performance testing. It can be used on a cluster, algorithms, as well as framework tests to obtain more important data in terms of volume and accurate content.

2/ How is it useful?

In addition to its function of measuring small sections of code, the microbenchmark also evaluates the performance and speed of multiple functions that perform the same task. The microbenchmark is a method that you feed into your API, or an algorithm that you administer to an input. The execution time of this one is counted in milliseconds or sometimes less.

Considered as the first step of performance tests, they ensure the smooth operation at the T-moment of the multiple data that make up a software.

Here is a sample File System Micro-benchmark Tests:

Source: https://www.informit.com/articles/article.aspxp=2144597&seqNum=2

3/ Microbenchmarking in BigData

Usually, a test on a lower level of code components was enough to speak of microbenchmarking. Big Data came to change the game. When we talk about microbenchmarking, it engages the server cluster and at the same time engages the algorithms or frameworks on which the developer relies to build an application.

Microbenchmarks in BigData contain, in addition to virtualization tests such as files or memories delegated by middleware within the software, monograms of the conventional lower-level system. In this area, several measures of interest are before us: the overall processing power, the speed of the cluster globally, a consistent addition of data is essential to be able to fully exploit the overall capacity of the system you are setting up.

4/ What problem could we encounter?

The Microbenchmarking is perfect for testing a specific function. However, it is not designed to check for pervasive changes as it only handles the very short term. For example, it is almost impossible, if not impossible, to use them to test an acceleration. Not based on a fixed code, the Microbenchmark performs a single operation several times.

Your benchmark can therefore be put into play by a simple factor change, without you realizing it.

5/ Microbenchmark vs Macrobenchmark

Now that we have defined the Microbenchmark, we will move to its opposite which can only be the Macrobenchmark.

The macrobenchmark is more global than the micro, it allows a more general study that passes the cap on the small details that the microbenchmarking deals with. Complementing each other, the macrobenchmark cannot do anything without going through the microbenchmark:

• If the Benchmark fails, it is almost impossible to know the source if a microbenchmark is not done in advance.
• Small-scale optimization would be difficult to detect with a macrobenchmark.
• Large errors are easier to detect with a macrobenchmark than a microbenchmark.
• The effect of a macrobenchmark’s work is more visible when there are many microbenchmarks.
• The successful microbenchmark is evaluated by a benign variation and irreproachable accuracy. The macrobenchmark proves its correctness when the workload is high.

To better illustrate our lyrics in a more basic and palpable way, here is a fictional graph that summarizes a code benchmark, including both macro and microbenchmark:

The lower part contains small lines, the middle part medium lines and the upper part large lines.

Although the lines reaching the middle part of the graph are important, they cannot be made without passing through the endless lines at the bottom. These represent the microbenchmarking. As you can see, the macrobenchmark cannot be completed without the support of the microbenchmark

6/ How to set your reference scale?

Before you perform your benchmark, you must first ask yourself the following question: “What size should it be?” If you make an unsuitable microbenchmark, you will not get the results you are looking for. If on the other hand your clue is too important in relation to your research, you might get lost.

The best thing to do would be to ask you the question “What is my target?”

If you are thinking of convincing your client, requester or even the internet, you are probably facing a microbenchmarking. Apply it on a small scale and you will quickly get results on the toughness and speed of your application.

Be careful, don’t forget that the microbenchmark is for many programs and software a race against time, encouraging to ensure the proper functioning of sub-programs in line with the timely delivery of multiple individual components of a software or application.

However, when properly implemented, microbenchmarks are the best way for a developer to test a specific language.

If you are looking to convince your end users, then the macrobenchmark is your savior. It will allow you to test your system with a user view and not your own. So you will be able to detect faults on a large scale.

As you can see, microbenchmarking is your first line of performance testing. It will allow you to ensure the smooth and very short-term operation of several individual sub-programs. However, the end result of your code may look like Rube Goldeberg’s machine when it is not well done.

References links:

https://authority-tech.website/the-impact-of-covid-on-low-no-code/

https://worksplatform.com/en/the-rise-of-citizen-developers/

http://adservio.fr/Clients.html

https://growth-authority.com/services-redaction/

http://adservio.fr/actualite.html

What is microbenchmarking : was originally published in ADSERVIO on Medium, where people are continuing the conversation by highlighting and responding to this story.