As things stand, FinTech solves problems and addresses pain points through innovative models, however, it is still reliant on legacy banking infrastructure. Cryptocurrency and its underlying technology, blockchain, has the power to revolutionise that infrastructure and, in doing so, revolutionise financial services work.

Even though Bitcoin has been making the big headlines, the main use cases in financial services are more practical — to store data in blockchain-based distributed databases for better efficiency and transparency. For example, we’ve seen our FinTech clients use blockchain, for storing financial operations logs and signature management. There are other uses too: for example, in lending blockchain can remove the need for centralised gatekeepers, making it more secure to borrow money and provide lower interest rates or in trade finance creating more transparency, security and trust among the trade parties.

There are many other use cases for the technology and we will explore them in detail as part of Fintech Week London for our sponsored session panel discussion ‘What’s Next For Blockchain In Financial Services’ on 12 July — you can get a free all-access online pass discount code here.

So, how does the landscape look right now? Firstly, there is a substantial difference in how blockchain can be used by players in different segments of financial services. Established banks and technology firms are primarily focusing on the applications for digital identities and regulatory compliance, whereas ‘start-up operators’ are mostly engaged in activities related to capital markets.

For banks blockchain’s power to reduce operational costs is the strongest business case while the current most relevant segment is that of payments. A decentralised ledger for payments can facilitate faster payments at lower fees, while clearance and settlement can get closer to real-time at reduced costs away from time-consuming and error-prone settlement processes.

Encouragingly for those working with blockchain, PoCs are being replaced with deployments to real production systems. While most use cases remain private and permissioned at this point, they are an encouraging indicator. The underlying technology of the network and protocol layers are now widely accepted as robust and stable allowing innovations to move to the application layer where the real value lies. As the higher stack application layer is more nascent, this is where using developers who are experienced becomes especially valuable to avoid wasting time and resources.

Technologies based on the BEAM VM and OTP such as Erlang and Elixir are often the right tool for the job when it comes to embarking upon a blockchain project. You can find the languages being used by a number of players in the space such as with ArcBlock’s blockchain developer platform that has a core infrastructure powered by BEAM technologies. They chose Erlang because it comes with built-in features that are critical to a blockchain system, like hot-upgrade, concurrency and high availability. This solid foundation has meant their team can focus more on building high-level features and making their platform developer friendly. Check out our recent webinar with Ty Chen, VP of Engineering, on building dApps on blockchain using Erlang.

Another example is POA Network, the Ethereum-based sidechain that offers an open-source framework for smart contracts to solve scalability, affordability, and interoperability issues. Erlang and Elixir give them the lightweight code needed to run thousands of processes concurrently on the same virtual machine, which is beneficial for scaling.

Our team consists of domain experts who have been involved in a wide variety of innovative blockchain projects with our clients. æternity blockchain is an Erlang-based scalable smart contract platform that looks to address some of the most fundamental​ elements of the technology — contract language, oracle and governance mechanisms and methods of consensus. æternity uses Erlang because it allows their team to write code that can respond to many requests in parallel and without crashing. Check out this talk from the Code BEAM conference on how and why æternity chose to use Erlang.

The last few years show blockchain technology stacks that can solve real problems in financial services. Ultimately, true advancements will emerge from the intersections of different disciplines. Interfaces of blockchain technologies with AI, ML and IoT need to be explored as we move to greater actual design and production engineering.

At Erlang Solutions we continue to collaborate closely with our partners in researching and delivering innovative solutions to support blockchain projects. This ranges from core blockchain technologies to more specific distributed applications supported by automated smart contracts. As experts with cutting edge know-how derived from direct exposure in building blockchain solutions and having been operating in this space for quite some time, we advocate that business leaders, regardless of industry, consider how to best use the technology to unlock value in their business models.

Join us to as we talk about the wider industry at Fintech Week London on July 12. Register for free.

Originally published at https://www.erlang-solutions.com on July 6, 2021.

It can hardly be described as ‘secret sauce’ anymore, given that Cisco ships 2 million devices per year running Erlang. That means a whopping 90% of internet traffic is going through routers and switches controlled by Erlang. But what is perhaps less well known within the tech community is that, away from telecoms, Erlang and other languages running on the BEAM virtual machine are powering some of the world’s biggest and most productive financial services systems. Here are just five to start with.

1. Vocalink

Why Erlang?

For ultra-reliable transaction delivery

Erlang is used for immediate payment switches by many companies facilitating instant bank transfers and bill payment services. Erlang, Mnesia and RabbitMQ are used for Vocalink’s global instant payment systems (IPS) which enables real-time payments in Singapore, Thailand, the US and other countries. The Vocalink IPS is now in use by The Clearing House — the banking association and payments companies owned by the largest commercial banks in the US. It has also been selected as the payment technology for the innovative P27 Scandinavian cross/border payment initiative.

Vocalink has been a driving force around the world in the evolution of real-time payments for over a decade. These real-time solutions sit at the centre of the global payments infrastructure and are designed to ‘run on their own rails’ at a massive scale. Their systems are relied on by financial institutions, corporates and governments to provide high availability and resilient payment solutions.

The ability to be able to securely and reliably handle payments requests at scale is thanks to Erlang, which, being originally designed for telecoms switching, is ideal for systems that need to be always on and always processing.

2. Æternity

Why Erlang and Elixir? For groundbreaking innovation with smart contracts and blockchain

æternity is one of the world’s most innovative open-source blockchains with an ambitious mission to bridge the gap between technology and society. They are using Erlang to scale a distributed peer-to-peer network and make their blockchain more reliable and available.

Founded in 2016 by Yanislav Malahov, known in some circles as “the Godfather of Ethereum”, æternity is designed to deliver productivity, transparent governance, and global scalability. Erlang Solutions were engaged to help them tackle the issues of scalability, which are inherent in modern blockchains.

The platform needed to handle large volumes of information in parallel. They wanted to increase transaction speed and scalability while also ensuring privacy by designing a unique state channel to enable the off-chain verification of data and smart contracts, allowing for all transactions to be independent of each other. The Erlang-based smart contract platform features oracles, a unique governance system, and a functional smart contract language called Sophia.

3. Solaris Bank

Why Erlang and Elixir? To be disruptive via fast and agile development

Solarisbank is a Berlin-based 400-person FinTech company making waves at a pace rarely seen before in the financial industry. Their Banking-as-a-Service platform enables businesses to offer their own financial products via Elixir powered APIs. It took Solarisbank less than three years to build the platform, scale up a team, and raise almost €100m in funding with investments.

Their full banking license enables companies to offer their own financial products. Partners can access Solarisbank modules in the fields of e-money, instant credit, lending, and digital banking as well as services from third-party providers integrated with the platform via an API.

Sign up for our FinTech newsletter >

4. Goldman Sachs

Why Erlang and RabbitMQ? To create system reliability for application innovation

The Erlang programming language is being used by Goldman Sachs in part of its hedge fund trading platform. Erlang is used as part of the real-time monitoring solution for this distributed trading system enabling changes to be made rapidly in response to market conditions.

Goldman Sachs leverages hundreds of applications communicating with each other. The Data Management and Distribution group provide messaging middleware services to the firm’s ecosystem. RabbitMQ, which is built in Erlang, is a key part of the messaging portfolio at Goldman Sachs.

RabbitMQ is a free, Open Source message broker that is highly available, fault-tolerant and scalable. As a system middleware layer, it enables different services in your application to communicate with each other optimally. The key value of using RabbitMQ at Goldman Sachs has been the freedom it has afforded its developers to innovate with minimal friction. This freedom helps Goldman Sachs to be one of the traditional financial services firms recognised as keeping pace with the most exciting FinTechs. You can learn more about how Goldman Sachs uses RabbitMQ in their talk at RabbitMQ Summit 2018.

5. Sum Up

Why Erlang and Elixir? Because of their 24/7 system availability and fault-tolerance

SumUP is the leading mobile point-of-sale (mPOS) company in Europe, enabling hundreds of thousands of small businesses in countries worldwide to get paid. They used Erlang to build their payment service from scratch.

They picked Erlang as the ‘right tool for the job’ because card processing is an industry requiring backend systems of a certain type, where availability and fault-tolerance is a must, and Erlang boasts a battle-proven track record in delivering this in telecoms and FinTech.

Nowadays, beyond the original hardware, mobile, and web apps, SumUp has also developed a suite of APIs and SDKs for integrating SumUp payment into other apps and services.

SUMMARY

As you can see Erlang, Elixir and other BEAM VM based technologies are being used in production across the FinTech landscape — this includes private and public blockchain, payment and credit card gateways, banking APIs and more traditional infrastructure management. In fact, it is likely you will find an Erlang team in all of the major banks and financial institutions.

It is easy to build prototypes with Erlang which is why it’s a great choice for fast-moving FinTech startups. While the Elixir programming language mixes the best of Erlang in terms of runtime, the concurrency model, and fault-tolerance, with powerful and battle-tested frameworks.

There are many more use cases for Erlang and other BEAM VM technologies from the biggest investment banks in the world to nimble FinTech challengers. Stay tuned to find out more soon.

Sign up for our FinTech newsletter >

Originally published at https://www.erlang-solutions.com on June 24, 2021.

Have you got the message? Chat is a critical feature for almost every business, in virtually every industry. Now, more than ever, digital communication is relied upon to share information and keep our contacts and users in touch. We’ve created bespoke chat applications for use cases as varied as large scale medical or digital health providers, industry-leading financial service providers and modern dating apps. For business-to-consumer uses, chat is a great way to turn your app or business into a community, keeping users engaged and adding a social element to your applications. On the other hand, in the B2B space, chat applications can be used to increase collaboration and productivity. In fact, external research conducted by one of our clients TeleWare found that instant messaging was the most in demand feature for a financial service app.

In this blog, we’ll look at some of the key considerations for an Instant Messaging service as well as the must-have features of the modern chat application and how MongooseIM 4.0 stacks up to deliver what you need.

Build vs buy

One of the first considerations a company needs to make when implementing a chat offering is whether to use an out-of-the-box product-as-a-service or software-as-a-service offering or build your own chat. Below we weigh up the pros and cons of each approach.

Benefits of buying

The key benefits of an out-the-box purchase solution is that you are able to deploy quickly. The bigger players in this space often offer a comprehensive set of integrations and require little to no development from your team. They also provide users with a familiar user-interface, which means they’re incredibly quick for anyone to learn how to use. All of this means you can be up-and-running quickly with the peace-of-mind that you’re using a tried and tested solution.

Cons of buying

Both product-as-a-service and software-as-a-service options create the ongoing overhead of a subscription fee by their very nature. Over time, this cost inevitably adds up, making it a more expensive offering. Another drawback is that bought options are designed as one-size-fits-all products and seldom offer flexibility for bespoke features and changes. These options offer next to no control and data ownership is often shared. This makes it hard for your users to control their privacy and hard for your chat solution to meet any needs other than the most vanilla offering.The customer service and support can also be variable. All of this creates a huge potential for complication if something stops functioning in what is essentially a blackbox solution.

Benefits of building

Building provides you with the flexibility to create a specific chat solution for your needs and own every step in the functionality. In theory, building can be more affordable over the long-term as it reduces the ongoing costs of a software-as-a-service offering. An owned solution also minimises the risk of major changes in your chat application no longer being compitable with the rest of your application.

Cons of building

When building goes wrong, it is the most costly option, with high upfront and ongoing maintenance costs. Building your own chat application can run into difficulties when the app starts to scale (which is exactly when you want them least). Lastly, building something bespoke means there is no support or community to help you troubleshoot.

The MongooseIM way

MongooseIM is a massively scalable, battle tested, open-source chat server that has been proven to be able to handle 10’s of millions of connections with ease. With MongooseIM you have the freedom and flexibility to use the open-source product and develop it to your needs, or you can engage our experts to build bespoke features for you. Our team also offers the peace-of-mind of support should you ever need it. This gives you the freedom and flexibility to develop and own your chat solution without the cost or risk of starting from scratch.

With over a decade’s experience in building chat applications, we know the features required to ensure a success, taking everyone from the end-user to the DevOps team into consideration. Below is a list of the most used and desired features and how MongooseIM stacks up.

Real-time chat

It goes without saying that a chat application should allow users to reliably send and receive messages in real-time. MongooseIM’s scalability ensures that no matter what the spikes or loads of your user-base is, no important message will be lost in transit.

Push notifications

Push notifications are one of the most valuable parts of a modern chat application. Even if your user is not logged into the application, they’ll still be informed of the message. For B2C applications, that increases the chances of bringing them back to your app and for B2B applications, it ensures no important message goes missed, without requiring your team to be logged into a chat application at all times. MongooseIM has an in-house developed push notification management system, MongoosePush, which is designed to integrate with MongooseIM to easily enable push notifications for your chat app.

External integrations

MongooseIM rarely works alone, usually it is coupled with other microservices. We offer a rest API that these services can call, and an event pusher for MongooseIM to notify them, thus providing a two-way communication with other microservices over the REST API.

API

An easy to use API makes your chat application faster and easier to embed and integrate into your chat. We offer a REST API, which is simple, modern and easily understood by most developers. This can be used for both backend integration and client / service development.

Multi-user Chat

Group chat is one of the most popular features in social settings, and one of the most in-demand features for business collaboration. MongooseIM offers a multi-user chat functionality that is reliable and seamless for users whilst minimising demands on your server. We also provide a light-weight implementation of multi-user chat, tailored for mobile devices.

File Transfer and sharing

For a majority of use cases, allowing users to share and transfer files makes a chat more usable, keeping them engaged on your platform longer. MongooseIM uses an out-of-band transfer method which reduces the workload on the server side whilst still enabling an easier to use experience for users to share files within the chat application.

Batch permission

Batch permissions allow for privacy and control of access to information. MongooseIM uses access control lists to offer this functionality. Our chat applications have been approved by regulatory bodies in the health care and financial services worldwide.

As an application built in XMPP, MongooseIM uses the tried and tested mod_roster functionality to allow for users to manage and customise their address books within the chat application.

History and version control

If something goes wrong, history and version control is vital. Having access to previous versions means you always have a proven version to fall back on. MongooseIM has a public history of its source code which you have access to at all times.

Contact sharing from within a chat application encourages connections between groups of users, helps to grow user bases and increase collaboration.

MongooseIM 4.0 has just been released. In this release, we’ve gone a step further to ensure an easy to use product for developers, users and a DevOps alike. Explore the changes on GitHub

If you need help with the perfect chat solution for your needs, talk to our team of scalability experts. We’re always happy to help.

You may also like:

Go back to the blog

Originally published at https://www.erlang-solutions.com on October 13, 2020.

The complete guide to Instant Messaging and in-application chat. was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

Code BEAM V

10–11 of September. This is the speaking line up that was originally organised for Code BEAM STO, now delivered online. Speakers include Viktória Fördős (Cisco NSO Core), Christopher Price (Ericsson Head of Software Technology), Robert Virding (Co-Creator of Erlang & Erlang Solutions, Senior Systems Architect) just to name a few. Early Bird tickets are available but closing soon. See the full speaking line up and buy tickets here.

ElixirConf EU Virtual

7–8 October. Our first ElixirConf EU Virtual in June had over 400 attendees, and the feedback was overwhelmingly positive. The second edition of ElixirConf EU Virtual will feature the same brilliant line-up of speakers that we had organised for this year’s physical event — only virtually. We’ve organised one of the most exciting events we’ve ever had with talks from Chris McCord, Linda Achieng’ Otieno, Sigu Magwa, Marlus Saraiva and heaps more. See the full speaking line up and buy tickets here.

Code MESH V

Code Mesh is London’s leading alternative tech conference, designed to be a melting pot for curious, inquisitive innovators. The conference connects programmers who believe in finding best-of-class solutions to any problem. Going virtual means their cross-language community goes global, allowing Code Mesh V to connect and inspire even more people. Very Early Bird tickets sell out within 24 hours every year, but you can join the waiting list here.

If you’d like to be part of the first ever global Code Mesh V event and inspire the cross-language community of ambitious programmers the call-for-talks and call-for-tutorials are now open.

Over the last ten years, we have helped A LOT of companies add value to their products using MongooseIM or XMPP based chat applications. This has allowed us to partner with, and get insights from companies spanning almost every industry and company size. You may wonder how much variety and complexity a team who specialises in scalable Instant Messaging is exposed to, but the truth is, the only thing that ties these projects together is MongooseIM, our scalable, customisable XMPP server. Once we dive deeper into the needs of our clients, we usually find deployments need new features, tailored to their specific need, to help them achieve their most critical business objectives.

Customisation is what sets the work we do apart. We make sure that your Instant Messaging is fit for purpose. Even when XMPP or MongooseIM offer a rich enough chat experience, there are other details we mare are right to deliver a reliable solution. This includes taking into consideration your workload and usage scenarios to ensure that you have the scalability to guarantee uptime. If you’d like a general introduction to our approach to building a chat application, watch our recent webinar.

How can we create or improve a chat experience for your business and its customers?

Building and maintaining a chat application is a continuous process. We join our customers at many different stages, and depending on their needs, we can help in different ways. Let’s take a closer look at some examples.

Building a bespoke solution.

We can become an embedded part of the team that builds the service. This allows us to architect and build the solution together from scratch. From the very first day, we learn about the unique properties of the customer and how instant messaging helps them to achieve their goals. As a result, we can use our expertise to make sure the system is built perfectly, from the start, before any less-than-ideal decisions are made. An added benefit of this for our customers is that their team gets to see how we go about architecting and designing a best-practice system. That way, there is a natural knowledge sharing of every step in the process, so when the product is launched and in-use, your team is familiar with how to operate and maintain your system. When possible, this is the perfect option for both the organisations we work with and us.

Improving and optimising before release.

We can’t always be there from the start. Sometimes, there is a proof of concept application ready which needs improving, often because there is a technical issue that needs to be solved before the release. At this stage, it’s still possible to help design the architecture of the entire chat system. We can also deploy it to the production environment and make sure it can cope with the increasing load introduced by new users. There are a number of common problems we help solve for companies looking to optimise their chat before release.

One of the most common reasons companies need help at this stage is that developers who have less experience with XMPP have implemented inadequate solutions which have already been solved by the existing, public XEP — XMPP extension.
We also often see problems arising when a custom extension is built on top of an existing one, which, when done incorrectly, can cause significant complications. In the lead up to the release of a chat solution companies often need help with custom integrations.
This is especially true for companies adding real time chat to an existing product. In many cases, this can be done outside of the MongooseIM code, on the other hand, it’s usually more scalable and efficient if it can be run within MongooseIM. Not every company has the need to hire full-time Erlang developers, especially for implementing a few integrations. Our team are experienced, ready and happy to help with custom integrations to ensure the reliability and scalability of the release.

To ensure success when working with an existing product that has yet to be released, we often need to take a step back and look at the product, its goals and its failures holistically. From there we can suggest the best solution. This may mean we need to rethink the existing implementation and change or reshuffle some of the code. In extreme cases, we may need to throw away large chunks of the existing solution to ensure the release is successful. This is the last resort and a decision that is taken collaboratively in the name of reaching an optimal solution.

Increasing the scalability of your chat to allow you to handle more users.

Your instant messaging is deployed and running. Your product is growing successfully. It should be a time to celebrate, but often increased adoption or use of a chat application comes with increased scalability issues. These need to be sorted fast and future-proofed to avoid giving your users a poor experience and damaging the growth you’ve worked hard to achieve.

Our role is to find the bottlenecks and fix them. We study the architecture, the server setup, configuration, enabled extensions and customised code. Then, we decide on the best possible solution together, in collaboration with our clients.

Improving or customising your chat application.

All right, the app is deployed, and users are chatting, so far so good. How can joining forces with us make things even better? There are many reasons companies will come to us to improve an existing instant messaging solution.

Many of our customers come to us to develop unique functionalities that are necessary for their success. We help by designing the extension on the protocol level and implementing it on the server. The public XEPs only cover use cases which are generic and where other people deploying chat services based on XMPP can also use them. So, if your chat needs a specific functionality, we’re always happy to help.

We also have projects where MongooseIM replaces another XMPP server. This is common when clients discover that the MongooseIM alternative technology they have chosen did not meet their expectations. A common reason for this is that out of the box functionalities can turn out to be black boxes that make it impossible to carry out necessary customisations or improvements.

Another common reason to switch to MongooseIM is to improve scalability when an existing solution reaches its capacity. In this case, we play the role of surgeons, carefully implementing a transplant to put MongooseIM at the heart, improving scalability but keeping the rest of the system running smoothly.

Tips to avoid common mistakes when using open-source MongooseIM:

Choose your XEP, and choose wisely.

X in XMPP stands for eXtensible. This means it might not be the simplest protocol, but there’s hardly a chat feature it doesn’t cover. There is a set of core RFCs on top of which custom extensions (XEPs) are built. There are usually no issues with the core functionality that is covered in RFCs. They are adopted by many client libraries and servers. XEPs are often isolated from each other, independent from the core functionalities. They add extra features or capabilities to the core of XMPP. Here comes the biggest challenge: deciding which XEP to choose (you can find more details in our XMPP use-cases guide). From our experience, developers starting their adventure with XMPP may have a hard time finding a suitable XEP. This can sometimes accidentally lead to your choice creating difficulties or limitations. Some examples:

Too many extensions are enabled on the server-side, but not used by the client app.
Certain XEPs like XEP-0012: Last Activity put extra load on the server, even when the client’s application is not using them. Enabling only used extensions helps to scale the MongooseIM cluster.

An outdated or rejected extension is used.
Often, different products with instant messaging functionality store the messages on the server. For this, there is extension XEP-0313: Message Archive Management. The majority of servers and client libraries have implemented the XEP for several years already. It replaces an old, more complicated and now deprecated XEP-0136: Message Archiving. Sometimes, developers choose the deprecated extension over the new one.

A custom extension is built even if an existing XEP covers the required functionality.
For instance, if a user wants to know if someone is active in a chat conversation. To achieve that, some extra XMPP stanzas need to be sent. Developers, me included, are creative creatures, and sometimes we rush to reinvent the wheel. In this particular case, XEP-0085: Chat State Notifications fulfils this need.

Reduce unknowns

When preparing for a wave of users using instant messaging functionality, we need to remember to load test. It’s very crucial to run load tests to know the capacity of your production setup. To my surprise, this is often neglected. Many people want an answer to the following simple question: “how many resources do I need to handle X number of users, sending Y messages per second”. There are too many factors and variables to accurately answer this question. To have a better idea about the capacity, you can run load tests simulating user behaviours. It usually starts with developing the scenario. After making sure that monitoring is in place, the load testing can start. Now we can learn how the system behaves and what to expect in the production environment.

Summing it up

It is very important:

• to know what functionalities are needed and enabled on the server
• to verify if a custom extension is already covered in a XEP
• to only build a custom extension if necessary
• to run load tests to find the capacity and be better prepared for the real traffic

All of the above may not be rocket science; many services run on MongooseIM without our help.
If you are up to the challenge, good, we are always happy to see our product in use.
But, when your time is critical, or if you stumble upon a problem you can’t fix, we’re happy to help. If hacking through 80K lines of unknown code is not your cup of tea, we can guide you through it. Stay safe and have fun adding real-time communication to your product, if required.

Originally published at https://www.erlang-solutions.com on June 26, 2020.

Add value to your app with Instant Messaging. Here’s where to start. was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

What should you look for when choosing to learn a new programming language? The answer might vary depending on what your project or career goals are, but as a basic starting point, you want a language that:

1. Is fun and easy to use
2. Has the ability to meet modern user demands
3. Has rewarding career progression
4. Has an active and supportive community
5. Has a range of helpful tooling
6. Has frameworks to allow full-stack development
7. Has easily accessible documentation
8. Ensures you grow as a programmer.

Over the course of the article, we’ll show you how Elixir rates in relation to all of the above dot points. Before we start, it’s worth letting you know a little bit about me. I like to consider myself a polyglot developer. Over the years I have worked with Elm, Lua, Rust, Dart, Go, Kotlin, Scala, C, C++, Perl, Ruby, PHP, Python, Java, JavaScript, Erlang and Elixir. If I were to only pick one language to learn as a brand new developer in 2020, it would be Elixir.

Why Elixir is fun and easy to use

Fun might not be the only important consideration, but it is one that should not be underestimated. Elixir’s syntax shares a lot of similarities with Ruby; both are high-level, readable, fun programming languages. As a senior developer from Pinterest once said of his decision to transition to Elixir:
“”I thought, ‘Wow, it’s as fun as Ruby, but it has some chops — it actually performs really well.’”
Personally, I love the ability to write less code. It is a language that is expressive enough to be understandable and fast to read. For example, using the Tesla library, I could implement integration with a provider like Paypal in 20 lines of code. I will be happy to schedule a hands-on workshop or a 1-on-1 call if there is a need for dive into this library.

How Elixir has the ability to meet modern usage demands

We all heard of the digital transformation, right? As populations grow, and more services move online there is increased pressure on systems to be able to handle huge spikes with billions of concurrent users.
Elixir runs on the BEAM VM. A system built for concurrency. As a result, it can handle these user spikes with ease. For those interested in what concurrency is, and how it works, we have a blog comparing the JVM and BEAM VM, which explains it well.
Pinterest and Bleacher Report are just two of widely popular and big companies who moved to Elixir in order to help solve issues of scalability. In the case of Bleacher Report, one of our clients, they were able to move from 150 servers, down to 5 servers. Not only did they reduce there physical infrastructure requirements, they were also able to handle more traffic with faster times. A language that can be relied upon for scalable, fault-tolerant will always be sought out by high profile companies.

Elixir’s rewarding career progression

Elixir is a growing language which is being adopted at a significant rate. As a result, Elixir developers have the opportunity to work for wide spectrum of companies such as PepsiCo, Pinterest, Lonely Planet or MBTA. And the list is continuing to grow. Check out this list for companies added in the last 12 months alone.

The supportive and active Elixir community

As you might expect with any new technology or programming language, the Elixir community started out small.
Nowadays Elixir has a thriving community that continues to grow, with fantastic events such as ElixirConf in Europe and in the US, EMPEX, Code Elixir LDN and Gig City Elixir, as well as Meetups all around the world. This community means the language is continuing to grow and evolve with new tooling, and that there is always someone to provide inspiration or help find solutions to a problem.

Elixir’s range of useful tooling

Tooling makes languages more versatile and tasks easier. They mean that you don’t have to reinvent the wheel with every new use case. Elixir already has a number of powerful tools, including Phoenix LiveView which enables developers to build real-time, front-end web applications without writing a line of JavaScript or Crawly which makes web crawling and Data Scraping in Elixir easy. Not only does Elixir have a wide-variety of tooling available, but as we mentioned earlier, there is also a passionate community of people continually creating new tools and improving the ones available.

Elixir frameworks allow for full-stack development

Given its scalability performance and its origins in Erlang, it is no surprise that Elixir is a popular backend choice. As mentioned above, Phoenix LiveView has provided an easy, time-efficient and elegant way for Elixir developers to produce front-end applications. Finally, the Nerves framework allows for embedded software development on the hardware end. As a result, Elixir is a language that can be adopted right throughout the tech stack. This doesn’t just make it an attractive choice for businesses; it also opens up the door for where the language can take you as a developer.

Elixir’s easily accessible documentation

A language and community where documentation is valued is one that makes knowledge sharing easy. The high standard of documentation in the Elixir community makes it easier for you to learn the language, but also, make it easier for anyone to contribute to its improvement and growth.

Learning Elixir can make you a better programmer in other languages

There are a number of stories, from people who come from object-oriented languages, who found that the process of learning Elixir made them a better programmer in their language of choice. Learning a new purely functional programming paradigm forces you to examine the way you approach programming. In doing so, it can help you identify habits that can be improved and introduce you to new ways of thinking about problems. This fresh perspective is invaluable for all future development you do, regardless of what language you are working with. For our friends using Ruby, this is particularly true, as the Elixir syntax will be familiar to you, allowing you an easy transition into functional, concurrent programming.

Everyone will have different motivation and considerations for what programming language to choose. But these are the reasons I would recommend you learn Elixir in 2020, for the years ahead.

Ready to get started in Elixir?

There are many ways to start. First, you should head to the getting started page of the official Elixir website. We also offer a number of free downloadable packages for Elixir.
Secondly, we offer a variety of Elixir training, including our Elixir Programming Training For Beginners.
To get started in the community ElixirForum is a great place to start, as well as searching the #Elixirlang and #MyElixirStatus hashtags on Twitter.

You may also like:

ElixirConf EU Virtual

Go back to the blog

Originally published at https://www.erlang-solutions.com on June 3, 2020.

Why Elixir is the Programming Language You Should Learn in 2020 was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

The success of any programming language in the Erlang ecosystem can be apportioned into three tightly coupled components. They are:
1) the semantics of the Erlang programming language, on top of which other languages are implemented
2) the OTP libraries and middleware used to architect scalable and resilient concurrent systems and
3) the BEAM Virtual Machine tightly coupled to the language semantics and OTP.

Take any one of these components on their own, and you have a runner up. But, put the three together, and you have the uncontested winner for scalable, resilient soft-real time systems. To quote Joe Armstrong, “You can copy the Erlang libraries, but if it does not run on BEAM, you can’t emulate the semantics”. This gets enforced by Robert Virding’s First Rule of Programming, which states that “Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang.”

In this post, we want to explore the BEAM VM internals. We will compare and contrast them with the JVM where applicable, highlighting why you should pay attention to them and care. For too long, this component has been treated as a black box and taken for granted, without understanding the reasons or implications. It is time to change that!

Highlights of the BEAM

Erlang and the BEAM VM were invented to be the right tool to solve a specific problem. They were developed by Ericsson to help implement telecom infrastructure handling both mobile and fixed networks. This infrastructure is highly concurrent and scalable in nature. It has to display soft real-time properties and may never fail. We don’t want our Hangouts calls on our mobile with our grandmothers dropped or our online gaming experience of Fortnite to be affected by system upgrades, high user load or software, hardware and network outages. The BEAM VM is optimised to solve many of these challenges by providing fine-tuned features which work on top of a predictable concurrent programming model.

Its secret sauce are light-weight processes which don’t share memory, managed by schedulers which can manage millions of them across multiple cores. It uses a garbage collector which runs on a per-process basis, highly optimized to reduce any impact on other processes. As a result, the garbage collectors do not impact the overall soft real time properties of the system. The BEAM is also the only widely used VM used at scale with a built-in distribution model which allows a program to run on multiple machines transparently.

Highlights of the JVM

The Java Virtual Machine (JVM) was developed by Sun Microsystem with the intent to provide a platform for ‘write once’ code that runs everywhere. They created an object oriented language similar to C++, but memory-safe because its runtime error detection checks array bounds and pointer dereferences. The JVM ecosystem became extremely popular with the Internet-era, making it the de-facto standard for enterprise server applications. The wide range of applicability was enabled by a virtual machine that caters for a wide range use cases, and an impressive set of libraries catering for enterprise development.

The JVM was designed with efficiency in mind. Most of its concepts are an abstraction of features found in popular operating systems such as the threading model which maps to operating system threads. The JVM is highly customisable, including the garbage collector (GC) and class loaders. Some state-of-the-art GC implementations provide highly tunable features catering for a programming model based on shared memory. The JVM allows you to change the code while the program is running. And, a JIT compiler allows byte code to be compiled to the native machine code with an intent to speed up parts of the application.

Concurrency in the Java world is mostly concerned with running applications in parallel threads, ensuring they are fast. Programming with concurrency primitives is a difficult task because of the challenges created by its shared memory model. To overcome these difficulties, there are attempts to simplify and unify the concurrent programming models, with the most successful attempt being the Akka framework.

Concurrency and Parallelism

We talk about parallel code execution if parts of the code are run at the same time on multiple cores, processors or computers, while concurrent programming refers to handling events arriving to the system independently. Concurrent execution can be simulated on single threaded hardware, while parallel execution cannot. Although this distinction may seem pedantic, the difference results in some very different problems to solve. Think of many cooks making a plate of Carbonara pasta. In the parallel approach, the tasks are split across the number of cooks available, and a single portion would be completed as quickly as it took these cooks to complete their specific tasks. In a concurrent world, you would get a portion for every cook, where each cook does all of the tasks. You use parallelism for speed and concurrency for scale.

Parallel execution tries to solve an optimal decomposition of the problem to parts that are independent of each other. Boil the water, get the pasta, mix the egg, fry the guanciale ham, grate the pecorino cheese. The shared data (or in our example, the serving dish) is handled by locks, mutexes and various other techniques to guarantee correctness. Another way to look at this is that the data (or ingredients) are present, and we want to utilise as many parallel CPU resources as possible to finish the job as quickly as possible.

Concurrent programming, on the other hand, deals with many events that arrive at the system at different times and tries to process all of them within a reasonable time. On multi-core or distributed architectures, some of the execution is run parallel, but this is not a requirement. Another way to look at it is that the same cook boils the water, gets the pasta, mixes the eggs and so on, following a sequential algorithm which is always the same. What changes across processes (or cooks) is the data (or ingredients) to work on, which exist in multiple instances.

The JVM is built for parallelism, the BEAM for concurrency. They are two intrinsically different problems, requiring different solutions.

The BEAM and Concurrency

The BEAM provides light-weight processes to give context to the running code. These processes, also called actors, don’t share memory, but communicate through message passing, copying data from one process to another. Message passing is a feature that the virtual machine implements through mailboxes owned by individual processes. The message passing is a non-blocking operation, which means that sending a message to another process is almost instantaneous and the execution of the sender is not blocked. The messages sent are in the form of immutable data, copied from the stack of the sending process to the mailbox of the receiving one. This is achieved without the need for locks and mutexes among the processes, only a lock on the mailbox in case multiple processes send a message to the same recipient in parallel.

The immutable data and the message passing enable the programmer to write processes which work independently of each other and focus on the functionality instead of the low-level management of the memory and scheduling of tasks. It turns out that this simple design not only works on a single thread, but also on multiple threads on a local machine running in the same VM and, using the built in distribution, across the network with a cluster of VMs and machines. If the messages are immutable between processes, they can be sent to another thread (or machine) without locks, scaling almost linearly on distributed, multi-core architectures. The processes are addressed in the same way on a local VM as in a cluster of VMs, message sending works transparently regardless of the location of the receiving process.

Processes do not share memory, allowing you to replicate your data for resilience and distribute it for scale. This means having two instances of the same process on two separate machines, sharing state updates among each other. If a machine fails, the other has a copy of the data and can continue handling the request, making the system fault tolerant. If both machines are operational, both processes can handle requests, giving you scalability. The BEAM provides the highly optimised primitives for all of this to work seamlessly, while OTP (the “standard library”) provides the higher level constructs to make the life of the programmers easy.

Akka does a great job at replicating the higher level constructs, but is somewhat limited by the lack of primitives provided by the JVM, allowing it to be highly optimised for concurrency. While the primitives of the JVM enable a wider range of use cases, they make programming distributed systems harder as they have no built in primitives for communication and are often based on a shared memory model. For example, where in a distributed system do you place your shared memory? And what is the cost of accessing it?

Scheduler

We mentioned that one of the strongest features of the BEAM is the ability to break a program into small, light-weight processes. Managing these processes is the task of the scheduler. Unlike the JVM, which maps its threads to OS threads and lets the operating system schedule them, the BEAM comes with its own scheduler.

The scheduler starts, by default, an OS thread for every core and optimises the workload between them. Each process consists of code to be executed and a state which changes over time. The scheduler picks the first process in the run queue that is ready to run, gives it a certain amount of reductions to execute, where each reduction is the rough equivalent of a command. Once the process has either run out of reductions, is blocked by I/O, is waiting for a message or completes executing its code, the scheduler picks the next process in the run queue and dispatches it. This scheduling technique is called pre-emptive.

We mentioned the Akka framework many times, as its biggest drawback is the need to annotate the code with scheduling points, as the scheduling is not done at the JVM level. By removing the control from the hands of the programmer, soft real time properties are preserved and guaranteed, as there is no risk of them accidentally causing process starvation.

The processes can be spread around the available scheduler threads and maximise CPU utilisation. There are many ways to tweak the scheduler but it is rare and needed only for edge and borderline cases, as the default options cover most usage patterns.

There is a sensitive topic that frequently pops up regarding schedulers: how to handle Natively Implemented Functions (NIFs). A NIF is a code snippet written in C, compiled as a library and run in the same memory space as the BEAM for speed. The problem with NIFs is that they are not pre-emptive, and can affect the schedulers. In recent BEAM versions, a new feature, dirty schedulers, was added to give better control for NIFs. Dirty schedulers are separate schedulers that run in different threads to minimise the interruption a NIF can cause in a system. The word dirty refers to the nature of the code that is run by these schedulers.

Garbage Collector

Modern, high level programming languages today mostly use a garbage collector for memory management. The BEAM languages are no exception. Trusting the virtual machine to handle the resources and manage the memory is very handy when you want to write high level concurrent code, as it simplifies the task. The underlying implementation of the garbage collector is fairly straightforward and efficient, thanks to the memory model based on immutable state. Data is copied, not mutated and the fact that processes do not share memory removes any process interdependencies, which, as a result, do not need to be managed.

Another feature of the BEAM is that garbage collection is run only when needed, on a per process basis, without affecting other processes waiting in the run queue. As a result, the garbage collection in Erlang does not ‘stop-the-world’. It prevents processing latency spikes, because the VM is never stopped as a whole — only specific processes are, and never all of them at the same time. In practice, it is just part of what a process does and treated as another reduction. The garbage collector collecting process suspends the process for a very short interval, often microseconds. As a result, there will be many small bursts, triggered only when the process needs more memory. A single process usually doesn’t allocate large amounts of memory, and is often short lived, further reducing the impact by immediately freeing up all its allocated memory on termination. A feature of the JVM is the ability to swap garbage collectors, so by using a commercial GC, it is also possible to achieve non-stopping GC in the JVM.

The features of the garbage collector are discussed in an excellent blog post by Lukas Larsson. There are many intricate details, but it is optimised to handle immutable data in an efficient way, dividing the data between the stack and the heap for each process. The best approach is to do the majority of the work in short lived processes.

A question that often comes up on this topic is how much memory the BEAM uses. Under the hood the VM allocates big chunks of memory and uses custom allocators to store the data efficiently and minimise the overhead of system calls. This has two visible effects:
1) The used memory decreases gradually after the space is not needed
2) Reallocating huge amounts of data might mean doubling the current working memory.

The first effect can, if really necessary, be mitigated by tweaking the allocator strategies. The second one is easy to monitor and plan for if you have visibility of the different types of memory usage. (One such monitoring tool that provides system metrics out of the box is WombatOAM).

Hot code loading is probably the most cited unique feature of the BEAM. Hot code loading means that the application logic can be updated by changing the runnable code in the system whilst retaining the internal process state. This is achieved by replacing the loaded BEAM files and instructing the VM to replace the references of the code in the running processes.

It is a crucial feature for no downtime code upgrades for telecom infrastructure, where redundant hardware was put to use to handle spikes. Nowadays, in the era of containerization, other techniques are also used for production updates. Those who have never used it dismiss it as a less important feature, but it is none-the-less useful in the development workflow. Developers can iterate faster by replacing part of their code without having to restart the system to test it. Even if the application is not designed to be upgradable in production, this can reduce the time needed for recompilation and redeployments.

When not to use the BEAM

It is very much about the right tool for the job. You need a system to be extremely fast, but are not concerned about concurrency? Handling a few events in parallel, and having to handle them fast? Need to crunch numbers for Graphics, AI or analytics? Go down the C++, Python or Java route. Telecom infrastructure does not need fast operations on floats, so speed was never a priority. Aided with dynamic typing, which has to do all type checks at runtime means compiler time optimizations are not as trivial. So number crunching is best left to the JVM, Go or other languages which compile to native. It is no surprise that floating point operations on Erjang, the version of Erlang running on the JVM, was 5000% faster than the BEAM. But where we’ve seen the BEAM shine is using its concurrency to orchestrate number crunching, outsourcing the analytics to C, Julia, Python or Rust. You do the map outside the BEAM and the reduce within the BEAM.

The mantra has always been fast enough. It takes a few hundred milliseconds for humans to perceive a stimulus (an event) and process it in their brain, meaning that micro or nano second response time is not necessary for many applications. Nor would you use the BEAM for microcontrollers, it is too resource hungry. But for embedded systems with a bit more processing power, where multi-core is becoming the norm, you need concurrency, and the BEAM shines. Back in the 90s, we were implementing telephony switches handling tens of thousands of subscribers running in embedded boards with 16mb of memory. How much memory does a RaspberryPi have these days? And finally, hard real time systems. You would probably not want the BEAM to manage your airbag control system. You need hard guarantees, something only a hard real time OS and a language with no garbage collection or exceptions. An implementation of an Erlang VM running on the bare metal such as GRiSP will give you similar guarantees.

Conclusion

Use the right tool for the job. If you are writing a soft real time system which has to scale out of the box and never fail, and do so without the hassle of having to reinvent the wheel, the BEAM is the battle proven technology you are looking for. For many, it works as a black box. Not knowing how it works would be analogous to driving a Ferrari and not being capable of achieving optimal performance or not understanding what part of the motor that strange sound is coming from. This is why you should learn more about the BEAM, understand its internals and be ready to finetune and fix it. For those who have used Erlang and Elixir in anger, we have launched a one day instructor-led course which will demystify and explain a lot of what you saw whilst preparing you to handle massive concurrency at scale. The course is available through our new instructor lead remote training, learn more here. We also recommend The BEAM book by Erik Stenman and the BEAM Wisdoms, a collection of articles by Dmytro Lytovchenko.

You may also like:

1.One of the authors also likes a similar looking dish not called carbonara but made with cream..↩

Go back to the blog

Originally published at https://www.erlang-solutions.com on May 11, 2020.

Throughout my professional career, I’ve had the chance to work with a few programming languages from different paradigms. Some of them were a joy to use, others, were bloated with features that could make even the simplest code unreadable. A little over two years ago I decided to give Erlang a try. Soon after, I started working at Erlang Solutions, and after a couple of months of intensive coding in Erlang I was introduced to Elixir. I’ve been working as an Elixir developer since then.

Over the last fourteen months I have been working with a client on mobility applications. We developed several parts of the backend of a corporate car sharing platform and built an application from scratch to conduct and analyze test drives at dealerships around the World. Development of the former had already started before we got there, so the tech stack was pretty much decided. We developed a handful of services exposing GraphQL APIs. For a number of reasons Elixir was the right fit for this purpose. Using the Absinthe library to craft the APIs was another good choice by their team. However, multiple services had to communicate with each other and making a responsive UI requires optimized queries whenever possible. We learned a lot from the experience which we will share for you today in this blog.

When we started working on the second application, we had more flexibility in choosing the tools. We decided to stick with a GraphQL API and used Absinthe, but paid attention to how we write the resolver functions. In the end, our API was blazing fast, because most of our resolvers ran only a handful of queries.

Designing a usable API is art, as is optimizing the backend that serves that API. In this blog post — and hopefully some upcoming ones — I’ll give you a few hints on what to avoid or what to strive for when working with Absinthe. We cannot always affect how other applications/services work, but we can do our best to make our service as fast as possible.

There are plenty of resources out there to help you get started with Absinthe, and the number of tutorials on how to use Dataloader are growing, but it is a recurring topic on Slack. I hope to give you an insight into how it works and how you can make use of it in your project.

What is Dataloader in a nutshell?

In short, Dataloader is a tool that can help your application fetch data in an optimal way by implementing batching and caching.

Dataloader in Elixir

In the Elixir world, it is provided by a library and integrates well with Ecto and Absinthe. Note that it’s not a silver bullet, your queries using Dataloader rely on the optimal implementation of the underlying library for the specific data source.

We are not going into details on how to use Dataloader on its own. Please consult the official documentation for that.

Instead, let’s dive into using Dataloader in Absinthe.

Using Dataloader in Absinthe Resolvers

For these examples, we are going to use rather simple schemas to avoid getting lost in the details. Assume we are building a database where we keep track of companies and their employees. One employee belongs to exactly one company, but each company may have multiple employees. The Ecto schemas may be defined as:

And the corresponding object definitions in Absinthe GraphQL may look like:

This is all good until we want to resolve the employees in the company. A naïve field definition of it may be:

Similarly, we can add the field :company to the employee object as:

If we now query a company through GraphQL and also ask for the employees on that field, then our backend will perform two SQL queries. This is not bad at all, but imagine a case where we have ten employee results. Moreover, each result asks for its own company field, which causes several duplicate queries to Ecto.

query($id: ID!) { company(id: $id) { id name employees { id name company { id name } } } }

This may not happen in this exact form, but it helps us imagine what happens when the same associated object has to be resolved for a list of results. We have to make several queries to Ecto for this to be answered. One query to resolve the company in the root, one query for each employee and one additional query for resolving the company of each employee. Twenty-one queries overall; we’re facing the infamous n+1 problem. This is where Dataloader comes to the rescue.

Adding Dataloader

The documentation of Absinthe is a good starting point for using Dataloader with an Ecto data source. In short, if we want to use Dataloader in our resolvers we have to do two things:

• Add a dataloader struct to the resolution context
• Add Absinthe.Middleware.Dataloader to the list of plugins in our schema

Let’s not forget to add the Dataloader plugin:

With these modifications, resolving the graphql query above requires only three Ecto queries. That is a significant improvement.

The query function

One of the useful features of the Dataloader.Ecto source is that we can pass a query function to it which can be used for filtering or processing parameters that are common to many fields, for example, pagination arguments.

Where we can define Repo.dataloader_query/2 to process parameters related to pagination and also leave room for extending it easily.

Note that so far we haven’t needed to write any Ecto queries, because we used the dataloader/1 helper from Absinthe.Resolution.Helpers.

We only see employees for one of the companies, while the list of employees for the other one is empty. This happens because the Ecto Dataloader tries to fetch the employees for both companies with a single query that includes an order_by on the company_id field. This works well when we don't want to add any limit or offset parameters. One workaround for this is to modify the argument list, which will force the loader to make separate queries for each company.

The query function is also useful if we want to have more flexibility, for example, filtering or ordering results. For this, we’ll have to extend our query function to process extra parameters. In general, I like to write query helpers that take a queryable object as their first parameter and return a queryable. For example (assuming we have a status field on the employee):

If we extend the query function ( Repo.dataloader_query/2) as below, we will be able to use these helpers easily:

And we can now resolve only active employees on companies if we specify query in the resolver:

Of course, other filtering is also possible, and we can also construct our queries based on the GraphQL parameters that the resolver receives in args.

Note that now :query becomes part of the key that is used in the dataloader cache, so querying the active and non-active employees of the same company in one GraphQL query might require two database queries.

Dataloader.KV

So far, we have only seen examples of how to use Dataloader.Ecto. But what if we need to collect data from another service to respond to the GraphQL query? We can use Dataloader.KV to retrieve and cache data based on keys.

For this, we will need a function that receives two parameters. The first parameter is a batch key that groups together different objects. We will return to this shortly. The second parameter is a list, usually a list of objects or IDs from which the required data is to be retrieved. The function should return a map where the keys are the elements of this list, and the associated values are the corresponding retrieved data. For example, if we store the addresses of employees in a different service, we may write a loader function as follows:

This function receives a list of employees and returns a map with each employee mapped to an address. How the call to another service and the lookup are completed are simply implementation details, but for the solution to be optimal, the other service should support querying data in batches (for all IDs at once instead of one-by-one).

To use this function as a load function for a Dataloader.KV source we may change the context function in the schema as follows.

Then we can resolve the address field for each employee using the :address dataloader source:

As you can see, we did not use the batch key in our loader, which means we will handle all employees in one batch. This is usually fine. The batch key can be useful if we intend to pass on certain arguments to the other service or refine the results. Perhaps we have a user token that we intend to supply for the service to check whether the user has the necessary access rights:

Then make the load function handle the additional arguments:

Note that for each different batch key we have to make a call to the other service, so we have to be careful when specifying the arguments. For example, if we pass in a unique ID (e.g., employee.id), then we lose the advantage of batching, the function is called for each employee.

In general, constructing the batch key provides flexibility, but it can also hide important details in the code. Use with caution.

More control over dataloader

Notice how we took advantage of the fact that we can embed on_load calls to optimize fetching the results. First, we tell dataloader to load the employees, then we use those employees to load their addresses. Finally, we fetch the addresses and count how many unique ones there are.

In general, this kind of resolver is useful when we want to move data one (or more) level up the tree with or without aggregation. In one project, I used the same solution to retrieve telematics data of vehicles to be aggregated and displayed on certain trips taken with those vehicles. Both the vehicles and telematics data needed to be queried from other services.

Conclusion

Dataloader is a powerful tool when it comes to optimizing queries, but we have to be aware of its limitations. We saw a few simple examples to get up and running with Dataloader and Absinthe. This is only the tip of the iceberg, and I am hoping to follow up with some more advanced tricks and tips.

Go back to the blog

Originally published at https://www.erlang-solutions.com on April 1, 2020.

Last year we launched #MyTopDogStatus to share and celebrate some of the fantastic companies using Erlang and Elixir technology. Since then, the Erlang and Elixir community has continued to thrive. Last year, our friends at Code Sync had a wave of brand new people using Erlang & Elixir attending their events, there were representatives of 53 new companies added to the list! And that’s on top of many repeat attendees. The BEAM community now has at least 1,300 companies using Elixir in their tech stack and there are 3 new Elixir jobs advertised on average per week.

The benefits of BEAM technologies remain the same as they were last year, the concurrency model means higher uptime, so less lost sales or angry users when something goes wrong. Both programming languages result in a reduction of code, making it easier to manage, reducing the cost spent on development and making it easier to update. Code reductions also allow companies to reduce the cost and energy demands of their physical servers, which is a benefit to the bottom line, and the planet. Both languages support hot coding so you can have changes made in minutes, not months allowing for a more agile system. Regardless of what benefit is the core motivator, BEAM-based technologies of Erlang and Elixir are almost always the secret sauce in the tech stack of large household name brands who need to handle large volumes of users and data.

So with all those benefits to take advantage of, who were the new top dogs to join the pack and start reaping the benefits of Erlang and Elixir in the last year? Let’s find out.

Samsung

The biggest of the new kids on the block is Samsung, a company which needs no introduction. With nearly 300 million mobile devices shipped and 2 billion dollars in revenue a year. Given the number of users, verticals and traffic Samsung handles, it’s no surprise that they are looking into incorporating BEAM technologies in their tech stack. This year, our very own Robert Virding, one of the co-creators of Erlang joined them in their Silicon Valley office to present a Meetup on the Erlang Ecosystem. Our team will be continuing to work with them on Erlang training and onboarding.

Cross River Bank

Cross River Bank is an innovative and ambitious take on banking. They combine the traditional features of a state-chartered bank with the flexibility and ambition of a FinTech. Their product features an extensive suite of services as a banking-as-a-platform offering, including lending, payments and risk management. Their smart work with technology partners allows them to focus on growing their business without compromising on technological innovation. As you can imagine, downtime in a sector dealing with large financial transactions needs to be minimised, so it’s no surprise to see BEAM-based technologies in their stack.

TubiTV

TubiTV is an entirely free alternative to video streaming services such as Netflix or Amazon Prime. It is America’s largest independently owned streaming services and operates on an ad revenue model as opposed to a user subscription model. There are over 15,000 videos available to stream at any one time. Dealing with that level of data is always going to be a good use case for a language like Elixir, you can learn more about some of their Elixir implementations over at the TubiTV Github page.

Handzap

Handzap is an agile, mobile-first app designed to make the gig economy easy available in 32 languages and over 100 countries. People can post tasks, review applicants and chat using instant messaging, video messaging, and voice calls, all from within the app. This makes it easier and more convenient than ever to post, publish, manage and ensure satisfactory delivery for applicants and job posters alike. Due to its ambitious nature, Handzap had a number of unique requirements for its chat functionality, particularly around user privacy. The MongooseIM team at Erlang Solutions know that no two sets of users are the same, and are specialists in creating customised, reliable instant messaging solutions. We helped them build a future-proof messaging platform that integrated seamlessly with their application whilst making sure all their specific requirements were met.

Danske Bank

Danske Bank is Denmark’s largest bank. A Fortune 500 company with over 145 years of history. It has grown to be one of the biggest banks in Northern Europe and the world. BEAM-technologies are perfectly suited to the high-value transactions of the financial services industry, and RabbitMQ (built in Erlang) specifically is often used in technology stacks to handle microservices.

OpenX

Digital advertising requires huge amounts of real-time decision making and needs to be able to handle significant spikes and loads. OpenX handles over 4.5 trillion data events per day, significantly more than 10 million per second. As we have seen with AdRoll in the past, Erlang’s concurrency is perfect for managing the high volumes of concurrent users required for the industry.

SnatchBot

SnatchBot is a fantastic new platform that aims to make the potential of chatbots available to everyone, including non-technical audiences who can’t write a single line of code. The SnatchBot Builder uses building block templates to allow you to publish your bot to mobile devices, web apps and chat services like Facebook Messenger, Slack and Skype. As we mentioned with Handzap, the MongooseIM team are world-class experts at developing innovative and ambitious instant messaging solutions and were happy to help on such an exciting project.

Siemens

Siemens is the largest industrial manufacturing company in Europe, with nearly half a million employees. They cover a diverse range of industries including Energy, Healthcare, Infrastructure and cities. If you look into any company of this size and scale, you’re likely to find a BEAM-based technology somewhere in their stack, and Siemens is no different.

PepsiCo

We mentioned PepsiCo’s use of Elixir in their tech stack to power business-critical solutions in our article last year. This year, our team will continue to work with them to train, upskill and grow the use of Elixir in their tech stack.

SITA

SITA (Société Internationale de Télécommunications Aéronautiques) a large scale organisation that specialises in telecommunications and IT for the aviation industry. It is a member-owned organisation comprised of over 400 airlines major airlines ensuring the technology and communications of all flights are up to safety standards.

Summary

With all these great companies taking advantage of Erlang, Elixir, RabbitMQ and the BEAM, isn’t it time you looked into how you can get a more reliable, fault-tolerant and dynamic solution. Talk to us; we’re always happy to help.

You may also like:

Which companies are using Erlang?
Which companies are using Elixir?
How to build machine learning in Elixir.
How to manage IoT edge data with Erlang.

Go back to the blog

Originally published at https://www.erlang-solutions.com on March 2, 2020.

MongooseIM offers a completely customisable enterprise Instant Messaging solution. This helps large organisations build solutions that meet the unique requirements of their users. As a result, we’ve built many best-in-class instant messaging platforms for household names in social media, telecoms and the health industry. In our latest release, MongooseIM 3.6.0, we’ve continued to empower our application owners, and their end users by improving and evolving the push notification functionality. Find out more about how MongooseIM can build a bespoke instant messaging solution for your needs, or read on to find out more about the improvements to push notifications.

Creating a better Push Notification Experience in MongooseIM

MongooseIM is continuing its evolution to improve the user experience for mobile applications. Push notifications aren’t just a standard requirement for instant messaging applications; they are also one of the most used features for end-users. As a result, there is an increased need to have a seamless experience for both application owners and the user themselves. Push notifications were added to MongooseIM 3 years ago. The implementation was based on an XMPP extension XEP-0357: Push Notifications and a dedicated, standalone component called MongoosePush. Since then, we have been lucky enough to work with several customers using this functionality. We have also been given great feedback from our community including people who use MongooseIM, MongoosePush or both in their projects. We listened to them, observed their issues and decided to create a better experience in MongooseIM 3.6.0 and MongoosePush 2.0.

How has the MongooseIM end-user experience changed?

Push notifications are delivered when the server observes a message addressed to a user who is offline. What does this mean exactly? From the server perspective, it means that the user has no connected devices. This works well enough in cases where users only have one device and don’t use session resumption as described in XEP-0198: Stream Management. Today, this is not the most common use case when it comes to mobile applications. First of all XEP-0198 is used because it helps to make sure that stanzas are delivered from the server to the client and the other way around. What is also important to note is that mobile applications are not always active. People switch applications or put their phone down. In this case, the app goes into the background and the network communication is limited to save battery. This leads to a situation where a device may not respond to regular TCP communication (because it’s disconnected or in the background) and the server still thinks the user is online. Push notifications are often used to notify the user that there is new information for them to check, so it is important that the notification is delivered in seconds, not minutes. In order to solve the problem, MongooseIM 3.6.0 and later versions, can send push notifications to such user sessions which are in the “undecided” state. More technical details and information on how to configure this behaviour can be found in the push notifications documentation.

Fine grained push notifications priority control

Not all Push Notifications will have the same level of priority. Sometimes you will want to change the priority in order to have a different behaviour on the client device. Before MongooseIM 3.6.0 it was not possible to change the priority level, even though MongoosePush, which talks to FCM or APNS, has that functionality. In order to address this limitation, we decided to give the control of the priority to the application developer. Now the priority can be set while enabling push notifications. Take a look at the enabling push notifications documentation to see the possible values and the other options you can set while enabling push notifications.

How has MongooseIM changed from for the operator?

No PubSub

The other thing we learned from our users was that they always use MongooseIM as both the instant messaging server and the service responsible for sending push notifications. The XEP-0357: Push Notifications is very generic and allows you to configure a PubSub service responsible for delivering push notifications on a different server. This was not the case in all the installations we were aware of. Because of that, we decided to bypass the PubSub communication in a situation where we know MongooseIM is responsible for sending push notifications. This makes the MongooseIM configuration easier, with no need to configure the PubSub service. What is more important is that it reduces the load on the nodes and minimises delays in delivering push notifications. In the PubSub-less configuration, MongooseIM can talk directly to MongoosePush in order to deliver push notifications. This was implemented in a way that allows smooth migrations for older versions of MongooseIM where push notifications were already used in the full version. More details about all the possible setups, their advantages and drawbacks can be found in our push notification tutorial.

RDBMS backend

In production systems it’s more convenient to store persistent data in an external database. It makes life easier when it comes to clustering MongooseIM and using it in containers. One of the previous releases introduced an RDBMS backend for PubSub but this kind of backend was still missing for push notifications. Now we have it, so you can start storing all the details, passed from the app when enabling push notifications and required to deliver them in Postgres, MySQL or even MSSQL.

Inactive device id removed

The device ids (in other words device tokens you get from APNS or FCM) are not valid indefinitely. Generating push notifications to expired tokens doesn’t make sense and introduces unneeded load in the system. Thanks to integrating with MongoosePush 2.0 and it’s new API v3 for push notifications, MongooseIM can detect if a push notification was sent to a device token which is no longer active and remove it from its database. Thanks to that we eliminate stale data as soon as we know it is no longer useful.

MongoosePush 2.0.0

The last 6 months were also busy around MongoosePush development. It’s a standalone service written in Elixir. It exposes a simple HTTP (1.1 and 2.0) API which can be used to trigger push notifications to FCM and APNS. The recent version added support for new authentication methods to APNS, support for new FCM v1 protocol and unified error codes which allows for instances to detect if the device token is active or not.

More information can be found in MongoosePush 2.0.0 changelog and its documentation on GitHub esl/MongoosePush.

Other changes

Push notification improvements are not the only change in MongooseIM 3.6.0. Take a look at 3.6.0 changelog to learn more and give it a try and share feedback!

Help us improve the MongooseIM platform:

1. Star our repo: esl/MongooseIM
2. Report issues: esl/MongooseIM/issues
3. Share your thoughts via Twitter
4. Download MongooseIM 3.6.0 Docker image
5. Download MongoosePush 2.0.0 Docker image
6. Sign up to our dedicated mailing list to stay up to date about MongooseIM, messaging innovations and industry news.
7. Check out our MongooseIM product page for more information on the MongooseIM platform.

Go back to the blog

Originally published at https://www.erlang-solutions.com on February 10, 2020.