<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[LumApps Experts - Medium]]></title>
        <description><![CDATA[The LumApps team shares their work processes and expertise on various topics, from engineering to product design. - Medium]]></description>
        <link>https://medium.com/lumapps-engineering?source=rss----e9909f99458b---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>LumApps Experts - Medium</title>
            <link>https://medium.com/lumapps-engineering?source=rss----e9909f99458b---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 06:05:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/lumapps-engineering" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Structured GCP logging in Rust]]></title>
            <link>https://medium.com/lumapps-engineering/structured-gcp-logging-in-rust-using-the-tracing-and-tracing-subscriber-crates-356fcb38e46e?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/356fcb38e46e</guid>
            <category><![CDATA[gcp]]></category>
            <category><![CDATA[logging]]></category>
            <category><![CDATA[rust]]></category>
            <dc:creator><![CDATA[Ludovic Vaugeois-Pepin]]></dc:creator>
            <pubDate>Mon, 21 Jul 2025 15:51:05 GMT</pubDate>
            <atom:updated>2025-07-24T08:52:47.970Z</atom:updated>
            <content:encoded><![CDATA[<h3>Structured GCP Logging in Rust</h3><h4>using the tracing and tracing-subscriber crates</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yKpV2tra7-DzYL6cjCc5zg.png" /></figure><p>Logging in Rust, using the <a href="https://docs.rs/tracing/latest/tracing/">tracing</a> and <a href="https://docs.rs/tracing-subscriber/latest/tracing_subscriber/">tracing-subscriber</a> crates, allows fine grain control of what gets logged where. Learning how <a href="https://www.youtube.com/watch?v=21rtHinFA40">these two crates work</a> has a steep-ish curve. Getting my head around how to use them took a day or two of learning and experimenting before being able to replace <a href="https://docs.rs/log4rs/latest/log4rs/">log4rs</a>.</p><p>The tracing-subscriber crate is used to “consume” logs emitted by the tracing crate. It provides ready-to-use objects in the <a href="https://docs.rs/tracing-subscriber/latest/tracing_subscriber/fmt/index.html">fmt module</a> for unstructured logging. Creating JSON structured logs using that module felt a bit forced and unnatural; at least I couldn’t find a clean enough way to do it. This is why I needed to get a better understanding of these crates.</p><h4>Structured logging in GCP</h4><p>A simple way to have applications in Kubernetes pods send logs to GCP is to write JSON objects to stdout or stderr. Among the different fields that would usually be included, such as a timestamp, a severity level, or a message, an optional trace to correlate entries can be added (look for field <strong>logging.googleapis.com/trace</strong> in the <a href="https://cloud.google.com/logging/docs/structured-logging#structured_logging_special_fields">documentation</a>)</p><p>Correlating log entries using a trace value can be done using <a href="https://docs.rs/tracing/latest/tracing/span/index.html">spans</a> from the tracing crate. Spans can be layered, and adding or updating their data can be done at different moments, meaning that their data can be set before they open, and can be modified afterwards. They seem to provide excellent support for elaborate structured logging scenarios. However, understanding how to obtain span data using the tracing-subscriber crate was a bit of a challenge for me, at least initially.</p><h4>Getting span data in subscribers</h4><p>A way to get the span data is to implement the Layer trait. I wrote a Layer that keeps track of the trace ID by getting the trace_id attribute of any span that opens. This is done by overriding the “on_new_span” method. Span attributes can change after the span is open. You need to override other methods of the Layer trait to capture those cases.</p><p>You can “save” the trace_id value by putting it in the extensions of the span. Otherwise the information is lost afterwards, which means we won’t have it anymore when getting log entries from tracing in the override of the “on_event” method.</p><pre>struct TraceId(String);<br><br>#[derive(Default)]<br>struct TraceIdVisitor {<br>    trace_id: Option&lt;String&gt;,<br>}<br><br>impl Visit for TraceIdVisitor {<br>    fn record_debug(&amp;mut self, field: &amp;Field, value: &amp;dyn std::fmt::Debug) {<br>        if field.name() == &quot;trace_id&quot; {<br>            self.trace_id = Some(format!(&quot;{value:?}&quot;))<br>        }<br>    }<br>}<br><br>struct GcpLayer {<br>    gcp_project_id: String,<br>}<br><br>impl&lt;S&gt; Layer&lt;S&gt; for GcpLayer<br>where<br>    S: Subscriber + for&lt;&#39;lookup&gt; LookupSpan&lt;&#39;lookup&gt;,<br>{<br>    fn on_new_span(&amp;self, attrs: &amp;Attributes&lt;&#39;_&gt;, id: &amp;Id, ctx: Context&lt;&#39;_, S&gt;) {<br>        let Some(span) = ctx.span(id) else {<br>            return;<br>        };<br>        let mut visitor = TraceIdVisitor::default();<br>        attrs.record(&amp;mut visitor);<br>        if let Some(trace_id) = visitor.trace_id {<br>            span.extensions_mut().insert(TraceId(trace_id));<br>        }<br>    }<br>   ...<br>}</pre><p>Here is how the trace ID value can be retrieved when log entries are received:</p><pre>impl&lt;S&gt; Layer&lt;S&gt; for GcpLayer<br>where<br>    S: Subscriber + for&lt;&#39;lookup&gt; LookupSpan&lt;&#39;lookup&gt;,<br>{<br>    ...<br><br>    fn on_event(&amp;self, event: &amp;Event&lt;&#39;_&gt;, ctx: Context&lt;&#39;_, S&gt;) {<br>        let mut trace = None;<br>        if let Some(scope) = ctx.event_scope(event) {<br>            for span in scope.from_root() {<br>                let extensions = span.extensions();<br>                let Some(trace_id) = extensions.get::&lt;TraceId&gt;() else { continue };<br>                let t = &amp;trace_id.0;<br>                trace = Some(format!(&quot;projects/{}/traces/{t}&quot;, self.gcp_project_id));<br>            }<br>        }<br>        ...<br>    }<br><br>    ...<br>}</pre><h4>Putting it together</h4><pre>async fn do_something() {<br>    info!(&quot;Doing something&quot;);<br>    // ...<br>    info!(&quot;Done doing something&quot;);<br>}<br><br>#[tokio::main]<br>async fn main() {<br>    init_logging().await;<br><br>    println!(&quot;With trace_id=456&quot;);<br>    do_something().instrument(info_span!(&quot;trace_id&quot;, trace_id = %&quot;456&quot;)).await;<br><br>    println!(&quot;Without a trace_id:&quot;);<br>    do_something().await;<br><br>    println!(&quot;With trace_id=789&quot;);<br>    do_something().instrument(info_span!(&quot;trace_id&quot;, trace_id = %&quot;789&quot;)).await;<br>}</pre><p>You can see the JSON logs with and without the trace ID:</p><pre>$ ./target/debug/with-tracing <br>With trace_id=456<br>{&quot;severity&quot;:&quot;INFO&quot;,&quot;message&quot;:&quot;Doing something&quot;,&quot;time&quot;:&quot;2025-07-21T07:58:48.417Z&quot;,&quot;logging.googleapis.com/trace&quot;:&quot;projects/PROJECT_ID_123/traces/456&quot;}<br>{&quot;severity&quot;:&quot;INFO&quot;,&quot;message&quot;:&quot;Done doing something&quot;,&quot;time&quot;:&quot;2025-07-21T07:58:48.417Z&quot;,&quot;logging.googleapis.com/trace&quot;:&quot;projects/PROJECT_ID_123/traces/456&quot;}<br>Without a trace_id:<br>{&quot;severity&quot;:&quot;INFO&quot;,&quot;message&quot;:&quot;Doing something&quot;,&quot;time&quot;:&quot;2025-07-21T07:58:48.417Z&quot;}<br>{&quot;severity&quot;:&quot;INFO&quot;,&quot;message&quot;:&quot;Done doing something&quot;,&quot;time&quot;:&quot;2025-07-21T07:58:48.417Z&quot;}<br>With trace_id=789<br>{&quot;severity&quot;:&quot;INFO&quot;,&quot;message&quot;:&quot;Doing something&quot;,&quot;time&quot;:&quot;2025-07-21T07:58:48.417Z&quot;,&quot;logging.googleapis.com/trace&quot;:&quot;projects/PROJECT_ID_123/traces/789&quot;}<br>{&quot;severity&quot;:&quot;INFO&quot;,&quot;message&quot;:&quot;Done doing something&quot;,&quot;time&quot;:&quot;2025-07-21T07:58:48.417Z&quot;,&quot;logging.googleapis.com/trace&quot;:&quot;projects/PROJECT_ID_123/traces/789&quot;}</pre><p>You can get the full code example here: <a href="https://github.com/ulodciv/rust-examples/blob/main/tracing-gcp/with-tracing/src/main.rs">https://github.com/ulodciv/rust-examples/blob/main/tracing-gcp/with-tracing/src/main.rs</a></p><p>There is also an equivalent sample app that uses log4rs: <a href="https://github.com/ulodciv/rust-examples/blob/main/tracing-gcp/with-log4rs/src/main.rs">https://github.com/ulodciv/rust-examples/blob/main/tracing-gcp/with-log4rs/src/main.rs</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=356fcb38e46e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/structured-gcp-logging-in-rust-using-the-tracing-and-tracing-subscriber-crates-356fcb38e46e">Structured GCP logging in Rust</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Skeleton on SwiftUI]]></title>
            <link>https://medium.com/lumapps-engineering/skeleton-on-swiftui-4664e9aef9ee?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/4664e9aef9ee</guid>
            <category><![CDATA[ios-app-development]]></category>
            <category><![CDATA[swift]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[ios-development]]></category>
            <category><![CDATA[swiftui]]></category>
            <dc:creator><![CDATA[Benjamin Dumont]]></dc:creator>
            <pubDate>Thu, 12 Jun 2025 13:13:53 GMT</pubDate>
            <atom:updated>2025-06-12T13:13:53.319Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*NM1Rakl7xY_yMAKH3IA6_A.png" /></figure><p>This article is about the loading operation rendering on iOS / iPad OS using a skeleton design.</p><h3>Skeleton implementation</h3><p>Let’s start with this example: we have a simple view containing some user informations and a button:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*jcmTG_bNHBEKbxHcaVLj6w.png" /><figcaption>Default view</figcaption></figure><p>The data model can be represented like this:</p><pre>struct UserModel: Equatable {<br>  let name: String<br>  let email: String<br>  let image: Image<br>}</pre><p>We will have 2 different states:</p><ul><li>the loading state: when the view appears or when we click on the Load user button</li><li>the loaded state: when we have the user informations</li></ul><p>Here is the state model:</p><pre>enum State: Equatable {<br>  case loading<br>  case loaded(UserModel)<br>}</pre><p>Let’s create our ViewModel:</p><pre>@Observable class UserViewModel {<br>  enum State: Equatable {<br>    case loading<br>    case loaded(UserModel)<br>  }<br>  var state: State = .loading<br><br>  func loadUser() async {<br>    state = .loading<br>    try? await Task.sleep(nanoseconds: 2_000_000_000) // Simulate an api call<br>    state = .loaded(<br>      UserModel(<br>        name: &quot;John Doe&quot;,<br>        email: &quot;johndoe@example.com&quot;,<br>        image: Image(systemName: &quot;globe&quot;)<br>      )<br>    )<br>  }<br>}</pre><p>To have a skeleton view during the loading state, we will use the redacted view modifier on the whole view. This view modifier will render the current UI (with data) using a skeleton template. Since we need some data to do this, we will need to mock it.</p><pre>struct UserView: View {<br><br>  @State private var viewModel = UserViewModel()<br>  @State private var user: UserModel = mockedUser<br><br>    var body: some View {<br>      VStack {<br>        HStack(alignment: .top) {<br>          user.image<br>            .resizable()<br>            .aspectRatio(contentMode: .fit)<br>            .frame(width: 40, height: 40)<br>            .clipShape(Circle())<br>          VStack(alignment: .leading) {<br>            Text(user.name)<br>              .font(.title2)<br>              .foregroundStyle(Color.red)<br>            Text(user.email)<br>              .foregroundStyle(Color.green)<br>          }<br>          Spacer()<br>        }<br>        Spacer()<br>        Button(&quot;Load user&quot;) {<br>          Task {<br>            await viewModel.loadUser()<br>          }<br>        }<br>      }<br>      .padding()<br>      .task {<br>        await viewModel.loadUser()<br>      }<br>      .onChange(of: viewModel.state) { _, newValue in<br>        switch newValue {<br>        case .loading:<br>          user = mockedUser<br>        case .loaded(let model):<br>          user = model<br>        }<br>      }<br>      .redacted(reason: viewModel.state == .loading ? [.placeholder] : [])<br>    }<br>}<br><br>// Our template data for skeleton UI<br>fileprivate var mockedUser: UserModel = UserModel(<br>  name: &quot;Firstname Lastname&quot;,<br>  email: &quot;email@apple.com&quot;,<br>  image: Image(systemName: &quot;person.circle.fill&quot;)<br>)</pre><p>So the user data will be filled by:</p><ul><li>mocked data when we are on loading state</li><li>real data when we have the informations</li></ul><p>The skeleton UI is rendered using the redacted view modifier. When we are on loading state, we want a placeholder (skeleton), otherwise we want the original rendering.</p><p>And the skeleton view looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*OPlDqbb0PgHfRuIkTNO3VQ.png" /><figcaption>Skeleton View</figcaption></figure><h3>Things to be aware of</h3><p>You may notice that the loading view takes into account the different colors of the original view. However, there are a few things you should pay attention to:</p><ul><li>The mocked data used for skeleton should represent real data in terms of UI. So try to have frames and text sizes near the real ones</li><li>Even when you are on a load state, the user can still interact with the application (e.g. tap on load user button again). It may be something you want to avoid.</li><li><strong>It won’t work on view created on UIKit (even using UIViewControllerRepresentable)</strong></li></ul><p>To avoid user interactions:</p><ul><li>You can use the disabled view modifier but you may have some issue about the color of disabled elements (see the button)</li></ul><pre>.disabled(viewModel.state == .loading)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*4SRIJ6FC2XosVxOTgEGDEQ.png" /></figure><ul><li>Or you can use allowHitTesting:</li></ul><pre>.allowsHitTesting(viewModel.state != .loading)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*gwAa2ZKszLc7aEftDeW-cA.png" /></figure><h3>UIKit Compatibility</h3><p>Unfortunately this is not compatible with Views thatt are drawn using UIKit (even using UIViewControllerRepresentable). But you can access to the redacted reasons.</p><p>Let’s create a simple UIViewController:</p><pre>import UIKit<br><br>final class UIKitViewController: UIViewController {<br><br>    private lazy var label: UILabel = {<br>        let label = UILabel()<br>        label.text = &quot;This is a UIKit View&quot;<br>        label.textAlignment = .center<br>        label.translatesAutoresizingMaskIntoConstraints = false<br>        return label<br>    }()<br><br>    override func viewDidLoad() {<br>        super.viewDidLoad()<br>        addComponents()<br>        addConstrains()<br>    }<br>}<br><br>private extension UIKitViewController {<br><br>    func addConstrains() {<br>        NSLayoutConstraint.activate([<br>            label.topAnchor.constraint(equalTo: view.topAnchor, constant: 64),<br>            label.centerXAnchor.constraint(equalTo: view.centerXAnchor),<br>        ])<br>    }<br><br>    func addComponents() {<br>        [label]<br>            .forEach { view.addSubview($0) }<br>    }<br>}</pre><p>We want to expose it to SwiftUI:</p><pre>import SwiftUI<br><br>struct UIKitView: UIViewControllerRepresentable {<br><br>    typealias UIViewControllerType = UIKitViewController<br><br>    func makeUIViewController(context: Context) -&gt; UIKitViewController {<br>      UIKitViewController()<br>    }<br><br>    func updateUIViewController(_ uiViewController: UIKitViewController, context: Context) {<br>       // Update the ViewController here<br>    }<br>}</pre><p>If we add the UIKitView to the User view, we will have this in the loading state:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*kg80V1a0HXfWzexqBXcVOw.png" /></figure><p>So, as said before, redacted view modifier is not compatible with UIKit. But we can access to its value using Environment Variables:</p><pre>struct UIKitView: UIViewControllerRepresentable {<br>  @Environment(\.redactionReasons) var reasons<br><br>    typealias UIViewControllerType = UIKitViewController<br><br>    func makeUIViewController(context: Context) -&gt; UIKitViewController {<br>      UIKitViewController()<br>    }<br><br>    func updateUIViewController(_ uiViewController: UIKitViewController, context: Context) {<br>       // Update the ViewController here<br>      uiViewController.reasons = reasons<br>    }<br>}</pre><p>And you can redraw the UI inside the UIViewController to look like a skeleton:</p><pre>final class UIKitViewController: UIViewController {<br>  var reasons: RedactionReasons = [] {<br>    didSet {<br>      // Reload the view<br>      addComponents()<br>      addConstrains()<br>    }<br>  }<br>  <br>    private lazy var label: UILabel = {<br>        let label = UILabel()<br>        label.text = &quot;This is a UIKit View&quot;<br>        label.textAlignment = .center<br>        label.translatesAutoresizingMaskIntoConstraints = false<br>        return label<br>    }()<br>  <br>  // View to simulate skeleton<br>  private lazy var skeleton: UIView = {<br>    let skeleton = UIView()<br>    skeleton.backgroundColor = .gray.withAlphaComponent(0.25)<br>    skeleton.translatesAutoresizingMaskIntoConstraints = false<br>    return skeleton<br>  }()<br><br>    override func viewDidLoad() {<br>        super.viewDidLoad()<br>        addComponents()<br>        addConstrains()<br>    }<br>}<br><br>private extension UIKitViewController {<br><br>    func addConstrains() {<br>      if reasons.contains(.placeholder) {<br>        NSLayoutConstraint.activate([<br>            skeleton.topAnchor.constraint(equalTo: view.topAnchor, constant: 64),<br>            skeleton.centerXAnchor.constraint(equalTo: view.centerXAnchor),<br>            skeleton.widthAnchor.constraint(equalToConstant: 145),<br>            skeleton.heightAnchor.constraint(equalToConstant: 21),<br>        ])<br>      } else {<br>        NSLayoutConstraint.activate([<br>            label.topAnchor.constraint(equalTo: view.topAnchor, constant: 64),<br>            label.centerXAnchor.constraint(equalTo: view.centerXAnchor),<br>        ])<br>      }<br>    }<br><br>    func addComponents() {<br>      if reasons.contains(.placeholder) {<br>        [label]<br>          .forEach { $0.removeFromSuperview() }<br>        view.addSubview(skeleton)<br>      } else {<br>        skeleton.removeFromSuperview()<br>        [label]<br>            .forEach { view.addSubview($0) }<br>      }<br>    }<br>}</pre><p>Here is the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/295/1*E7V1Qyv-PqlPVZbdDojtkw.gif" /><figcaption>Redacted with UIKit</figcaption></figure><h3>Conclusion</h3><p>Redacted view modifier is easy to use and powerful on a SwiftUI project but y<strong>ou must be meticulous when using it</strong>. You can access to its value using environment variables that permits to create your own implementation for UIKit code but it is quite painful.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4664e9aef9ee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/skeleton-on-swiftui-4664e9aef9ee">Skeleton on SwiftUI</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform at LumApps: Part 3]]></title>
            <link>https://medium.com/lumapps-engineering/terraform-at-lumapps-part-3-daa3c869f0f4?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/daa3c869f0f4</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[terragrunt]]></category>
            <category><![CDATA[self-service]]></category>
            <dc:creator><![CDATA[Jérôme Pin]]></dc:creator>
            <pubDate>Mon, 10 Jun 2024 14:16:25 GMT</pubDate>
            <atom:updated>2024-06-10T14:16:25.054Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mbaxWBNb_mjqT_PX" /><figcaption>Photo by <a href="https://unsplash.com/@jcraice?utm_source=medium&amp;utm_medium=referral">Julia Craice</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In <a href="https://medium.com/lumapps-engineering/terraform-at-lumapps-part-1-f37660b4ed95">our first</a> <a href="https://medium.com/lumapps-engineering/terraform-at-lumapps-part-2-27494897def4">two articles</a> we talked about the structure of our code and our platform.</p><p>As a reminder, we currently have 15 cells (where the entirety of the LumApps product is running) and around 60 services in each cell.</p><p>It means we have :</p><ul><li>15 terragrunt.hcl files for every service.</li><li>Around <strong>900 </strong><strong>terragrunt.hcl files</strong> (15*60) in total.</li></ul><p>We saw how we built our repo to manage such a scale and how we rely on our in-house cli infracli to handle tedious tasks such as editing all terragrunt.hcl files for any given service.</p><p>So… Life is better in our repository, but life is not great (yet). We still had two issues that arise when we make a change on our main Terragrunt module :</p><ul><li>How can we refactor the terragrunt.hcl’s inputs and avoid both the tediousness and the human error ?</li><li>How can we move resources in the Terraform state when their ID is dynamic ?</li></ul><p>The answer is simple and elegant. We made a new addition to our infracli tool : <strong>migrations</strong> !</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2Fl3nWqzkMR5diFKkkU%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fbrooklynninenine-brooklyn-nine-nine-nine-99-l3nWqzkMR5diFKkkU&amp;image=https%3A%2F%2Fmedia4.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExbWZydnRjeDRtZnZ5a3RkeHA0NnQ4NG16dXJyenFpdDZsNm01MGxuaiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2Fl3nWqzkMR5diFKkkU%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="246" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c9d84ddfdd8ff541aad5d50220dc486/href">https://medium.com/media/3c9d84ddfdd8ff541aad5d50220dc486/href</a></iframe><h3>Variable refactoring &amp; human error</h3><p>Let’s take an example for this one.</p><p>In front of our database instances, we have PGBouncer running. Each service has its own PGBouncer pods that can be configured:</p><ul><li>has_pgbouncer (bool, False): whether to deploy PGBouncer for the service.</li><li>pgbouncer_transaction_mode (bool, False): To use transaction pooling instead of <a href="https://www.pgbouncer.org/features.html">session pooling</a>.</li><li>pgbouncer_resources_requests (map): Describe the CPU and RAM requests to assign to the pods for the Kubernetes scheduler.</li><li>…</li></ul><p>Now, we want to modify these variables according to a few rules:</p><ul><li>Group them under a single map like :</li></ul><pre>pgbouncer = {<br>  transaction_mode = ...<br>  requests = {...}<br>  ...<br>}</pre><ul><li>Remove the pgbouncer prefix before each variable</li><li>Rename has_pgbouncer to pgbouncer.enabled</li><li>Always set pgbouncer.enabled to True</li></ul><p>It’s not very hard to do by hand obviously. But it’s very tedious, it’s easy to forget one of the rules, and doing it ourselves doesn’t add any value.</p><p>Additionally, since a service configuration could be spread into multiple files (because of possible per-environment and per-cell overrides, as shown in the “<em>Overridable inputs</em>” part of <a href="https://medium.com/lumapps-engineering/terraform-at-lumapps-part-2-27494897def4">our second article</a>), it adds even more places to make these changes.</p><p>We need a solution to make it easy and less error-prone.</p><h3>State manipulation</h3><p>Then we have an issue with state manipulation : moving, importing and deleting resources.</p><p>Terraform currently provides two ways of manipulating its state safely :</p><ul><li>Through a terraform subcommand : <a href="https://developer.hashicorp.com/terraform/cli/commands/import">import</a> and <a href="https://developer.hashicorp.com/terraform/cli/state">state *</a></li><li>Through configurations blocks : <a href="https://developer.hashicorp.com/terraform/language/import">import</a> and <a href="https://developer.hashicorp.com/terraform/language/modules/develop/refactoring">moved</a></li></ul><p>Import and moved blocks seem very cool from an IaC standpoint because they can be written next to the code they pair with, and everyone knows what’s been done to the state. However, references in the state must be static. Here is an example:</p><p>We have a simple resource :</p><pre>resource &quot;local_file&quot; &quot;file&quot; {<br>  content  = &quot;LumApps is a cool product!&quot;<br>  filename = &quot;my_file&quot;<br>}</pre><p>But then, we want to change its reference in the state to take into account a variable :</p><pre>variable &quot;cell_name&quot; {<br>  type    = string<br>  default = &quot;foo&quot;<br>}<br><br>resource &quot;local_file&quot; &quot;file&quot; {<br>  for_each = toset([var.cell_name])<br>  content  = &quot;LumApps is a cool product!&quot;<br>  filename = &quot;my_file&quot;<br>}</pre><p>Applying this will force a re-creation of our resource. So of course, we create a moved block :</p><pre>moved {<br>  from = local_file.file<br>  to   = local_file.file[var.cell_name]<br>}</pre><pre>│ Error: Invalid expression<br>│<br>│   on main.tf line 19, in moved:<br>│   19:   to = local_file.file[var.cell_name]<br>│<br>│ A single static variable reference is required: only attribute access and indexing with constant keys. No calculations, function calls, template expressions, etc are allowed here.<br>╵</pre><p>Terraform cannot do this move operation by itself unfortunately, so we must rely on the terraform state mv command.</p><p>For this simple case we would need to write a terragrunt command to make the move. And sometimes, even worse, two commands : one to remove the old resource from the state and another to import the new resource.</p><p>Since we have 15 cells at LumApps, we would need to do this 15 times, each time with a little change to var.cell_name.</p><p>Multiply this by 60 if we have to do the change for each service and we’d be spending our whole week watching Terraform commands run.</p><p>Manipulating the state with such commands has quite a few drawbacks:</p><ul><li>It’s error-prone</li><li>It cannot be reviewed in advance</li><li>It is tedious to write multiple times and adapt every time</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FkJxEM9PmVcNtOYwkGz%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fbrooklynninenine-nbc-brooklyn-nine-99-kJxEM9PmVcNtOYwkGz&amp;image=https%3A%2F%2Fmedia2.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExN3FpYnQzbWxuanZyY3I3bHRrZ2hzOTk2ODltcHRtbGFwcWJidGJwNiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FkJxEM9PmVcNtOYwkGz%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/b63620062ab268d83deea0cb617821ba/href">https://medium.com/media/b63620062ab268d83deea0cb617821ba/href</a></iframe><h3>Infracli models</h3><p>In infracli, every “thing” related to terraform is described by a Python class.</p><p>Here is an overview of our implementation :</p><pre>class Cell:<br>    name: str<br>    provider: CloudProvider<br><br>    get_subfolder(self, name: str) -&gt; Subfolder: ...</pre><pre>class Subfolder(TerragruntFolder):<br>    path: pathlib.Path<br>    terragrunt_file: TerragruntFile</pre><pre>class TerragruntFile(TerraformFile):<br>    dependencies: dict[str, str]<br>    source: str | None<br>    inputs: dict[str, str | dict]<br>    generate: dict[str, Any]<br>    remote_state: dict[str, Any]<br>    before_hooks: dict[str, Any]<br><br>    def read_file(self) -&gt; str: ... # read the file&#39;s content<br>    def load(self) -&gt; dict: ... # parse the HCL content and store into attributes<br>    def write(self) -&gt; TerragruntFile: ... # write the attributes back into the file as HCL</pre><p>From any larger element, we can easily get deep down the stack. Starting from a cell, it’s easy to reference a subfolder’s input :</p><pre>cell.get_subfolder(&quot;foo&quot;).terragrunt_file.inputs.get(&quot;has_pgbouncer&quot;)<br># =&gt; True</pre><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FGyUk5PPkr7KdGZUNPN%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fbrooklynninenine-nbc-brooklyn-nine-99-GyUk5PPkr7KdGZUNPN&amp;image=https%3A%2F%2Fmedia0.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExYnU3aXltcnlmbzJ4d3FtcnE5cWh0aWl5eHM2MGs3ejR4eG84N3p1eiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FGyUk5PPkr7KdGZUNPN%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c0b5b53fde5d43b197a1e1590fdd0d6/href">https://medium.com/media/3c0b5b53fde5d43b197a1e1590fdd0d6/href</a></iframe><h3>Infracli migrations</h3><p>With our two issues in mind, we settled on running some kind of command(s) before Terraform would run, with access to the whole Terraform context : state, variables, etc.</p><p>Armed with our Python classes, our life is quite easy when we want to modify anything in our repository.</p><p>Based on our previously-explained issues, we ended up building two classes for our migrations : one to modify the state (TerraformSubfolderStateMigration) and one to modify a module’s inputs (InputsFileMigration).</p><p>Then we can create subclasses from each of them based on our needs, and write our migration. Here are two migrations corresponding to our two earlier examples :</p><pre>class MigratePGBouncerFlatVariablesToObject(InputsFileMigration):<br>    &quot;&quot;&quot;Convert pgbouncer_* variables to a pgbouncer object.&quot;&quot;&quot;<br><br>    TERRAGRUNT_MODULE_MIN_VERSION = &quot;6.0.0&quot; # The version of our terragrunt module required to run this migration<br><br>    # What will happen with our current variables <br>    VARIABLES_MAPPING = {<br>        &quot;has_pgbouncer&quot;: None, # Remove has_pgbouncer<br>        &quot;pgbouncer_transaction_mode&quot;: &quot;pgbouncer.transaction_mode&quot;, # Put transaction_mode inside the pgbouncer map<br>        &quot;pgbouncer_resources_requests&quot;: &quot;pgbouncer.requests&quot;, # Same<br>    }<br><br>    # Run only if the subfolder is recent enough AND if the keys declared in the mapping above are in the terragrunt.hcl file.<br>    def is_necessary(self, input_file: TerragruntFile) -&gt; bool:<br>        return (<br>            self.is_above_minimal_module_version()<br>            and input_file.has_variables_requiring_migration()<br>        )<br><br>    # Change the variable according to the mapping above then re-write the terragrunt.hcl file and format it.<br>    def migrate(self, input_file: TerragruntFile) -&gt; None:<br>        input_file.inputs = self.remap_variables(input_file)<br>        TerragruntFile.write(input_file.path, input_file)<br>        input_file.format()</pre><pre>class MoveLocalFileInState(TerraformSubfolderStateMigration):<br>    &quot;&quot;&quot;Move the local_file to support the for_each addition.&quot;&quot;&quot;<br><br>    # Manipulate the state for the subfolder<br>    def migrate(self, cell: Cell, subfolder: Subfolder) -&gt; None:<br>        self.terragrunt_state_mv(<br>            old_id=&quot;local_file.file&quot;,<br>            new_id=f&quot;local_file.file[{cell.name}]&quot;,<br>        )<br>        ...</pre><h3>Conclusion</h3><p>We made a few proof-of-concept about such migrations over the years, with more-or-less satisfaction.</p><p>Most of the time, we thought it would require a small bash script written in 5 minutes. And we would end up with an unmanageable monstrosity, feared by everyone. It was never safe to run and re-run and was always missing some features.</p><p>With the design of infracli well thought out, we finally had all the cards we needed to write great migrations, and we did !</p><p>Nowadays, this is a very appreciated part of our toolbox which solves a lot of our previously tedious tasks and we are very happy with it !</p><p>In addition to the peace of mind we gained, the investment is paying off and it saved us countless hours already.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=daa3c869f0f4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/terraform-at-lumapps-part-3-daa3c869f0f4">Terraform at LumApps: Part 3</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform at LumApps: Part 2]]></title>
            <link>https://medium.com/lumapps-engineering/terraform-at-lumapps-part-2-27494897def4?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/27494897def4</guid>
            <category><![CDATA[self-service]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[terragrunt]]></category>
            <dc:creator><![CDATA[Jérôme Pin]]></dc:creator>
            <pubDate>Tue, 19 Dec 2023 15:21:19 GMT</pubDate>
            <atom:updated>2023-12-19T15:21:19.728Z</atom:updated>
            <content:encoded><![CDATA[<h4>Managing terragrunt.hcl files at scale</h4><p><a href="https://medium.com/lumapps-engineering/terraform-at-lumapps-part-1-f37660b4ed95">In the first article</a>, we talked about who we were, our goal for the LumApps’ platform and how we organized and managed the infrastructure.</p><figure><img alt="Timeline from 2018 to 2020 of the self-service deployment at LumApps" src="https://cdn-images-1.medium.com/max/1024/0*zTO2x3N9vQI3Vvmj" /><figcaption>Timeline from 2018 to 2020 of the self-service deployment at LumApps</figcaption></figure><p>In the summer of ’20, we were very satisfied with the state of LumApps’ infrastructure. However we noticed we were spending more and more time doing repetitive tasks on behalf of the developers. Incidentally, it meant we were slowing down the velocity of every team each time they needed a change.</p><p>We started thinking about how we could provide the developers with self-service on their service’s infrastructure.</p><p>In LumApps’ context, “self-service” means the ability for developers to deploy, configure and maintain their services and their infrastructure by themselves using an abstraction layer.</p><p>In the words of Platform’s Director Jan Villeminot : “automation alone allows the Platform team to handle high-value tasks, but self-service will increase developers’ velocity and agency on the infrastructure”.</p><h3>History</h3><p>GitOps then self-service dates back to 2018 at LumApps.</p><p>At first it was a mix of self-service and GitOps. Our first guinea pig was the DNS repository. This repository was using Terraform and CircleCI to manage our DNS zones and records. Making a change meant opening a PR and looking for CircleCI’s output. Upon approval and merge, CircleCI applied the change.</p><p>In 2019, we went on GitOps-ing two other repositories because we (the Platform domain) were slowing down the development teams : our api-gateway and our ArgoCD repositories. Both repositories had huge amounts of repetitive and complex code. Our goal was then to provide some kind of abstraction layer (in the form of easier YAML code blocks) for the developers to configure themselves their applications’ api-gateway and deployments.</p><p>Then we tackled Github’s management : one of our repositories hosts all the resources needed for Terraform to manage all our Github objects (organization, teams and sub-teams, repositories and their configuration, branch protections, etc…). Every developer was able to open pull-requests but the Platform domain was responsible for applying them. To keep track of what Terraform did, we started applying the requested changes with CircleCI instead of our laptops. We were still impacting velocity (more on that later) but at least we gained some safety.</p><p>In 2021, we found out about <a href="https://github.com/runatlantis/atlantis">Atlantis</a>. Atlantis can be plugged on a Github repository using web hooks. It can listen to Github’s PR’s comments, plan and apply Terraform resources and write back its output inside a PR’s comment.</p><p>We used the repository holding our Github resources to build our first proof-of-concept using Atlantis. And it was a huge success for both the Platform domain and the development teams !</p><p>A developer is now able to open a PR and handle its lifecycle (except for the approval, of course) all by himself :</p><ul><li>I have a new team mate ? I can add him to our organization.</li><li>I need my team to access another repository ? I can grant additional privileges to my team.</li><li>I want to make some CI checks required ? <em>(You guessed it)</em></li><li>…</li></ul><p>The Platform domain can easily review proposed changes along with Terraform’s plan’s output directly inside the PR. We don’t have to switch context to apply a change and we don’t impact velocity anymore.</p><h3>What did we learn ?</h3><p>Through this GitOps/self-service journey, we learnt a few important things that were of tremendous importance for the success of our project of providing infrastructure as a self-service function :</p><ul><li>Making developers write Terraform code isn’t possible. Learning a new syntax for the sole purpose of a small and infrequent change isn’t a good thing;</li><li>Building opinionated infrastructure and tooling works well;</li><li>We can provide an abstraction layer over infrastructure-related code but it must be both simple and future-proof;</li><li>Relying on modified files in a PR isn’t enough to understand the entirety of the proposed change. Terraform’s output must appear in the pull request;</li></ul><h3>Aligning the stars</h3><p>Early on, we had defined our requirements for this project. We needed (in no particular order):</p><ol><li>A tool to automate the infrastructure deployment</li><li>A way to handle secrets securely</li><li>A way to hide sensible informations in the plan</li><li>To simplify our Terraform code</li><li>A tool to ease the configuration</li><li>A prototype to use as a sandbox</li></ol><p>Requirements 1 and 6 have been covered earlier in this article.</p><p>Requirement 4 was the hardest to match. We did that by being tremendously smart 😜.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Ftenor.com%2Fembed%2F22065379&amp;display_name=Tenor&amp;url=https%3A%2F%2Ftenor.com%2Fview%2Fbrooklyn-nine-nine-jake-peralta-peralta-you-genius-genius-smart-gif-22065379&amp;image=https%3A%2F%2Fmedia.tenor.com%2F0Vf4Kd9NpicAAAAF%2Fbrooklyn-nine-nine-jake-peralta.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=tenor" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/4dd0f4dad7bb7bcba016b55ed87b2021/href">https://medium.com/media/4dd0f4dad7bb7bcba016b55ed87b2021/href</a></iframe><p>Let’s focus on the other points.</p><h3>A way to handle secrets securely</h3><p>We don’t want to store our secrets in Git (even encrypted) but we need a way to have them stored safely, versioned, and audited. The best way for us to do this is with <a href="https://www.vaultproject.io/">Hashicorp Vault</a>.</p><figure><img alt="A schema showing Terraform reading secrets from Hashicorp Vault and writing them into Kubernetes Secrets." src="https://cdn-images-1.medium.com/max/1024/0*z3yBMhJ385ZgjGpy" /></figure><p>Terraform reads necessary secrets from one of our Vault cluster using a <a href="https://registry.terraform.io/providers/hashicorp/vault/latest/docs/data-sources/generic_secret">generic Vault datasource</a> and put them inside every service’s Kubernetes secret.</p><p>Any developer can manipulate a map named additional_secret_entries to select which keys should be injected from the Vault cluster into the service’s secret :</p><pre>additional_secret_entries = {<br>  &quot;foo/bar&quot; = [<br>    &quot;baz&quot;,<br>  ]<br>}</pre><p>In this case, it means the key named baz and its value will be pulled from the Vault cluster at the foo/bar path and stored inside the service’s Kubernetes secret.</p><p>The corresponding code for this behavior looks like this :</p><pre>data &quot;vault_generic_secret&quot; &quot;additional_secrets&quot; {<br>  for_each = toset(keys(var.additional_secret_entries))<br>  path     = each.value<br>}<br><br>locals {<br>  # - We iterate on the path provided by the devs in the inputs<br>  # { path = [secret_key, secret_key2], path2 = [secret_key3, secret_key4] }<br>  # - We generate a list of maps with secrets contained in secret_keyN<br>  # - `...` is the spread operator. It transforms func([ a, b, c ]...) into func(a,b,c)<br>  additional_secrets = merge([<br>    for path in toset(keys(var.additional_secret_entries)) : {<br>      for key in var.additional_secret_entries[path] :<br>      key =&gt; data.vault_generic_secret.additional_secrets[path].data[key]<br>    }<br>  ]...)<br>}<br><br>resource &quot;kubernetes_secret&quot; &quot;service_secret&quot; {<br>  metadata {<br>    name      = &quot;${local.service_name}-secrets&quot;<br>    namespace = module.namespace.namespace_name<br>  }<br><br>  data = merge([local.additional_secrets]...)<br>}</pre><h3>A way to hide sensible informations in the plan</h3><p>On earlier 0.x versions of Terraform, there was no way to explicitly tell Terraform that a given string was a secret. It was all in the hands of the provider to decide whether a resource’s attribute was always sensitive or never.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FYemPO3cVf6jKA3cd3c%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FYemPO3cVf6jKA3cd3c%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia1.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExZW5jYncycnZhbm5kMTYxYnkxZHZzcnZ0bm12ODljYzMxams1OHdkcSZlcD12MV9naWZzX2dpZklkJmN0PWc%2FYemPO3cVf6jKA3cd3c%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c5e9d922534d2a55fa4b80c094086f8/href">https://medium.com/media/3c5e9d922534d2a55fa4b80c094086f8/href</a></iframe><p>Fortunately, during our quest for a solution, the IaC’s gods heard our prayers (maybe sacrificing a few clusters made them listen) and provided us with the 0.15 release of Terraform !</p><p>This release brings the function sensitive() (and its friend nonsensitive ). That way, we can be sure that every secret’s element won’t be shown in a plan !</p><p>There is a little issue though… Since Terraform won’t show the content of a secret, there is no way to determine which key will be modified when reading the plan. This is an intended behavior but is quite scary.</p><p>To work around this, we used a little trick in our book : each value in the Kubernetes secret is hashed and stored in a configmap. The configmap is then made non-sensitive explicitly by Terraform.</p><p>This allows us to understand which key from the secret will be modified without showing the sensitive value in the plan !</p><h3>A tool to ease the configuration</h3><p><a href="https://medium.com/lumapps-engineering/terraform-at-lumapps-part-1-f37660b4ed95">In our first article, Barney didn’t like editing 750+ terragrunt.hcl files</a> and we neither.</p><p>As a reminder, our infrastructure (and its Terraform representation) is organized around cells and subfolders. Every cell contains the same set of subfolders as the others : one subfolder for our Kubernetes cluster, one subfolder for our Elasticsearch cluster, one subfolder for the in-house service X, etc…</p><p>It means a few things regarding the scale of our repository:</p><ul><li>A huge part of every terragrunt.hcl is identical to the 750+ others</li><li>A developer must edit around 15 (one per cell) terragrunt.hcl with almost the same content when he wants to change something for its service</li><li>We have no easy way to manage all these files</li></ul><p>We found no way to resolve all these pain points.</p><p>So we built the way.</p><p>Introducing… <strong>infracli</strong> !</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Ftenor.com%2Fembed%2F22443601&amp;display_name=Tenor&amp;url=https%3A%2F%2Ftenor.com%2Ffr%2Fview%2Fbrooklyn-nine-nine-amy-santiago-we-did-it-mission-accomplished-b99-gif-22443601&amp;image=https%3A%2F%2Fmedia.tenor.com%2FLzSvhU4JMAgAAAAF%2Fbrooklyn-nine-nine-amy-santiago.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=tenor" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/79881aa252ec16ba822c44c666db3d53/href">https://medium.com/media/79881aa252ec16ba822c44c666db3d53/href</a></iframe><h3>Infracli</h3><p>Infracli is both a library and a CLI written in Python.</p><p>Its purpose is to ease the management of common (or repetitive) tasks on our repository, for both developers and the Platform domain.</p><h3>Features</h3><ul><li>Plan and apply a subfolder on all cells at once</li><li>Create a new cell, update an existing cell’s file</li><li>Connect to a GCP/Azure-managed PostgreSQL database</li><li>Manipulate secrets inside our Vault clusters</li><li>Create a new service, update an existing service</li><li>Bump the Terragrunt module’s version of a service</li><li>Run migrations on a service’s Terraform’s state or on its inputs</li></ul><p>Manipulating secrets inside our Vault clusters will likely be the subject of another article and will not be addressed now. Instead, we will focus on service management.</p><p>The three main subcommands for this usage are :</p><ul><li>infracli service create</li><li>infracli service update</li><li>infracli service bump</li></ul><h3>Our requirements</h3><p>Here is what almost all terragrunt.hcl files look like :</p><pre>locals {<br>  ...<br>}<br><br>include {<br>  path = find_in_parent_folders()<br>}<br><br>dependency &quot;kubernetes&quot; {<br>  config_path = &quot;${get_parent_terragrunt_dir()}/kubernetes&quot;<br>}<br><br>dependency &quot;network&quot; {<br>  config_path = &quot;${get_parent_terragrunt_dir()}/network&quot;<br>}<br><br>...<br><br>terraform {<br>  source = &quot;vcs.com:repo//svc-go?ref=v1.18.0&quot;<br>}<br><br>inputs = {<br>  ...<br>  network_name = dependency.network.outputs.vpc.network_name<br>  additional_secret_entries = {<br>    &quot;foo/bar&quot; = [<br>      &quot;baz&quot;,<br>    ]<br>  }<br>  ...<br>}</pre><ul><li>A set of locals</li><li>An include block</li><li>Optionally some dependency blocks</li><li>A terraform block</li><li>A set of inputs</li></ul><p>These elements may be identical or may vary a little (depending on the environment or the cell, for example), but differences are minimal.</p><p>Knowing this, we need a way to handle a generic file content and the ability to override some parts of it.</p><h3>How infracli works ?</h3><p>Beside infracli itself, we added two important concepts : a <em>per-terragrunt-module template</em> and a <em>hierarchy of inputs</em>.</p><h3>Terragrunt.hcl template</h3><p>Every terragrunt module contains a file named terragrunt.hcl.tpl which contains all the static code common to every subfolder instantiating said module. Basically, it contains everything the final terragrunt.hcl will have except the inputs.</p><h3>Overridable inputs</h3><p>We needed a solution to append per-service, per-cell, per-environment inputs to this static file, without repeating every inputs.</p><p>At the root of our repository, we added an inputs/ directory. Each service has its inputs defined in a dedicated folder. We require at least a file for Google and a file for Microsoft cells.</p><pre>inputs/<br>  ...<br>  svc_foo/<br>  ├── go/<br>  │   └── default.hcl<br>  └── ms/<br>      ├── default.hcl<br>      └── dev/<br>          ├── default.hcl<br>          └── dev-ms-cell-001.hcl<br>  ...</pre><p>If needed, one can add or override inputs per-environment (for example dev/default.hcl) or even per-cell (dev/dev-ms-cell-001.hcl).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/0*lwhymuUT5RG9-K6-" /></figure><p>With these two concepts (and infracli gluing them together) it became easy to modify every terragrunt.hcl. Whether the user wants to edit 750+ files at once or only a specific one for its service, the process is the same !</p><p>Now, let’s demonstrate infracli in action with a fake service.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Ftenor.com%2Fembed%2F10153385379469800051&amp;display_name=Tenor&amp;url=https%3A%2F%2Ftenor.com%2Ffr%2Fview%2Fhot-damn-gif-10153385379469800051&amp;image=https%3A%2F%2Fmedia.tenor.com%2FjOgSNSrxMnMAAAAN%2Fhot-damn.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=tenor" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/8c85602aab2bb84acb90a2488dcbe431/href">https://medium.com/media/8c85602aab2bb84acb90a2488dcbe431/href</a></iframe><h3>Example</h3><p>Let’s say we have a service named foo relying on the svc-go Terragrunt module.</p><h4>The template</h4><p>We define the template inside the Terragrunt module. It will be used by every service instantiating the module. We store it inside the module (vcs.com:repo/svc-go/infracli/terragrunt.hcl.tpl) :</p><pre>locals {<br>  ...<br>}<br><br>include {<br>  path = find_in_parent_folders()<br>}<br><br>dependency &quot;kubernetes&quot; {<br>  config_path = &quot;${get_parent_terragrunt_dir()}/kubernetes&quot;<br>}<br><br>dependency &quot;network&quot; {<br>  config_path = &quot;${get_parent_terragrunt_dir()}/network&quot;<br>}<br><br>...</pre><h4>The inputs</h4><p>Then we need to define the inputs. As LumApps is multi-cloud, we need to provide at least one file for Google and one file for Microsoft cells (see the directory structure before).</p><p>To keep our example short, we are going to define two files : one for all Google cells, and one for a specific google cell, which is going to improve and override the first file.</p><p>The first one would apply to all Google cells : inputs/svc_foo/go/default.hcl</p><pre>terraform {<br>  source = &quot;vcs.com:repo//svc-go?ref=v1.18.0&quot;<br>}<br><br>inputs = {<br>  ...<br>  network_name = dependency.network.outputs.vpc.network_name<br>  additional_secret_entries = {<br>    &quot;foo/bar&quot; = [<br>      &quot;baz&quot;,<br>    ]<br>  }<br>  ...<br>}</pre><p>Then we define a file specifically for a given cell : inputs/svc_foo/go/dev/dev-go-cell-001.hcl :</p><pre>inputs = {<br>  foo = &quot;bar&quot;<br>  additional_secret_entries = {<br>    &quot;some/secret/${var.environment}&quot; = [<br>      &quot;key&quot;,<br>    ]<br>  }<br>}</pre><h4>The run</h4><p>Finally, we run the command infracli service update svc_foo. For every cell, it will merge and overwrite all inputs and append the resulting content to the content of the template. Then write all of this in the terragrunt.hcl file.</p><p>We are closing the second part of this series on this. As of now, every LumApps developer can alter its service’s infrastructure by opening a PR which modifies a few files. This is a tremendous improvement on the time spent for both the developers and the Infra team.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Ftenor.com%2Fembed%2F24868445&amp;display_name=Tenor&amp;url=https%3A%2F%2Ftenor.com%2Ffr%2Fview%2Fbrooklyn99-amy-santiago-and-thats-how-you-improve-morale-morale-improve-gif-24868445&amp;image=https%3A%2F%2Fmedia.tenor.com%2F0e-HTGr0w50AAAAF%2Fbrooklyn99-amy-santiago.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=tenor" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/6063abae899d5f481db5cfe3efa4a857/href">https://medium.com/media/6063abae899d5f481db5cfe3efa4a857/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27494897def4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/terraform-at-lumapps-part-2-27494897def4">Terraform at LumApps: Part 2</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform at LumApps: Part 1]]></title>
            <link>https://medium.com/lumapps-engineering/terraform-at-lumapps-part-1-f37660b4ed95?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/f37660b4ed95</guid>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Jérôme Pin]]></dc:creator>
            <pubDate>Mon, 23 May 2022 12:18:33 GMT</pubDate>
            <atom:updated>2022-06-06T07:41:38.441Z</atom:updated>
            <content:encoded><![CDATA[<h3>Context</h3><p>During the early years, our product was running entirely on Google App Engine with Google Datastore as the database engine. App Engine allows for a planet-scale autoscaling of our web applications without the hassle of managing the infrastructure.</p><p>In 2018, we started seeing the limits of our model: running a monolith was causing issues. Using a PaaS solution wasn’t very cost-effective and scaling our Engineering organization was going to be a struggle.This is where we started our journey towards a <em>Service-Oriented Architecture</em>. The goal was to overcome our current struggles and to be future-proof.</p><p>Now that the migration to our new architecture is well underway, we want to show what we have done on the infra side, over a series of articles.</p><h3>The Platform domain</h3><p>The Platform domain is responsible for building the platform hosting LumApps. It brings together developers and platform engineers. Our focus is not only on stability, but on cost-efficiency and developer velocity too.</p><p>We chose to build our platform together with the feature teams to tailor it to the LumApps product and its developers needs.</p><p>The Platform domain’s roles are pretty diverse:</p><ul><li>Build and maintain the platform hosting LumApps. <br>This is a traditional CloudOps function: infrastructure setup and maintenance, CI/CD toolchain, SaaS solutions analysis and implementation, etc.;</li><li>Define (together with the feature teams) and enforce best practices (through tooling) for development, deployment and run of the different services composing the platform;</li><li>Allow feature teams to focus on building features with added value by removing as much complexity as possible from infrastructure-related processes;</li><li>Monitor and improve continuously the developer experience;</li><li>Work tightly with features teams to provide architecture advices early in a service development;</li></ul><h3>Use cases and requirements</h3><p><em>Our first requirement is to automate and secure the building of the platform.</em></p><p>The platform is built around Kubernetes, PostgreSQL, Redis and Elasticsearch. The last two running on Kubernetes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nwIvTACQguZqXK2M" /><figcaption>High-level view of a LumApps cell</figcaption></figure><p>Each customer is deployed into a <em>cell</em> which is a whole deployment of LumApps (with its Kubernetes, PostgreSQL, Redis and Elasticsearch clusters). By correctly sizing the cells, we aim to remove the noisy neighbor phenomenon and limit the blast radius in case of an outage. This is obviously a trade-off with maintenance and infrastructure cost.</p><p><em>Our second requirement is to provide developers with autonomy using GitOps practices.</em></p><p>From day one, we wanted to provide feature teams with self-service infrastructure following the motto “<em>you build it, you run it</em>”. We also needed to enforce best practices and be opinionated about what we allow (or not) on the platform.</p><p>Giving feature teams more autonomy on their services built-up their confidence and allowed the Platform domain to focus on providing expertise and improving the platform itself.</p><h3>Implementation</h3><p>LumApps is available on both Google Cloud Platform and Microsoft Azure as our customers are usually linked to one or the other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*y2AkalxIhSWPUOK5" /><figcaption>Self-service flow for infrastructures and services deployment</figcaption></figure><p>Cells are independent, located in different regions, and run the entirety of LumApps. They are all deployed the same way and they all run the same version of our services.</p><h3>The infrastructure</h3><p>The Platform domain is composed of several teams, including the infra team which is responsible for the deployment of all the underlying infrastructure. We chose to use <a href="https://www.terraform.io/">Terraform</a> as our means of industrialization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*w2f5_lZhCHkgFdWm6WMCag.jpeg" /><figcaption>Drake endorses Infrastructure-As-Code</figcaption></figure><p>Deploying everything with Terraform (and versioning in Github) allows us to ensure consistency across cells, perform code reviews, audit our code and its changes, and quickly revert a change if needed.</p><h3>Common infrastructure</h3><p>First, using Terraform, the Infra team creates the cell itself (the GCP project or the Azure Resource Group) and all the storage for later use by Terraform for its states. Every cell and every service has its own state stored on the provider’s object storage service.</p><p>Then we create all the common infrastructure components:</p><ul><li>The virtual network and the subnets we will use;</li><li>A Kubernetes cluster (Google Kubernetes Engine or Azure Kubernetes Service);</li><li>The API-Gateway;</li><li>All Message Bus elements (PubSub or EventHub);</li><li>An Elasticsearch cluster for our search engine;</li><li>An HashiCorp Vault cluster;</li><li>The Datadog agent;</li></ul><h3>Services infrastructure</h3><p>Finally the service infrastructure can be deployed. Each service have (or may have, depending on its needs):</p><ul><li>A Kubernetes namespace;</li><li>A set of Network Policies to restrict access to the services on a need-to basis;</li><li>A Redis cluster;</li><li>A PostgreSQL cluster (Google CloudSQL or Azure Database for PostgreSQL) with PGBouncer in front;</li><li>An access to third-party services: Elasticsearch, PubSub/EventHub and Vault;</li></ul><p>This part is GitOps-ed using Atlantis (more on that in our second article). Every developer has the ability to create a service’s infrastructure and modify it to have it tailored to the service’s needs.</p><h3>The service</h3><p>The service itself is deployed using ArgoCD, reading its configuration from the corresponding repository on Github.</p><p>This gives feature teams a total agency to deploy whatever they want and whenever they want. We do limit the available Kubernetes objects to an extent through ArgoCD configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7PlxoAfCRsmj8mhe" /><figcaption>Architecture of a typical in-house service</figcaption></figure><p>A classic service usually creates several Kubernetes resources: a deployment for the main API, some event processors deployments to subscribe to the message buses, a Kubernetes service (obviously), multiple ConfigMaps and secrets, some HPAs and various (Cron)Jobs (depending on the service and its infrastructure).</p><h3>Our main repository</h3><h3>Structure</h3><p>Most of our infrastructure-related code is stored in a single repository. It works around two main components: our cells and our modules.</p><p>Since we are in an era of big changes at LumApps, having everything in a mono-repo lets us perform quick iterations and deployments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*qQf-XTHN-1byIR1s1CFRUQ.png" /><figcaption>Organization of our infrastructure mono-repo</figcaption></figure><p><a href="https://terragrunt.gruntwork.io/">Terragrunt</a> is a huge component in reducing our code base by providing reusable elements, code include and dependencies between modules.</p><h3>Modules</h3><p>We rely on both Terraform and Terragrunt modules to deploy our infrastructure. Terraform modules are used to package a set of tightly-coupled Terraform resources while Terragrunt modules are used to manage all resources for a given service (including Terraform modules).</p><h3>Cells</h3><p>The <em>cells</em> folder contains all our cells spread out into their respective environments. Every cell contains the same structure which is a set of subfolders. Each subfolder is in charge of calling a Terragrunt module to create something specific.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*1R-RKIE37BzMvgRX1sCElg.png" /><figcaption>Subfolders in a cell</figcaption></figure><p>A subfolder doesn’t contain much beside a <em>terragrunt.hcl</em> file which instantiates the Terragrunt module with the appropriate inputs.</p><p>All in-house services are deployed in their own subfolder (with a name starting with svc_). They all instantiate the same Terragrunt module (<em>svc-go</em> for Google cells and <em>svc-ms</em> for Microsoft cells).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lqj2U3Y5oVfZYcQC" /><figcaption>Call graph of a subfolder, its Terragrunt module and its Terraform modules</figcaption></figure><p>We created a Terragrunt module generic enough to be able to handle all our use cases for these in-house services:</p><ul><li>database deployment, provisioning and configuration;</li><li>Redis configuration;</li><li>message queues access;</li><li>synchronous call to other services;</li><li>custom secrets;</li><li>etc.</li></ul><h3>Timeline towards the self-service</h3><p>While we chose to do self-service from the get-go, we iterated and improved a lot of things throughout the whole project, which is still ongoing for that matter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aCR49Ud0vCD2_r0G" /></figure><ul><li><strong>Late 2018:</strong> The beginning of the project with our first Terraform modules at LumApps;</li><li><strong>Mid 2019:</strong> Design is still ongoing for the new architecture. Realization that we will need to do a lot of copy/paste to make Terraform work;</li><li><strong>Early 2020:</strong> Terragrunt prototyping and implementation to reduce said copy/paste;</li><li><strong>Late 2020:</strong> Heavy effort towards factorization and generalization of our Terragrunt modules;</li><li><strong>Early 2021:</strong> Atlantis prototype and MVP;</li><li><strong>Mid 2021:</strong> Infracli prototype (see below) and self-service for the Platform domain only;</li><li><strong>Late 2021:</strong> Self-service MVP go-live for the whole Engineering department;</li></ul><p>We chose to assume the role of an Ops team for the first half of this timeline in order to really understand what the platform should be. Working together with the feature teams, we completed our design of what is a LumApps Service.</p><p>Only in mid-20, we started to focus our efforts on making the self-service project a reality. That’s very important as it allowed us to answer 95% of the needs of the feature teams through the self-service alone.</p><h3>Issues</h3><p>We encountered a few roadblocks during our journey towards deploying and automating our infrastructure.</p><h3>Bootstrapping</h3><p>Currently, bootstrapping anything requires human intervention.</p><p>The infra repository’s initial remote state and the Google project storing it (along the associated service accounts) all have to be created and stored by a human being.</p><p>We said earlier that each cell stores its Terraform state (and the state of its subfolders) in a dedicated bucket. But where does the cell store its state when creating its own bucket? This is a chicken or the egg dilemma.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/0*MHcyIj4HydYtXsMu" /></figure><p>Every cell contains a subfolder named <em>cell_bootstrap</em> which is used to bootstrap the cell itself. That means it creates either a new project along with a bucket to hold the cell’s state (and the cell’s subfolders’ states) for GCP, or for Microsoft cells, it creates a new resource group and a storage account with the associated container.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*o-ZA2iz3XfnVMe4oWdunXg.png" /><figcaption>Terraform remote state storage</figcaption></figure><p>The state of the cell_bootstrap folder is, however, stored in an environment-specific bucket outside of the cell’s project or resource group.</p><h3>How to authenticate on Azure using AAD’s service principals ?</h3><p>We use Azure Active Directory to authenticate both machines (using service principals) and humans (using our own accounts) when performing Terraform operations on Azure.</p><p>However, granting permissions to Service Principals on certains services requires a delegation that only a human can grant. To simplify the delegation process to its maximum, and like we did for the cell bootstrapping, every Azure cell has a special subfolder handling this and printing the URL for a human to click.</p><h3>How to handle a lot of terragrunt.hcl?</h3><p>As we’re writing this, we have 15 cells and north of 50 subfolders in each cell. Even if a subfolder contains almost nothing, it still means we have around <strong>750 terragrunt.hcl files !</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FRHOwWKH5OY7buuGHNi%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Flaff-tv-comedy-himym-how-i-met-your-mother-RHOwWKH5OY7buuGHNi&amp;image=https%3A%2F%2Fmedia4.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExa3hxMDhienFsamtrejVhd2Yya3U2MTNxajZsaDU5MTJteHh2ZGVmeSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FRHOwWKH5OY7buuGHNi%2Fgiphy_s.gif&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="435" frameborder="0" scrolling="no"><a href="https://medium.com/media/8dabc56d00551adb23aa9ccd6c5ef053/href">https://medium.com/media/8dabc56d00551adb23aa9ccd6c5ef053/href</a></iframe><p>If we want to modify even a small part of these files, it’s still a tremendous amount of work and that would greatly reduce the adoption of the self-service as we conceived it.</p><p>We designed a small command-line interface called “infracli” to make this procedure as simple as possible while reducing code duplication. More on this command in the second article of this series :)</p><p>We are <a href="https://www.lumapps.com/company/job-offers/">hiring</a> :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f37660b4ed95" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/terraform-at-lumapps-part-1-f37660b4ed95">Terraform at LumApps: Part 1</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to create a lean, tree-shakeable React Design System library]]></title>
            <link>https://medium.com/lumapps-engineering/how-to-create-a-lean-tree-shakeable-react-design-system-library-1fe4c72da01b?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/1fe4c72da01b</guid>
            <category><![CDATA[react]]></category>
            <category><![CDATA[design-systems]]></category>
            <category><![CDATA[ecmascript]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[rollup]]></category>
            <dc:creator><![CDATA[Guillaume Cornut]]></dc:creator>
            <pubDate>Thu, 15 Jul 2021 12:04:24 GMT</pubDate>
            <atom:updated>2021-07-15T12:04:24.854Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Co-authored with: </em><a href="https://medium.com/u/43e36ed87a2e"><em>Pierre Dupuis</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sfcoUUpw6B-XTyqF" /><figcaption>The big rock (our old monolithic UMD bundle) and the small tree (our new small tree-shakeable lib)</figcaption></figure><h3>Introduction</h3><h4>LumApps Design System</h4><p>At LumApps, we are developing LumX, our own design system which is a central repository containing not only our components but also our design patterns and guidelines. This is a huge project that dates back to the end of 2018, aiming to replace our home-made UI Kit while doing so much more in the process.</p><p>If you want to know more about this project, you should take a look at our article on how we started building a design system <a href="https://medium.com/lumapps-engineering/how-we-started-building-a-design-system-at-lumapps-f360fcc9a8ce">[1]</a>.</p><p>Since our UI Kit was in AngularJS and since we are currently migrating our entire product from AngularJS to React, we do need to maintain our Design System in both these frameworks. Our road to React is a great story itself, check it out in our dedicated article <a href="https://medium.com/lumapps-engineering/lumapps-road-to-react-ee0cc96af26e">[2]</a>.</p><p>For the purpose of this introduction, let’s assume we are in 2020. It has been almost two years since the beginning of the project and our Design System is still in a 0.x alpha version. Up until this moment, we have developed a lot of components and we could continue to add more, but since our partners needed to use our Design System, we needed to release its first stable version.</p><p>Apart from having to finish some components, to improve the overall codebase and to be more accessible, we need to improve our performance. One way of doing it is to improve our bundling configuration.</p><h4>What was wrong with our configuration</h4><p>At the time of the 0.x alpha version, our Design System setup is not that bad. Since our component library is developed using multiple javascript frameworks, our Design System follows a monorepo architecture with separated independent packages for each framework and one for our demo site.</p><p>We decided to use Typescript soon enough to have our entire codebase typed. And we can benefit from a pretty neat storybook environment to ease our development.</p><p>The architecture is pretty clean and the development environment is efficient, so what could be wrong? Our bundling configuration. We had a complex webpack configuration leading to <strong>a single browser-ready UMD (Universal Module Definition) bundle full of polyfills.</strong> This is doubly wrong because we knew we would never need to directly import LumX into a browser and we need to handle the browser compatibility at an application level already.</p><p>Also, the monolithic UMD bundle meant that someone wanting to only use our button component using the following syntax would end up having our entire Design System in their app bundle.</p><h4>Tree-shaking in javascript</h4><p>When it comes to building a Javascript application, there are a lot of different module formats available from CJS (CommonJS) to ESM (ES2015 Modules) through UMD (Universal Module Definition).</p><p>We won’t explain every specificity of those modules. The important thing to know is that ESM modules can be statically analyzed to allow build tools (like Webpack or Rollup) to perform tree-shaking on the code — tree-shaking being a process that removes unused code from bundles <a href="https://code-trotter.com/web/understand-the-different-javascript-modules-formats/">[3]</a>. This means that, in the case previously stated, if someone wants to use our button component using the same syntax as before, only this component would end up in their app bundle.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6f593acd303530097198aef7d8bd038d/href">https://medium.com/media/6f593acd303530097198aef7d8bd038d/href</a></iframe><p>Another approach would have been to create one bundle (UMD for example) for each component or resource we want to expose. This solution, used by Lodash and MaterialUI for instance, can be a good alternative to ESM modules if you want your modules to be compatible with every browser without transpilation.</p><p>However, this solution requires a complex bundle configuration to avoid including duplicate code inside the different micro-bundles and it asks the developer to be more precise during the import like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5c11bdcbb1d80b9c7e9e9ab446c134d2/href">https://medium.com/media/5c11bdcbb1d80b9c7e9e9ab446c134d2/href</a></iframe><p>We decided to go with the tree-shakeable ESM solution. If you want to learn how to create a tree-shakeable component library step by step, you should have a look at the amazing resource that helped us a lot along the way <a href="https://dev.to/lukasbombach/how-to-write-a-tree-shakable-component-library-4ied">[4]</a>.</p><h3>What did we want to achieve</h3><p>Our goal was pretty straight-forward: we wanted to move from our ugly browser-ready UMD bundle full of polyfills to the <strong>smallest possible tree-shakeable ESM library</strong>.</p><p>As we are developing in TypeScript <strong>we do need some transpilation</strong> but we want to keep ESM modules for tree shaking and reduce polyfills.</p><p>Also, and this objective will have a direct impact on our bundle configuration, we need to be able <strong>to decide precisely what we want to expose</strong> as the API of a library should be kept under surveillance for potential breaking changes that we absolutely want to avoid after releasing the first version.</p><p>Finally, we want to watch what libraries are bundled with our lib and <strong>avoid re-exporting libraries</strong> commonly used outside the lib.</p><h3>Methods</h3><h3>Rollup, ESM and React-TS transpilation</h3><p>As we need to continue to use advanced bundling configuration for the Lumapps app (using Create React App and Webpack) we would rather keep the bundling of the LumX project as simple as possible.</p><p>As such, we chose to migrate from Webpack to Rollup because it is more suitable for ESM libraries by design and was simpler to configure for our needs. It tends to be pretty common to use Rollup for libraries and Webpack for apps <a href="https://medium.com/webpack/webpack-and-rollup-the-same-but-different-a41ad427058c">[5]</a>.</p><p>The project being a React component library written in TypeScript, we decided to use a simple babel transpilation config with `<em>@babel/preset-env`</em> for basic EcmaScript transpilation and browser polyfills, `<em>@babel/preset-react`</em> for React TSX/JSX transpilation and of course `<em>@babel/preset-typescript`</em> for TypeScript transpilation.</p><p>Concerning polyfills, we decided to reduce them at the minimum with the Browserslist’s default browsers (&gt; 0.5%, last 2 versions, Firefox ESR, not dead) <a href="https://github.com/browserslist/browserslist#full-list">[6]</a>.</p><p>The resulting rollup configuration goes as followed:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/422e86505c668bc9296625d9e681c63e/href">https://medium.com/media/422e86505c668bc9296625d9e681c63e/href</a></iframe><h3>Splitting the output chunks</h3><p>For an even better tree-shakeable library, we decided to also split the generated bundle in multiple files. One file per component.</p><p>To do this, we list all component files and add them in the rollup input indexed by component name:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d764b592525e1111664c54ee3f1b4dd/href">https://medium.com/media/5d764b592525e1111664c54ee3f1b4dd/href</a></iframe><p>By doing that, we also created multiple chunks in output but we want to only expose the `<em>@lumx/react`</em> module to the outside world. The chunks for each component and the chunks automatically generated by rollup should not be accessible.</p><p>One simple solution was to rename all the chunks that are not the root <em>index</em> module so that they are moved to an `<em>_internal`</em> directory which will discourage users of the library from importing them in their code.</p><p>The code for the output files is the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/266c826f6157c17fa56e7b7b8c245f4e/href">https://medium.com/media/266c826f6157c17fa56e7b7b8c245f4e/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dhk-BK5x9Wra4pP5" /><figcaption>Well chopped up ESM modules!</figcaption></figure><h3>Avoiding re-exporting libraries</h3><p>One trick we also used to reduce the generated bundle size was to look at the libraries we imported in LumX that we also used in the main application and externalize them.</p><p>The obvious library that does not need to be re-exported is, of course, `<em>react`</em> and `<em>react-dom`</em>. We previously already excluded react from the bundle but we pushed this further by also externalizing `<em>lodash`</em> and `<em>moment`</em> from the bundle.</p><p>To externalize these dependencies, the cleanest way is to declare them as peer dependencies in the `<em>package.json</em>`:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/15bb344f15ef63a2a09619eb0dba262c/href">https://medium.com/media/15bb344f15ef63a2a09619eb0dba262c/href</a></iframe><p>Moving dependencies in the `<em>peerDependencies`</em> field of the `<em>package.json`</em> prevents them to be installed automatically when installing LumX and can be useful to update these libraries without having to update them in LumX first.</p><p>One down side of this is that the dependencies are not installed when working in the LumX repository. A solution to this is to add the <a href="https://www.npmjs.com/package/npm-install-peers">npm-install-peers</a> util in the `<em>prepare`</em> npm script that only executes after a `<em>npm install</em>` or `<em>yarn install</em>` locally and not in the project in which you import LumX.</p><p>Example of use in the `<em>package.json</em>`:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/62d20a1a1e1d5da1bc51d019e2349e8d/href">https://medium.com/media/62d20a1a1e1d5da1bc51d019e2349e8d/href</a></iframe><p>Then, to externalize these peer dependencies from the bundle, you can use this rollup plugin:</p><p>import `<em>peerDepsExternal</em>` from `<em>rollup-plugin-peer-deps-external`</em>;</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6478008199499f7bcc07a8876c47c9a5/href">https://medium.com/media/6478008199499f7bcc07a8876c47c9a5/href</a></iframe><h3>Exporting type definitions</h3><p>With the previously described configuration we can publish an ESM module in which we choose what to expose and how. But with the transpilation we also lost the TypeScript type definitions that we would want to publish in the NPM package alongside the ESM module.</p><p>If you just generate declaration files with the typescript compiler, you will get a declaration file for each TypeScript file in the repository. But, like mentioned before, we only want to expose one module, the `<em>@lumx/react`</em> module.</p><p>To do this, we used the <a href="https://www.npmjs.com/package/dts-bundle-generator">dts-bundle-generator</a> utility and configured it as a `<em>postbuild`</em> npm script in the `<em>package.json`</em>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/42c4c37b31349be3da151d8d0ebd0541/href">https://medium.com/media/42c4c37b31349be3da151d8d0ebd0541/href</a></iframe><h3>Results</h3><p>We compared the `<em>dist`</em> folder content and size before and after the new bundling configuration and the results speak for themselves.</p><p>The previous UMD bundle looked like this:<br><a href="https://bundlephobia.com/result?p=@lumx/react@0.28.1">https://bundlephobia.com/result?p=@lumx/react@0.28.1</a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b24e3e0d5626894523c513c6377fc350/href">https://medium.com/media/b24e3e0d5626894523c513c6377fc350/href</a></iframe><p>The final tree-shakeable ESM bundle looks like this:<br><a href="https://bundlephobia.com/result?p=@lumx/react@1.0.0-alpha.5">https://bundlephobia.com/result?p=@lumx/react@1.0.0-alpha.5</a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6599e90c02bb2b1532b02a059e00f53d/href">https://medium.com/media/6599e90c02bb2b1532b02a059e00f53d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/659/1*jd01YKoXBR3PRiO9KGWxyQ.png" /></figure><h3>References</h3><ol><li><a href="https://medium.com/lumapps-engineering/how-we-started-building-a-design-system-at-lumapps-f360fcc9a8ce">https://medium.com/lumapps-engineering/how-we-started-building-a-design-system-at-lumapps-f360fcc9a8ce</a></li><li><a href="https://medium.com/lumapps-engineering/lumapps-road-to-react-ee0cc96af26e">https://medium.com/lumapps-engineering/lumapps-road-to-react-ee0cc96af26e</a></li><li><a href="https://code-trotter.com/web/understand-the-different-javascript-modules-formats/">https://code-trotter.com/web/understand-the-different-javascript-modules-formats/</a></li><li><a href="https://dev.to/lukasbombach/how-to-write-a-tree-shakable-component-library-4ied">https://dev.to/lukasbombach/how-to-write-a-tree-shakable-component-library-4ied</a></li><li><a href="https://medium.com/webpack/webpack-and-rollup-the-same-but-different-a41ad427058c">https://medium.com/webpack/webpack-and-rollup-the-same-but-different-a41ad427058c</a></li><li><a href="https://github.com/browserslist/browserslist#full-list">https://github.com/browserslist/browserslist#full-list</a></li><li><a href="https://github.com/lumapps/design-system/blob/v1.0.0-alpha.5/packages/lumx-react/rollup.config.js">https://github.com/lumapps/design-system/blob/v1.0.0-alpha.5/packages/lumx-react/rollup.config.js</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1fe4c72da01b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/how-to-create-a-lean-tree-shakeable-react-design-system-library-1fe4c72da01b">How to create a lean, tree-shakeable React Design System library</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implicit ALS for community recommendation]]></title>
            <link>https://medium.com/lumapps-engineering/implicit-als-for-community-recommendation-77510dbe1844?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/77510dbe1844</guid>
            <category><![CDATA[sparkml]]></category>
            <category><![CDATA[als]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Jean-Baptiste Delafosse]]></dc:creator>
            <pubDate>Fri, 23 Apr 2021 08:52:51 GMT</pubDate>
            <atom:updated>2021-04-23T08:52:51.031Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BVe-65YCBAH8XGrt" /><figcaption>Photo by <a href="https://unsplash.com/@dylandgillis?utm_source=medium&amp;utm_medium=referral">Dylan Gillis</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>At Lumapps, we believe horizontal communication can have a massive impact on people and organization. It helps people learn more, faster and better. It makes innovation easier by encouraging interdisciplinary cooperation. It helps organizations do what is important for their customers more efficiently.</p><p>We implemented this idea within Lumapps in what we call “Communities” where individuals can communicate and collaborate within their organization in any way they deem necessary.</p><p>We’ve seen great use cases for communities implemented by our customers : from the usual community of practice that foster creativity and expertise to community about people that want to talk about their pets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/279/0*fi_Ybi-02SaEei9W" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/0*ocB2eg1dBWXCC7Ov" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/285/0*oOgLTAfEZ09TpoTh" /><figcaption>Many kind of communities exists within lumapps</figcaption></figure><p>Having too many communities on your intranet can be a burden though : users can experience fear of missing out. A recommendation system for communities is one of the many solutions that could help. This system should be able to help users discover new communities they might like.</p><p>We wanted to assess how hard it would be to implement such a recommendation system using Spark MLLib. While the math is fairly straightforward, we found some roadblocks along the way that we wanted to share.</p><h3>Starting the project</h3><p>After taking a look at the state of the art for recommendation algorithms, We quickly decided to use a collaborative filtering approach using the Alternating Least Square (ALS) method. This algorithm suits our needs:</p><ul><li>Fast to put in place when compared to content-based approach</li><li>Well documented</li><li>Already tested with success for many use cases</li></ul><p>What is great about collaborative filtering is that you only need data that tells you a user is interested in an item: in our case, a community.</p><p>We don’t really have reliable explicit feedback from users on a community in Lumapps: you can’t give a 5 star review to a community. We do have implicit feedback metrics already in place though. We used the number of views of a user on a community as an implicit feedback of his interest. The more views a user has on a community, the more likely he is to be interested in that community.</p><p>We could start the project.</p><h3>Evaluating the model</h3><p>Evaluating the model and tuning its hyperparameters was harder than anticipated.</p><p>All <strong>precision based metrics </strong><em>(exemple: Mean Square Error)</em><strong> </strong>make no sense with implicit feedback as the input label represents “Views” and the output label is a preference calculated by the ALS. One could argue we could scale views and preferences from 0 to 1 but that would be hacky at best. It should also be noted that precision based metrics requires knowing which items were disliked which is not possible in an implicit feedback situation : a workaround could be done by adding randomly user-community tuples with 0 views.</p><p><strong>Binary Classification metrics</strong> (exemple: Accuracy) cannot be used directly either, as we don’t output a <em>class</em> but a <em>preference</em>. We could convert that preference to a class by defining a threshold above which the recommendation system recommends a community or not. The threshold itself could be considered a hyperparameter. Even doing so, binary classification methods are hacky because we still don’t know which items were disliked.</p><p><strong>Recall based metrics</strong> such as ranking oriented ones are the best for such a recommendation system. We wanted to use a Ranking Evaluator conjointly with cross validation to find the right hyperparameters. If you want a Ranking Evaluator in SparkMl that is directly usable with cross validation and ALS, you need to implement it yourself.</p><p>At first, we tried the usual implementation : rank the communities recommended for a user and compare it to the ones the user actually viewed. We quickly came to the conclusion that it was not the right approach. Our users were interacting with very few communities. Let’s say “John” interacted with 3 communities. If you perform a 3-fold cross-validation, you are evaluating your rank metric on 2 communities at best.<br>Our second approach was building a ranking metric of users for a given community. For a given community, we rank users based on the number of times they viewed the community. We then turn the model upside down by making it predict users for a community instead of communities for user. It works well because each community is viewed by a lot of users.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/840f234e94864d0fb02b305a1b42f6a1/href">https://medium.com/media/840f234e94864d0fb02b305a1b42f6a1/href</a></iframe><p>Knowing we were going to use the previous Rank Based Metrics. We decided to stratify our whole dataset by community before performing a 80%/20% split for testing purposes.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/68a3620d99214da3a3dfde965dbce1c5/href">https://medium.com/media/68a3620d99214da3a3dfde965dbce1c5/href</a></iframe><p>In the end our dataset is used is the following way:</p><ul><li>Test Set : A 80%-20% split stratified by communities</li><li>Validation: through a 3-Fold Cross Validation</li></ul><h3>Results</h3><p>Despite all these roadblocks, we managed to train a decent recommendation system on our own Lumapps deployment (dogfooding at its finest).</p><p>While the algorithm was not using data regarding the business unit of users, we definitely observed a correlation between the latent variables learned and the business units of users. It was expected : a lot of communities within Lumapps are work related. The biggest separation we observed at Lumapps in community consumption is between the “Sales” and the “Engineering”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aT4moPJxR1VM1sdoKYYdOw.png" /><figcaption>Engineering and Sales team use different community and are separated by the ALS</figcaption></figure><p>Our UX team also conducted user interviews with a selected customer that showed that the recommendations provided by such a system would be a great addition to our product.</p><h3>Conclusion</h3><p>We want to give our users great community recommendations so they can discover what their peers are up to.</p><p>The collaborative filtering approach available natively in Spark is a great way to build such a system as it provides horizontal scalability out of the box. We had yet to test how it behaves on production data : it’s now done.</p><p>We faced a few challenges when using this algorithm : evaluating the recommendation system was not straightforward with the data we had. We found an appropriate solution by implementing a Ranking Evaluator that ranks users instead of communities.</p><p>The results observed are consistent with what was expected: we found that the user factors learned were highly correlated with business units and that the community factors are correlated with the topic of each community.</p><p>User interviews performed on the field by our UX team confirm the recommendation system could be of great value for our users.</p><p>We also realised ALS could be used as a decent feature engineering method before applying other models.</p><p>The next step is to bring this POC to production 🚀</p><h3>References</h3><ul><li><a href="https://stats.stackexchange.com/questions/226825/what-metric-should-i-use-for-assessing-implicit-matrix-factorization-recommender">https://stats.stackexchange.com/questions/226825/what-metric-should-i-use-for-assessing-implicit-matrix-factorization-recommender</a></li><li><a href="https://stackoverflow.com/questions/46462470/how-can-i-evaluate-the-implicit-feedback-als-algorithm-for-recommendations-in-ap/46490352#46490352">https://stackoverflow.com/questions/46462470/how-can-i-evaluate-the-implicit-feedback-als-algorithm-for-recommendations-in-ap</a></li><li><a href="https://towardsdatascience.com/large-scale-jobs-recommendation-engine-using-implicit-data-in-pyspark-ccf8df5d910e">https://towardsdatascience.com/large-scale-jobs-recommendation-engine-using-implicit-data-in-pyspark-ccf8df5d910e</a></li><li><a href="https://vinta.ws/code/build-a-recommender-system-with-pyspark-implicit-als.html">https://vinta.ws/code/build-a-recommender-system-with-pyspark-implicit-als.html</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=77510dbe1844" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/implicit-als-for-community-recommendation-77510dbe1844">Implicit ALS for community recommendation</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Notification center : Comment Lumapps a retravaillé la fonctionnalité vers les standards actuels]]></title>
            <link>https://medium.com/lumapps-engineering/notification-center-comment-lumapps-a-retravaill%C3%A9-la-fonctionnalit%C3%A9-vers-les-standards-actuels-7d99bf1bd78e?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/7d99bf1bd78e</guid>
            <category><![CDATA[project-management]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[notification-center]]></category>
            <category><![CDATA[rework]]></category>
            <category><![CDATA[front-end-development]]></category>
            <dc:creator><![CDATA[Victor Tram]]></dc:creator>
            <pubDate>Wed, 17 Feb 2021 13:18:34 GMT</pubDate>
            <atom:updated>2021-02-17T13:18:34.484Z</atom:updated>
            <content:encoded><![CDATA[<h3>Notification center : Comment Lumapps a retravaillé la fonctionnalité vers les standards actuels</h3><p>Les notifications dans une application web sont de nos jours une fonctionnalité cruciale. Dans LumApps, elles permettent à l’utilisateur de se tenir au courant des dernières informations d’entreprise, des nouveaux posts de communautés ainsi que de leurs activités liées, comme les commentaires ou les réactions.</p><p>La fonctionnalité des notifications était initialement limitée sur LumApps. Elle permettait uniquement de voir la notification et cliquer dessus pour atteindre le contenu concerné, puis la notification disparaissait.</p><p>Dans le cadre du travail fait sur la partie du header du produit (aussi appelé Topbar) qui devait évoluer techniquement, nous avons également repensé le centre de notifications, lui donnant à la fois un nouveau design et de nouvelles actions possibles, plus en phase avec ce qui est attendu de nos jours.</p><p>Cet article revient sur le déroulement du développement de ces nouvelles évolutions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*93yEctGM4H4SxzTr" /><figcaption>Le header (ou Topbar), en haut</figcaption></figure><h3>Feature team, task force team</h3><p>LumApps fonctionne en Feature team, c’est-à-dire que chaque équipe gère l’ensemble d’une ou de plusieurs fonctionnalité(s) au sein de l’application LumApps, sur l’ensemble des processus et couches techniques, (conception, design, développement backend, développement frontend, QA…). Chaque équipe (aussi appelé pod) est responsable de sa partie du code, qui peut cohabiter avec celui des autres pods, contrairement à une organisation en composants où chaque fonctionnalité métier est gérée de manière transverse par toutes les équipes tout en restant techniquement autonomes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DuxC9kKspwj8z4iKZ9yPAw.png" /></figure><p>Dans un premier temps, le besoin était de refondre visuellement et techniquement l’ensemble du header Topbar, afin de le rendre plus rapide au chargement et légèrement remis au goût du jour, sans le changer fonctionnellement. Le centre de notifications faisant partie intégrante du header, il est également inclus dans ce travail de refonte.</p><p>La majorité du travail se faisait au niveau du design et de la couche front-end ; il n’était donc pas nécessaire de mobiliser un pod entier, d’autant plus que la top bar contient des fonctionnalités gérées par plusieurs équipes. Une “task force” a donc été créée, regroupant plusieurs designers et développeurs de différents pods.</p><p>Dans un second temps, une fois le nouveau header dans un état stable, un travail de refonte fonctionnelle pour la partie du centre de notifications a été prévu. Le besoin était de laisser l’utilisateur gérer plus simplement ses notifications :</p><ul><li>Pouvoir les marquer comme lues/non lues</li><li>Les filtrer selon leurs statuts et retrouver les contenus plus tard</li><li>Et améliorer la réactivité générale de l’interface.</li></ul><p>Cette partie nécessitant un travail backend et n’étant pas directement en lien avec les objectifs de la task force, elle a été gérée par le pod social, responsable de la fonctionnalité du centre de notifications.</p><h3>Gestion de projet</h3><p>Le cycle de développement de LumApps au moment de cette refonte se décomposait en releases toutes les 6 semaines, elles même décomposées en 3 sprints de 2 semaines. La raison pour laquelle nous avons décidé de diviser l’évolution du centre de notifications en deux étapes est parce que d’un point de vue gestion de projet, la refonte du header était prioritaire par rapport à l’ajout de nouvelles fonctionnalités. Il était également plus simple et moins risqué pour la release de diviser le travail en faisant la refonte technique et visuelle tout en gardant un fonctionnel identique d’abord, puis ensuite d’ajouter les nouvelles fonctionnalités dans la release suivante. De plus, les éléments techniques comme le <a href="http://design.lumapps.com/">le design system Lumx</a> ont pu être adaptés et réutilisés, il n’y a donc pas eu trop de développements d’états intermédiaires amenés à être remplacés par la suite.</p><p>Le projet s’est ainsi divisé de la manière suivante :</p><ul><li>En premier lieu, la task force a été chargée de faire évoluer le header Topbar dans son ensemble, centre de notifications compris, en reproduisant son fonctionnel existant.</li><li>Environ deux sprints avant la release du travail de la task force, les développeurs backend du pod social ont entamé le travail préparatoire nécessaire aux évolutions fonctionnelles prévues pour la release suivante.</li><li>La première release sort ainsi avec le centre de notification migré visuellement et techniquement, et le backend prêt pour son évolution fonctionnelle.</li><li>Les trois sprints suivants, les développeurs frontend du pod social ont pu s’attaquer à la refonte fonctionnelle, en reprenant le travail de la task force et en s’appuyant sur le travail backend déjà réalisé.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y7nM5qLCZ1tCCXIVQkeMVw.png" /></figure><p>Une gestion de projet la sorte n’est pas évidente à gérer, avec plusieurs équipes et plusieurs sprints en ligne de mire. Le travail a été conséquent et plusieurs points de difficultés ont dû être surmontés, notamment :</p><ul><li>Le travail de la task force nécessitait un travail important, et on a préféré privilégier la qualité à une date fixée de release, ce qui a décalé celle-ci d’une release par rapport à ce qui était prévu, et de fait, a également décalé le planning de la refonte fonctionnelle.</li><li>Le travail backend en amont était nécessaire mais en raison du timing et des nouveaux composants créés côté frontend, il était difficile de le tester pendant son développement, ce qui fait que des bugs backend ont été remontés tardivement.</li><li>En raison de conflits de plannings, la migration technique et visuelle de la task force a été réalisée par des développeurs frontend qui ne venaient pas du pod social, et qui n’étaient pas amenés à travailler sur son évolution fonctionnelle ; un transfert de savoir a été nécessaire, ce qui prend forcément du temps supplémentaire.</li></ul><p>Néanmoins, dans l’ensemble, le travail réalisé à été à la hauteur de ce qui était attendu.</p><h3>Release 1 : Refonte du header Topbar et du centre de notifications</h3><p>Le header historique a été développé en angularjs, et afin de le migrer dans la nouvelle architecture en React, l’équipe Task force “Frontend Core Topbar” a été spécialement constituée pour que des développeurs normalement de différents pods travaillent à temps plein sur ce chantier. Parmi les tâches de cette refonte, se trouvait celle de la partie centre de notifications. Le travail de migration était double :</p><ul><li>Technique pour migrer le code historique angularJS vers React</li><li>Visuel pour avoir une première mise à jour du style, plus en phase avec le nouveau header.</li></ul><p>La migration d’angularjs vers React nécessite une réécriture complète du code, car la structure même du code est totalement différente entre ces frameworks. Ce travail a été effectué par des développeurs non membres du pod social, pour des raisons de planning.</p><p>Bien que non idéal, cela n’a pas posé de problème majeur car ils avaient la connaissance technique nécessaire pour le faire ; c’est l’avantage des feature teams, même en faisant partie d’une autre équipe, ayant travaillé la même couche technique, l’appropriation du code est simplifiée.</p><p>Visuellement, le design a été repensé pour utiliser la bibliothèque de composants React open source créé par LumApps : <a href="http://design.lumapps.com/">le design system Lumx</a>. Les changements peuvent paraître mineurs mais ceci facilite également le développement et la maintenance du produit car elle évite de créer des composants à utilisation unique.</p><p>Finalement, le résultat est fonctionnellement identique à l’ancien centre de notifications, tout en ayant été adapté à un nouveau design. Sous le capot, la migration vers React permet également une meilleure maintenance future, et une performance accrue, deux objectifs qui ont initialement motivé le travail de refonte.</p><p>Cette évolution a été intégrée dans la première release, avec l’ensemble de la refonte du header Topbar. Le tout a pu être testé en interne, afin de pouvoir l’ajuster et fixer les bugs dans la release suivante, avant son déploiement dans les environnements clients.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*xenr_4qwtMlBpG5t3gepwA.png" /><figcaption>Avant/Après : Centre de notifications originel et centre de notifications dans le nouveau header, pour la Release 1</figcaption></figure><h3>Release 2 : Nouvelles fonctionnalités dans le centre de notifications</h3><p>Pour la release suivante, le besoin principal était de faire évoluer le centre de notifications vers des standards plus actuels : au lieu d’avoir uniquement une liste de notifications qui disparaissaient au clic, nous voulions ajouter un système de notifications lues ou non lues, qui permettaient de garder la notifications visible même après la visite du contenu. L’utilisateur pourra dorénavant faire la distinction entre marquer comme lu, marquer comme non lu et supprimer entièrement la notification. Il pourra également tout marquer comme lu ou tout supprimer, et pourra filtrer la liste pour n’afficher que les notifications non lues. Enfin, le centre de ne comptera pas la totalité des notifications (information non utile), mais uniquement celles non lues.</p><p>Les différentes tâches nécessaires à cette évolutions étaient les suivantes :</p><ul><li>Côté backend, faire évoluer le service pour ajouter un statut aux notifications pour distinguer entre non lu, lu et supprimé, et migrer les notifications déjà existantes en base pour les rendre compatibles avec le nouveau format, tout en restant utilisables par le système actuel, pour éviter une rupture de service.</li><li>Préparer un script qui va sanitiser la base de données actuelle, pour ne plus garder les notifications supprimées après un certain temps, pour des raisons de performances.</li><li>Côté frontend, depuis les changements faits par la task force, faire encore évoluer visuellement les notifications pour qu’elles suivent les nouvelles maquettes.</li><li>Ajouter les nouvelles fonctionnalités prévues en retravaillant les composants existants ou en créant de nouveaux, puis en les branchant au backend.</li></ul><p>Les tâches backend ont été faites par le pod social en même temps que la refonte du header, lors de la release précédente, puis mis en attente pour son utilisation dans la release suivante. Quant aux tâches frontend, elles n’ont pas été réalisées par les mêmes développeurs que pendant la refonte du header, mais grâce à des standards de développement communs ainsi qu’à l’utilisation du design system, la transition s’est faite sans accroc majeur.</p><p>Il y avait encore beaucoup de travail à abattre. Néanmoins il a pu être réalisé dans les temps et trois sprints plus tard, le nouveau centre de notifications a donc été amélioré pour permettre tout ce qu’on peut attendre d’un centre de notifications moderne. Ces évolutions ont pu donc être comprises dans la release suivant celle du nouveau header, 6 semaines plus tard, et seront déployées chez les clients au fur et à mesure que les clients adoptent le nouveau header Topbar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*MlqBGReRO2Tuoe8wIlyHlw.png" /><figcaption>Avant/Après : Centre de notifications du nouveau header Release 1 (sans évolution) et centre de notification Release 2 (avec nouvelles fonctionnalités)</figcaption></figure><h3>Evolutions futures</h3><p>Le chantier de la refonte du centre de notifications est assez conséquent. Le code ayant été changé en backend et totalement réécrit en frontend, la stratégie de déploiement a été donc en premier lieu de ne pas imposer cette nouvelle refonte aux clients, mais de le tester intensivement en interne, sur l’intranet LumApps. Un certain nombre de bugs et petites améliorations ont ainsi pu être corrigés avant le déploiement chez les clients.</p><p>Certaines petites améliorations ont ainsi été ajoutées après la fin du cycle de développement :</p><ul><li>Une version optimiste des fonctionnalités tout marquer comme lu et tout supprimer, qui sont effectifs immédiatements au lieu d’attendre le retour du backend</li><li>Un peaufinement des états de chargements lors du passage entre toutes les notifications et les notifications non lues uniquement</li><li>Quelques mises à jour visuelles et mises à jour de texte</li><li>Ou encore une amélioration des performances backend.</li></ul><p>Et d’autres possibles évolutions futures pourraient être à venir, par exemple :</p><ul><li>Avoir plus de contrôle et de finesse sur les notifications envoyées</li><li>Avoir des actions contextuelles sur les notifications (par exemple, aller directement sur la zone Commentaires, accepter la demande d’un membre à rejoindre une communauté, etc.)</li><li>Avoir des notifications en temps réel, grâce à l’utilisation de websockets.</li></ul><p>Après 2 mois de travail, et l’apport de plusieurs équipes, le nouveau centre de notifications est maintenant plus en phase avec un centre de notifications moderne. Les premiers retours lors de la release interne pour les employés LumApps sont très positifs, et nous espérons que l’adoption de LumApps sera améliorée en partie grâce au centre de notifications lors du déploiement chez les clients.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d99bf1bd78e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/notification-center-comment-lumapps-a-retravaill%C3%A9-la-fonctionnalit%C3%A9-vers-les-standards-actuels-7d99bf1bd78e">Notification center : Comment Lumapps a retravaillé la fonctionnalité vers les standards actuels</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Developing a new product quickly in a fast pace environnement]]></title>
            <link>https://medium.com/lumapps-engineering/developing-a-new-product-quickly-in-a-fast-pace-environnement-287679dcccd0?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/287679dcccd0</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[teamwork]]></category>
            <category><![CDATA[mobile]]></category>
            <dc:creator><![CDATA[Thomas Lombard]]></dc:creator>
            <pubDate>Tue, 19 Jan 2021 09:14:17 GMT</pubDate>
            <atom:updated>2021-01-19T09:14:16.993Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*spvWekFZSsjlBURIQdfH-w.jpeg" /></figure><h3>Introduction</h3><p>In June 2020, we started to hear about a new mobile application in a strategic meeting at LumApps. This new app was intended to target a new kind of users.</p><p>It involves new features that have never been done at LumApps and also a short deadline, end of September (that is to say 3 months only to develop a new application!).</p><p>During our development process we have strong standards such as units tests, reviews, testing to always have a good quality in our releases. In this specific scenario the main objective was the short deadline so we had to use some shortcuts while keeping a good quality.</p><p>At the end, we managed to have an MVP at the end of September while keeping (almost) all our standards. This article will cover different topics which can explain how we were able to do it. Throughout this article, I’ll never mention iOS or Android platform even if, for different reasons, we’re not at the same level on every subject we do have the same philosophy and goals.</p><h3>CodeBase</h3><p>Probably the most important point is the codebase. If you’re working on a project with strong technical debt or/and no defined architecture there is no chance for you to get things done quickly. You’ll face too many difficulties such as regression, strong dependencies, duplicate code which will get in your way and slow you down or even prevent you from doing the enhancement or feature you want to achieve.</p><p>From the beginning of the mobile applications development (3 years ago), architecture, clean code and refactor have always been our main priorities. We’re proud after all this time to have a quality project which allows us to implement new features easily without affecting the rest of the project. Every team member is involved in this task and we keep improving our architecture as the project grows.</p><h3>Team</h3><p>In the mobile team, we are lucky enough to have mostly senior developers which is very helpful to keep a clean project thus not mandatory. There is some very important subject that shouldn’t be underestimated.</p><p>Although you might come from different companies, might have learned different frameworks, architecture, coding patterns. All those experiences are useful only if all team members are going in the same direction. If anyone is coding on its own without paying attention to what has been done or should be done, your team is wasting a lot of time and energy.</p><p>In the mobile team, we have a defined architecture (<a href="https://medium.com/lumapps-engineering/how-we-ensure-long-term-quality-for-ios-apps-series-1-3-9f87d6444819">don’t hesitate to read this article from last year</a>) that we always follow. We review each other all the time to avoid duplicate code, to make sure that all our standards are applied and just to be up-to-date of what the others are doing. It could seem a bit tight. It’s just the opposite, we’re always discussing how we can make our project better, what new tools can we use from update we get from Google/Apple and we look for better solutions as our project keep getting more and more features.</p><p>It’s impossible to have the perfect architecture or coding patterns because it depends on a lot of elements such as the complexity of the application you’re working on, the size and experience of your team, the time you have to achieve your feature and much more. But it’s important to keep in mind that your project should always evolve and you should solve issues you can have before it’s too late and you can’t turn back. Of course, this should be done by discussing with your team members.</p><h3>Minimise human mistakes</h3><h4>CI</h4><p>We have been using continuous integration for a while now and it helps us for different tasks. If you want more technical details about it you can check <a href="https://medium.com/lumapps-engineering/how-we-ensure-long-term-quality-for-ios-apps-series-2-3-6e1ca5c48d06">this article</a>.</p><p>We now have multiple targets on the same project and it can be difficult to maintain. Every time we have a new PR, GitHub actions check that every test pass and that all targets are compiling without any errors. It did prevent us to merge a PR with a target not compiling a couple (and more) of times.</p><p>We now give the possibility for a client to have a specific logo for their app and being able to deploy it without using LumApps generic one. From a mobile point of view, it is a difficult task because it means having to deploy every custom app and also to do it every time we want to update to have the latest features. It would be impossible to do it without a CI process which we now have!</p><p>CI requires a lot of investments but it is a big asset for us and it keeps involving with our project.</p><h4>Unit tests</h4><p>We always see unit tests has a great tool to develop. It’s right, unit tests help you to have flexible code (if you can’t easily test a specific part of your code you’re probably having too many dependencies which lead to rigidity), spend less time debugging and can also explain a complex algorithm just by reading all different tests scenarios.</p><p>But we often forget that it’s one of the best tools to avoid regressions. If you want to refactor a part of your project, how can you be sure at the end that everything is working as it was before if you don’t have tests on it? There is no way you can anticipate all behaviours of a class or a feature if it’s not 100% tested. Because we’re working on a product which is available for many years now and will be for years to come, regression is probably our worst enemy and units tests (and even UI tests) are the tools we must have.</p><h3>Keep a good focus</h3><p>One of the biggest challenges we faced during this project development was the unknown. We had to quickly start development to catch up with the concurrency and this was contradictory with the fact that we weren’t sure of what we wanted…</p><p>We started by focusing on one of the main features: Chat. After a few days trying the first SDK, we quickly switched to an other service which is the SDK we use on the MVP. On this feature we managed to progress quickly thanks to the documentation and the SDK which had all the features we needed, it’s only after 2 or 3 weeks that we focused on design.</p><p>The other big feature was Workflow, we decided to use a web view to get help from another team. It was a lot of work for them and we made sure to help them as quickly as we could to avoid wasting time. The final implementation was near the end of the project but it wouldn’t have been possible without discussing and being available during several weeks to test the feature on mobile when needed.</p><p>The last feature was Documents, this came very late in the process and we only worked on it in the last weeks. It was difficult because this feature wasn’t ready on backend and frontend side. Working with the backend team we managed to get something going just before the end of the project.</p><h3><strong>Organization</strong></h3><p>An Agile Scrum organization was implemented.</p><h4>Sprint:</h4><ul><li>2 weeks</li><li>Unique Jira board for all the developments teams</li></ul><h4>Meetings:</h4><p>Every day:</p><ul><li>Written daily</li></ul><p>Every week:</p><ul><li>Project weekly (progress reports concerning the deals with the customers and the developments, highlight on the blocking points)</li></ul><p>Every 2 weeks:</p><ul><li>Demo (every feature available in the new application was showed, even if it was not 100 % ended or validated)</li><li>Sprint planning</li></ul><p>This organization was facilitating the communication between all the teams, as explained just after.</p><h3><strong>Communication</strong></h3><p>An important skill we used during this project was the communication between the teams (technical, product, project management, CTO, etc.).</p><p>As soon as a developer was facing a blocking point or difficulty, he warned immediately the developer manager, Product Owner and Product Manager. They discussed the issue :</p><ul><li>If they succeeded to found a solution, they shared it with the teams;</li><li>If they failed to do so, they contacted the CTO and when a solution was found (most of the time within the day), it was shared with the team.</li></ul><p>A meeting minutes was delivered just after each important meeting, so that the decisions taken could be checked in case of doubt.</p><p>These reports were also useful to prepare the QA session at the end of the project. Several coordination meetings were also organized during the week (see « Organization » section), so that each stakeholder was aware of the advancement of the project.</p><p>Another great example would be the test environment. We faced the difficulty of having all features deployed at the same time. We needed to synchronise a lot with the other teams (not only those working on this project) to test everything on a single environment which at the end didn’t delay us that much.</p><h3>Conclusion</h3><p>Of course, everything wasn’t perfect, we first created a new project to finally come back to our main project but use a different target. We had a lot of back and forward mistakes at the end of the project. We had to move to a new chat SDK. We worked on a community feature to finally use the community post feed we have inside the app without changes. And also we ran into lot of trouble to get a stable environment. Thanks to a big focus on reporting, and discussion between all the teams involved, all those mistakes didn’t impact the deadline and were discovered early in the process.</p><p>To summarise what help us during this time</p><ul><li>A solid codebase</li><li>A great team</li><li>Focus on the things we knew won’t change</li><li>Apply design only at the very end</li><li>Help other teams as quickly as we could to avoid being stuck (Workflow)</li><li>A structured organization with important steps, such as demos every 2 weeks</li><li>Reports and communication between teams</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=287679dcccd0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/developing-a-new-product-quickly-in-a-fast-pace-environnement-287679dcccd0">Developing a new product quickly in a fast pace environnement</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LumApps — Changing our frontend interview process]]></title>
            <link>https://medium.com/lumapps-engineering/lumapps-changing-our-frontend-interview-process-37086f998f4a?source=rss----e9909f99458b---4</link>
            <guid isPermaLink="false">https://medium.com/p/37086f998f4a</guid>
            <category><![CDATA[frontend]]></category>
            <category><![CDATA[continuous-improvement]]></category>
            <category><![CDATA[interview]]></category>
            <dc:creator><![CDATA[Juani Galán]]></dc:creator>
            <pubDate>Tue, 12 Jan 2021 08:42:27 GMT</pubDate>
            <atom:updated>2021-01-12T08:42:27.629Z</atom:updated>
            <content:encoded><![CDATA[<h3>LumApps — Changing our frontend interview process</h3><p>At the beginning of 2020, we decided to change our interview process for our frontend candidates by introducing a new frontend test that tries to take away the pressure of coding challenges.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uzMUPMlSNSD5fKhwQJ_faQ.jpeg" /><figcaption>It is time to say goodbye to the whiteboard interview</figcaption></figure><h3>A little bit about LumApps</h3><p>If you do not know about <strong>LumApps</strong>, we are a global tech company with R&amp;D teams in France that provides our customers a <strong>SaaS Digital Workplace</strong> solution, which creates a holistic workspace, integrated with several suites and collaboration tools. Want to know a little bit more about us? Head over to <a href="https://www.lumapps.com/">LumApps.com</a> and take a look!</p><h3>What was our frontend interview process like before?</h3><p>Up until 2020, our frontend interview process consisted of several interviews that tried to evaluate the technical knowledge of the candidates as well as the cultural fit with the company:</p><ul><li>An <strong>initial interview with our tech recruiter</strong> as a the initial step of the interview process</li><li>An <strong>interview with the team&#39;s manager</strong> in order to get to know the candidate a little bit more and evaluate their tech background as well as their cultural fit.</li><li>An online questionnaire with technical questions for <strong>JavaScript</strong>, <strong>Angular</strong> <strong>JS</strong> and <strong>React</strong>. These questions would be quite theoretical (like <em>What is “hoisting” in JavaScript?</em>) or tricky (like what is the output of <em>(function() { var a = b = 5; })(); console.log(b);</em>), or involved specific framework concepts such as digest cycle for Angular JS or explain the reconciliation process for React.</li><li>We then had a <strong>live coding challenge</strong> where the candidate would have a series of exercises to do while another engineer followed their steps.</li><li>If those tests went well, we then had a <strong>final round of interviews</strong> with several members of different teams, each of them with a different technical background.</li></ul><h3>What were things that led us to change the process?</h3><h4>Technical perspective</h4><p>From a technical standpoint, our technical test was not ideal for either our candidates as well as our company.</p><ul><li>Our online form had mostly questions that tried to evaluate very specific technical knowledge but <strong>ended up being</strong> questions that were <strong>tricky, nit-picky</strong> and really <strong>did not help us evaluate a candidate</strong>. We really did not need to know if a candidate knows what is the output of <em>(function() { var a = b = 5; })(); console.log(b);</em>), it proves little to nothing 😅</li><li>We also asked questions about Angular JS, which is really a technology that we are currently using in our Legacy application, but <strong>we are slowly transitioning into full React</strong>. Having questions that evaluate Angular JS knowledge <strong>will not help us in the future</strong>, since what we really want is to have a full React application. And usually when we need to change something on the legacy application, is it basically to fix a bug or add a React component, <strong>so we really did not need to evalute a candidate&#39;s knowledge on Angular JS</strong>.</li><li>Then there was the <strong>coding challenge</strong>. This is a current subject in the frontend community, with several opinions and perspectives. From our end, we believe that <strong>we need to evaluate our candidate on how well they will perform their daily tasks</strong>, <strong>how they can code using the technologies that we are using</strong>, and which are their strengths and which are the topics that they need a little bit more help. <strong>A coding challenge does not represent any of those things</strong>, since you are not usually coding a binary search (on LumApps frontend at least), you are not usually coding with someone evaluating your every move, and you do not code without Google or an IDE.</li><li>Furthemore, as a company, LumApps grows everyday, with a lot of features under development, some of them with a very important weight on their shoulders. We needed to <strong>start evaluating candidates on whether they were capable of leading entire initiatives</strong>, or they could be part of those initiatives with more of a contributor role. This meant that we needed to evaluate our candidates on an architectural and long term perspective, and with <strong>our current tests, were not able to</strong>.</li></ul><h4>Cultural perspective</h4><p>As for our cultural evaluation of the candidate, what we were really missing is the fact that the candidate <strong>did never encounter other developers from their own team</strong>. They had interviews with their managers, and other coworkers from other teams, but they never talked with someone that will be working with them on a daily basis.</p><h3>Which is our interview process now?</h3><p>Taking all of those topics into consideration, we decided to restructure our frontend interview process:</p><ul><li><strong>Our initial interviews with the recruiter and the manager are still in place</strong>, we found that those steps are quite essential for explaining the candidate what we do at LumApps, what they could possibly do in their team and how the interview process will unfold.</li><li><strong>We ditched the online form, all the technical questions and the coding challenge</strong>, and replaced it with a <strong>frontend test that each candidate can do from their home</strong>. We have a <a href="https://github.com/lumapps/frontend-tech-test">public repository</a> with a base application, with several utilities and scaffolding already in place in order to mimic the tools that we use at LumApps on our daily basis. The candidate needs to fork that repository, and code the provided exercise, which is basically a <strong>search results and details page of superheroes from Marvel</strong>, using Marvel&#39;s public API. The test usually takes around 4 hours to complete, we already provide a developement environment, with no setup needed. Just hit <strong><em>yarn</em></strong> and you are good to go.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/1*WJeIXJJlXNGhljBFlI4Rrg.png" /><figcaption>We provide pseudo mockups so that the candidate has an idea on what it is expected of them</figcaption></figure><ul><li>Once the candidate is satisfied with their test (the candidate can take all the time they want to do it, add tests, iterate, etc.), they provide us access to their forked repository and <strong>we start the review</strong>. In order to do that, we have a series of <strong>guidelines and key aspects that we want to evaluate</strong>, and depending how the candidate has performed on each aspect, we can have an idea of what their seniority would be. These topics go from <strong>code readability, project scaffolding, maintainability, performance, accessibility, usability and user and developer experience</strong>.</li><li>After that test, we do three interviews (usually the same day) with that candidate. The first one is <strong>a debrief of the technical test</strong> where we discuss a few of the implementations and decisions that the candidate took for their test. We want to have a moment to <strong>discuss these topics personally with the candidate</strong>, in order to evaluate what they wanted to do on the test, how they did it, and what they could add in the future.</li><li>The second one is more of a <strong>cultural fit interview</strong> where the candidate discusses with <strong>two of their future/possible team mates</strong>. We try to get someone from the backend team and someone from the frontend team in that meeting, so they we have several different points of view.</li><li><strong>The third one is with our CTO</strong>. Our cultural fit is very important and our CTO loves to exchange with the future candidates on this topic, and this allows the candidates to have an exchange with a co-founder, with provides a lot of insights and opportunities to discuss the long term vision of the company.</li></ul><h3>What did we learn from this change?</h3><p>As of today, <strong>we have already changed for good our frontend test</strong>, and all of our frontend candidates from now on will take the new format. From it, we learned that allowing candidates to <strong>take the test in a normal working environment</strong>, where you do not have someone breathing on your neck and where you are implemeting <strong>something cool and fun</strong> rather than testing your skills for coding a complex function can provide a set of results that allows us not only to <strong>evaluate their skills and how can they work on a daily basis</strong>, but also allow candidates to <strong>express what they can do</strong>, what technologies they can use, and how they can use them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37086f998f4a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lumapps-engineering/lumapps-changing-our-frontend-interview-process-37086f998f4a">LumApps — Changing our frontend interview process</a> was originally published in <a href="https://medium.com/lumapps-engineering">LumApps Experts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>