Composition without fragments

Here is an example of how we might reuse a GraphQL string with traditional interpolation:

const DAYS = `
  monday tuesday wednesday thursday friday saturday sunday
`

const GET_STORE_INFO_QUERY = `
  query {
    getStoreInfo {
      schedule {
        ${DAYS}
      }
    }
  }
`

This will certainly work if we simply try to use it to call the GraphQL server, but we’ll eventually run into some weird issues if we try to use any sort of static analysis tools like graphql-inspector (more on that later).

Composition with fragments

Here is the same example from above but using fragments:

const DAYS_FRAGMENT = `
  fragment days on Schedule {
    monday tuesday wednesday thursday friday saturday sunday
  }
`

const GET_STORE_INFO_QUERY = `
  ${DAYS_FRAGMENT}

  query {
    getStoreInfo {
      schedule {
        ...days
      }
    }
  }
`

Yes, we are still using template literals to add the fragments to the query string, but this is completely valid and how fragments are meant to be used.

If you want to experiment with fragments in the GraphQL playground, you can simply recreate the expression in its composed form:

fragment days on Schedule {
  monday tuesday wednesday thursday friday saturday sunday
}

query {
  getStoreInfo {
    schedule {
      ...days
    }
  }
}

Fragments with variables

You can even use variables with fragments just like other types.

fragment days on Schedule {
  monday tuesday wednesday thursday friday saturday sunday
}

fragment employees on StoreInfo {
  employees(status: $status) {
    id
    name
  }
}

query GetStoreInfo($status: EmployeeStatus = ACTIVE) {
  getStoreInfo {
    schedule {
      ...days
    }
    ...employees
  }
}

As you can see in this example, fragments can be queries or any other GraphQL struct.

Fragments are the canonical mechanism provided by GraphQL to compose types in clean, reusable ways. They give us the flexibility to breakup complex queries, reduce redundant code, and still leverage powerful static analysis tools in our projects. In a future article, I will demonstrate one of those tools, graphql-inspector which might save you from deploying broken queries to production.

RESTful patterns fit best into services that are rigid and basic in how they implement CRUD methods. The “U” in CRUD — the update method — is often the most complicated. On the surface, it is simple in concept (just update some data) but there are many ways to interpret that meaning and fit it into your system. The easiest update design to implement is to PUT new data in place of the old, and the most difficult is to PATCH new data over the old. A PATCH resource could itself be designed various ways, but this article is going to focus on the merge patch.

What is a merge patch?

The best explanation of what merge patch means would be the original RFC from 2014. The tldr is: a merge patch means modifying existing content with a literal merge of partial incoming data into the preexisting data. We are going to focus on doing this with JSON since it is the most common approach, but the same concepts could apply in other formats as well.

We can exemplify this with a simple scenario. If you have a resource with some data about a person and you want to make a modification, your original data might look like this:

GET /people/{personId}
{
  "firstName": "John",
  "lastName": "Doe"
}

You want to be able to change some of the data with a request that includes only explicit modifications, like this:

PATCH /people/{personId}
{ 
  "firstName": "Jane" 
}

The modified data will ideally include the preexisting data with those new modifications (in this case, only the “lastName” property) applied on top of it:

GET /people/{personId}
{
  "firstName": "Jane",
  "lastName": "Doe"
}

So this all probably seems intuitive, and it really is. But the unintuitive part is actually implementing the design in whatever language/framework, and doing so such that it doesn’t wipe out unmodified data. You need your system to be able to know the difference between absent values and newly modified null values (e.g. "lastName": null versus the complete absence of the “lastName” property), and this needs to be able happen at any depth in the tree. Well, you get to decide where things are literally merged versus replaced still, but we’re going to focus on making it possible to merge to any extent/depth necessary.

The Language-Agnostic Details of an Implementation

Let’s think through this and design as much as we can without focusing on a specific language. As I mentioned before, we need a way to distinguish between present and absent values in our merge request. This means, we need a mechanism that traverses/parses the incoming request and gives us the details about what is new. Ultimately, we don’t even care about what isn’t changed. In fact, if we are really designing a clean system, the request isn’t designed around the internal data structure at all, but rather a structured request which just might happen to mirror the internal design.

Let’s say that we are using an object-oriented language; this means we can have some object like “UpdatePersonRequest”, an instance of which gets deserialized from a JSON entity. We want to be able to find out from this object what new data has been provided/modified. In pseudo-code, that could look something like this:

UpdatePersonRequest {
  string firstName
  string lastName
}

Here, I’m left with not knowing whether any null values in this were actually set or not. In other words, if I deserialized an instance of this object from a JSON payload like { "firstName": "Jane" }, I’d get an instance that looks like this:

UpdatePersonRequest {
  firstName: "Jane",
  lastName: null
}

When I proceed to apply this request to my internal data, I have to assume that the “lastName” was updated to null. But that’s not what I intended, so let’s try something else. Instead of these literal values in the request object, I can set some sort of generic object type which contains information on whether each property was updated (and if so, what’s the new value).

UpdatePersonRequest {
  MaybeUpdated<string> firstName = MaybeUpdated() // default
  MaybeUpdated<string> lastName = MaybeUpdated() // default
}

MaybeUpdated<T> {
  boolean wasUpdated = false // default
  T value
}

Don’t get hung up on the notation here — remember, this is just pseudo code. The <T>-like syntax is a common way of notating type parameters in statically typed languages. If that’s confusing, you can visualize this example with all of those removed, and it conceptually means the same thing for our use case.

The takeaway in this example is that I now have a request object that I can somehow parse from the incoming JSON payload, and then use to apply changes for any of the values with the wasUpdated property set to true. I won’t go into how we actually do that, but the gist of it is: traverse the raw JSON and create an instance of UpdatePersonRequest with the sub-properties that are present (in our example, “firstName”) set in the object.

UpdatePersonRequest {
  firstName: { wasUpdated: true, value: "Jane" }
  lastName: { wasUpdated; false, value: null }
}

Caveats and Alternatives

In the best case scenario, you are using a framework that provides serialization and can do most (or all) of this work for you. You may be able to get mostly there and only need to provide some function/class to the framework that tells it how to parse/deserialize into these new objects (MaybeUpdated) specifically, but that’s usually not a heavy lift and is something you only have to do once.

If you aren’t working with types or classes, you can still simulate the approach here using whatever the language’s implementation of maps is (e.g. Maps in Java, object literals in Javascript, dictionaries in Python, and so on). However, those have their tradeoffs. Using generic maps/objects means that you are bound to doing reflection at some point or otherwise working with generic structures on both ends of the exchange (between updates and the resulting work you do with them), and that’s just not ideal.

If you are working in a language where you typically use generic maps/objects, I’m first going to suggest migrating towards a more strongly typed approach (even if you aren’t literally using types in your language, there are various schema validation libraries that are helpful for both the server and client side sanity). If that isn’t an option, then you just might have a little more work to do to simulate what we’ve done here; you’ll have to iterate/reflect over every provided updated value as generic data (or otherwise over the target object), and that just has the possibility of getting expensive (time-wise) fast and really isn’t best practice.

Conclusion

Note that I didn’t go into how you deal with the data once it is parsed from the request. Whether you are just applying the changes to an internal representation and then saving that in a single node in a database, or using a fancy ORM to magically do that for you, or even converting the updates into small bite-sized events to then be processed/published before ultimately resurfacing in your read model, the complexity in merge patches lies mostly within the interpretation/model of the data received from the outside. As soon as possible, you want to know and hold onto the difference between provided values and absent values, and let the rest of your system react to those explicit changes.

In this article, I’ll demonstrate how to create a public API with gRPC. The application code itself will be relatively simple and won’t include authentication, with more focus instead on the tooling and methodology around building the service as a whole and exposing the protobufs so that consumers can generate clients from the same protos to use the service.

Learning Objectives

• Create a Protocol Buffer library
• Create a gRPC service implementation (Java)
• Create a gRPC client implementation (Node)

Requirements

• Java 9+
• Node 8+
• protop

Create a Protocol Buffer Library

We first want to create the protobufs that will describe the API we want to build. In this example, I’m creating a service to convert money between currencies. Let’s say this is part of a larger numbers-oriented service.

Initialize the project with protop

If you haven’t installed protop before, you can do so either from Homebrew ( brew tap protop/io && brew install protop) or from the source following the instructions here.

$ mkdir numbers-protos && cd $_
$ protop init

Following the prompt, I’ll create a basic protop manifest:

Organization name (required): awesomelabs
Project name (required): numbers
Initial revision (default 0.1.0):

We won’t have to do anything with it after this point, but here’s the protop.json that was generated:

{
  "organization" : "awesomelabs",
  "name" : "numbers",
  "version" : "0.1.0",
  "include" : [ "." ],
  "dependencies" : { }
}

Create some protos

Now, I’ll create a small proto file to define a money service. Since this tutorial isn’t about writing protos themselves, I’ll skip going through it in depth; here’s the proto I’ve created:

This basically just defines a few messages that hold information about money and a service with a single RPC to convert a value from one currency to the other.

To make it available to other projects, I can either publish locally (protop link), publish to an external registry (protop publish), or publish to a VCS platform like Github (just create a normal Git project, and push it up). I’ll do the latter — and you can find that repository here. Now, the library is available to any other project that wants to use it. In the next part, I’ll demonstrate doing that to create a gRPC service that implements the MoneyService proto.

Create a gRPC Service Implementation (Java)

Initialize the application with Gradle

Setting up a Java project is a little more involved than the previous step, but I’m using Gradle to make it easier. Here’s what that looks like for me using Gradle 6.2.1:

$ mkdir numbers-service && cd $_
$ gradle init

Select type of project to generate:
  1: basic
  2: application
  3: library
  4: Gradle plugin
Enter selection (default: basic) [1..4] 2

Select implementation language:
  1: C++
  2: Groovy
  3: Java
  4: Kotlin
  5: Swift
Enter selection (default: Java) [1..5] 3

Select build script DSL:
  1: Groovy
  2: Kotlin
Enter selection (default: Groovy) [1..2] 1

Select test framework:
  1: JUnit 4
  2: TestNG
  3: Spock
  4: JUnit Jupiter
Enter selection (default: JUnit 4) [1..4] 4

Project name (default: numbers-service):
Source package (default: numbers.service): awesomelabs.numbers

> Task :init
Get more help with your project: https://docs.gradle.org/6.2.1/userguide/tutorial_java_projects.html

BUILD SUCCESSFUL in 48s
2 actionable tasks: 2 executed

Make it a protop project as well

Next, I’ll make this into a protop project so that I can import the numbers-protos library. Instead of going through the protop init interface, I can also of course just create a protop.json manually. Here’s what that looks like so far:

{
  "organization" : "awesomelabs-internal",
  "name" : "numbers-service",
  "version" : "0.1.0",
  "include" : [ "." ],
  "private": true,  
  "dependencies" : { }
}

Now, I just need to add my protos library as a dependency. Since the awesomelabs/numbers library was published to a Github repository, I’ll use that URL instead of a version, prepended with “git:” so protop knows that it’s definitely a Git repository (note: Medium is wrapping the lines here):

"dependencies": {
  "awesomelabs/numbers": "git:https://github.com/jefferyshivers/numbers-protos"
}

Sync the external protobuf dependencies

This should be enough to run protop sync and import the dependencies for the project:

$ protop sync
Syncing external dependencies.
Done syncing.

And now if I look inside the project’s protop path in the project directory, I can see that the dependency was imported as expected from Github:

$ ls .protop/path/awesomelabs/numbers/
README.md money.proto protop.json

I’ll add that path to my Gradle configuration so that the protoc compiler can use those protos to generate the classes needed for the service implementation. After setting up my build.gradle to use the protobuf plugin, I just need to add the protop path as a proto source directory:

sourceSets {
    main {
        proto {
            srcDirs += ".protop/path"
        }
        ...
    }
}

To make things easier so that I don’t have to run protop sync manually, I can create a simple command line task in Gradle, and make it execute before protoc generates anything:

You’ll notice that I added a flag --git-refresh; this tells protop to refresh the Git sources every time sync is run. When it does this, it will re-clone the repository rather than pulling. While this might seem expensive, it means you can be as specific as you need with any dependencies down to the commit (e.g. https://github.com/<repo_name>/tree/<commit_sha>) without worrying that using --git-refresh will pull newer revisions if you don’t want them. Since I wasn’t that specific with this dependency, I’m ok with getting the latest version from the master branch every time.

When we build the project now, the output logs should look something like this:

$ gradle clean && gradle build

BUILD SUCCESSFUL in 720ms
1 actionable task: 1 executed

> Task :protop
Syncing external dependencies.
Done syncing.

BUILD SUCCESSFUL in 4s
12 actionable tasks: 12 executed

Implement the gRPC service

Finally, the generated protos (in this case, Java classes) should have been generated such that I can actually write out the service. Here’s a simple implementation of the service with the convert RPC call:

You can see the full project here; the main files to look at are the build.gradle, awesomelabs.numbers.App, and awesomelabs.numbers.MoneyService. And now I should be able to start the service:

gradle clean && gradle run

BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed

> Task :protop
Syncing registered dependencies.
Done syncing.

> Task :run
Server started on port 8080
<===========--> 87% EXECUTING [15s]
> :run
|

Create a gRPC Client Implementation (Node)

Now that the money service is running, it should be possible to create a gRPC client and interact with it. Since it doesn’t matter what language the service or client are in, I’ll do this part in Node.

mkdir numbers-node-client && cd $_

Create a new protop project

As with the service implementation, I’ll create a protop manifest and add the awesomelabs/numbers package as a dependency.

{
  "name": "numbers-node-client",
  "version": "0.1.0",
  "organization": "awesomelabs",
  "include": [ "." ],
  "dependencies": {
  "awesomelabs/numbers": "git:https://github.com/jefferyshivers/numbers-protos"
  }
}

If you’re new to gRPC (or to using it with Node), I created the client to pretty closely mirror the example Node client in the official gRPC documentation. Here’s what it ended up looking like:

As you can see, I am dynamically loading the protos from the protop path, and then creating a service client. The gRPC library allows me to use JSON to build the request protos and will handle serializing them in the RPC. Now, running the script in the project directory should result in console output similar to the following (with different numbers since they are always randomized):

$ node .
Original: 7.3406037480265685 USD
Converted: 5.33649466631217 MXN

And that’s it! I can continue to create any number of service implementations or clients using that original protobuf library without adding any fancy pipelines to generate artifacts.

Conclusion

I hope that this example helps demonstrate that protocol buffers and gRPC can be easier to work with than they initially might seem. They are great technologies with somewhat intimidating first impressions, but ultimately, with the right tools and development patterns, they can make your life building robust projects — whether it’s a public API, an internal microservice mesh, or even just a simple library of protocol buffers for others to build upon.

Examples on Github

I’ve published all the examples used in this tutorial to Github. Feel free to clone them, or use them as a basis for a new project. I’m also interested to see other examples out there, so please let me know if you’d like for me to share what you’ve created and I’ll gladly link to it here!

• “numbers-protos” - protobuf library
• “numbers-service” - gRPC service implementation (Java)
• “numbers-client” - gRPC client implementation (Node)

Additional resources

• protop - protocol buffer package manger
• protocol buffers - official website
• gRPC - official website

Protop (“proto p”) is a new tool for working with protocol buffers. The core focus of the project is the challenge of dependency management and resource sharing, especially for projects that span beyond a single code base — think: distributed systems, public APIs, or even just common protobuf definitions.

Motivation

Monorepos for developing/building protobufs have become convention for a few reasons — mainly easy sibling dependency and pipeline aggregation. But this quickly becomes an anti-pattern when the monorepo itself becomes a complicated bottleneck for workflows. Unless your organization is all in on this pattern, it can be painful, for example, to release new versions of a protobuf or to differentiate between small changes and significant ones.

On the other hand, we can look at tools like Maven, pip and NPM and see that a good pattern for resource sharing — or package management — has already been established and proven. With the potential simplicity of proto project structures and the availability of existing tools like protoc and Prototool, the foundation has been laid for robust project development, and the only missing piece is a mechanism for sharing/retrieving protos with an access pattern that can include external registries.

Goals

Protop is aimed at solving the problem of sharing protobufs. The tool attempts to be powerful but invisible — it generally stays out of the way and leaves only the smallest footprint in your projects. At a much higher level, I’m hoping to bring the community together with the common goal of building something that allows us to create/distribute protos with a standard approach that will facilitate more collaboration in general, as well as promote an easier path towards using protocol buffers and gRPC altogether.

Deliverables

The two main components of protop are:

• the protop CLI: This is the main interface that allows you to compose projects and retrieve protos from other projects. The most significant features include the abilities to publish (protop publish), sync external dependencies via a simple manifest (protop sync), and link a project locally (protop link) — this is analogous to npm link or gradle publishToMavenLocal.
• Nexus repository integration: This plugin allows anyone to add a protop repository to a new or existing Nexus instance.
• A public repository: I haven’t focused on creating a public repository yet, but that is something that’s very high on the roadmap. I’m partly hoping that your feedback/collaboration will help inform good decisions about how that will/should come to be.

Outcomes

Here are the most significant ways that protop will impact projects and the biggest takeaways:

• Artifacts don’t have to be shared. When working with protobufs, the protos themselves are the contracts that should be shared, not the artifacts that they generate.
• Pipelines are simplified. Protop removes the need to build pipelines that do overly complex tasks to compose/track protos in an organization.
• Protop works seamlessly with existing build tools. Because it doesn’t try to replicate any part of the protoc compiler or other build tools like Gradle, and due to its simple API, it integrates with minimal effort.
• Proto projects can become the small, composable units that they were meant to be.

RFC (Request for Comments)

This project was motivated by my own observations and wishes as a developer working with protobufs and gRPC, both on the job and in my personal projects. But that doesn’t mean that I’ve necessarily made something that is the most useful for other developers. With that said, I’d love to get feedback on the project.

RFC (Request for Contributors)

I need help building this tool and making it better! There is a huge roadmap ahead, and besides trying to build features and get to 1.0 faster, having more hands on deck will keep the project direction in check and ensure a better probability of building the best quality and most useful tool we can. Please visit the projects at https://github.com/protop-io and join in on the fun.

I recently began working with the Lightning Component Framework (Salesforce’s latest front-end toolset). As a Javascript — and particularly a React + Redux — developer at heart, of course one of the first questions I had when entering this new space was how it lends itself to flux architectures.

Out of the box, Lightning comes with a hefty load of useful features that can be uniquely composed to create a unidirectional data flow, but it doesn’t come with any specific suggestions for exactly how to set something like that up. Perhaps that’s because it really isn’t that difficult, or maybe they want to leave it open for developers to decide because the subtle differences between approaches could affect big changes that are hard to detect if one isn’t forced to consider them (oof); or maybe it just isn’t a big enough concern for Salesforce, compared to the broader idea of simple reusability, for example, which seems to have been the overarching theme of the framework from the beginning.

In any case, I figured the relatively low number of existing resources regarding this topic was the perfect excuse for me to share my own understanding of how flux might be successfully achieved in a Lightning component project. I’ll also discuss at a higher level why flux can in general be useful for Salesforce developers and things to keep in mind as one adopts such a pattern.

A Brief Background of a Few Important Lightning Features

Just to cover all my bases, these are some things you should definitely know about before trying to extend what Lighting is already capable of doing:

Attributes: Enough can’t be said about the importance and usefulness of attributes in Lightning components. This includes the traditional HTML flavor of attributes that web developers are all familiar with, as well as the <aura:attribute ...> attributes which are set inside the body at the component level (well, technically not in the {!v.body}, but between the component tags at the top level) and can of course be reached/set through parent components in that traditional manner. Attributes allow us to chain values between parent/child components, they can hold a changing state at the individual component level, and they can be dynamically changed or referenced in javascript. That last part is a powerful feature because of its simplicity (which makes a world of difference when you are talking about building an API around existing functionality).

Events: The Lightning event API is another super handy feature of the framework, and it is probably the one of which I have the fewest complaints (I really don’t have many overall, to be fair). It carefully distinguishes between two types of events: component and application. The gist of that difference is that component events bubble, like the traditional DOM events we are used to (such as the almighty “click”) that are propagated up the stack (until/unless propagation is explicitly halted) and not received by sibling or descendent components. Application events, on the other hand, are broadcast application-wide; any component related in the application stack is eligible to receive the event, and all dispatchers and subscribers are entirely decoupled (thus, there is no order for who receives the event, and no recipient can change anything about that event once it has been fired).

Components: These are the workhorses of the Lightning framework. They may or may not contain actual HTML markup. At its core, a component is an engine that may have attributes (and other things like handlers and methods, but we’ll brush over that for now) and a body, which is an array of zero or more elements/components. It delegates javascript to its controller (and optionally a separate handler.js), and can connect to a specific Apex controller as well. It knows how to update its own state whenever the value of a bound attribute ({!v.myAttribute}) is changed, so that you don’t have to. Just hook it up, and let the powerful lightning engine handle the rest.

Why Flux?

Let’s talk about application state. Say there is some value, such as the amount of time that has passed since page load, which two or more components care about (and display)— is it efficient, or safe, to have the same logic repeated across components to handle the updating value? The answer to that is clearly no. And in this case, we could simply put our clock in the top level component and chain it down to each child attribute that cares through good ole attributes. But eventually, if we have enough of these shared values and a scaling application, we’ll be left with quite a mess to pick through with our eyes (and our brains), so maintainability is something we want to keep in mind as well.

The biggest danger in our clock example is illustrated here: say we didn’t delegate the timing to one top level attribute but rather to the individual components, that one of the clocks could, for example, get out of sync; there is no way for it or any of the other components to know that this happened, and so the application would continue onward as if everything is fine. Clocks are pretty accurate, so let’s make this problem a little more real: one of the components has a pause button. Now, we have to notify all of the other components that time has stopped, and we still don’t know for sure that they are even synchronized. If we go back to having the clock at the top level, and simply chain the time value down to each child, this last problem is all of a sudden much less intimidating because now we only have to notify one component about the change, and we can count on that component to update all the children that care about it. And that is the beauty of flux: unidirectional data flow from a single source of truth.

Disclaimer(-ish)

I want to top this last part off and prepend the next section with a warning about using flux. It is unbelievably tempting to put everything in the application state when you start programming with this pattern. It seems awesome and smart and efficient and clean to just put it all in one place, but that really isn’t the idea behind unidirectional data flow. Use flux to only handle shared application state. That is, the stuff that is crucial to the success of some core application functionality (like persisting un-submitted form data across changing routes in a SPA, or more typically sharing data between components)— not something that only the little widget in the left side bar cares about and no one else. Don’t flood the flux highway with unnecessary pieces. You’ll end up creating way more work for yourself that way, and the value of flux will be lost.

Leverage component attributes when you can; Lightning has such a great way of handling stateful components through attributes (even more simply than React does in basic cases), and that should be taken advantage of. That is not to say that you can’t have overlapping flux hierarchies if you really need to; maybe that widget is super complex and comes with a bunch of children components that share data — just make sure that the pattern is justified. Otherwise, if you need it in only one place, keep it in that one place.

An Example in Lightning

Ah, at last, code. I know, these things usually come with so much more copy-and-pastable code examples, and I’m sorry for holding out. But my hope is that this article will have revealed enough that you can easily imagine how to set something like this up without much strain or need for further research, even as a newcomer to the Lightning framework like me.

In this solution, I am creating the following pieces:

• a base FluxContainer component, which is extensible so that other components can use it to manage their application state
• a FluxContainer_getState application event which will be broadcast by the container
• a FluxContainer_setState component event that can be fired by any component to update state
• a parent component, which extends the FluxContainer
• two instances of a child component, which has a button and shows the number of times the button (in either instance) has been clicked

FluxContainer.cmp

<aura:component exstensible="true">
  <aura:registerEvent name="getState"
                      type="e.c:FluxContainer_getState" />
  <aura:handler name="setState" 
                event="c:FluxContainer_setSTate" 
                action="{! c.setState }" />
</aura:component>

FluxContainer.controller.js

({
  setState: function (component, event, helper) {
    var update = event.getParam('update');

    if (
      update.containerName &&
      component.get('v.containerName') &&
      update.containerName == component.get('v.containerName')
    ) {
      helper.setState(
        component,
        update.containerName,
        update.state,
        update.callback
      );
    } else {
      console.error('There was a problem pairing containerNames.');
    }
  }
})

FluxContainer.helper.js

({
  setState : function (component, containerName, state, callback) {
    var res = {
      unavailable_attributes: []
    },
      updates = [];

    for (var param in state) {
      var attr = 'v.' + param;
      var val = state[param];

      if (component.get(attr)) {
        component.set(attr, val);
        updates.push({
          attribute: attr,
          value: val
        });
      } else {
        res.unavailable_attributes.push(param);
      }
    }

    // post the update as an application event
    if (updates.length > 0) {
      var updateEvent = 
          component.getEvent('e.c:FluxContainer_getState');
      updateEvent.setParams({
        containerName: containerName,
        state: updates
      });
      updateEvent.fire();
    }

    // call the optional callback
    callback && callback(res);
  }
})

FluxContainer_getState.evt

<aura:event type="APPLICATION" description="FluxContainer_getState">
    <aura:attribute name="update" type="Object" default="{}"/>
</aura:event>

FluxContainer_setState.evt

<aura:event type="COMPONENT" description="FluxContainer_setState">
    <aura:attribute name="newState" type="Object" default="{}"/>
</aura:event>

Parent.cmp

Make a parent component that acts as the whole application container. Invoke two child components.

<aura:component description="parent"
                extends="c:FluxContainer">
  <aura:handler name="init" 
                value="{! this }"
                action="{! c.onInit }"

  <aura:attribute name="containerName" 
                  type="String" 
                  default="parentContainer"/>
  <aura:attribute name="someAttribute" 
                  type="Integer" 
                  default="0"/>
  
  <Child />
  <Child />

</aura:component>

Parent.controller.js

Add the onInit function which will initialize its state and broadcast it.

({
  onInit : function (component, event, helper) {
    var init = component.getEvent('c:FluxContainer_getState');
    init.setParams({
      containerName: component.get('v.containerName'),
      state: [
        {
          attribute: 'count', 
          value: component.get('v.count')
        }
      ]
    });
    init.fire();
  }
})

Child.cmp

<aura:component>
  <aura:registerEvent name="addOne" 
                      type="c:FluxContainer_setState"/>
  <aura:handler name="update" 
                event="e.c:FluxContainer_getState" 
                action="{! c.onUpdate }"/>
  <aura:attribute name="count" 
                  type="Integer"
                  default="0"/>

  <button onclick="{! c.onButtonClick }">
    You clicked me or my twin {! v.count } times.
  </button>

</aura:component>

Child.controller.js

The onButtonClick function sends a new count (previous count plus one) up via the FluxContainer_setState event, which takes one parameter, “newState”, an object of just the updated items (if we had other stateful properties in the container, they would not be affected by this unless we explicitly added them).

({
  onButtonClick : function (component, event, helper) {
    // dispatch the new count
    var update = component.getEvent('addOne');
    update.setParams({
      newState: {
        count: component.get('v.count') + 1
      }
    });
    update.fire();
  },

  onUpdate : function (component, event, helper) {
    var state = event.getParam('update').state;
    
    state.count && component.set('v.count', state.count);
  }
})

This child component explicitly handles getting state updates via onUpdate, but just like the FluxContainer base component, we could abstract that functionality to a base component which any child like this one could extend to automatically handle simple state updates. I wanted to exemplify how flexible this composition could be though; if your project is small enough, maybe you don’t need to abstract anything, and maybe the logic can remain in your immediate controller (and/or helper).

So the summary of things here goes like this: The top level container is initialized with a count of zero clicks. On initialization, it broadcasts that event application-wide through the getState event. The two child components subscribe to that event and update their own attributes then and any other time the getState event is fired with that attribute. When either of the children’s buttons are clicked, they dispatch the setState component event with a new count value. That value is received by the flux container, updated in its own attribute, then reposted application-wide with getState so that the other button (and technically the original button too) can have the latest value from state.

I hope that this example illuminates some of the possibilities with Lightning components and the flux pattern. The flexibility of the framework lends itself well to a multitude of possibilities, and it really comes down to what your project needs in order to succeed and be maintainable. Consider all the reasons for and against reducing state whenever you are setting each piece up. Likewise, consider all the reasons for why you set it up the way you do. If you want everything to be wireless, use events whenever you can. If your project is small, consider just directly chaining properties and actions through attributes. If you have a complex component nested somewhere inside the larger container component, consider making it a container too (this is where containerName would be useful, so different containers and children don’t get confused with updates from the wrong hierarchy).

Thanks for reading, and I hope to learn other ways people are using flux in Lightning.

This article will go through the process of building a chatbot from scratch using Amazon Lex for language processing, Loopback for creating a REST API and serving up the front end, and React for building the front end. The aim of this example is to be as reduced as possible in terms of how much goes into each step in order to get a full stack bot framework up and running. Of course, you could reduce the process much further with different tools entirely, but I’ve chosen these because they will probably scale very well.

The following steps are subdivided into three basic parts, corresponding to those larger phases of the project.

1. Amazon Lex

To get started, you’ll need an AWS account. Once that is setup, navigate to Lex and click “Create”. Select “Custom Bot”, and give it a name; I’m going to call this one “CatBot”. Choose the rest of the options on that page, and, at the bottom, “Create”.

You’ll be redirected to your new bot! Before you can test it out, you’ll need at least one intent. I created one called “Meow”. Add one or more utterances with phrases that should trigger this intent. For the Meow intent, I want to trigger it with the phrases “meow” or “prrr”, so those are my two utterances. For now, skip the Slots section[1]; expand the Confirmation Prompt, and fill in the fields with whatever you’d like the bot to say back to the client to confirm (or escape) the intent. I wrote “Oh, are you a cat?” for Confirm and “No problem, I must have misheard you.” for Cancel. “Save” at the bottom, and select “Build” at the top.

Once the build process completes, you can type one of the triggering utterances into the test prompt on the right, and you should get the confirmation message(s) you wrote in return. If you respond “yes” to the prompt, you’ll get a new message in return with something like “Intent Meow is ReadyForFulfillment:”. If you respond “no”, you’ll get the cancellation message like “No problem, I must have misheard you.”.

Click “Services” in the navigation bar at the top of the page, then “Users” from the left menu, then “Add User” at the top. Give this AWS user a name and Programmatic access. In the next step, select “Attach existing policies directly” and select “AmazonLexRunBotsOnly”. Create the user, then you’ll see a success message with the credentials for the new user. Take note of the Access Key ID and Secret Access Key, and we’ll need them when we connect our backend to Lex.

This is only the very tip of the iceberg with Lex, but in the interest of moving on to the next steps, click “Publish” and give it any alias you’d like (for simplicity you can just make it the same as the name).

[1] You’ll definitely want to start looking into slots, if nothing else. Here is an awesome video tutorial series on using Lex; it goes into slots and many other considerations that didn’t make it into this example.

2. Loopback

To use Loopback, a Node.js API Framework, you’ll first need to have Node installed. Then install the Loopback CLI with npm (npm comes with Node by default):

$ npm i -g loopback-cli

Next, navigate in the console to a directory where you want to install your project, and go through the loopback setup procedure as follows:

$ lb
? What's the name of your application? catbot
? Enter name of the directory to contain the project: catbot
? Which version of LoopBack would you like to use? 3.x (current)
? What kind of application do you have in mind? empty-server (An empty LoopBack API, without any configured models or datasources)

Once Loopback finishes its part, it should prompt you to cd into the new project directory and create a model.

$ cd catbot
$ lb model

This brings up a prompt to configure a model; since we created an empty API, it didn’t come preconfigured with any datasources, so you’ll see a warning about this which you can just ignore.

$ lb model
Warning: Found no data sources to attach model. There will be no data-access methods available until datasources are attached.
? Enter the model name: utterance
? Select model's base class Model
? Expose utterance via the REST API? Yes
? Custom plural form (used to build REST URL):
? Common model or server only? common
Let's add some utterance properties now.

Enter an empty property name when done.
? Property name: message
   invoke   loopback:property
? Property type: string
? Required? No
? Default value[leave blank for none]:

Let's add another utterance property.
Enter an empty property name when done.
? Property name: sessionid
   invoke   loopback:property
? Property type: string
? Required? No
? Default value[leave blank for none]:

Let's add another utterance property.
Enter an empty property name when done.
? Property name: author
   invoke   loopback:property
? Property type: string
? Required? No
? Default value[leave blank for none]:

Let's add another utterance property.
Enter an empty property name when done.
? Property name:
$

We’ll come back to that model, but while you’re still at the command line do:

$ mkdir client/dist && touch client/dist/index.html

That is going to be the entry point for our front end. Open your project in your preferred editor, and delete or comment everything out in server/boot/root.js. Then, in server/middleware.json, edit the “files” node to this:

"files": {
  "loopback#static": {
    "params": "$!../client/dist"
  }
},

Now when we run node ., the root route at localhost:3000 will serve the contents of client/dist/index.html. You can put whatever you want in there for now to test it, but later we’ll set it up so that it’ll load our bundled React app.

When we created the Utterance model earlier, it generated two files in common/models. You can see or update the current schema for the model in the .json file; open utterance.js to add some custom methods to the model.

After doing npm i --save aws-sdk, update the file to look like this:

"use strict";

// use something like dotenv to load environment variables at
// runtime. otherwise you can hardcode the following variables
// for quick testing
const access_key_id = process.env.AWS_access_key_id;
const secret_access_key = process.env.AWS_secret_access_key;

const AWS = require("aws-sdk");
const awsRegion = "us-east-1";

AWS.config.update({
  accessKeyId: access_key_id,
  secretAccessKey: secret_access_key,
  region: awsRegion
});

const lexruntime = new AWS.LexRuntime();
const crypto = require("crypto");

const params = {
  botAlias: "CatBot" /* required */,
  botName: "CatBot" /* required */
};

module.exports = Utterance => {
  // methods...
}

Where you see “methods…” above, we’ll add some custom methods, which will be routed to specific endpoints to be used by our application.

When a chatbot is first loaded, we might want to greet the user with a welcome message. Create an instance method called firstOpen, which will accept one argument, a callback function.

Utterance.firstOpen = cb => {
  // create a unique sessionid to keep track of the session
  let sessionid = crypto.randomBytes(20).toString("hex");

  let response = { message: "Hi there. Say something a cat might say!" }

  cb(null, sessionid, response)
};

We generate a new sessionid in order to keep track of the session as the conversation progresses. We’ll send this back and forth with the client until they stop responding or begin a new session. The nice part about this id is that it doesn’t get associated with the user (or doesn’t have to be), so their sessions are entirely anonymous by default.

Next, connect that to a remoteMethod like so:

Utterance.remoteMethod("firstOpen", {
  returns: [
    {
      arg: "SessionKey",
      type: "string"
    },
    {
      arg: "Response",
      type: "string"
    }
  ]
});

Now we need another method that will accept an incoming message from the user, send that to Lex, and then send the response from Lex back to the user. Call this method “message”:

Utterance.message = function (sessionkey, message, cb) {
  let req_params = Object.assign(params, {
    userId: sessionkey,
    inputText: message
  });

  lexruntime.postText(req_params, (err, data) => {
    cb(null, sessionkey, data);
  });
};

I took out the error handling to condense that excerpt, but you should definitely refactor those into the methods in order to streamline any debugging processes later.

Utterance.remoteMethod("message", {
  accepts: [
    {
      arg: "SessionKey",
      type: "string"
    },
    {
      arg: "Message",
      type: "string"
    }
  ],
  returns: [
    {
      arg: "SessionKey",
      type: "string"
    },
    {
      arg: "Response",
      type: "string"
    }
  ]
});

You can try out the REST endpoints by going to localhost:3000/explorer. If the app is running already, restart it, then proceed with our changes now in place.

In the API explorer, expand “POST /utterances/firstOpen”, and click “Try it out!”. This should generate a Response Body a few sections below that with the “SessionKey” and “Response”. If you copy the SessionKey and paste it into the so-named parameter field at “POST /utterances/message”, along with a Message that matches one of the intent triggers we setup in Lex. The “Response” here should include some other attributes sent to us from Lex. Most importantly at the moment, the “message” should be our confirmation prompt for the intent we triggered; that intend name should appear in this object as well.

Everything you see in the Response here will be super useful later on, but for now this is enough to justify moving on to build a front end so that we can display / interact with this stuff in a nice user interface.

3. React

The front end can be done without React, but just for fun I’m going to use it anyway. Using it will allow us to create a component that can be inserted into any project without changing any of the existing code. If you haven’t read my article on creating a React app using Webpack and Express, that’ll explain how I setup some of this part of the project. The general summary is that you’ll need to create a webpack.config.js file in the root of your project directory that looks like this, and your React source code in client/src. Webpack will bundle everything into a generated file in client/dist.

$ npm i --save-dev react react-dom webpack babel-core babel-loader babel-preset-env babel-preset-react babel-preset-stage-2 style-loader css-loader sass-loader node-sass
$ touch webpack.config.js
$ mkdir client/src && mkdir $_/styles && mkdir $_/../containers
$ touch client/src/index.js && touch client/src/styles/App.scss

Write the following in client/src/index.js:

import React from "react";
import ReactDOM from "react-dom";

import "./styles/App.scss";
import Bot from "./containers/Bot";

ReactDOM.render(
  <div className="App">
    <Bot />
  </div>,
  document.getElementById("root"));

And the styles you want to scope to the app in client/src/styles/App.scss.

Now let’s make our Bot component:

$ touch client/src/containers/Bot.js && touch client/src/styles/Bot.scss

Here are the main goals we have going into this component:

• when the bot is first loaded, get the welcome message from our backend
• have an input field where the user will type, and from which we’ll submit their messages to the bot
• keep track of all the messages that have been sent or received
• display all of the messages in the view, and update it automatically as the dialog progresses

Start the component out with an initial state. sessionkey will hold the session key, input will hold the user’s message input, and messages will hold all of the messages from the conversation.

import React, { Component } from "react";

import "../styles/Bot.scss";

class Bot extends Component {
  state = {
    sessionkey: null,
    input: "",
    messages: []
  };

  componentDidMount = () => {
  }

  render() {
    return(
      <div className="Bot"></div>
    )
  }
}

Add some logic to componentDidMount to make a request to our backend when the Bot is mounted in the application.

componentDidMount = () => {
  fetch('/api/utterances/firstOpen', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json'
    }
  }).then(res => res.json())
    .then(body => {
      this.recordMessage('BOT', body.Response.message)
      this.setState({ sessionkey: body.SessionKey })
    })
}

The block above factors out a function called recordMessage to persist our messages:

recordMessage = (sender, message) => {
  this.setState({
    messages: this.state.messages.concat({
      SENDER: sender,
      MESSAGE: message
    })
  })
}

Add a script to ./package.json to build what we have so far with Webpack and test it out:

"scripts": {
  ...
  "build": "./node_modules/.bin/webpack"
},

Now you can run npm run build to generate the bundled code into the client/dist directory. You can add the --watch flag to that command to have it automatically rebundle when changes are saved. Make client/dist/index.html so that it will load the generated bundle.js:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8">
    <title>lex + loopback bot demo</title>
  </head>
  <body>
    <div id="root"/>
    <script src="./bundle.js"></script>
  </body>
</html>

Now when you go to localhost:3000, you should see nothing in the DOM yet. Open developer tools in Chrome and make sure the console is clear with no errors. If you have React Dev Tools installed, go to that tab and select the Bot component. On the right, you should be able to expand the State object and see that the initial message from the Bot was saved into state.

Back to the code, inside your Render method in the Bot component, add an input element.

const input = (
  <div className="input-container">
    <input
      value={this.state.input}
      onChange={this.changeInput}
      onKeyPress={this.handleKeyPress} />
  </div>
);

return (
  <div className="Bot">
    {input}
  </div>
);

This element uses two new functions called changeInput (updates the state and inherently the DOM when the user changes the input text) and handleKeyPress which will trigger submission of the message upon hitting the Return key.

changeInput = e => {
  let input = e.target.value;
  this.setState({ input: input });
};

handleKeyPress = e => {
  (e.key === 'Enter') &&
  (this.state.input !== "") &&
  this.submitMessage();
}

With the project rebundled, you should see this input field in the browser now. Before testing it out further, add the submitMessage function which is called above when the user submits a non-empty message, and a function to persist those messages in state:

recordMessage = (sender, message) => {
  this.setState({
    messages: this.state.messages.concat({
      SENDER: sender,
      MESSAGE: message
    })
  })
}

submitMessage = () => {
  let message = this.state.input;
  this.setState({ input: "" })
  this.recordMessage('USER', message)

  fetch('/api/utterances/message', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      Message: message,
      SessionKey: this.state.sessionkey
    })
  }).then(res => res.json())
    .then(body => {
      this.recordMessage('BOT', body.Response.message)
  })
}

Now we should be fully communicating with Lex from our front end! Check the browser console again to make sure no errors are showing up (and it’s always a good idea to keep an eye on the backend as well). You should see an updated list of messages in State (if you’re using React Dev Tools) as you submit new ones. Make sure the MESSAGE and SENDER values are what you’d expect.

Add an element to show the messages in the DOM like so:

const messages = (
  <div className="messages">
    <div className="messages-inner">
      {
        this.state.messages.map((message, index) => {
          return (
            <div 
              key={message.SENDER + index} 
              className={`message ${message.SENDER}`}>
              {message.MESSAGE}
            </div>
          )
        })
      }
    </div>
  </div>
)

// const input = ...

return (
  <div className="Bot">
    {messages}
    {input}
  </div>
);

You can add some styles to make the view a little less dry; I also added some functionality to scroll the chat to the bottom every time the conversation is updated, which you can see here.

And that’s it!

Conclusion

Now your chatbot is alive and ready to grow / change with any new features you might come up with. Consider looking deeper into Lex to see how you can really expand the logic that goes into your chat conversations. You can think about persisting chats with Loopback (read about PersistedModels, which you can connect to many different kinds of datasources). On the front end, maybe you’ll want to do something more interactive like have a certain response (or input) affect the DOM, or perhaps cache the conversation and session key so that the chat can be persisted locally on the client side and perhaps picked back up later.

I’d love to hear how else others might tackle parts of this project, and otherwise hope that this has been a helpful run-through for those who are just getting into some of these tools like myself.

If you want to clone the finished project (minus the stuff that happens in Lex), view my repository on Github here, and please star it if you find it a useful resource!

In an attempt to ease the process of learning some of the new tools I’m particularly interested in, I thought it might be useful to someone out there (including future me, to be honest) to put together a little tutorial on how I would setup a new React+Express+Webpack project. I’m going to be a little redundant sometimes in this one because I remember what it was like to be at a stage where most of this was way over my head, and I certainly would have appreciated having a guide like this to follow.

Without further ado, let’s jump right into it! To follow this tutorial, you’ll need to have Node and npm installed, and a command line interface and text editor open.

tl;dr A super simple and verbose guide to creating a web app using React, Express and Webpack.

Project Structure

Using the command line, create a new project directory and the necessary files and subdirectories we’ll need. Note that this is a relatively arbitrary setup; you can adapt the structure however you need to fit your project, but this is typically how I’d arrange things for a new or minimal app.

mkdir myproject && cd $_

# the Express server file:
touch index.js

# Webpack configuration:
touch webpack.config.js

# React source directory and file
mkdir src && touch src/index.js

# Static files
mkdir dist && touch dist/index.html

We should have things setup now like so:

.
├── dist
│   └── index.html
├── index.js
├── src
│   └── index.js
└── webpack.config.js

We will eventually serve our project from the dist folder; src contains the React source code that Webpack will transpile (using Babel) so that it can be loaded in index.html and understood by any modern browser.

Setting Up React

To create the React part of the project, first install react using npm:

npm i -D react react-dom

A more verbose version of the above would be:

npm install --save-dev react react-dom

I’ll use the previous abbreviations for the rest of the tutorial, but if this is your first experience using npm, I recommend reading the docs to learn about all the flags / options available.

We should now have a package.json file in our root directory with those two packages listed under devDependencies.

Next, code a simple React component in src/index.js:

import React, { Component } from 'react';
import ReactDOM from 'react-dom';

class App extends Component {
  render() {
    return (
      <div>
        Hello world.
      </div>
    )
  }
}

ReactDOM.render(
  <App />,
  document.getElementById('root'));

If you’re not already super comfortable with React and the rest of these tools, I recommend moving forward and not trying to extend any of the above code quite yet. Get everything up and running first, then go crazy. Always.

Setting Up Webpack

We are going to use the Babel transpiler to morph our React project into code that is browser-ready, so install a few packages for that:

npm i -D babel-core babel-loader babel-preset-env babel-preset-react

Note the inclusion of babel-preset-env and the exclusion of babel-preset-es2015 which most tutorials out there still use. The latter is now deprecated.

We’ll use Webpack to actually coordinate the use of Babel here, so install it next:

npm i -D webpack webpack-dev-server

Next, configure Webpack in webpack.config.js:

const path = require('path');

module.exports = {
  entry: './src/index.js',
  output: {
    filename: 'bundle.js',
    publicPath: '/',
    path: path.resolve(__dirname, 'dist')
  },
  module: {
    rules: [
      {
        test: /\.js$/,
        exclude: /node_modules/,
        use: {
          loader: 'babel-loader',
          options: {
            "presets" : ["env", "react"]
          }
        }
      }
    ]
  }
};

This is a minimal Webpack configuration, specifying an entry point for our source code, the output destination, and the packages we are running against code that matches the test patterns and their loaders and presets.

Bundling the Project

In the package.json file in the root of project directory, add scripts so that we can run webpack and webpack-dev-server:

  // ...
  "scripts": {
    "start": "node index.js",
    "start-dev": "./node_modules/.bin/webpack-dev-server --content-base dist/",
    "build": "./node_modules/.bin/webpack"
  },
  // ...

Now running npm run build should render dist/bundle.js. No need to open that new file; it’s rather big, and full of code that you and I will never need to really know about. The important thing is that we can now load this bundled code in our html (dist/index.html), and that can simply be set up like so:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8">
    <title>React Express Boilerplate</title>
  </head>
  <body>
    <div id="root"></div>
    <script src="./bundle.js"></script>
  </body>
</html>

The empty div with the id of “root” will be replaced by our React component. That happens because of the last three lines in src/index.js. “Root” is of course an arbitrary name; you can use whatever name you’d like, and this is also a good moment to understand, if you’re new to React, that you can render more than one top-level component in a project. In other words, you could have another div with the id of “Root-2”, and copy the render method in the React code to render the same component (or perhaps another) into that element. We won’t do that here, but it’s good to keep in mind how flexible / extendable many of these tools really are at the end of the day!

Setting Up Express

Finally, install Express, which we’ll use to serve the project:

npm i -S express webpack-dev-middleware

And write the Express server in ./index.js:

const express = require('express');
const app = express();
const path = require('path');

const webpack = require('webpack');
const config = require('./webpack.config');
const compiler = webpack(config);

app.use(require('webpack-dev-middleware')(compiler, {
  publicPath: config.output.publicPath
}));

app.get('*', (req, res) => {
  res.sendFile('dist/index.html', { root: __dirname });
});

app.listen(3000);

You can now serve your project with npm run start! Open localhost:3000 in a browser to view the project as is.

With the start-dev script that we wrote earlier, we use webpack-dev-server to hot-reload the project in the browser as assets are updated. To also automatically rebundle the project anytime a change is made to the React source code, add --watch to the build command (note: you'll need two tabs open to have this running in one while the Express server is running in another).

Conclusion

And that’s it! Or, that’s the beginning of it. There is so much more you can do with React, Express, Webpack and any other tools that might fit into your project. But hopefully this is a solid starting point, or at least a resource that you can cherrypick something relevant to factor into a different setup that you’re developing.

P.S.

I usually add another babel preset babel-preset-stage-2 so that I can use class properties within my React components. You can use that or babel-transform-class-properties to handle this (npm install whichever, then add it to the Babel presets object in your Webpack config, like: "presets”: ["env", "react", "stage-2"]).

P.S.S.

I’ve uploaded the result of this tutorial to a repository on Github here.