<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Liberocks on Medium]]></title>
        <description><![CDATA[Stories by Liberocks on Medium]]></description>
        <link>https://medium.com/@liberocks?source=rss-95a794b6046------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*l-7yMxXTIhiwPT8IUaRW8g.jpeg</url>
            <title>Stories by Liberocks on Medium</title>
            <link>https://medium.com/@liberocks?source=rss-95a794b6046------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 13:36:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@liberocks/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AWS API Gateway Websocket + Lambda = ]]></title>
            <link>https://liberocks.medium.com/aws-api-gateway-websocket-lambda-efcddbc128e2?source=rss-95a794b6046------2</link>
            <guid isPermaLink="false">https://medium.com/p/efcddbc128e2</guid>
            <category><![CDATA[aws-api-gateway]]></category>
            <category><![CDATA[websocket]]></category>
            <category><![CDATA[redis]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[lambda]]></category>
            <dc:creator><![CDATA[Liberocks]]></dc:creator>
            <pubDate>Sat, 29 Feb 2020 15:24:32 GMT</pubDate>
            <atom:updated>2020-03-02T02:03:50.832Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VWsQ-M9xBjkT_BGRMuKl-A.png" /></figure><p>Websocket is used for a long running and stateful communication, while Lambda is used for a very short and stateless process. It seems counter-intuitive to couple them altogether, don’t they? But really AWS API Gateway Websocket and Lambda pose great possibility if we dare to combine them.</p><h3>TL;DR</h3><p>All the code in this article can be seen in this repository : <a href="https://github.com/linerocks/medium-websocket-serverless">medium-websocket-serverless</a>.</p><h3>Overview</h3><p>We’re gonna start this with definitions. According to <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API">Mozilla</a> ,</p><blockquote>The <strong>WebSocket API</strong> is an advanced technology that makes it possible to open a two-way interactive communication session between the user’s browser and a server. With this API, you can send messages to a server and receive event-driven responses without having to poll the server for a reply.</blockquote><p>To elaborate, whilst REST only responds if there is any request, websocket can update the data when there is event fired regardless users request it or not. Historically, building websocket required setting up fleets of hosts that were responsible for managing the persistent connections that underlie the websocket protocol.</p><p>Websocket itself is not a new technology, but it’s not more famous than REST API. This might be because all the hassle to maintain architecture using websocket, however AWS makes it easier since it was launched back in 2018. With API Gateway Websocket, it’s no longer necessary to administer the connections, thus we can focus on business logic. We just need to write down the flow upon inbound request on each route.</p><p>The very bare minimum typical application of websocket comprises at least three routes :</p><ul><li><strong>$connect </strong>— This route is used when user first connects to our API</li><li><strong>$disconnect</strong> — This route is used when user disconnects from our API</li><li><strong>$default</strong> — This route is used to handle inbound request when there are no matching defined route key</li></ul><p>On this project, I employ Redis as a intermediary between the websocket connection and the lambda to store clients state. Suppose we want to create a simple chat application.</p><h3>The Connection Manager</h3><p>This is the most important part of this article, the connection manager. Here is the complete code,</p><pre>const AWS = require(&#39;aws-sdk&#39;)<br>const redis = require(&#39;async-redis&#39;)</pre><pre>export class Connection {<br>  constructor (params = {}) {<br>    this.host = params.host<br>    this.port = parseInt(params.port)<br>    this.password = params.password<br>  }</pre><pre>  init (event) {<br>    this.client = redis.createClient({<br>      host: this.host,<br>      port: this.port,<br>      password: this.password<br>    })</pre><pre>    this.gateway = new AWS.ApiGatewayManagementApi({<br>      apiVersion: &#39;2018-11-29&#39;,<br>      endpoint: event.requestContext.domainName + &#39;/&#39; + event.requestContext.stage<br>    })<br>  }</pre><pre>  addConnection (key, connectionId) {<br>    return this.client.sadd(key, connectionId)<br>  }</pre><pre>  removeConnection (key, connectionId) {<br>    return this.client.srem(key, connectionId)<br>  }</pre><pre>  getConnections (key) {<br>    return this.client.smembers(key)<br>  }</pre><pre>  async publish (key, event, message = null) {<br>    if (!message) message = event.body<br>    const connections = await this.getConnections(key)</pre><pre>    for (const connectionId of connections) {<br>      if (event.requestContext.connectionId === connectionId) continue<br>      try {<br>        await this.gateway.postToConnection({ ConnectionId: connectionId, Data: message }).promise()<br>      } catch (e) {<br>        this.removeConnection(connectionId)<br>      }<br>    }<br>  }<br>}</pre><p>I wrap this connection manager in a class which is basically in charge storing user connectionId and publish the data to all the members of the establishing connection. Each connected client has unique identity namely <em>connectionId</em>. This is auto-generated and we can obtain them when there is incoming event.</p><pre>constructor (params = {}) {<br>    this.host = params.host<br>    this.port = parseInt(params.port)<br>    this.password = params.password<br>  }</pre><p>The connection class contructor only require the Redis connection parameters, nothing happens.</p><pre>init (event) {<br>    this.client = redis.createClient({<br>      host: this.host,<br>      port: this.port,<br>      password: this.password<br>    })<br>    this.gateway = new AWS.ApiGatewayManagementApi({<br>      apiVersion: &#39;2018-11-29&#39;,<br>      endpoint: event.requestContext.domainName + &#39;/&#39; + event.requestContext.stage<br>    })<br>}</pre><p>Immediately after instantiating the object, we initiate both the Redis client and AWS API Gateway Management.</p><pre>addConnection (key, connectionId) {<br>    return this.client.sadd(key, connectionId)<br>  }</pre><pre>removeConnection (key, connectionId) {<br>    return this.client.srem(key, connectionId)<br>  }</pre><pre>getConnections (key) {<br>    return this.client.smembers(key)<br>  }</pre><p>This class own three Redis related methods : adding connection, removing connection, and retrieving connections. Because the connectionId is unique, we could use the Redis set. SADD will push the connectionId to the set. SREM will remove the specified connectionId from the set. And SMEMBERS will return the set value.</p><pre>async publish (key, event, message = null) {<br>    if (!message) message = event.body<br>    const connections = await this.getConnections(key)</pre><pre>    for (const connectionId of connections) {<br>      if (event.requestContext.connectionId === connectionId) continue<br>      try {<br>        await this.gateway.postToConnection({ ConnectionId: connectionId, Data: message }).promise()<br>      } catch (e) {<br>        this.removeConnection(connectionId)<br>      }<br>    }<br>  }</pre><p>Finally for publishing an event to all the connected member but the event owner using gateway’s <em>postToConnection</em>. As a note, if the gateway find that a user is no longer with us, an error will be thrown. We need to catch it and remove the non active user from the group.</p><h3>serverless.yml</h3><p>Before moving to route handler, first we handle the serverless configuration on serverless.yml as below,</p><pre># NOTE: update this with your service name<br>service: websocket-serverless</pre><pre># Create an optimized package for our functions <br>package:<br>  individually: true</pre><pre>plugins:<br>  - serverless-bundle # Package our functions with Webpack<br>  - serverless-dotenv-plugin # Load .env as environment variables</pre><pre>provider:<br>  name: aws<br>  runtime: nodejs10.x<br>  stage: dev<br>  region: ap-southeast-1</pre><pre>environment:<br>    REDIS_HOST: ${env:REDIS_HOST}<br>    REDIS_PORT: ${env:REDIS_PORT}<br>    REDIS_PASSWORD: ${env:REDIS_PASSWORD}</pre><pre>functions:<br>  connect:<br>    handler: functions/connect.handler<br>    events:<br>      - websocket:<br>          route: $connect<br>  default:<br>    handler: functions/default.handler<br>    events:<br>      - websocket:<br>          route: $default<br>  disconnect:<br>    handler: functions/disconnect.handler<br>    events:<br>      - websocket:<br>          route: $disconnect</pre><p>Overall the configuration is trivial, but the functions in the bottom. These functions specify which handler should handle which route. As we discuss above, we will be using three basic routes.</p><h3>$connect</h3><pre>const { Connection } = require(&#39;../libs/connections&#39;)</pre><pre>let connection<br>const host = process.env.REDIS_HOST<br>const port = process.env.REDIS_PORT<br>const password = process.env.REDIS_PASSWORD</pre><pre>exports.handler = async (event, context, callback) =&gt; {<br>  if (!connection) {<br>    connection = new Connection({ host, port, password })<br>    connection.init(event)<br>  }</pre><pre>await connection.addConnection(&#39;myRoom&#39;, event.requestContext.connectionId)<br>  await connection.publish(&#39;myRoom&#39;, event, `${event.requestContext.connectionId} joined to the room.`)</pre><pre>return { statusCode: 200 }<br>}</pre><p>When users join to the connection, we want to store their connectionId and inform all the existing users that they are joining. Assume we call the connection as <em>myRoom</em>. All the connectionId will be stored in a set in Redis called <em>myRoom</em>. Once the connection is successfully kept, publish a notification to all the member using publish. Note that all variables outside the handler will be cached by lambda, so next time we need, the connection had been already available.</p><h3>$disconect</h3><pre>const { Connection } = require(&#39;../libs/connections&#39;)</pre><pre>let connection<br>const host = process.env.REDIS_HOST<br>const port = process.env.REDIS_PORT<br>const password = process.env.REDIS_PASSWORD</pre><pre>exports.handler = async (event, context, callback) =&gt; {<br>  if (!connection) {<br>    connection = new Connection({ host, port, password })<br>    connection.init(event)<br>  }</pre><pre>  await connection.removeConnection(&#39;myRoom&#39;, event.requestContext.connectionId)<br>  await connection.publish(&#39;myRoom&#39;, event, `${event.requestContext.connectionId} left the room.`)<br>  return { statusCode: 200 }<br>}</pre><p>In contrast to $connect event, $disconnect event will remove the user’s connectionId and inform to all member that this user leave the room.</p><h3>$default</h3><pre>const { Connection } = require(&#39;../libs/connections&#39;)</pre><pre>let connection<br>const host = process.env.REDIS_HOST<br>const port = process.env.REDIS_PORT<br>const password = process.env.REDIS_PASSWORD</pre><pre>exports.handler = async (event, context, callback) =&gt; {<br>  if (!connection) {<br>    connection = new Connection({ host, port, password })<br>    connection.init(event)<br>  }</pre><pre>  await connection.publish(&#39;myRoom&#39;, event)<br>  return { statusCode: 200 }<br>}</pre><p>This routes only relay the incoming message from a user to all member. That’s all.</p><h3>Testing</h3><p>For testing, we need to install wscat first to simply try the ws connection.</p><pre>$ sudo npm install -g wscat</pre><p>After we set the redis connection parameter and deploy the serverless application, we will see a prompt like this</p><pre>Serverless: DOTENV: Loading environment variables from .env:<br>Serverless:   - REDIS_HOST<br>Serverless:   - REDIS_PORT<br>Serverless:   - REDIS_PASSWORD<br>Serverless: Bundling with Webpack...<br>Serverless: Packaging service...<br>Serverless: Creating Stack...<br>Serverless: Checking Stack create progress...<br>........<br>Serverless: Stack create finished...<br>Serverless: Uploading CloudFormation file to S3...<br>Serverless: Uploading artifacts...<br>Serverless: Uploading service connect.zip file to S3 (158.9 KB)...<br>Serverless: Uploading service default.zip file to S3 (158.7 KB)...<br>Serverless: Uploading service disconnect.zip file to S3 (158.91 KB)...<br>Serverless: Validating template...<br>Serverless: Updating Stack...<br>Serverless: Checking Stack update progress...<br>.....................................................................<br>Serverless: Stack update finished...<br>Service Information<br>service: serverless-nodejs-starter<br>stage: dev<br>region: ap-southeast-1<br>stack: serverless-nodejs-starter-dev<br>resources: 24<br>api keys:<br>  None<br>endpoints:<br>  wss://1ejk8cib44.execute-api.ap-southeast-1.amazonaws.com/dev<br>functions:<br>  connect: serverless-nodejs-starter-dev-connect<br>  default: serverless-nodejs-starter-dev-default<br>  disconnect: serverless-nodejs-starter-dev-disconnect<br>layers:<br>  None<br>Serverless: Run the &quot;serverless&quot; command to setup monitoring, troubleshooting and testing.</pre><p>Save the returned endpoints and connect to it using wscat.</p><pre>$ wscat -c wss://5acet940oc.execute-api.ap-southeast-1.amazonaws.com/dev</pre><p>Here is the screenshot on my machine</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xs_YSVCCuz0SiFq_Iv22Ig.gif" /></figure><h3>Summary and Lesson Learned</h3><p>We can easily create real time application using AWS API Gateway and Lambda Serverless. Yet we have to maintain the state in persistent volume so that we can maintain the connection context.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efcddbc128e2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ticketing System : A Case Study of MongoDB Transactions and Atomicity]]></title>
            <link>https://liberocks.medium.com/ticketing-system-a-case-study-of-mongodb-transactions-and-atomicity-883e4f655905?source=rss-95a794b6046------2</link>
            <guid isPermaLink="false">https://medium.com/p/883e4f655905</guid>
            <category><![CDATA[mongoose]]></category>
            <category><![CDATA[atomicity]]></category>
            <category><![CDATA[express]]></category>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[transactions]]></category>
            <dc:creator><![CDATA[Liberocks]]></dc:creator>
            <pubDate>Fri, 28 Feb 2020 00:42:10 GMT</pubDate>
            <atom:updated>2020-02-28T00:55:52.538Z</atom:updated>
            <content:encoded><![CDATA[<h3>Ticketing System : A Case Study of MongoDB Transactions and Atomicity</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*V6tzimMx4Kc1oshi" /><figcaption>Photo by <a href="https://unsplash.com/@nhillier?utm_source=medium&amp;utm_medium=referral">Nick Hillier</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>A ticketing system holds crucial duty on guaranteeing each taken slot belongs to exactly a passenger. This typical system under the hood requires atomicity and rollback capabilities of the database; where in MongoDB, rollback is analogous to transactions. In this article, I want to discuss these MongoDB capabilities through ticketing-system case study.</p><h3>TL;DR</h3><p>All the code in this article can be seen in this repository : <a href="https://github.com/linerocks/medium-mongodb-atomicity-transaction">medium-mongodb-atomicity-transaction</a>.</p><h3>Definition And Concept</h3><p>I start this post with definitions of atomicity and transactions. According to MongoDB Documentation, atomicity is described as</p><blockquote>In MongoDB, a write operation is atomic on the level of a single document, even if the operation modifies multiple embedded documents <em>within</em> a single document.</blockquote><p>I found an article nicely recapped the atomicity operation in Mongoose in on table as following,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*niQeJQfqMfaBqGUc2QDZfQ.png" /><figcaption><a href="https://masteringjs.io/tutorials/mongoose/update">https://masteringjs.io/tutorials/mongoose/update</a></figcaption></figure><p>In general, you should use save() to update a document in Mongoose, unless you need an atomic update.</p><p>While transactions is described as,</p><blockquote>For situations that require atomicity of reads and writes to multiple documents (in a single or multiple collections), MongoDB supports multi-document transactions. With distributed transactions, transactions can be used across multiple operations, collections, databases, documents, and shards.</blockquote><p>The transactions is noted by these interrelated operations : <em>establishing session</em>, <em>starting transaction</em>, <em>committing transaction</em>, <em>aborting transaction</em>, and <em>ending session</em>. Each operation in a transaction use same session, thus the session has to be started in the very beginning. Then the transaction is started afterwards. The business process happens in this stage. If the process runs as we want, the db operation needs to be committed. And if in the middle of process something unwanted happens, we need to immediately abort the transaction and consequently all the db operation won’t even touch the db. Finally, we end the transaction no matter the status of the process.</p><h3>The System Overview</h3><p>My version of basic ticketing system includes at least two function : creating schedule and booking the slot. I call them as schedule-endpoint and ticket-endpoint. While the schedule-endpoint demonstrates transaction operation, the ticket-endpoint demonstrate both of atomic and transaction operations.</p><p>The schedule-endpoint will create the schedule for appointed train and slots afterwards. The schedule won’t be committed to db if there is any of slot creation fails and vice-versa.</p><p>As for ticket-endpoint, the slot booking is queued for concurrent requests. If the request successfully grab the slot, the ticket will be made. Otherwise, if the request unsuccessfully grab the slot, the ticket won’t be made and the request is rejected.</p><p>To accommodate these needs, here are the db schema designs; nothing specials.</p><ul><li>Schedule schema</li></ul><pre>const mongoose = require(&#39;mongoose&#39;)<br>const { Schema } = mongoose</pre><pre>const schema = new Schema({<br>  name: { type: String, required: true },<br>  quota: { type: Number, required: true },<br>  departure: { type: Date, required: true }<br>})</pre><pre>module.exports = mongoose.model(&#39;Schedule&#39;, schema)</pre><ul><li>Slot Schema</li></ul><pre>const mongoose = require(&#39;mongoose&#39;)<br>const { Schema } = mongoose<br>const schema = new Schema({<br>  scheduleId: { type: mongoose.ObjectId },<br>  seatNumber: { type: Number, required: true },<br>  available: { type: Boolean, default: true },<br>  bookedBy: { type: mongoose.ObjectId, default: null }<br>})<br>module.exports = mongoose.model(&#39;Slot&#39;, schema)</pre><ul><li>Ticket Schema</li></ul><pre>const mongoose = require(&#39;mongoose&#39;)<br>const { Schema } = mongoose</pre><pre>const schema = new Schema({<br>  seatNumber: { type: Number, required: true },<br>  passengerId: { type: mongoose.ObjectId, required: true },<br>  scheduleId: { type: mongoose.ObjectId, required: true },<br>  slotId: { type: mongoose.ObjectId, required: true }<br>})</pre><pre>module.exports = mongoose.model(&#39;Ticket&#39;, schema)</pre><h3>Schedule Endpoint</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/925/1*UL0tE67BCZEwMXlEVqI8Tg.png" /><figcaption>Multi-document creation for schedule and slots</figcaption></figure><pre>app.post(&#39;/schedule&#39;, async (req, res, next) =&gt; {<br>  const session = await mongoose.startSession()<br>  session.startTransaction()</pre><pre>try {<br>    // Schedule collection<br>    const data = {<br>      name: req.body.name,<br>      quota: req.body.quota,<br>      departure: DateTime.fromFormat(req.body.departure, &#39;yyyy-MM-dd HH:mm ZZZ&#39;).toJSDate()<br>    }<br>    const schedule = await Schedule.create([data], { session })</pre><pre>// Slot collection<br>    const slots = [...Array(data.quota).keys()].map(i =&gt; ({<br>      scheduleId: schedule[0]._id,<br>      seatNumber: i + 1<br>    }))<br>    await Slot.create(slots, { session })</pre><pre>await session.commitTransaction()<br>    res.json({ data: schedule[0] })<br>  } catch (error) {<br>    await session.abortTransaction()<br>    next(error)<br>  } finally {<br>    session.endSession()<br>  }<br>})</pre><p>Here are some note on code above:</p><ul><li>First we collect user request data by parsing request body. This request contains <em>schedule name</em>, <em>slot quota</em>, and <em>departure date</em>. This data is then used to make the schedule document. If the schedule is smoothly created, we proceed creating slot as much as specified quota.</li><li>The slots are created accordingly regarding to incremental seat number. Other than seat number, slots also use schedule object id as reference. Then the <em>create</em> operation creates the slots at once.</li><li>If there is no exception thrown, we agree to commit these documents. Otherwise, the transaction is aborted and the error is informed to requester.</li></ul><h3>Ticket Endpoint</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OjFCyaQtCbSeHU_wXEo6Pw.png" /><figcaption>Atomic operation on booking slot</figcaption></figure><pre>app.post(&#39;/ticket&#39;, async (req, res, next) =&gt; {<br>  const session = await mongoose.startSession()<br>  session.startTransaction()</pre><pre>try {<br>    // Find slot<br>    const query = {<br>      scheduleId: req.body.scheduleId,<br>      seatNumber: req.body.seatNumber,<br>      available: true<br>    }<br>    const slot = await Slot.findOneAndUpdate(query, {<br>      $set: {<br>        bookedBy: req.body.passengerId,<br>        available: false<br>      }<br>    }, { useFindAndModify: false })<br>    if (!slot) throw new BadRequest(&#39;Slot is not available&#39;)</pre><pre>    // Create ticket<br>    const ticket = await Ticket.create([{<br>      seatNumber: req.body.seatNumber,<br>      passengerId: req.body.passengerId,<br>      scheduleId: req.body.scheduleId,<br>      slotId: slot._id<br>    }], { session })</pre><pre>await session.commitTransaction()<br>    res.json({ message: &#39;Ticket is successfully booked&#39;, ticket: ticket[0] })<br>  } catch (error) {<br>    await session.abortTransaction()<br>    next(error)<br>  } finally {<br>    session.endSession()<br>  }<br>})</pre><p>Here are some note on code above:</p><ul><li>As shown in table above, we could use <em>findOneAndUpdate</em> operation for conducting atomic operation. We won’t get this atomicity behaviour if for instance we choose to update the slot as below</li></ul><pre>const query = {<br>      scheduleId: req.body.scheduleId,<br>      seatNumber: req.body.seatNumber,<br>      available: true<br>    }<br>const slot = await Slot.findOne(query)<br>slot.available = false<br>slot.bookedBy = req.body.passengerId<br>await slot.save()</pre><ul><li>We try to find specified slot with particular query. If there are concurrent request at same time, only the first request gets the document, while the subsequent request won’t find the specified slot due the slot availability has been updated to be <em>false</em>.</li><li>In the end, if slot is managed to be booked by user whom <em>passengerId</em> is specified, the ticket will be issued.</li></ul><h3>Testing</h3><p>Assume we work using microservices paradigm where this ticketing service runs in multiple container. To cut the long short, we run the express using bash background process instead of docker/kubernetes. For example, we run 5 container at once.</p><pre>for port in {3000..3004}; do<br>    nodemon index ${port} &amp;<br>done</pre><p>Then create the schedule only on one container :</p><pre>curl -X POST -H &quot;Content-Type: application/json&quot; -d &#39;{&quot;name&quot;:&quot;Argo Wilis&quot;,&quot;quota&quot;:4,&quot;departure&quot;:&quot;2020-02-29 20:07 +0700&quot;}&#39; 0.0.0.0:3000/schedule</pre><p>yielding response:</p><pre>{<br>    &quot;data&quot;: {<br>        &quot;_id&quot;:&quot;5e5852d781636c3e4bc72083&quot;,<br>        &quot;name&quot;:&quot;Argo Wilis&quot;,<br>        &quot;quota&quot;:4,<br>        &quot;departure&quot;:&quot;2020-02-29T13:07:00.000Z&quot;,<br>        &quot;__v&quot;:0<br>    }<br>}</pre><p>Then simultaneously book the same seat number:</p><pre>curl -X POST -H &quot;Content-Type: application/json&quot; -d &#39;{&quot;scheduleId&quot;:&quot;5e584fc617f7d2279ec1f1f2&quot;,&quot;seatNumber&quot;:4,&quot;passengerID&quot;:&quot;5e584ba2b76ce81243782255&quot;}&#39; 0.0.0.0:3000/ticket &amp; curl -X POST -H &quot;Content-Type: application/json&quot; -d &#39;{&quot;scheduleId&quot;:&quot;5e584fc617f7d2279ec1f1f2&quot;,&quot;seatNumber&quot;:4,&quot;passengerID&quot;:&quot;5e584ba2b76ce81243782256&quot;}&#39; 0.0.0.0:3001`/ticket &amp; curl -X POST -H &quot;Content-Type: application/json&quot; -d &#39;{&quot;scheduleId&quot;:&quot;5e584fc617f7d2279ec1f1f2&quot;,&quot;seatNumber&quot;:4,&quot;passengerID&quot;:&quot;5e584ba2b76ce81243782257&quot;}&#39; 0.0.0.0:3002/ticket &amp; curl -X POST -H &quot;Content-Type: application/json&quot; -d &#39;{&quot;scheduleId&quot;:&quot;5e584fc617f7d2279ec1f1f2&quot;,&quot;seatNumber&quot;:4,&quot;passengerID&quot;:&quot;5e584ba2b76ce81243782258&quot;}&#39; 0.0.0.0:3003/ticket &amp; curl -X POST -H &quot;Content-Type: application/json&quot; -d &#39;{&quot;scheduleId&quot;:&quot;5e584fc617f7d2279ec1f1f2&quot;,&quot;seatNumber&quot;:4,&quot;passengerID&quot;:&quot;5e584ba2b76ce81243782259&quot;}&#39; 0.0.0.0:3004/ticket &amp;</pre><p>resulting response :</p><pre>&gt; {&quot;message&quot;:&quot;Slot is not available&quot;}<br>&gt; {&quot;message&quot;:&quot;Slot is not available&quot;}<br>&gt; {&quot;message&quot;:&quot;Slot is not available&quot;}<br>&gt; {&quot;message&quot;:&quot;Slot is not available&quot;}<br>&gt; {<br>    &quot;message&quot;:&quot;Ticket is successfully booked&quot;,<br>    &quot;ticket&quot;: {<br>        &quot;_id&quot;:&quot;5e585f8918fdd40a6bf343ee&quot;,<br>        &quot;seatNumber&quot;:4,<br>        &quot;passengerId&quot;:&quot;5e584ba2b76ce81243782258&quot;,<br>        &quot;scheduleId&quot;:&quot;5e5852d781636c3e4bc72083&quot;,<br>        &quot;slotId&quot;:&quot;5e5852d881636c3e4bc72087&quot;,<br>        &quot;__v&quot;:0<br>    }<br>}</pre><h3>Summary and Lesson Learned</h3><p>While MongoDB provides atomicity and transactional operation with battery included, we have to design the process to comply how these operation works.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=883e4f655905" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Starting Serverless Project with Mongoose]]></title>
            <link>https://liberocks.medium.com/starting-serverless-project-with-mongoose-5c95a7b5bb92?source=rss-95a794b6046------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c95a7b5bb92</guid>
            <category><![CDATA[mongoose]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[mongodb]]></category>
            <dc:creator><![CDATA[Liberocks]]></dc:creator>
            <pubDate>Wed, 19 Feb 2020 09:00:33 GMT</pubDate>
            <atom:updated>2020-02-19T09:13:29.297Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fkRj1XLWkA9b_XDD" /><figcaption>Photo by <a href="https://unsplash.com/@casparrubin?utm_source=medium&amp;utm_medium=referral">Caspar Camille Rubin</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In this post, we’re gonna build, test locally, and deploy a Serverless API, that can access a MongoDB using mongoose</p><h3>TL;DR</h3><p>All the code in this article can be seen in this repository : <a href="https://github.com/linerocks/medium-serverless-mongoose">medium-serverless-mongoose</a>.</p><h3>Setting up serverless</h3><p>Before starting project using AWS services, we should configure the <a href="https://aws.amazon.com/cli/">AWS CLI</a> first. We need Python to install <strong>awscli</strong>; Python 2 version 2.65+ or Python 3 version 3.3+. Now we install the <strong>awscli </strong>by running:</p><pre>$ sudo pip install awscli</pre><p>It’s gonna take a moment. Once it has completed, we need to configure our AWS credential. The credential looks like this:<br>- Access key ID : <strong>AKIAIOSFODNN7SAMPLEID</strong><br>- Secret access key : <strong>wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY</strong></p><p>Run the awscli with our access key id and secret key :</p><pre>$ aws configure</pre><p>We may leave the <strong>Default region name</strong> and <strong>Default output format</strong> the way they are.</p><p>For the MongoDB database, we can use the Atlas Free Tier Cluster. See the following link if you need help complete tutorial setting up the database : <a href="https://docs.atlas.mongodb.com/getting-started/">get started with Atlas</a>. All we need for this article is connection string for the lambda to connect to. The connection string for MongoDB should look like this :</p><pre>mongodb+srv://&lt;username&gt;:&lt;password&gt;<a href="http://twitter.com/cluster0">@clusterx</a>xx-cobrxxx.mongodb.net/test?retryWrites=true&amp;w=majority</pre><p>Now we are going to initiate the serverless project. To initiate NodeJS based serverless project, run the following :</p><pre>$ serverless install --url <a href="https://github.com/linerocks/serverless-nodejs-starter">https://github.com/linerocks/serverless-nodejs-starter</a> --name medium-serverless-mongoose</pre><p>and then install the modules:</p><pre>$ cd medium-serverless-mongoose<br>$ npm install</pre><p>Note that you need to install the serverless module globally : <strong>npm install serverless -g</strong>. On command above, we use the bare minimum serverless template on specified url.</p><p>Finally install the mongoose and other modules :</p><pre>$ npm install mongoose mongoose-paginate luxon lodash</pre><p>Then, to contain mongoose schema and model, we will create new directories <strong>database</strong>.</p><pre>$ mkdir -p database/schema<br>$ mkdir -p database/model</pre><p>At this point, the project structure will look like:</p><pre>medium-serverless-mongoose<br>├── database<br>│   └── model<br>│   └── schema<br>├── env.example<br>├── functions<br>│   └── hello.js<br>├── libs<br>│   └── response.js<br>├── LICENSE<br>├── package.json<br>├── package-lock.json<br>├── README.md<br>├── serverless.yml<br>└── tests<br>    └── hello.test.js</pre><h3>Building REST API</h3><p>In this article we’re gonna make a CRUD API to administer medical doctor list as in <a href="https://carbonhealth.com/">carbonhealth</a>. Our data structure will be as following mongoose schema :</p><pre>// database/schema/doctor.js</pre><pre>import mongoose from &#39;mongoose&#39;<br>import mongoosePaginate from &#39;mongoose-paginate&#39;</pre><pre>const doctor = new mongoose.Schema({<br>  firstName: { type: String, required: true },<br>  middleName: { type: String, default: null },<br>  lastName: { type: String, default: null },<br>  credentials: [{ type: String, required: true }],<br>  medicalSchools: [{ type: String, required: true }],<br>  gender: { type: String, enum: [&#39;male&#39;, &#39;female&#39;], default: null },<br>  isActive: { type: Boolean, default: false },<br>  licenses: [{ type: String, required: true }],<br>  birthDate: { type: Date, required: true }<br>}, { collation: &#39;doctor&#39;, timestamps: true })</pre><pre>doctor.plugin(mongoosePaginate)</pre><pre>export default doctor</pre><p>From this schema, we’re gonna make the doctor model :</p><pre>// database/model/doctor.js</pre><pre>import mongoose from &#39;mongoose&#39;</pre><pre>import schema from &#39;../schema/doctor&#39;</pre><pre>export default mongoose.model(&#39;Doctor&#39;, schema)</pre><p>Using these database model, let’s get started on our first API to create a doctor. This API will capture the POST request data from the user and store them in MongoDB. Here is our function :</p><pre>// functions/createDoctor.js</pre><pre>const mongoose = require(&#39;mongoose&#39;)<br>const pick = require(&#39;lodash/pick&#39;)<br>const pickBy = require(&#39;lodash/pickBy&#39;)<br>const identity = require(&#39;lodash/identity&#39;)<br>const { DateTime } = require(&#39;luxon&#39;)</pre><pre>const { DoctorModel } = require(&#39;../database/model/doctor&#39;)<br>const response = require(&#39;../libs/response&#39;)</pre><pre>let cachedDB = null</pre><pre>async function connectToDatabase (uri, options = {}) {<br>  if (!cachedDB) cachedDB = await mongoose.connect(uri)<br>}</pre><pre>export const handler = async (event, context) =&gt; {<br>  try {<br>    await connectToDatabase(process.env.MONGO_URI)</pre><pre>const data = pickBy(pick(JSON.parse(event.body), [<br>      &#39;firstName&#39;, &#39;middleName&#39;, &#39;lastName&#39;, &#39;credentials&#39;,<br>      &#39;medicalSchools&#39;, &#39;gender&#39;, &#39;isActive&#39;, &#39;licenses&#39;,<br>      &#39;birthDate&#39;<br>    ]), identity)<br>    if (data.birthDate) data.birthDate = DateTime.fromFormat(data.birthDate, &#39;yyyy-MM-dd&#39;).toJSDate()</pre><pre>const doctor = await DoctorModel.create(data)</pre><pre>return response.success({<br>      message: &#39;Doctor had been created successfully.&#39;,<br>      data: doctor.toObject()<br>    })<br>  } catch (e) {<br>    return response.fail({ message: e.message })<br>  }<br>}</pre><p>Since the request data will be contained in <strong>event.body</strong>, we pick the data from parsed JSON using <strong>lodash/pick</strong>. For handling date and time data we use <a href="https://moment.github.io/luxon/"><strong>luxon</strong></a>’s DateTime module. And for the convenience, our handler returns response via <strong>libs/response.js</strong> that cover success (code 200), fail (code 500), and badRequest (code 400) case. The rest flow regarding to mongoose remains the same as developing REST API in express or other framework.</p><p>The important note in here is everything outside <strong>handler</strong> function will be cached by the lambda. Thus for best practice connecting to MongoDB, we first check if db connection had been cached or not. If the db connection hadn’t been established, connect to MongoDB then as in variable <strong>cachedDb</strong>.</p><p>Once our <strong>createDoctor</strong> handler is done, we need to define this handler in <strong>serverless.yml</strong>. Add the handler as following:</p><pre># NOTE: update this with your service name<br>service: medium-serverless-mongoose</pre><pre># Create an optimized package for our functions <br>package:<br>  individually: true</pre><pre>plugins:<br>  - serverless-bundle # Package our functions with Webpack<br>  - serverless-offline<br>  - serverless-dotenv-plugin # Load .env as environment variables</pre><pre>provider:<br>  name: aws<br>  runtime: nodejs10.x<br>  stage: dev<br>  region: us-east-1</pre><pre>environment:<br>   MONGO_URI: ${env:MONGO_URI}</pre><pre>functions:<br>  createDoctor:<br>    handler: functions/createDoctor.handler<br>    events:<br>      - http:<br>          path: doctor<br>          method: post<br>          cors: true</pre><p>Here we are adding our <strong>createDoctor</strong> function to the configuration and associating <strong>createDoctor</strong> function to <strong>/doctor</strong> endpoint. This tells to lambda to accept the request coming from API Gateway specific to the endpoint alongwith the POST method. This pattern of using a single Lambda function to respond to a single HTTP event is very much like the <a href="https://en.wikipedia.org/wiki/Microservices">Microservices architecture</a>.</p><p>The same thing goes with <strong>getDoctor</strong> function.</p><pre>const mongoose = require(&#39;mongoose&#39;)</pre><pre>const { DoctorModel } = require(&#39;../database/model/doctor&#39;)<br>const response = require(&#39;../libs/response&#39;)</pre><pre>let cachedDB = null</pre><pre>async function connectToDatabase (uri, options = {}) {<br>  if (!cachedDB) cachedDB = await mongoose.connect(uri)<br>}</pre><pre>export const handler = async (event, context) =&gt; {<br>  try {<br>    await connectToDatabase(process.env.MONGO_URI)</pre><pre>const doctor = await DoctorModel.findById(event.pathParameters.id)<br>    if (!doctor) return response.badRequest({ message: &#39;Doctor with specified id is not found&#39; })</pre><pre>return response.success({<br>      data: doctor.toObject()<br>    })<br>  } catch (e) {<br>    return response.fail({ message: e.message })<br>  }<br>}</pre><p>Instead of parsing <strong>event.body</strong>, this GET function use <strong>event.pathParameters</strong> to specify which record of a doctor we want to see. Then edit serverless.yml for this function:</p><pre># NOTE: update this with your service name<br>service: serverless-mongoose</pre><pre># Create an optimized package for our functions <br>package:<br>  individually: true</pre><pre>plugins:<br>  - serverless-bundle # Package our functions with Webpack<br>  - serverless-offline<br>  - serverless-dotenv-plugin # Load .env as environment variables</pre><pre>provider:<br>  name: aws<br>  runtime: nodejs10.x<br>  stage: dev<br>  region: us-east-1</pre><pre>environment:<br>   MONGO_URI: ${env:MONGO_URI}</pre><pre>functions:<br>  createDoctor:<br>    handler: functions/createDoctor.handler<br>    events:<br>      - http:<br>          path: doctor<br>          method: post<br>          cors: true<br>  getDoctor:<br>    handler: functions/getDoctor.handler<br>    events:<br>      - http:<br>          path: doctor/{id}<br>          method: get<br>          cors: true</pre><p>In this <strong>serverless.yml</strong> we use environment block to define sensitive and configurable variables so that the secret aren’t gonna be committed to git repository. To access the variable, we are using <strong>process.env</strong>. This is demonstrated in our <strong>createDoctor</strong> and <strong>getDoctor</strong> functions to define the MongoDB URI.</p><p>To make the environment available, create new file called <strong>.env</strong> containing your MongoDB URI :</p><pre># adjust the connection string<br>MONGO_URI=mongodb+srv://&lt;username&gt;:&lt;password&gt;<a href="http://twitter.com/cluster0">@clusterx</a>xx-cobrxxx.mongodb.net/test?retryWrites=true&amp;w=majority</pre><p>Now we are ready to test our newly created API. We can test them locally at this point. It’s made available using <strong>serverless invoke</strong> local command. But first let’s create our mock data in directory <strong>mocks</strong> on a file called <strong>createDoctor-event.json</strong> as follows</p><pre>{<br>  &quot;body&quot;: &quot;{\&quot;firstName\&quot;:\&quot;George\&quot;,\&quot;lastName\&quot;:\&quot;Wonton\&quot;,\&quot;credentials\&quot;:[\&quot;MD\&quot;],\&quot;medicalSchools\&quot;:[\&quot;Harvard Medical School\&quot;],\&quot;gender\&quot;:\&quot;male\&quot;,\&quot;isActive\&quot;:true,\&quot;licenses\&quot;:[\&quot;SIP 2000/y70.george\&quot;],\&quot;birthDate\&quot;:\&quot;1970-03-17\&quot;}&quot;<br>}</pre><p>And to invoke the function, run this command in the project root directory.</p><pre>$ sls invoke local --function createDoctor --path mocks/createDoctor-event.json</pre><p>The response should look similar to this.</p><pre>{<br>    &quot;statusCode&quot;: 200,<br>    &quot;headers&quot;: {<br>        &quot;Access-Control-Allow-Origin&quot;: &quot;*&quot;,<br>        &quot;Access-Control-Allow-Credentials&quot;: true<br>    },<br>    &quot;body&quot;: &quot;{\&quot;message\&quot;:\&quot;Doctor had been created successfully.\&quot;,\&quot;data\&quot;:{\&quot;middleName\&quot;:null,\&quot;lastName\&quot;:\&quot;Wonton\&quot;,\&quot;credentials\&quot;:[\&quot;MD\&quot;],\&quot;medicalSchools\&quot;:[\&quot;Harvard Medical School\&quot;],\&quot;gender\&quot;:\&quot;male\&quot;,\&quot;isActive\&quot;:true,\&quot;licenses\&quot;:[\&quot;SIP 2000/y70.george\&quot;],\&quot;_id\&quot;:\&quot;5e4cec2e0ea671389038d2b4\&quot;,\&quot;firstName\&quot;:\&quot;George\&quot;,\&quot;birthDate\&quot;:\&quot;1970-03-16T17:00:00.000Z\&quot;,\&quot;createdAt\&quot;:\&quot;2020-02-19T08:05:02.713Z\&quot;,\&quot;updatedAt\&quot;:\&quot;2020-02-19T08:05:02.713Z\&quot;,\&quot;__v\&quot;:0}}&quot;<br>}</pre><p>The other CRUD function, can be clone from the repository <a href="https://github.com/linerocks/medium-serverless-mongoose">medium-serverless-mongoose</a>.</p><h3>Deploying the API</h3><p>Now that our APIs are complete, let’s deploy them. To deploy them, simply run :</p><pre>serverless deploy</pre><p>If the deployment is success, at the bottom of terminal output, we’re gonna see this important information about our API look similar as following.</p><pre>Serverless: Stack update finished...<br>Service Information<br>service: serverless-mongoose<br>stage: dev<br>region: ap-southeast-1<br>stack: serverless-mongoose-dev<br>resources: 34<br>api keys:<br>  None<br>endpoints:<br>  POST - <a href="https://omge2px3hl.execute-api.ap-southeast-1.amazonaws.com/dev/doctor">https://omge2px3hI.execute-api.ap-southeast-1.amazonaws.com/dev/doctor</a><br>  GET - <a href="https://omge2px3hl.execute-api.ap-southeast-1.amazonaws.com/dev/doctor">https://omge2px3hI.execute-api.ap-southeast-1.amazonaws.com/dev/doctor</a><br>  GET - <a href="https://omge2px3hl.execute-api.ap-southeast-1.amazonaws.com/dev/doctor/%7Bid">https://omge2px3hI.execute-api.ap-southeast-1.amazonaws.com/dev/doctor/{id</a>}<br>  PUT - <a href="https://omge2px3hl.execute-api.ap-southeast-1.amazonaws.com/dev/doctor/%7Bid">https://omge2px3hI.execute-api.ap-southeast-1.amazonaws.com/dev/doctor/{id</a>}<br>  DELETE - <a href="https://omge2px3hl.execute-api.ap-southeast-1.amazonaws.com/dev/doctor/%7Bid">https://omge2px3hI.execute-api.ap-southeast-1.amazonaws.com/dev/doctor/{id</a>}<br>functions:<br>  createDoctor: serverless-mongoose-dev-createDoctor<br>  getDoctorList: serverless-mongoose-dev-getDoctorList<br>  getDoctor: serverless-mongoose-dev-getDoctor<br>  editDoctor: serverless-mongoose-dev-editDoctor<br>  deleteDoctor: serverless-mongoose-dev-deleteDoctor<br>layers:<br>  None<br>Serverless: Run the &quot;serverless&quot; command to setup monitoring, troubleshooting and testing.</pre><p>After deployed to the cloud, we can access the API directly.</p><pre>curl <a href="https://omge2px3hi.execute-api.ap-southeast-1.amazonaws.com/dev/doctor/%7Bid">https://omge2px3hI.execute-api.ap-southeast-1.amazonaws.com/dev/doctor/</a>5e4cd837daa680511f6399b1</pre><p>Resulting the same data as in local API.</p><pre>{&quot;data&quot;:{&quot;middleName&quot;:null,&quot;lastName&quot;:&quot;Wonton&quot;,&quot;credentials&quot;:[&quot;MD&quot;],&quot;medicalSchools&quot;:[&quot;Harvard Medical School&quot;],&quot;gender&quot;:&quot;male&quot;,&quot;isActive&quot;:true,&quot;licenses&quot;:[&quot;SIP 2000/y70.george&quot;],&quot;_id&quot;:&quot;5e4cd837daa680511f6399b1&quot;,&quot;firstName&quot;:&quot;Jorge&quot;,&quot;birthDate&quot;:&quot;1970-03-16T17:00:00.000Z&quot;,&quot;createdAt&quot;:&quot;2020-02-19T06:39:51.812Z&quot;,&quot;updatedAt&quot;:&quot;2020-02-19T07:08:53.482Z&quot;,&quot;__v&quot;:0}}</pre><p>That’s all about getting started with lambda and mongoose. There are still a lot of aspects to take care of like AWS resources, authorization, etc. Yet for the simplicity of the article, we don’t cover them all here.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c95a7b5bb92" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>