Contributed by Talia Trilling, Senior Software Engineer @ eatsa

Most engineers have experience working on legacy codebases. Unless you’re an early hire at a startup, you probably don’t have much of a choice; even those of us asked to build out new codebases are usually first asked to cut our teeth on an existing codebase. These facts are in direct tension with technology culture on the whole, and the obsession with anything new and radical. I’ve always been a bit of a skeptic on that front (in that sometimes I think people overly favor what is new versus what is established), but my recent experiences as a backend engineer at eatsa led me to challenge this assumption.

eatsa was founded in 2014, within a very different technological landscape. Ruby 2.1.1 was used with Rails 4.1.1, which at the time were the most recent releases of the language and framework. As our codebase grew rapidly over the next couple of years, other than to move ahead two minor releases (to Ruby 2.1.5 and Rails 4.2.8), our stack stayed on the original versions. By the time I joined the company in mid-2018, my team was actively developing with this setup. After not much time, I became pretty sick of working on something that followed outdated standards and was difficult to troubleshoot. Additionally, it was a common problem to want to use a specific gem that was not supported by our version of Ruby, which led us to have to use less practical implementations.

My predecessors had investigated updating Ruby and Rails, but had come to the conclusion that it was unlikely that our stack would ever be updated given the size of our codebase, and the fact that we couldn’t take time away from feature development to invest in a potentially insurmountable improvement. Still, I had a nagging feeling that the issue could be solved, with the right approach.

When my boss announced we were doing a tech debt sprint and we could tackle anything we thought could improve our monolith’s performance, I jumped at the opportunity to upgrade our Ruby and eventually Rails versions. Although I found a large number of articles discussing literally how to increment the configuration files for an out of date Ruby version (as in, change your Gemfile to reflect the new version, run `rvm install NEW_VERSION`, re-run `bundle install` and ta-da, you’re done!), I didn’t see much out there about the more cumbersome reality of updating a massive application to actually be compatible with a significantly newer version of Ruby, which made me less confident I would be successful in my attempts.

However, after a couple of months of confusion, and interruptions to work on features, I have brought our system from running on Ruby 2.1.5 and Rails 4.2.8 to using Ruby 2.5.5 and Rails 5.0.7.2. I’ve learned immensely from this journey, and I wanted to share some of the lessons I’ve gleaned from this process (beyond that yes, you will need to update your Gemfile and re-run `bundle install`). There’s definitely a certain sense of shame associated with working on outdated tech, especially working at a startup, which I think keeps people from talking about it, so I am really excited to have this opportunity to discuss the issue openly and share some of the things that helped me.

Make sure you understand your deploy process

Part of my job is running code deployments for customers, and I considered myself to be pretty familiar with how our build process worked. Upgrading our Ruby demonstrated to me just how much I actually misunderstood, or took for granted, in our build process. We have a custom build framework written in Bash, and while it worked well when we never updated our Ruby or Rails, it quickly became my enemy. Our Travis CI instance seemed incapable of pulling the correct Ruby because of the changes, and then once that hurdle was overcome, some of our boxes started crashing because of an issue with how our build framework communicated with AWS Code Deploy (TL;DR, symlinks do not always play well with Code Deploy, see https://github.com/aws/aws-codedeploy-agent/issues/152). Prior to working on this upgrade, I had never written bash or even really read much of it, and I can’t exactly say I’ve become a pro in this department, but with some help from others, I was able to take out some of the more hacky logic that broke when the Ruby version changed, and make it so that in future upgrades only our `RUBY_VERSION` or `RAILS_VERSION` would need to be updated for our build process to pull the correct version.

This part of the process was particularly painful, and at times, I found myself frustrated with my lack of system-level knowledge. I do think that putting in the work to understand what was broken (and when that failed, reaching out to those around me for support) made me a stronger engineer.

Don’t assume your build passing on local == passing on a production (or production-like) environment

A somewhat related discovery for me was that my upgrades successfully functioning on my local environment did not actually indicate success in production or production-like environments in AWS. Prior to this upgrade, I had worked primarily on backend feature development, and while once or twice I had written code that worked locally and not on the server, it was usually because of faulty data or another external factor. These upgrades were an entirely different beast, and I learned that the hard way when I declared victory prematurely. At minimum, I now try to put these types of changes on our internal testing server before opening a PR, so that I can look for the most glaring errors and compatibility issues.

Accept that you may have to throw out some of your gems

An unfortunate reality of Ruby development is that if you are keeping your tech stack up to date, you are likely to discover that some of the gems you rely upon have been deprecated or left unmaintained, resulting in them not working as expected on newer versions of Ruby or Rails. The bad news here is that you have to become a little bit of an archeologist. Sometimes deprecated gems will link to gems that fill the same purpose (which is greatly appreciated on my part), but some gems seem to die a silent death, leaving you wondering, “did someone decide that this gem hit the epitome of greatness in 2013 and has thus left it as is? That can’t be right…” This can lead to a general sense of panic and confusion, but I can promise you that someone else has already run into the same issues, and you can usually follow their lead. For every gem I found that was no longer usable, I found a replacement gem, often one that captured the previous gem’s functionality fully without requiring changes to our codebase.

Start small, or, embrace incremental improvements!

Part of what delayed people on my team from working on these upgrades was that by the time the idea was raised, not only the current version of Ruby was deprecated, but the following version as well, with the version following that on track to be EOLed soon. However, the approach that I took was to try to incrementally upgrade, even with the knowledge that if we only got from Ruby 2.1.5 to 2.2.x, we would still be working with a deprecated stack. This piecemeal approach is what made the work fathomable, instead of a giant undertaking with no end in sight. In the end, I split the work as going from 2.1.5 to 2.3.8, then 2.3.8 to 2.4.4, and finally 2.4.4 to 2.5.5. Each of those splits was based on trial and error, and discovering the distinct section of work required for each upgrade, so that I wouldn’t drown in the work (and perhaps more importantly, so that the peers reviewing my PRs wouldn’t hate me).

Monitoring and testing are key

This should go without saying, but once you’ve made these huge upgrades, it is crucial to put the work in to test the functionality of your app, and to make sure that monitoring is in place in case of that process failing to capture issues. In our testing, we even moved the code to a production environment and created a complete transaction, including payment processing, in order to feel confident in the changes.

When all else fails, just give it a genuine effort

At the end of the day, the only thing that distinguishes me from the people who said the upgrade was impossible is my willingness to try — I had no idea if the upgrade was going to be possible, but I tried anyways, and eventually succeeded. As an engineer, learning to push myself to solve problems that feel impossible without letting the threat of failure dictate my actions has been immensely valuable, and I would challenge others to do the same if they are ever in a similar situation. At a minimum, you will learn more about your code base and increase your knowledge of Ruby, and you may actually find the situation is simpler than you feared. I now look at our code base with a less fearful eye — as absurd and self-indulgent as it may sound, I feel like the person who wrangled the Hydra, and I feel more ownership over all future development in this codebase. It is not an uncommon issue for a company to have an outdated tech stack, but I hope my experience can show people that what matters less is the problem and what matters more is the effort put in to fix it.

Reflections on an “Impossible” Upgrade was originally published in Brightloom on Medium, where people are continuing the conversation by highlighting and responding to this story.

Contributed by Trisdan Leyson, Platform Engineer @ eatsa

Our restaurant partners work hard to keep customers engaged and happy, and that means they need to be able to update their menus. Perhaps that seasonal offering of liver and onions isn’t selling quite as well as they had hoped. Or maybe a recipe simply needs some adjustments; that vegan chili would be great if it didn’t have those pickles on top. If you write software for restaurants, like we do at eatsa, you need some way to delete items from the menu.

Most of the time, however, you don’t really want to delete the item. You want to make it impossible to order it, but still want to keep a record of its existence. Maybe you want to do some analysis, figure out who ordered it, possibly see what kind of profile they fit, or whatever else you’d like to know. Maybe the purple fingerling potato and truffle oil quinoa salad had a small, devoted following that you’d like to better understand. Maybe you simply want to let customers see what they’ve ordered in the past. There’s no reason to purge an order from history just because it contained an item you don’t serve anymore.

We on the eatsa API team chose to do this by updating our data model in Postgres to support soft deletes. Soft deletes are a fairly simple idea: instead of data actually getting removed from the database, it gets marked as “deleted.” When the mobile app requests the current menu, the code filters out all the soft deleted items so customers can’t order them. However, when customers want to see all the orders they’ve ever made, the code leaves in the soft deleted items. That way, if an eatsa partner lets the CEO’s nephew invent a recipe and then they remove it soon after, customers can still see the time they ordered that brown sugar, yam, marshmallow, and jello salad on a dare.

For such a simple concept, soft deletes have a lot of subtleties, starting with how you mark the data as deleted. The simplest way is to add a true / false flag to each record. If the record is deleted, set the flag to true; otherwise set it to false. This works, but sometimes you realize later that you want multiple types of “deletion.” Perhaps there are some menu items that you want to remove from both the current menu and the order history, but you don’t want to fully delete them. You can’t represent this with just true and false. Instead you could store a character or a short string with each record that represents its current state. With this solution you can invent new codes every time you want a new type of deletion.

Another subtlety is how to track when an item was deleted. One option would be to do this with a separate date field on each record, but you could also use the date to determine if a record is deleted. If the date isn’t set, the record has not been deleted; if it is set, the record has been deleted, and you know when that happened. This is the approach we went with. The part of our API that manages menus is in Ruby, which has a library called Discard for working with this approach to soft deletes.

You could also completely remove the deleted item from the main database table and store it in a separate table. We could have had an “items” table and a “deleted items” table. This approach avoids some of the challenges mentioned below. But it makes the code more complex since you have to search two different tables whenever you need soft deleted items. Searching two tables can also slow down the code when it needs to look up an item that can be either soft deleted or not; now it has to do two lookups from the database instead of one.

Once we’d chosen a “deleted at” date as our implementation of soft deletes, we had to address a few other subtleties. Before, I said “your code filters out all the soft deleted items,” which sounds simple, but can easily get complicated. You have to track down everywhere in your code that looks up data that can be soft deleted and decide whether that context calls for filtering out the soft deleted data or not. If you forget to do this filtering somewhere, it can result in bugs. When we first rolled out soft deletes, we had a bug like this that went uncaught since no one had soft deleted anything from the table that was affected, even during pre-release testing. Luckily, it was caught and fixed the first time someone tried to soft delete an item from that table, which happened in a pre-production environment, so no customers were ever impacted.

A bigger challenge came when we couldn’t support something everyone takes for granted: deleting a record and then creating it again. When you hard delete a record from a database table, it’s completely gone. If you want it back, you can create the same record again. You lose this ability with soft deletes since the record still exists in the table, it just appears deleted from a user’s perspective, so trying to recreate it can run afoul of data integrity checks that the database does. There are two approaches you can take to getting back this behavior, both more complicated from an implementation perspective than what you get with hard deletes. You can allow multiple records with the same data, or you can force there to be only one record.

Either way, you have to define an identity for records so you know when a new record is the same as an existing one. Allowing multiple identical records makes defining an identity trickier. You can’t rely on a generated ID, because each “identical” record will have its own ID. You have to rely on other values to define a record’s identity, which can get very complicated. If you use menu item name as part of the identity, you have to decide if “Burrito Bowl” and “burrito bwol” are the same thing. You have to decide if “No-Worry Curry” is the same thing as “no worry curry” and “no worry, curry.”

Ensuring data consistency can get a lot harder when you allow multiple records with a single identity. You need to write extra code to check validity, and a lot of integrity checks that your database can do become hard or impossible to use. On one project I worked on at a previous company, users were identified by email address. The database forced every record to have a unique email address. When I started on the project, it was impossible to delete users except by going into the database and directly removing the records. Since it was an internal tool used by a small team, this wasn’t a problem for a long time, but eventually an employee with the same name as a previous employee needed to log into the tool. I instituted soft deletes for users, allowing multiple rows with the same identity since that was how it was done in other parts of the code whose example I followed. To make this work I had to remove the uniqueness constraint on the email address. Even though it was valid for multiple records to have the same email address, only one of those records was allowed to be active; the others had to be soft deleted. There was no way to tell the database to enforce this constraint, though, so it had to be checked in code. This made the code more complicated, and removed the last line of defense against inconsistent data being stored in the database.

You also have to be careful about letting your generated IDs get outside your code. Suppose someone uploads a menu with a “Perfect Salad” on it and assigns the “Perfect Salad” an ID. The ID is given to various clients so they can send requests to the eatsa platform to order a “Perfect Salad.” But then it turns out the “Perfect Salad” was a mistake; it was supposed to be a “Pear-fect Salad.” If you’ve implemented soft deletes in a way that allows multiple rows per unique record, deleting the “Perfect Salad” and replacing it with a “Pear-fect Salad” creates a new record that has a new ID, and the one that was given to clients for making orders is now wrong. You need to make sure they all get the new ID for the “Pear-fect Salad” and stop using the one for the “Perfect Salad,” which will fail now because the “Perfect Salad” has been taken off the menu. Even if the process of distributing this ID to the clients is fully automated, there could be a time period where the “Perfect Salad” is deleted but the new IDs haven’t been synched yet due to caching or other factors. During this period, clients who try to order the “Perfect Salad” will see errors.

For all these reasons, we decided to restrict each record to exist only once in a table, and use UUIDs as the identity mechanism. If two records have the same UUID, they’re considered identical. This was straightforward, since our tool for updating menus at eatsa passes a UUID for every item, so the code always knows what record it’s supposed to be looking for. If it doesn’t find the record, it creates one. If it finds the record and that record already exists in the database, but is soft deleted, the soft deleted record is resurrected and modified to whatever state the user requested. If the record exists in the database and is not soft deleted, it’s merely updated to the requested state.

Getting to a good implementation of soft deletes was a lot more challenging than I expected, but in the end we had something that allowed our restaurant partners to update their menus without losing the data that will let them track the sales performance of temporary seasonal items, search for commonalities among their most popular dishes, or just help customers find that amazing burger they ordered six months ago.

Soft Deletes: Harder Than you Think was originally published in Brightloom on Medium, where people are continuing the conversation by highlighting and responding to this story.

Contributed by Talia Trilling, Software Engineer @ eatsa

For many software engineers, the products we build and work on are completely foreign to us. At some companies, we may not be consumers of the product, and in worse case we may not fully understand the functionality we are implementing. I am guilty of this one myself, having worked for a little over a year on accounting software as someone who barely knows how to balance a checkbook. It’s not just that I wasn’t a user of the product, I couldn’t even relate to the end consumer, and that massively narrowed my ability to get excited about what I was building and invested in the reliability of it. Furthermore, I couldn’t make basic inferences about how to best develop the user experience or automate it, because the use cases were so foreign to me. That’s not to say that I didn’t try to learn about accounting, but it never was meaningful to me in the way I needed it to be. I was not a part of the intended product’s audience (accounting professionals), and I never would be, no matter how much effort I put into learning about it. I think I was so excited to be employed as an engineer that I didn’t realize the impact my feelings (or lack thereof) towards the product limited my ability to interact with it.

Engineering can be confusing in its own right, but combined with complex accounting rules and use cases, it took on another level of unfamiliarity for me. Testing required following pages of detailed excel spreadsheets, instructing me on how the product should behave for capital expenditures or long term leases, or accounting adjustments, or any number of other configurations. There was never a world in which I organically would have been able to test functionality, and while that’s ok, it made concepts like test driven development feel more to me like a nice idea than a feasible reality.

When I first interviewed with eatsa, I knew that on the most basic level, I could relate to the goal of improving the restaurant experience. Despite scores of technological innovation, the way customers engage in the dining experience has mostly stayed the same over the past couple of decades, and eatsa aims to change that. I was excited to work for a company that was building something so innovative, but I thought that was mostly a bonus, not something that would directly impact me on a daily basis. What I began to realize as I became more familiar with the company, was how much more I could grow as an engineer when I related to the product being built. I began to see products as massive living ecosystems rather than a series of engineered tools that in an ideal world would interact. Being in a small startup environment further enhanced this feeling, as the constant speed of and iteration on development wouldn’t and couldn’t happen in a silo, but in constant conversation with people responsible for other parts of the system.

The products that eatsa is building affect me on a deeply personal level. As someone with severe food allergies and at times limited mobility, eating out is not always a pleasant or even safe experience. In my time at eatsa, I’ve glimpsed into the types of improvements future restaurants could potentially make to improve these experiences, and that knowledge makes my work feel meaningful and impactful, something I believe engineers often struggle with.

I come to the office excited about what I am building, knowing that my engineering work has the potential to fix problems I and many others face. By streamlining how someone might use a delivery service when they are unable to leave their home, or enhancing the ease with which a user can check the allergens in a dish, I know I’m having a tangible impact on things I care about. I also relish applying programming concepts to this context, because I’m not only moving from the theoretical to the practical but I’m also more cognizant of how the practical application impacts the end goal.

Although not all engineers will be able to have the experience of building a product they are so invested in, I hope that my peers begin to recognize that this aspect of employment can be just as valuable as desirable tech stacks or a strong company culture. When my motivation to build systems is combined with my passion for the product, the experience takes on an entirely new meaning to me.

Engineering for Yourself was originally published in Brightloom on Medium, where people are continuing the conversation by highlighting and responding to this story.

Contributed by Jason Fruy, Physical Design Director @ eatsa

When you experience a moment that is truly magical, you know it. You know it, because you FEEL it.

I recently felt magic when visiting Disneyland. My wife, two-year-old son and I were up way past bedtime waiting in line for a ride for what felt like forever. As we approached the front of the line, exhausted, a kind ride operator asked how our visit was going. When we told him that it was our first visit ever, he smiled and told us that after we rode, we could stay on the ride for a second round.

This seemingly simple gesture made our night. No matter the motivation, this employee was guided by the core principle sharing joy, and he did. I carry this moment with me and remember it when I think Disney.

Because a magic moment is something you FEEL, it is something that cannot be faked — it must be genuine. In our eatsa stores, customers smile with joy or take videos of friends when they experience picking up their food from our cubbies. This break from what is expected during lunch embodies the magic that work to bring to our store guests, and it continues to stoke the curiosity and delight of our customers.

More than ever, people are hypersensitive to their experiences and engagement with products and brands. As a result, brands are becoming increasingly aggressive in their attempt to garner attention. The table stakes of customer service are no longer enough. Brands need to identify early on, what makes them unique and “magical.” We must ask ourselves “how can we make the short time a guest spends in a restaurant the best time of their day?” How do we garner reactions like “that’s so cool!!” and “how the heck did they even do that?!?”

Here are seven ways to infuse magic into your brand’s products and experiences:

1. Tell genuine stories with substance. At The Museum of Ice Cream in San Francisco, you are asked to say one thing you’re are worried about in life before you enter the museum. You are then asked to let go of that worry and smother it with the joys of ice cream. This act offers a simple yet compelling narrative and an amazing way to frame what you are about to experience.
2. Create next level “surprise and delight.” We must genuinely fulfill the phrase “surprise and delight.” Push beyond merely what is expected. For example, eatsa’s design focuses on merging digital and physical technology in a way that is unexpected, joyful and engaging, thus much more exciting than your average coupon or free side of guacamole.
3. Get to know your customers even better. Find ways to get your guests to say “somehow they just know me!” Employing impressive personalization features will set your business apart and keep people coming back.
4. Optimize, don’t just modernize. Don’t apply design for the sake of design, be useful. One of my favorite examples of this is the Kazbrella upside down umbrella. At first this design seems flashy, but its features are thoughtful and incredibly useful. You’re left wondering why it took this long for someone to think of this.
5. Help, but don’t hover. Enjoying a meal can be a very personal thing and hovering over guests is a non-starter. Give customers the space they desire, but always be findable and available if needed.
6. Understand the value of your customer’s time. Consumers are busy and want to use their time as they see fit. Empower guests with clearly defined pickup instructions and mobile apps that provide details during the ordering process to maximize the guests’ precious time. For example, eatsa’s mobile app tells customers when their food will be ready down to the exact minute.
7. Lead. Don’t follow. Be original. In the world of sequels, spin-offs and substitutes, separate your business model and offerings from the pack with new and provocative customer engagements.

Pushing ourselves beyond the obvious answers in today’s restaurant industry is imperative. Rather than thinking outside the box, consider instead that there is no box. In whatever you create, work to help people to FEEL the magic.

This post was also published on Modern Restaurant Management.

7 Ways to Put the Magic Back into your Restaurant was originally published in Brightloom on Medium, where people are continuing the conversation by highlighting and responding to this story.

Contributed by Mark Fleschler, Software Engineer @ eatsa

Here at eatsa, we are working to revolutionize the restaurant industry. This is no small feat and brings with it a barrage of unique engineering challenges. As we work to solve a number of problems with the current process of ordering and serving food in the fast casual restaurant industry, we also are trying to create a magical experience that no other restaurant is currently delivering. If you are unfamiliar with our restaurant experience and how we solved the problem of efficient and lightning fast ordering and delivery of meals, check out this video.

One of the more visually engaging components of our restaurant ordering experience is our visual queue, the order Status Screen. The screen is located above the cubby wall in our partners’ and our restaurants. The Status Screen is a dynamic and constantly animating queue of customer names and their order status. It highlights when an order is ready and displays a cubby number guiding you to where you can pick up your meal.

The development of the Status Screen produced a number of fascinating and exceptional engineering challenges and was a hugely collaborative effort with our Design, TechOps, and DevOps teams. In order to create the most immersive and intuitive customer experience possible, initial requirements demanded the screen seamlessly animate vertical lists of names and display a rotating series of brand focused images behind it.

In addition, this would be displayed on a large video wall setup that could be comprised of at least three and up to nine displays. As you can see in the images above and below, the design went through a number of visual iterations on how our display would look, but the overall layout remains largely consistent with what you see in our restaurants.

Our Design team challenged the Software Engineering team to utilize animation as a mechanism to smoothly and seamlessly express transitions, which would help contribute to the magic experienced in our restaurants. We needed to support animation of the customer’s name when the order was marked ready, the expansion and contraction of the name area, smooth vertical movement of names across columns, and the fade transition of images in the background. This was a very daunting task for the Software Engineering team.

In addition to these design requirements, there were also the constraints relating to cost, maintenance, and scalability. After an initial trial and demo was created using a proprietary and expensive visual development platform, we chose to switch to web technologies as the platform to scale with. With web as the platform, it would be easier for different team members to work on it, give us flexibility in tooling, and make it easier to work across different hardware and operating systems. That’s right, our in-store displays are kiosk mode web pages displaying full screen.

To support animations, CSS transforms were utilized along with other technologies to handle instantaneous push updates (websockets) and simple state management and rendering of components (backbone.js and marionette.js).

Warning — I’m about to get a bit technical, but it’s hard to tell this story without doing so. What follows are a series of unique engineering challenges we faced during development of the Status Screen.

Challenge 1: Rendering the names

The first problem the final design presented was how we were going to display and animate customer names on the screen. It’s a subtle and seamless effect you may not pay attention to. You’ll notice how as a name moves up to the top of a column (as orders are fulfilled), it will seamlessly appear at the bottom of the column to the left while still present in the column to the right. Simply rendering the names singularly proved impossible to generate this effect in HTML, but was solved with redundant rendering of the entire name list for each column rendered (see illustration above). The trick was to set the vertical position via the unordered list element’s scrollTop value differently for each column, so that when a name disappears in one column it will begin to appear in another.

Challenge 2: Expanding and contracting columns

This challenge was a matter of visualizing what design wanted within the context of HTML structures. By simply rendering our list of columns all the time, we could wrap our horizontally laid out list with a viewport and a subtle border radius. This could then be made to expand and contract dynamically based on how many orders were present. In addition to this, we would also need to calculate and animate the appropriate width as the last column to display would need to be slightly thinner to leave a bit of distance between the list of names and the edge of the screen for visual effect. Thinking of the name lists as different viewports into the same list helped inform this approach.

Challenge 3: Screen layout of the display wall

This was probably the most difficult of the challenges we faced when building the Status Screen. I was under the impression that the display wall monitors were capable of 1x5 or 1x9 orientations, which would natively display odd aspect ratios. Our screen needed 6480x1280 pixels for the 1x9 setup we have at our 121 Spear restaurant location.

Unfortunately, this is not how wall displays work. They are configured to display an ultra high-def resolution (4k or 3840x2160 pixels) with the corresponding aspect ratio. As you can see in the image to the left, the displays are laid out in an unintuitive manner, with the leftmost row sitting at the bottom and the rightmost at the top. To allow for the seamless horizontal animation of the expansion and contraction of the column viewport, we would again utilize redundant rendering. We had two paths for this: render the components repeatedly on the same layout or simply render the previously built Status Screen URL redundantly over three iframes (see above illustration of iframe layout) utilizing the same viewport approach to focus only on a subset of the overall view (i.e. bottom row renders only the left third of the screen, etc). The latter solution was faster to implement, requiring no restructuring of the existing code and could be prototyped and executed in a day. After testing this out on a small UHD display we were able to validate its look on the video wall displays with success!

Challenge 4: Performance optimization

At this point we were ready to start installing the Status Screen in our stores and perform more active testing with the rest of our technology stack and finally process some orders. We were targeting very affordable hardware: an Intel NUC, to display the Status Screen. However, as we began testing on the targeted hardware, we noticed how slow and choppy the animations were, especially if dozens of orders were being added and fulfilled simultaneously. This was not surprising given how much redundant rendering was occurring, and some optimizations were made. Ultimately, that wasn’t the root of the issue — it was how we were animating, using CSS transforms. What surprised me was how much slower and less efficient they were when compared to using a dedicated javascript animation library, such as velocity.js.

After reimplementing all the animations using velocity.js, which had the added benefit of combining animation logic with the rest of the rendering logic, we were amazed to discover an order of magnitude performance improvement. Hundreds of names could now go nuts animating and fulfilling at the same time while maintaining close to the required 30fps necessary to ensure a smooth visual experience. A win win!

Challenge 5: Synchronizing background image transitions

One of the last challenges wasn’t discovered until our stores were open and using this technology. Over time, the background image transitions would slowly become out of sync. This was due to the fact that when each of the three iframes rendered, they would do so asynchronously, and the background images would only start to transition once loaded. Over time they would fall out of sync.

To solve this, we moved the logic to render and transition the background images out of each of the iframes and into the view that renders the iframes themselves. One thing that is nice when rendering a web page with iframes from the same origin is that you can descend into the iframe and manipulate its contents directly. This allowed us to detect when each iframe had finished loading, and initialize the rendering and transition logic of the background images across all the iframes simultaneously. Problem solved.

Lessons Learned

This project was a massive learning experience for myself and our whole team, and these lessons helped to streamline our processes moving forward. First and foremost, it’s important to know the limitations of your hardware setup, including how the chosen display hardware is configured and what its capabilities are. CSS transforms were much slower than previously thought and are now avoided when considering animating HTML dynamically. It’s remarkable how powerful velocity.js is as a tool in this regard. Javascript is much faster than we thought. We found that even on constrained hardware using it to handle animations felt counterintuitive, but ended up being the way to go.

Engineering at eatsa

This story is just one of many at eatsa. Across all of our products and projects, we tackle challenging and uncommon engineering problems you don’t normally encounter at traditional mobile or cloud based technology companies. We not only have to scale our mobile and cloud infrastructures, but also how we develop and maintain our restaurant technology. We also build our own hardware and need to be able to interface with it. Every day, we are faced with questions and entirely novel product ideas that require lateral thinking and creative solutions. Never a dull day here!

Anticipating unforeseen challenges. Engineering the solutions. was originally published in Brightloom on Medium, where people are continuing the conversation by highlighting and responding to this story.

Contributed by Nora Naranjo, Mechanical Engineer @ eatsa

The concept of robotics has been intrinsically tied with social evolution, working as a mirror of human and cultural desires, fears, and passions at the given time. Since its definition in 1920 in K. Čapek’s play, R.U.R. ‘Rossum’s Universal Robots,’ the mediatic portrayal of robots in mainstream media has changed from soulless production muscle to soulful embodiments of human kindness such as in ‘The Iron GIant’ and ‘Wall-e,’ to an array of invincible vigilantes (Terminator, RoboCop), or mass-manufactured sentient human replacements (A.I. Artificial Intelligence, Blade Runner).

Exponential growth in processing power per unit of size helped robotics evolve from a technological sideshow to a main component of daily life, making consumer facing robots a constant topic not only in industry publications but also in non-technical news sources. Concepts like artificial intelligence, machine learning, and industrial automation are now pervasive marketing buzzwords, while self-driving cars and automated manufacturing facilities have become a conflicting reality: humanity has built a world of overwhelming complexity and is pushing technology to simplify it on a quest for speed, efficiency and economy, while the human impact of these developments, such as job displacement and income inequality, take a backseat.

The cycles on policy-making and legislation are significantly slower than the tech industry’s innovation pace, making policy outdated or obsolete almost immediately after it is passed. Companies should not wait for governments to regulate automated systems, instead they should hold themselves accountable for the impact of their actions and their inventions. I believe engineers have the power to drive the decisions to hold their systems, and to a larger extent, their company accountable.

Since I joined eatsa a year and a half ago, there have been significant changes to the company’s mission and values. eatsa evolved from a restaurant company looking to cut the costs of healthy and sustainable food by the means of automation, to a tech company striving to help partners scale their operations by building intuitive hardware and software systems that significantly reduce restaurant employee turnaround, order queues, pickup times, and increase overall customer and employee happiness. But it is one eatsa’s values, defined by employee consensus that acts as a moral compass for system development: Be Human.

The morality of a company should not only focus on immediate positive results such as: faster store throughput, increased ROI for partners, and improved customer experiences, but also the impact those systems have on society:

• Faster throughput makes food more affordable and reduces stress on restaurant employees.
• Automated systems make food preparation more ergonomic and drastically reduces the risk of work related injuries.
• Intuitive systems ease the learning curve for a broader range of technical and cultural backgrounds, enabling restaurants to have a more diverse employee base.

Striving to facilitate human-machine symbiosis and enhance the capabilities of both machine and human must be an essential part of the product development cycle. Resourceful Design and Engineering teams will thrive on the challenge to create a real human-centric system, and build systems that will enhance the productivity and performance of both business and worker alike. I am proud to be part of a company building a better future for humans, and sincerely hope eatsa’s approach to robotics and automation becomes a trend in both the industry and media.

The Human Side of Automation was originally published in Brightloom on Medium, where people are continuing the conversation by highlighting and responding to this story.