Background

I worked for a small successful startup that created the Barracuda web application, mobile apps, and other tools such as business intelligence in the retail vertical. Barracuda was a typical Ruby/Rails monolith that was quick to build but was reaching some architectural limitations.

One of Barracuda’s features was around photo processing. An integration with a third party vendor was used to add metadata to the photos via an API call from Barracuda. For fun, let’s name the vendor “Atlantis”. Some of the responsibilities related to this feature include:

• Associating metadata to customer photos
• Accurately billing customers for photo processing on a per-photo basis
• Error handling when photos needed to be re-processed
• Display of the metadata in a separate web application for BI purposes

The feature was at a pivotal turning point where it had proven successful for one customer and the company wanted to scale it to all their customers.

The Problems

The existing implementation prevented widespread adoption because of several limitations.

Manual Configuration

To enable a new customer for photo processing, a combination of code changes and database updates were required. The update process was error prone and tedious. In addition, code changes required a code deployment and integration testing with Atlantis in production.

Testing

All new processing models were tested in production because of a lack of an integration sandbox with Atlantis. Code changes around this area were high risk because the lack of sufficient automated tests created the need for manual regression testing.

Billing

In order to calculate billing for customers with photo processing enabled, database queries were manually run at the end of each month to calculate costs. The billing logic had to correctly calculate costs based on the number of photos processed in the time period and exclude re-processing under certain conditions, such as processing mistakes. Some processing mistakes were categorized as mis-configuration errors, and would not incur additional charges to the customer. Others were passed along to the customer as additional fees. Many of these exceptions required manual intervention to determine the category.

Architecture

Because the photo processing was part of monolithic Barracuda, the data was stored in a single database. This data was used for Barracuda, for the photo display web app, and for billing calculations. Having the data commingled caused the separate concerns to be difficult to manage. Barracuda might want to store the data a certain way to optimize the configuration, while the billing would be easier by changing the data to a different format.

The below diagram shows some of the photo logic that Barracuda was responsible for. Although the photo logic is shown as separate processes, the code was commingled with Barracuda’s other concerns which are not included in the diagram:

As demand for the photo processing feature grew, we knew that the existing design was reaching its limits. The company had two high level goals: Enable photo processing for additional customers and expand the feature set of photo processing. To support these goals, the team identified these immediate needs:

1. Eliminate manual configuration
2. Extract photo processing to its own application
3. Automate billing

Approach

Our long term vision was to extract a separate application (code named “Guppy”), create a new product team, and expand the feature to all our customers. Getting there was going to be a long process and we were dedicated to these principles:

Test Coverage

The manual process of testing with every release was severely inhibiting feature requests. In addition, we lacked tribal knowledge of how the photo code worked. So we were committed to increasing test coverage. We employed TDD for all new code and added tests for all legacy code we planned to refactor or redesign.

Releasability

We had other responsibilities in Barracuda including bug fixes and other features. So it was important to work in a fashion that allowed continued releases and short feedback loops.

Maintain functionality

We operated within the constraint that we couldn’t disrupt or change our current customer’s expectations nor our agreement with our third party vendor, Atlantis. We had to move towards our new vision while keeping features backwards compatible.

Working with our product owner, we had a whiteboard session to identify dependencies between our three immediate needs. For example, improving billing would be dependent upon getting the data in a new format, so we decided the app extraction should happen first. Similarly, we could easily automate the configuration without making the app extraction harder, so we moved that to an early stage. As people raised issues to be resolved and work to be done, we decided as a team which phase fit best for each item. Phase I moved away from manual configuration. Phase II was refactoring to create a seam to extract the new app. Phase III involved the extraction, and Phase IV added features to the billing process.

Phase I

The goal of the first phase was to remove the hardcoded configuration that required code releases to add a customer to the photo processing feature. Additional whiteboard planning and design sessions allowed us to drill down into the details of the first phase. These are the steps we came up with:

1. Backfill tests

Because there were no automated integration tests for this behavior, we began by writing characterization tests to provide cover for our upcoming refactor. We wrote these at a low enough level to focus on just the changes we were about to make, but high enough to allow flexibility in how the code was written beneath the test umbrella.

2. Move hardcoded configuration

Keeping our long term vision of moving the configuration into Guppy in mind, we created completely new tables with only minimal references to existing foreign keys.CUSTOMER_IDwas used as a foreign key to tie our new tables to existing customer data.

3. Populate tables

Using our tests to protect us, we moved the hardcoded values to the database. Relying on the newly created tests ensured we didn’t break our existing customer’s experience.

4. Automate configuration

We planned to give our internal users a UI to edit the customer configuration data, but that logic needed to live in the new app, Guppy, which didn’t exist yet. For now, we created a rake task to allow those changes to be made by developers, but without the need for a code deployment.

The new code in the rake task was covered by TDD which helped drive our design and improved our test coverage.

Phase II

The focus of the next phase was to separate concerns within the monolith in preparation for creating the new app. Extracting the logic into Guppy was going to be a high risk change which required coordination with Atlantis, so we began with more test coverage. Pair programming, frequent rotation between pairs, and mini-design sessions were key elements that allowed us to respond to unexpected challenges we encountered along the way. It was during this time that we decided during one retro to use “Clicking start on a user story” as a trigger to pull the dev team together for a mini-design session. This was extremely helpful to keep us all on the same page and avoid getting bogged down into implementation details during story grooming.

1. Breaking external dependencies

Up to this point, the only way to verify functionality with our external vendor was to “test in prod” in close coordination with Atlantis. This was another obstacle to scaling — if we couldn’t change our architecture without involving the vendor, we would be forever dependent upon their availability. Breaking this external dependency in test was critical. We added test coverage for the photo processing logic by stubbing the calls to and from Atlantis. We covered all happy paths and edge cases with additional characterization tests. We uncovered several bugs in the process but prioritized those bug fixes separately to keep us focused on our long term vision.

2. Extract service class

We identified a seam in Barracuda where we could separate photo processing from its other responsibilities and pulled that into a class in a new service layer. We called this a service class as an indication of a separate layer. At this point, it was all still part of Barracuda’s monolithic architecture.

Pulling the logic in a single controller apart was messy. We had to separate the parts of the procedure that involved the photo processing from the other steps. We began by identifying the if clause that checked if a customer was enabled for photo processing and followed that logical seam.

As we worked, an interface between Barracuda and future-Guppy started to emerge. In addition to CUSTOMER_ID, we could see that some minimal details of the photos themselves would be required to coordinate between the two apps. We kept the service’s parameter list to that minimal interface. It would serve as our RESTful API later on.

As part of extracting the service class, we added unit tests to cover its existing behavior. Although the service we extracted was rather big and ugly, we knew that was good enough for now and we would have plenty of opportunity to improve it further.

Phase III

With the logic cleanly separated in Barracuda, it was time to create Guppy! This phase saw a lot of emergent design as we worked with the new data model. We had a “DO NOT ERASE!” whiteboard with our data model and lots of arrows to show how the legacy data would be mapped to the new data. We referred to this diagram daily and made updates as needed. It was a highly collaborative and fun process.

1. Guppy is born

We created a new Rails app for Guppy. We chose to stay with Rails as the framework because the code we wanted to move was already RoR and would speed up the extraction. We were focused on getting Guppy separated from Barracuda as quickly as possible without getting sidetracked by additional architectural improvements (yet). We chose HTTP Basic Authentication for inter-app communication between Barracuda and Guppy as it was simple to implement and we did not have more stringent security requirements driving us yet.

2. Separating responsibilities

At this point we could see that our configuration tables needed to be split further to support Barracuda’s and Guppy’s separate responsibilities. We decided that Barracuda would be responsible for knowing which customers had photo processing enabled (the what) and Guppy would be responsible for knowing how to do that processing. Barracuda maintained the list of customers enabled, and we created new tables in Guppy for the details of how to integrate photo processing with Atlantis.

The service class we had extracted moved to Guppy and Barracuda’s controller call to the service changed to a RESTful call to Guppy.

Part of Guppy’s responsibilities would be to receive the callback from Atlantis, but that required Guppy to be fully deployed. So we wrote code for Barracuda to forward those requests to Guppy and put that logic behind a feature flag.

3. Release management

The above changes could not be made wholesale without introducing downtime for the photo processing feature. Instead, we coordinated deployment between the two apps with the following strategy:

1. Add new feature to Guppy alongside existing behavior
2. Start using new feature in Barracuda
3. Remove old behavior from Guppy

For example, to move a new responsibility from Barracuda to Guppy, we would:

1. Modify Guppy to accept an optional parameter sent to it from Barracuda. If Guppy saw the parameter, it used it, but otherwise did not rely on it. Release Guppy with this optional capability.
2. Modify Barracuda to start sending the parameter to Guppy. Since Guppy was already released, it could count on the parameter being handled. Release Barracuda with this change in functionality.
3. Modify Guppy to require the new parameter and remove the optional branch in the code. Release Guppy with this updated contract now complete.

This involved many small releases and steps but allowed us to have 0 downtime and to work with small feedback loops to ensure everything was working perfectly. Note that the releases were order dependent, but otherwise the timing allowed for flexibility. They didn’t have to be released at the same time or even on the same day. We could have also used feature flags but they provided no additional benefit given our deployment process.

3. Atlantis callback

Guppy was ready for prime-time! We changed our internal routing of the callback URL to point to Guppy’s API and the third party vendor was unaware of any differences from their end.

4. Syncing data

Separating the photo processing code proved to be far easier than separating the data. Poor dependency management on Barracuda’s database meant that there were many dependencies on the data outside Barracuda’s control. We had to maintain that data contract. But we also wanted some of that data to move to Guppy. Thus we introduce data synchronization between the two apps. Guppy would receive the metadata from Atlantis’ callback, store it internally as it saw fit, and then call Barracuda with the data in the legacy format Barracuda expected.

Phase IV

At this point, Barracuda and Guppy had clearly defined responsibilities with the logic and data separated and we were in a place to grow the feature.

1. Billing reports

Using the old queries as our baseline for comparison, we created a new billing endpoint in Guppy. We used manual comparisons to ensure that we captured the correct logic in the new code and then wrote automated tests to lock down that logic.

2. Data migration

In order for Guppy to accurately create billing reports, it needed old data that lived in Barracuda’s database. We created backfill tasks to migrate the data to Guppy. The amount of data was too large to migrate in a single batch operation so we sliced it by date range. Beginning with the most recent data, we moved backwards in time incrementally until it was all moved over.

Now that Guppy was unhindered by Barracuda’s concerns, we were free to create data structures that made sense for creating billing reports. As the data was being migrated, it was also transformed into tables optimized for billing purposes.

As each date range was migrated, we were able to validate old billing reports from Barracuda against Guppy’s new report to ensure correctness. It was a cool way to validate our theories in real time instead of relying on exhaustive analysis.

3. Billing Enhancements

Whew, we were close! With Guppy handling the photo processing, data, and billing responsibilities, we were in a place to make the requested enhancements.

To finish off our highest priorities, we enhanced the billing API to allow date ranges (in addition to just end-of-month), scoping by customer, and more analytics.

New Architecture

Our final architecture after extracting Guppy looked something like:

Future

We had accomplished our high level goals of eliminating manual and error prone processes which set us up to turn the feature on for additional customers.

In the future, Guppy would see a new team created which grew the little app into a suite of tools and customer enhancements that enabled the company to grow a whole new line of business.

Lessons Learned

We learned a lot about how to work with legacy code. One of the secrets of our success was the continued focus on high level goals. While adding tests to existing functionality and migrating data, we encountered a ton of areas for improvement. But we knew that if we fixed them all, we would delay getting to the place where Guppy could be extracted and improved. We kept a clear focus on either refactoring or adding features and never commingling the two actions.

We were surprised at how hard it was to separate the data. As mentioned earlier, allowing others to be dependent upon Barracuda’s data structure caused a lot of pain. Even though these dependencies were internal company processes, they were almost impossible to remove or change. (Obviously, avoid this situation by creating a layer of abstraction as soon as possible!) Direct database reads made our lives very difficult! We accomplished the separation with extra processes to copy data and a commitment to which app owned which part of the data.

Legacy data is difficult to work with and we encountered a lot of surprises when we moved it. Slicing the data by date range allowed us to make course corrections as we discovered data anomalies instead of having to predict and fix all of them at once. Iteration for the win — in code and data!

Another win for us was to keep a long term vision without nailing down all the details too soon. Every step was a mini-design and discovery process which worked well. We knew what we wanted to accomplish in each phase, but not exactly how to do it until we got there.

The whole process took about four months to complete. I credit much of the success of this project to the highly collaborative nature of the team and willingness to adapt to changes. We were fortunate to be working with a product organization that accepted the unknown timeframe and willingness to work with us in a highly transparent way.

Breaking up Barracuda was originally published in EarthVectors on Medium, where people are continuing the conversation by highlighting and responding to this story.

How I Got Here…

I’m currently employed as a Frontend Instructor at Turing School of Software & Design, where I primarily teach React, Redux, React Router and testing with Jest and Enzyme. In an effort to promote professional development, Turing recently partnered with EarthVectors, Inc. to give instructors an opportunity to roll off of the instructional staff and spend 6 weeks with a company getting experience working on a dev team and with a production codebase. Sadly, my 6 weeks is coming to an end this week, but I want to share a few things have learned during this short time.

Day to day life…

I think it’s important to share some context about what my day to day work life has been like with EarthVectors (spoiler alert… it’s been fantastic!). I have been working with two other developers on an application for a genetics company who is researching childhood cancer. There is also a Product Owner/Manager who is remote, but very present via Zoom for daily stand-ups and Iteration Planning Meetings (IPMs). The atmosphere is extremely collaborative. For instance, during IPMs, the dev team is collaborating with the Product Owner/Manager in writing stories, grooming stories, and assigning a point value to each story (difficulty level of the feature). This is done in a very democratic fashion… it’s basically like Paper, Rock, Scissors, but you throw out your point estimation (1, 2 or 3). If everyone is consistent in their estimation, the point value is set. If not, each person on the team has the opportunity to share their reasoning behind their vote. Once everyone has been heard, a collective decision is made on the point value and the story is then placed in the backlog for the dev team to pick up when they are ready to start working on that particular feature.

This collaborative mindset also carries over into the development process. EarthVectors is a company that strongly believes in pair programming (as do I now). I found working in a pair to be super efficient. As the saying goes, two heads are better than one. It’s easier to catch mistakes when they pop up instead of chasing down errors later. We are typically all in the office together, with the exception of one day a week when we all work remote. No matter if we are in the office or working remotely, we always mob (all 3 of us) or pair on the same problem. This is nice because everyone stays up-to-date on what is happening in the codebase. Every decision that we make, from which story to pick up, setting goals for our pairing session, design decisions, etc., everything is decided as a team. I’ve now seen first hand the benefits of this work style, such as building trust and communication amongst the team.

Another core principle EarthVectors adheres to is Test Driven Development (TDD). I was familiar with TDD coming in and try to instill this practice in my students, but had never actually been part of a team that works in 100% TDD fashion. This is an incredible skill that brings a lot of value to a project. When the team begins to work on a new feature, we look at the story to gather any context and details we might need. The first goal is to write a failing test (easy). Then you want the errors in the test to instruct the code you write. It’s a constant red-green feedback loop. Basically, we always want to be writing tests that will drive out our functionality. I have found that this method produces lean and clean code. It gives us purposeful direction and prevents us from writing unnecessary code. I have officially drank the TDD Kool-Aid.

A few takeaways…

Coming in on day 1, I knew very little about the application we would be working on. I knew it was a Rails app with GraphQL and React, so I was just hoping I wouldn’t be totally lost. Diving into a new, large and unfamiliar codebase can be scary. And I’m not going to lie, it was pretty intimidating. But even though I didn’t know much about Rails and GraphQL at the time, I found that I could still follow along and understand what was going on. I attribute part of this to the team’s unwavering commitment to writing excellent tests. When you have good tests in place, they will tell a story of what the code is actually doing. So no matter the language, when you are just trying to get familiar with the codebase, tests are a great place to start.

When problem solving, always try and find the smallest possible step you can take that will move you in the right direction. Even if that step is pseudo-coding and drawing a picture. These things can be incredibly helpful! The same goes for refactoring. Start with the smallest possible step and follow that red-green feedback loop with tests.

Always strive for short feedback loops. Ideally, you are creating stories that can be delivered back to the Product Owner/Manager in the same day. Once the story is delivered, it is either accepted by the Product Owner/Manager or rejected with specific feedback to iterate on. If a story takes longer than a day to complete, consider reflecting on this story and thinking about how it could have been broken down into multiple smaller stories.

Great developers don’t stop at just getting something to work. You should strive to know exactly what every line of code is doing and why. This is where the most valuable learning takes place that separates you from the rest of the pack.

And speaking of great developers, I hit the jackpot when it comes to the people I had the pleasure of working with over the last 6 weeks. I learned a lot about what kind of mentor I want to be to others in the future. I have to give a huge shoutout to Kim Hendrick and Mihir Parikh at EarthVectors. They are so knowledgeable, so patient, so kind, great teachers, even better humans, and just overall badass developers! I can’t say enough good things about them and will truly miss being their sidekick!

Thank you!

Lastly, I want to thank Turing School of Software & Design and EarthVectors, Inc. for making this opportunity possible. I am so grateful for this experience and the people I have met along the way!

Jumping In At The Deep End… was originally published in EarthVectors on Medium, where people are continuing the conversation by highlighting and responding to this story.

I work for a small consultancy, EarthVectors, Inc, and we get to work in a variety of domains and spaces where we help clients solve both business and technical problems every day. I recently ran into an interesting case around ActiveStorage and validations and figured I would share my experiences.

The application is a standard Rails application using ActiveStorage and erb templates to render bootstrap forms and handle your typical CRUD (Create Read Update Delete) operations. During some manual testing of the app we uncovered an issue where after a failed validation, the files that were uploaded were removed and upon resubmission the user would get another failed validation that their file was missing. Definitely not the most intuitive user experience, so we decided it needed to be fixed.

When uncovering an issue like this I like to take a step back and understand why something is a problem before jumping into coming up with solutions. There are usually many solutions to a problem so understanding the tradeoffs between each is key to building great software. This particular issue was a problem because the user did not have a clear indication of what fields were required and in filling out the minimum fields needed for form submission would get into a bad state potentially causing a lot of frustration. We need to provide a solution that will allow the user to understand our expectations and to have a seamless experience without unexpected errors. Let’s explore a few solutions that could help make the workflow more clear to the user.

Solution 1

We can let the user know what fields are required before they submit the form. If they know what we expect them to enter into the form before it is complete then we wouldn’t need to even worry about whether or not the photo is lost after a saved validation because they shouldn’t get into that state. Below is the file field that we need to add styling to and make sure the user can clearly see that this field is required. The ujs_target is used to trigger some javascript to load an image preview after the file input is changed.

<%= form.file_field :photo, hide_label: true, data: { ujs_target: "photo-input" }, help: "PNG or JPG only" %>

First let’s start by adding a required class into our application.scss file. A convention for form fields is to make required fields red so let’s add a class to append some red text after the field input help text.

.required small:after {
  color: red;
  content:" *Required";
  font-size: 0.5rem;
  text-transform: uppercase;
}

Now that we have our CSS class, we need to add it to our file field. Because we are using a bootstrap_form_with we can do that by adding our class as a wrapper_class. After adding that class we should now see our Required text after the help for the file input text.

<%= form.file_field :photo, wrapper_class: "required", hide_label: true, data: { ujs_target: "photo-input" }, help: "PNG or JPG only" %>

Success! Now that we have a working example of applying styling to indicate required fields, what are the tradeoffs for implementing only this solution? First, we have clearly denoted which field is required and at a minimum the user should see this field and understand they have to fill it in before submitting the form. This solution was quick to implement and we could reuse it across all our file fields in the entire app relatively quickly. However, we are still dependent on the user doing the right thing. What happens if they STILL don’t see the text and submit an empty form? What if they are an evil user and try to break the system by submitting nothing? Let’s discuss another solution in Solution 2 that will give us more confidence that the system will work correctly even if the user submits empty data.

Solution 2

Another solution to the problem would be to cache the photo so that the user does not have to re-upload after a failed validation and the photo is available even if the page refreshes. This gets at the root of the problem regardless of whether the user is evil or is just in a hurry and glances over any indications we have to denote that a field is required. Let’s see what it would take to implement something like that. Below is our file field that we would like to modify.

<%= form.file_field :photo, hide_label: true, data: { ujs_target: "photo-input" }, help: "PNG or JPG only" %>

If we were using CarrierWave as our image uploading solution then we would add a hidden field called photo_cache and that would provide the caching mechanism we would need. We can do something similar using ActiveStorage but need to provide the value to the hidden field. We need to get the signed_id from the photo so that ActiveStorage wires everything up correctly upon save. That information is available to us through the form using form.object.photo.signed_id. Additionally, we only want to add that hidden field if the photo has been attached, otherwise we expect the value to be attached using the file field. Let’s add that hidden field to the form so that we can start caching.

<%= form.hidden_field :photo, value: form.object.photo.signed_id if @model.photo.attached? %>
<%= form.file_field :photo, hide_label: true, data: { ujs_target: "photo-input" }, help: "PNG or JPG only" %>

Success! The user may not know that this field is required but at least they will not be forced to re-upload their photo if they received a validation error. As mentioned earlier, this solution gets at the root of the problem and is not dependent on human behavior for the system to work correctly. There are a couple drawbacks worth mentioning with this solution. The code is a bit more complicated than simply adding a CSS class and we would need to remember to do this across all places that we use file_fields. In order to make reuse easier we could extract this into a partial or form helper but that would also increase the complexity of the code for the sake of reuse. If there are many places that we want to reuse this code then that may be a good tradeoff to make.

Solution 3

Another solution to the problem would be to prevent the user from submitting the form using front end validations. That way they know immediately that they are missing required fields before they even make a request to the server. Let’s see what adding the required attribute to enable the front end validation looks like.

<%= form.file_field :photo, hide_label: true, required: true, data: { ujs_target: "photo-input" }, help: "PNG or JPG only" %>

Success! This was pretty painless to add and we have prevented requests from even getting to our system if the user forgets to add required information. However, if we miss adding required to all the fields that are required then we can still get into a weird state where the file gets lost.

In the end we went with a mix of all 3 solutions to provide the most amount of information to the user and to prevent losing files that they had just uploaded. Below is the final solution that we added to the codebase.

Final Solution

<%= form.hidden_field :photo, value: form.object.photo.signed_id if @model.photo.attached? %>
<%= form.file_field :photo, wrapper_class: "required", hide_label: true, required: true, data: { ujs_target: "photo-input" }, help: "PNG or JPG only" %>

Happy coding!

Validations and File Caching using ActiveStorage was originally published in EarthVectors on Medium, where people are continuing the conversation by highlighting and responding to this story.

I see many developers start by designing their model/table and work their way outward to the view. In fact, Rails even encourages this by focusing on “resources”. But what if, instead of working inside-out, we tried working outside-in? That’s what this technique is all about.

Advantages of this technique are:

• Faster Feedback
• Fewer assumptions
• Clearer separation of responsibilities
• Works for any front-end framework
• It’s fun!

Faster Feedback

Fast feedback is awesome and makes programming easier. It helps maintain sanity by limiting the number of mental balls that you need to juggle. Fast feedback alerts you early and often to mistakes that you make and makes debugging those mistakes a no brainer. Outside-In Development starts with “working” software and a feature spec and ensures that it continues to work as you make small changes that improve the design and move from “working” to working.

Fewer Assumptions

One of the big downsides of inside-out development is that, sometimes, when you finally get to the UI, you find that it’s difficult to mold the UI to the expectations of your model. This can contribute to a difficult to use UX. It can also contribute to compromises made to the model (unless you’re willing to start over and re-think the model based on what you’ve learned at the UI layer). Outside-In Development doesn’t make any assumptions about the model and it has a tendency to create cleaner models and separation of responsibility.

Clearer Separation of Responsibilities

In layered architectures (like web-development), it is important to make boundaries between layers clear — separation of responsibilities. Views should not query the database. Controllers should route requests to the business domain layer. Outside-In Development pushes business logic into the business layer and keeps the View and Controller logic clean and simple.

Works for Any Front-End Framework

In my experience this works with any front-end (and back-end) framework. I used it extensively in Ruby and Rails environments — ERB, HAML, React (with or without Redux), and Backbone. I also used it with Elixir and Phoenix. It also works well with APIs (and is not just for front-end development). I’m not naive enough to claim it works for every situation, but I have yet to find a situation that it can’t help with.

It’s Fun!

Outside-In Development is fun! It makes programming simple and the clean design at the end makes me feel good. It also allows me to conserve brain cycles for things that are truly difficult.

Overview

The idea is relatively simple. I start with hard-coded values in HTML. For example, I hard-code each row in a table. Next, I extract the hard-coded values into a variable of hard-coded values. Then I push the hard-coded values down the stack until I can’t push it down any farther. Finally, I introduce a persistence layer. Throughout the process, I add unit tests where applicable.

Example

I’m writing a simple application for my favorite local used car dealership (DMad Motors). They want a page that lists their inventory of cars.

Caveat: This is a simple application, but it’s enough to demonstrate the technique. I often use a “simple” application to learn new techniques, languages, and tools (sounds like a future blog!).

Step One: Introduce Hard-Coded List

I start by hard-coding a simple list of cars. When doing a list like this I like to start with 2–3 items. One item isn’t a list. And more than three is cruel and unusual punishment. 😉

Note that at this stage it is critical that I write a feature spec to verify the table contents. The feature spec is what makes this technique work and allows me to treat the upcoming changes as a large-scale refactoring.

Step Two: Extract Hard-Coded List into Variable

Next up, I extract the hard-coded values into an Array and introduce a loop for the rows in the table.

I’m choosing to use OpenStruct here. I prefer this to a Hash because it allows me to convert to some more concrete type of object later (e.g., ActiveRecord objects). Some people may prefer Struct for performance, but I know from experience that using OpenStruct is temporary and I find it easier to work with.

Also, I could use a plain variable (i.e., cars) instead of an instance variable (i.e., @cars), but using @cars prepares me for moving this to the controller.

No changes to specs are necessary!

Step Three: Push Data Down a Layer (i.e. from View to Controller)

Then I move the hard-coded list into the controller.

I also introduce a controller spec (yes, I still write controller specs).

Step Four: Introduce Persistence

Finally, there’s nowhere else that I can push the hard-coded list of cars down to and it’s time to introduce some persistence. I create a migration, create a Car model, and update both the controller spec and feature spec to persist Car instances and remove the use of OpenStruct.

Here is the final list:

And the specs:

Summary

This is a simple technique that has a lot of upside for very little cost. It works great when pairing because you and your pair can take turns implementing steps and the process of moving from hard-coded values to persistence ensures that you and your pair are on the same page. Give this a try. I think you’ll like it and I hope you have as much fun using it as I do.

Happy Coding!

Code: https://github.com/dmadouros/dmad_motors

App: https://dmad-motors.herokuapp.com/

Outside-In Development was originally published in EarthVectors on Medium, where people are continuing the conversation by highlighting and responding to this story.

While engaged with ATHN, a non-profit serving the hemophilia and thrombosis community I was a part of the small 6-person development team located in Denver, CO. We previously worked with Pivotal Labs, and we are lucky enough to share space with them as well as the Pivotal Tracker team. We get some amazing perks.

In fact, we start our day with catered breakfast.

After breakfast, a cowbell rings signaling us that it’s time for Pivotal standup where we get to meet new people, share helps, interestings, and events with each other. I love being involved in Pivotal standup. We often hear things about technologies that we would otherwise have no exposure to, especially given that we are such a small team. It’s amazing how much you can pick up just from overhearing other people’s problems and solutions.

Next, we head to our EarthVectors standup, where the six of us join a meeting with the remote portion of the technical team. Standup is a time to briefly share what we did yesterday, but mostly talk about anything interesting or surprising that came up. It’s also a time to share design decisions we made and get input from the rest of the team. We have ad-hoc “dev meetings” throughout the day to help make design decisions as well to foster joint ownership of the code.

Because we work in pairs, standup is presented as what “we” did, “we” discovered, “we” would like input on. It’s a subtle but amazingly effective way to build a team and ownership of the code and product.

Something we do a little differently is IPM (iteration planning meeting). Instead of a formal, scheduled meeting to review stories, we do it as needed. After standup is a good time to do a checkpoint and make sure we all have enough work to do. If not, we discuss and estimate enough stories to make sure we keep the flow going during the day. We only have three point sizes, and some of them don’t count.

We estimate stories as 0, 1, 2 or 3. But a 0 and a 3 are exceptions. Most stories are 1’s or 2’s. We don’t argue about point sizes for very long, we just make sure everyone has the same understanding on level of effort, complexity, and unknowns, and then reach a consensus between a 1 or 2. In the end, we know that the points sizes are just estimates. Accuracy is not the end goal. Splitting a story that’s either bigger than a 2 or that would be easier to deliver in pieces is common. This helps us get feedback as quickly as possible. Yesterday we split two stories. Although their point sizes didn’t change, we divided them into different slices so that we could deliver the most complicated part first. We could then make sure it looks and works correctly. Subsequently we can deliver the next story that just adds more of the same type of logic.

Speaking of iterations… we have them, technically. I think our Pivotal Tracker project is setup for weekly iterations starting each Monday. But it really doesn’t impact how we work. Velocity tracking is just for rough planning purposes. We don’t commit to any stories or numbers of stories in any given timeframe. We estimate stories. Our PO uses those estimates to plan releases, and we release whenever it makes sense. If there is a critical deadline, we work together with our PO to figure out the best way to prioritize those stories first. Functionality is pruned if required. The biggest goal is visibility. If a story is estimated at a 1 and we realize it is more complex than we thought, we have a conversation. That story could be split further, the complexity could be removed, or a simpler way suggested.

After standup, we have a habit of walking across the street to a nearby Metropolis coffee. It inspires jealousy in other teams, but there is something about taking a few minutes to connect with each other before we dig into our workday in earnest. Difficult problems from the previous day are often resolved on this walk.

When we return, we have a quick discussion about the pairs for the day. We work in pairs all the time (except when someone’s out). Rotation is a balance between keeping someone on the “stream” that has context and filling our dance card for the week. We try to make sure we all are exposed to every part of the system. The pairs are mixed up to keep fresh ideas flowing and maintain balance within the force. Streams are feature sets that group stories together. Since there are six of us, and we work in pairs, there are three streams going at any given time. Sometimes, two pairs will jump on the same stream to get it done sooner. Usually, each stream is separate and this allows us to avoid merge conflicts and inter-pair dependencies. We’re all working on the same system, but different vertical slices of functionality.

With pairs decided, we get to work. Mihir and I paired today on a new feature that David and I worked on yesterday. Yesterday, David shared context with me, and today I passed that along to Mihir. Tomorrow, I expect Mihir will stay on that stream to share context with someone new while I join someone else and pickup context from another stream. This is how we avoid silos, jointly own all the code, and pass context of new ways to do things to each other. It’s amazing.

Throughout the day, whenever a pair feels they could use more input, they ping the entire team. We sit at open pairing workstations, so an impromptu dev meeting involves finishing your current thought and swiveling your chair to form a circle. We’ve somehow managed to create an egoless environment where everyone values and wants input from everyone else. Today, Lew and Joe pinged the whole team to share tips about the rspec upgrade they were working on.

We usually play ping-pong once or twice during the day. It’s a great way to take a break, especially when stuck on a hard problem. Although it’s hard to tear yourself away, usually one of us will be thinking clearly enough to suggest a pong break. We almost always come back with a better plan of attack, even if that plan is to call a dev meeting for help.

When lunchtime comes around, sometimes we grab food and find a conference room to watch an episode of something interesting on Netflix. We informally reserve Fridays for team lunch out — a good way to connect and unwind.

In the afternoon, it’s back to work! We use TDD to design and develop. Finding better ways to design code and test it are frequent dev meeting conversations. When to stub? What to cover at a unit-level vs. a higher-level test? We’re constantly making new “rules” (my word, others will dislike that word choice) that we try out and revisit often. We’re not afraid to experiment, but we make sure we’re doing it as a team. We’ve posted our team values if you want to know more.

That’s our typical day, but I may not have described the fun very well. If Andrew’s not cracking us up, then we’re poking fun at each other or ourselves. It’s definitely a balance we’ve achieved between having fun and producing a ton of high quality work. We do both really well.

An EarthVectors developer’s life was originally published in EarthVectors on Medium, where people are continuing the conversation by highlighting and responding to this story.