A Shared Context Optimization to Eliminate 75% Service Calls
Reflections on DRY Principle & Chatty Services Anti-Pattern
Personalization lies at the heart of modern recommendation systems. To deliver relevant suggestions, a recommendation engine must understand who the user is — what they’ve browsed, what they’ve purchased, and what patterns emerge from their behavior. At Insider, our Recommendation API relies heavily on the Unified Customer Database (UCD) to fetch this contextual information in real time.
As the system evolved and new features were added, we noticed a growing inefficiency: the number of HTTP calls to UCD had increased as new use cases were introduced to the product pipeline and the API side. It caused the HTTP requesting interaction between the Recommendation API and the UCD to resemble the Chatty Services Anti-Pattern. The number of unnecessary requests to the personalization source decreased the overall success of the feature quality and hurt the health metrics of the Recommendation API.
The existing implementation approach in the Recommendation API violated the common Don’t Repeat Yourself (DRY) Software Development Principle. As new recommendation use cases were introduced to the API, we often employed very similar implementation blocks to perform very similar needs. It resulted in introducing nearly implementation blocks to the API and having unnecessary repetition in the codebase.
This post walks through how we identified the architectural bottlenecks, the refactoring strategies we applied, and the performance gains we achieved as a result.
We perform various HTTP calls to the UCD’s Profile API, which returns User Profile and User Events information to other services. UCD is the primary source of all user events and attributes of Insider, and the most important source of personalization features that we present to end users. In our cases, HTTP calls from the Recommendation API to UCD allow us to gather,
- Products that a user interacted with recently,
- Categories that a user had a recent interest,
- General interests and affinities of a user, extracted with predictive machine learning models, which enhance recommendation accuracy,
These insights enable the Recommendation API to personalize an incoming API request based on the given user ID.
While maintaining a healthy connection between these two services is crucial to preserve the personalization quality of the Recommendation API, the overall design choices within it have led to redundancy in the total number of UCD Profile API calls required to meet the personalization needs of requests.
Next, we discuss these architectural problems, applied solutions, and the results after applying refactoring.
Current Architecture
Recommendation API is one of the cornerstone components of the Smart Recommender product of Insider that provides real-time product recommendations to end users and other internal products of Insider. It is written as a Scala API that benefits from the Akka framework’s Actor models.
Recommendation API has various actor implementations, each encapsulating the set of operations that is required to fetch product recommendations that the end user can request for different scenarios.
To serve as an example, the AnalyticsRecommendationActor is implemented to perform;
- Fetch the products that the given user interacted with,
- Fetch the interests and affinities that the user is attributed with,
- Fetch and filter products from the Product Catalog Database with the given recommendation strategy,
- Introduce personalization based on the parameters in the API request. These personalization options can be,
- Rank the products that match the user’s interests and affinities higher,
- Recommend products from the last category that the user interacted with,
- Exclude products that the user recently interacted with.
As a different example, the RecentlyViewedActor is responsible for fetching the products that the user recently viewed. As suggested by its name, the response we obtain from this actor often ends up on the websites or mobile apps of our partners as “Last Visited” product recommendations. To provide these recommendations, it performs;
- Fetch the products that the given user interacted with,
- Fetch the interests and affinities that the user is attributed with,
- Fetch and filter products from the Product Catalog Database obtained from the UCD Profile API response of the user.
- Introduce personalization based on the parameters in the API request. These personalization options can be,
- Rank the products that match the user’s interests and affinities higher,
- Recommend products from the last category that the user interacted with,
- Exclude products that the user recently interacted with.
While encapsulating these operations benefits the project in terms of readability, we may need to invoke multiple actors to return a personalized product recommendation response to end users. As each actor implements very similar personalization features, they are likely to perform the same UCD Profile API HTTP calls. For a long time, this resulted in introducing an important redundancy and inefficiency for the sake of simplicity.
Solution
As the development team, resolving the depicted problem was one of the tech debt items that we had listed for a while. It was a known tech debt item to refactor the actors in the API to share the user details that we fetch from UCD to optimize the total number of Profile API calls we perform.
Get Mert Gurkan’s stories in your inbox
Join Medium for free to get updates from this writer.
As the first step of the improvement process, we needed to decide on the implementation method. The Controller level of the API orchestrates all of the actors responsible for fetching recommendations. We decided to put the orchestration of sharing user information to that level as well. To this end, we introduced a fetchUserContextOnce method to the Controller class of the API, introducing the ability to access the UCDService. As presented in the code snippet below, the fetchUserContextOnce method is now called as an asynchronous function prior to the operations used to fetch product recommendations from various actors.
// Simplified version of the Controller level Actor orchestration
// getActorResponses often performs multiple actor invocations
def getRecommendationsFromActors(router: ActorRef, message: BaseRequestMessage): Future[RecommendationResponse] = {
for {
userContext <- fetchUserContextOnce(message)
enrichedMessage = enrichMessageWithUserContext(message, userContext)
resultingRecommendations <- getActorResponses(router, enrichedMessage)
} yield RecommendationResponse(resultingRecommendations)
}The UserContext is the new class that returns from the fetchUserContextOnce function.
case class UserContext(
user: User,
purchasedItems: List[String],
viewedItems: List[String],
lastVisit: UserLastProduct,
partner: Partner,
userId: String
)
case class User(
userId: String,
val affinities: List[AttributeAffinities] = List.empty[AttributeAffinities],
userBasedRecommendations: List[Product] = List.empty[Product]
)It includes the user details we may need to perform all of the existing personalization features in the actors.
When a single request can’t fully satisfy personalized recommendation needs, we invoke multiple actors to fetch recommendations from different strategies. This fallback approach ensures we return a sufficient number of products in the API response. Each additional actor receives a new message class created by copying details from the original request’s message object.
// The first recommendation message is enriched with the UserContext
def enrichMessageWithUserContext(
message: BaseRequestMessage,
userContext: UserContext
): BaseRequestMessage = {
message match {
case msg: UserContextAware => msg.withUserContext(userContext)
case _ => message
}
}
// withUserContext function converts the previous actor's typed message to a
// new type and copies the UserContext
def withUserContext(context: UserContext): AnalyticsRecommendationMessage =
this.copy(userContext = Some(context))As part of the solution, we added the UserContext as a field to message classes. When new message objects are generated, the UserContext of the initial message is copied to the new message to keep the user details. It allows us to reuse the Profile API response we got for the user in different actors. On the actor side of the solution, we introduced a new trait called UserContextAware. This trait includes a common implementation that checks whether the current personalization request needs a new UCD Profile API call or if it can use the UserContext. Message classes that actors receive extend this trait to have the UserContext capabilities. This perspective change allowed us to remove the repeating implementation blocks and transition them into a common reusable trait. Thus, it was our action to adapt the DRY Principle to this problem.
The introduction of the UserContext and enriching message classes with these objects resolves our chatty service anti-pattern. If we gather user details and store them in the UserContext, we do not perform additional Profile API requests. In scenarios where multiple actors are used to return recommendations, only the first actor performs an actual HTTP call to the UCD Profile API. It is the primary positive impact of our solution in terms of HTTP request optimization.
val userContext = UserContextUtil.extractUserContext(params)
val recentlyViewedItemsFuture = if (userContext.nonEmpty && userContext.viewedItems.nonEmpty) {
Future.successful(userContext.viewedItems.take(getRecentlyViewedSize(params.size)).toArray)
} else { getRecentlyViewedItems(params) }This is the UserContext integration of the RecentlyViewedActor that is described earlier in the post. In cases where the user details are already fetched, we do not perform an additional UCD request to obtain the recent items that the user visited. Many of the actors that is responsible for addressing a use case now implement similar code blocks that
Results
After establishing the refactored version of the Recommendation APIs controller, observing the implications of the changes was not challenging. Recommendation use cases that required invoking multiple actors resulted in only requesting the user details for the first actor’s personalization needs. The remaining sequential actor invocations utilized the UserContext objects and stored user details.
The case depicted above is one of the common use cases that we encounter in the Recommendation API. It used to have a high rate of fallback recommendation strategy usage and caused a high number of UCD Profile API HTTP calls to return API responses to end users.
These optimizations improved the Recommendation API’s overall performance as well. By streamlining the HTTP dependencies between the Recommendation and UCD Profile APIs, we reduced p50 latency by nearly 7 ms. And while p95 and p99 latencies are influenced by other edge cases, we also saw fewer timeouts and circuit breaker failures related to the UCD Profile API requests.
The trade-off in readability at the Controller and Actor levels proved worthwhile. In some scenarios, we reduced UCD Profile API requests from 3 or 4 down to just 1. Seeing these optimizations translate into measurable latency improvements is satisfying — and more importantly, that improved performance will be felt by our customers and end users.
To summarize, with these changes
- We decreased the p50 latency of the Recommendation API from ~35 ms to ~28 ms,
- We decreased the number of UCD calls during the most complex use cases to 1 from the total number of actor invocations needed for these requests
- Our total number of Circuit Breaker warnings and errors (corresponding to timeouts and request failures) decreased nearly 14 times
As important as the positive results above, we also applied implementations that follow correct software development principles. Our reflection after the solution was: We should not tie the improvements we observed to the details of the project of the actor classes. In fact, actor classes of the API can be considered use case implementation blocks. Reshaping the crucial part of the API according to best practices & correct software development patterns allowed us to obtain these results.
Our improvements with the Recommendation API and the Smart Recommender are not limited to this post. Curious readers are welcome to check our recent real-time inference-based personalization feature called User Engagement Recommendations. While my colleague Can’s post focuses on this new feature, my recent post focuses on its integration with the Recommendation API.
Thank you for your time and attention. Stay tuned for the upcoming blog post, where we will discuss scaling and improvement actions performed on the Product Catalog Database side of the Recommendation System.
