Sitemap

Java Data API Annotation Processor

6 min readMar 6, 2022

--

A developer toolkit that reduces time to build Data APIs by 75% through rapid API generation (REST & GraphQL), using Java compile-time annotations.

Press enter or click to view image in full size
Photo by Phil Shaw on Unsplash

What problem does this product solve?

With the advent of microservices, the overall scope of a system is increasingly being segregated into multiple services.

Consider a big enterprise that has several systems, each focusing on a specific set of responsibilities. There is a need for a single view of the data generated by these systems, organized by business domains. At a large organization, a simple 360-degree view of a farmer may require integration from a dozen data sources.

Data Mesh is an approach that enables an organization’s data to be easily accessible and interconnected across the entire business. Data Mesh is found on the following base principles:

  1. Data as a Product
  2. Data is available everywhere, Self-Serve
  3. Data is governed wherever it is

The data mesh approach provides standard data guantees and access patterns by exposing a canonical set of data sets to consumers, thus obviating them from dealing with the complexities of the source systems.

Within a Data Mesh, a Data API is an interface that provides a self-serve consumption layer and solves the objective of making data discoverable, and secure, so other teams can make use of it.

The manual development of Data APIs for such read services severly impacts the agility & standardization of the consumption layer.

This is where the Java Data API Annotation Library comes into play.

I recently led a micro-team in designing, building, and building out this product. Through the product, we accomplished a 75% reduction in time to roll out a new Data API.

Get Jeyabalaji Subramanian’s stories in your inbox

Join Medium for free to get updates from this writer.

Let’s look at the salient features, high-level design & technologies behind this product in the sections below.

Salient Features

  • Ability to express query or sorting ability on fields through annotations
  • Allow search across hierarchical table structure
  • Database agnostic
  • Ability to plug in this as a library within Maven or Gradle dependency management providers.

Express Query or Sorting ability on fields through annotations

  • Allows developers to use a handful of annotations (similar to the familiar JPA annotations such as Table, Column etc.) to express their query/sort requirements and produce a full-fledged Data API.

Here’s an example of how users would enable query and sort on a field:

@Entity 
@Table(name = "employee")
@DataApi
public class Employee implements Serializable {
@Column(name = "employee_name")
@EnableQuery
@Sortable
private String employeeName;
}

Allow search across hierarchical table structure

  • Ability to search across hierarchical table structure by exposing all fields enabled for a query across an entire tree of entities.

For example, let us say you have enabled DataApi for Entity A, which has a One To Many relationship with Entity B. Entity B has a One to Many relationship with Entity C. The search class for entity A will have filter fields belonging to Entity A, Entity B, and Entity C.

Press enter or click to view image in full size

This enables consumers to develop powerful Data APIs that support complex queries across tables, without breaking a sweat.

Database agnostic

  • Ability to enable Data API on any relational database. The product does not have a lock-in with any specific database. The queries constructed are ANSI SQL compliant.

High-Level Design

  • At a high level, the product employs compile-time annotation processing that enables developers to express their API query requirements as annotations. These are similar to the JPA Annotations developers are already familiar with. Annotations are provided on the field level to enable querying and/or sorting.
  • The product then extracts the metadata through these annotations & builds a meta-data schema that houses developer specifications.
  • The meta-data schema is further leveraged to build a full-fledged API codebase through Java code generation at compile time.
  • The generated codebase exposes the following:
  • Search classes are exposed for developers to express complex queries or sort requirements
  • Service classes are exposed for passing in the search objects constructed above & retrieving search results.

The high-level design schematic is as follows:

Press enter or click to view image in full size

End-User flow

  1. The user includes api-annotation-processor as a maven dependency in the package manager.
  2. The user decorates entity classes & fields inside these classes with appropriate annotations (for query, sorting, etc.).
  3. The user compiles the code.
  4. The system reads annotations from entity classes, generates metadata, and generates an API codebase under the target folder, exposing search classes & service processors.
  5. The user uses the search classes generated above, composes search objects, and implements API requirements by calling the service processor.
  6. The user can choose to unit test the API generated.

From end users perspective, they just need to annotate entities & compile the code. They can then freely express any complex query by constructing appropriate search objects, and not worry about the underlying implementation.

Note on Query parameters

The system generates appropriate qualifiers based on the data type of the field on which EnableQuery is used. Specifically,

  • String fields: like and in are exposed
  • Numeric & Date fields: equals, not equal, greater than equals, less than equals are exposed

Just by annotating a field with EnableQuery, developers are able to express the full range of query possibilities, without writing a single line of code.

Note on Annotations

Entity level annotations:

  • DataApi: Use this annotation on any Entity Class and watch it getting transformed into a REST Service and/or a GraphQL endpoint.

An entity must contain all of DataApi, Entity and Table annotations to be considered by the processor.

An entity must contain a Primary Key field. I.e. a field annotated with Id or EmbeddedId.

Field level annotations

  • EnableQuery: Use this annotation to enable search on a field within an entity. The system automatically generates qualifier fields for Equals, NotEquals, In, Not In, Like, Greater than Equals, Greater Than, Less than Equals, and Less Than criteria.
  • Sortable: Use this annotation to enable sorting on a field within an entity.
  • InnerJoin: Use this annotation on relations (One to One or One to Many or Many to One) to specify the inner join relation.
  • LeftJoin: Use this annotation on relations (One to One or One to Many or Many to One) to specify the left join relation.

A field must contain one of Column, OneToMany, ManyToOne, OneToOne, EmbeddedId or Id annotation to be considered by the processor.

Technology components

  • Java Poet: A Java API for generating .java source files. We used Java Poet to translate metadata schema into code.
  • QueryDSL: A powerful high-level abstraction for SQL queries. We used QueryDSL to compose powerful yet readable fluent queries by dynamically joining entities based on metadata, adding dynamic predicates (i.e. search criteria), and sorting by clauses.
  • GraphQL SPQR: GraphQL SPQR (GraphQL Schema Publisher & Query Resolver, pronounced like speaker) is a simple-to-use library for rapid development of GraphQL APIs in Java. It uses a code-first approach to generate GraphQL schema and APIs from the existing models.

Conclusion

By composing a set of powerful libraries, we were able to produce a solid developer product that reduced the time to build Data APIs by 75% and enabled developers to channel their time for other value-adding activities.

--

--

Written by Jeyabalaji Subramanian

Experienced Technology Professional | Writer.