Bringing structure to your express app

Here at Leapsome, we have been using NodeJS since the very early days of the company as there are a number of benefits in using the same language across the stack. One of the most common web frameworks for NodeJS, even today, is Express. It provides a simple and clean API and doesn't enforce any type of code structure, which meant that our codebase was able to evolve organically.

Some challenges we faced with Express

That flexibility allowed the product to be developed fast, taking into account the needs of the market. Nevertheless, as the Leapsome team scaled and the product matured, the organic evolution of the codebase was causing a few challenges.

No separation of concerns

Input formatting, validation, error handling, business logic and response formatting were all mixed in the same function.

Code hard to test, debug & reuse

The tight coupling between handlers and routes meant that, for the most part, handlers didn't return any values and internally called the response object from Express, making it particularly challenging in some cases to test, debug and even reuse them.

No clear code structure

No consistent style across the codebase, as some handlers returned Promises, some didn't return anything (response handled inside the handler) and some returned an array of [error, response]. That lead to the code being sometimes hard to navigate, as there was no common enforced style of organising the code.

No consistent error handling or input validation

As there was no enforced way of handling errors, it lead in some cases to errors bubbling up to the client without a proper message or status code. Besides, input validation was left to the sole discretion of the handler, which in some cases lead to inputs not being properly validated, and, ultimately, bugs.

Being a bootstrapped company, it was not feasible for us to freeze the development of new features and take on a major re-write of our codebase.

Thus, while solving the challenges stated above, the solution also had to obey a certain number of constraints:

• Not require a full rewrite
• Have little overhead while still providing some common structure
• Be flexible enough to be painlessly replaced / evolved as needed

Layered Architecture

To bring a lightweight structure to the backend logic, we applied a layered architecture to our code. The main guiding principle for our backend code follows the separation of concerns displayed in the following layered architecture.

📥 Check out the entire code here.

‍

Authentication

Authentication is performed on most client-facing endpoints via a JWT token (API token if accessing the Leapsome API).

The authentication is implemented as a middleware, that also packs i18n, structured logging and metrics.

Basic Authorization

Basic authorisation against user's rights stored in the JWT token are also performed on each relevant route via a middleware.

In cases where authorisation rules are more complex than the user possessing some specific rights, subsequent authorisation will be performed at the Business Validation / Business Logic layer.

Request Parsing

Next step on the layer is request parsing. This is only relevant when a handler is called in the context of a route.

Request parsing is handled at the Resolver level in the framework.

Schema Validation

On top of parsing the request in a consistent way, we will also add some basic input validation at the Resolver level, where we make sure that the handler gets well-formed data.

The Schema Validation runs even if the handler is called outside of a route context!

Note: the data can still be invalid from a business point of view (for example, trying to update a field in a resource the user has no access to), but from a structural point of view, the data should be a valid object.

Business Validation

In some cases, having basic authorisation and well-formed inputs is not enough to ensure that a certain action is allowed (for example, trying to update a specific field in a resource). As such, some more validation, closer to the business logic, is needed at the Handler level.

Business Logic

This is where most of the code will live, including the data layer and any specific business logic. The code is implemented at the Handler level.

Note: handlers always return Promises.

Client Response Formatting

To allow re-usability of the handlers outside of a router context, we separate the return value from the handlers and the client response formatting.

The latter happens at the route level directly, so in our case, in Express routers.

We do provide a list of formatting functions to simplify the response formatting and harmonise status codes (json(), nocontent(), ...).

Error Handling

Finally, we perform error handling at the very top level of our router hierarchy. In practise, we bubble up any error that might occur anywhere in the different layers.

To harmonise and instrument errors, we provide a list of specific errors that can be thrown.

Framework Artefacts

To provide structure and abstraction, the framework provides 3 distinct artefacts:

Resolvers

A resolver is created via the resolver(name, resolve, validate) factory function.

The name parameter is required and allows handlers to identify the resolver in their ctx.

The resolve parameter is require and allows handlers to parse the request req when called in a route context. resolve is a function that takes req as only parameter and returns a value from that parameter.

The validate parameter is optional (defaults to an identity function) and performs schema validation and type coercion when needed. This function is run anytime the handler is called, be it inside a route or inside another handler. This function always returns a value for the resolver.

Example:

‍

Handlers

A handler is created via the handler(resolvers, func) factory function.

The resolvers parameter is an Array of resolvers. Those resolvers will be ... resolved before the handler function is call.

The func parameters is required. It contains all the business validation and business logic and is called after all the resolvers are resolved. It returns a Promise.

To cater to the route context, we also provide a utility function on the handler called .fromRequest(req) which takes an Express request object and runs the resolver resolve function to select the appropriate field.

Example:

Errors

Finally, we also provide some utility functions to easily handle errors.

All errors at the handler or resolver level are thrown and bubbled up the route hierarchy until they are handled at the top level.

We also provide specific types of errors which represent specific HTTP response code and deal smartly with stack trace and logging.

Final Implementation

Tying everything up together, we ended up with the following core implementation of our framework.

📥 Check out the entire code here.

With that implementation, we have now successfully resolved all of the issues we were facing at the beginning:

• Clear separation of concerns:Thanks to the layered approach the frameworks gives us, we now have a clear separation between input validation, business logic, error handling and response formatting.
• Code easily testable, debuggable and reusable:With the clear separation of concerns, we can now test each block separately. Handlers are just functions that return a value, making them easily observable.Being functions with no dependencies to the response object, they are also easily reusable in other handlers, while still providing input validation.
• Consistent code structure:By enforcing a certain way of architecting our code, new code using the framework yields more consistently styled code, which is ultimately easier to read, understand and maintain.
• Consistent and built-in error handling and input validation:Finally, with error handling and input validation built into the framework, it becomes easier to debug potential issues and clears up the communication with the client code.

Ultimately, every technical endeavour we undertake at Leapsome is geared towards achieving our purpose of making work more fulfilling for everyone, and being able to rapidly build high-quality features aligns with that purpose (by the way, we are hiring!).