Improving Web Application Error Diagnosis with Request IDs

Error handling and logging go hand in hand in web application development. Whenever we encounter unexpected situations in web applications, we want to log something. We also want to return meaningful error messages to the users of the web application.

Imagine the following scenario:

A user of our web application tries to log in, but after entering the correct username and password, the login fails. They contact customer support but are only able to tell the error code to trace the error. As developers, we find in the error logs individual errors corresponding to the same error code from the same time period, but we do not know which of them corresponds to the particular request sent by the user.

We might be able to deduce the correct request based on the user ID. But what if the error message doesn’t define the user and there is a huge number of overlapping request-response threads in the error log? 

Or maybe the cause of the error seems to originate from another service.

Maybe, in this case, the authentication service was down? Perhaps the service bus we were using to pass down the authentication request was timing out? Whatever the reason was, without something to connect the error to the request, it would be hard to understand what happened.

Let’s spend some time trying to understand how we can improve error diagnosis with request IDs.

Request ID to the Rescue

What is a Request ID?

A request ID is a unique identifier for each request. It is a value that is generated for each request and is passed along with the request and the response. The request ID is used to identify the request in the logs and to connect the request to the response and everything happening between them.

A natural place to add the request ID to the headers is whenever our web application receives a request. This way, the request ID is available for the whole request-response cycle.

There are several benefits to following and logging a request ID:

  1. Debugging: In case of any issues or errors, having a request ID makes it easier to track and debug the problem by connecting logs and traces for a specific request.
  2. Monitoring: Request IDs can help monitor the application’s performance and behaviour and identify any trends or patterns that may require attention.
  3. Improved User Experience: With request IDs, users can receive more accurate and helpful support from customer service, as the support team can easily identify the user’s request and provide them with more relevant information.
  4. Collaboration: Request IDs can also be used to enable collaboration between different teams, such as development and operations, by providing a common identifier for a specific request.
  5. Auditing: Request IDs can be used to create an audit trail for regulatory compliance or security purposes, allowing for tracking and monitoring of sensitive or critical actions taken within the application.

Could we do this automatically? Well, it depends on the web server, and the platform one is using. For example, in Heroku, the request ID is automatically added to the request headers.

But even without support for request IDs in the platform, this is easy to implement by ourselves. The process is the same for all frameworks and platforms. We just need to generate a unique string for each request and add it to the request headers. If one likes to return the request ID to the user of the web application, it can be added to the response headers too.

Example: How to Generate Request IDs on Node Express?

Let’s see how we could implement request ID generation on Node Express. In principle, the process would be the same in other web frameworks, too, just generate a unique value for each request and add it to the request headers.

In Express, we can use middleware to generate a request ID for each request. To put simply, a middleware is a function that has access to the request and response objects and also to the next middleware function in the web application’s request-response cycle. Basically, we are able to manipulate request and response objects in middleware as we like.

A middleware that generates a request ID for each request could look like this:
/middlewares/requestId.js

import { v4 as uuidv4 } from "uuid";

const requestId = (request, response, next) => {
  const requestId = uuidv4();

  request.headers["x-request-id"] = requestId;
  response.set("x-request-id", requestId);

  next();
};

export default requestId;

The middleware uses the uuid package to generate a unique request ID. The request ID is then added to the request headers and the response headers. And finally, middleware calls the next middleware function in the request-response cycle.

Now the request ID is available later in the request-response cycle, for example, in the error handling or logging middleware.

How to Log Requests and Errors with Request IDs?

Typically, request logs and error logs are written to different files or locations. With the request ID, we can easily connect the request to the error log and follow the request through the logs.

Let’s see how we could implement request logging and error logging with request IDs in Node Express. Again, we can use middleware for this.

A request logging middleware could look like this:
/middlewares/requestLogger.js


const requestLogger = (request, response, next) => {
  const requestId = request.headers["requestId"];

  console.log(
    `[Request]: ${requestId} ${request.method} ${request.url} ${JSON.stringify(
      request.params
    )}`
  );

  next();
};

export default requestLogger;

And similarly, an error logging middleware might look like this:

/middlewares/errorLogger.js

const errorLogger = (error, request, response, next) => {
  const requestId = request.headers["requestId"];

  console.error(`[Error]: ${requestId} ${error.stack || error.message}`);

  next(error);
};

export default errorLogger;

Notice how the first parameter of the error logging middleware is the error object. We want to pass this object to the next error handler in the stack. In this case, we would like at least to return the error to the user of the web application.

How to Return Request IDs to the Users of the Web Application?

As we already have set the request ID in the response headers in our request ID middleware, the request ID is returned to the user of the web application automatically.

How this request ID would be used by the user of the web application depends on the application. For example, the user could send the request ID to the support team if they encounter an error. With the request ID, the support team could easily find the related request and the error log from the logs.

The decision on whether the request ID should be visible to the user could be made based on the HTTP response code. Typically, if the error status is 5xx, we would show the request ID to the user. This way, the user could pass the request ID to the support team for closer examination.

How to pass Request IDs to Microservices?

In our APIs, we might also make calls to external services. If these services are managed by us, we could add the request ID to the request headers of the calls to the services. This way, we could follow the request through the logs of these services too.

We might or might not have the logs aggregated in one place from all these services. In any case, having the request ID available will make it easier to resolve issues.

Conclusion

In web application development, error handling and logging are essential for tracking unexpected situations and returning meaningful error messages to users. A request ID is a unique identifier for each request that can be used for easy identification of requests and errors in the logs.

We have seen here an example of how to implement a simple Node Express middleware for generating a request ID for each request. Depending on the platform used, the principle is the same. We have also seen how to log requests and errors with request IDs and discussed how to make use of these principles in real applications.

All in all, request IDs are a simple but powerful tool for tracking requests and errors in web applications.

Author Information

Sami Ruokamo is a software developer and works at Buutti. He is interested in software development, especially web development. He has been working with web application development for over ten years. He is also interested in DevOps and cloud technologies.

To see a more thorough presentation of implementing these ideas on Node Express, check out https://github.com/samiru/articles/blob/main/node-express-logging-error-handling/tutorial.