Introduction:

When running a Django Application in production, we will log the message into a log file, and when in need of debugging, we’ll go through the logs and find or track the issue.

But when multiple requests are handled by the Django Application, the messages will be logged to the file in order. Hence, we cannot track the sequential flow of the API call.

So that we are going to use custom middleware to attach a unique ID to the thread's local variables, and create a filter that accesses that variable and attaches it to any log record.

Flow Diagram:

Prerequisites:

Python 3.8

Django Project

Steps:

Step 1: Configure loggers with Django

In case you didn’t have a Django application configured with the logger, please refer to the following article Logging with Django.

Step 2: Creating middleware

Now, you can simply navigate to the Django core app directory and create a folder named 'middleware'.

cd <django_project_directory>/<core_app>
# in my case
cd /home/ubuntu/django_example_repo/core/
mkdir middleware
cd middleware

Now, create a file named unique_logs.py by running the following command.

touch unique_logs.py

Please refer to the following example image for reference, execution results and folder structure of the above-executed commands.

Open the file unique_logs.py and add the following code to the file.

from threading import local
import uuid

_locals = local()

class UniqueLog():

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        request.META["unique-request-id"] = uuid.uuid4()
        _locals.unique_request_id = request.META["unique-request-id"]
        response = self.get_response(request)
        return response

The above code will create a middleware named UniqueLog, which will be called when every request is received by the Django Application.

UniqueLog middleware will create a uuid4 into unique_request_id and load it into the local() function. And when every time request is received, new unique uuid is created every time.

Step 3: Create a Logger filter

Then again, open the same unique_logs.py file and add the following code.

import logging

class UniqueLogFilter(logging.Filter):

    def filter(self, record):

        if not hasattr(record, 'unique_request_id'):
            record.unique_request_id = ""
        if hasattr(_locals, 'unique_request_id'):
            record.unique_request_id = _locals.unique_request_id
        return True

The above code will create a filter named UniqueLogFilter, which will create a format named unique_request_id from the local() function’s unique_request_id created by the middleware UniqueLog.

Note: Now the unique_logs.py file will finally look like the code snippet below.

import logging
from threading import local
import uuid

_locals = local()

class UniqueLog():

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        request.META["unique-request-id"] = uuid.uuid4()
        _locals.unique_request_id = request.META["unique-request-id"]
        response = self.get_response(request)
        return response

class UniqueLogFilter(logging.Filter):

    def filter(self, record):
        if not hasattr(record, 'unique_request_id'):
            record.unique_request_id = ""
        if hasattr(_locals, 'unique_request_id'):
            record.unique_request_id = _locals.unique_request_id
        return True

Step 4: Adding middleware to settings.py

Open the settings.py file and look for MIDDLEWARE.

Now add the middleware with the path to the settings by adding the following code.

Now, append the following code to the middleware configuration at the end of the

 .... # other middlewares
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    # our custom middleware
    'core.middleware.unique_logs.UniqueLog',
]

Step 5: Configure the logger

In the settings.py file, look for the logger configuration under the variable.

Create filters by adding the following code.

The filter refers to the filter class UniqueLogFilter we created in step 3.

'filters': {
    'uniqueID_filter': {
        '()': 'core.middleware.unique_logs.UniqueLogFilter',
    }
},

Create a formatter named unique_format by adding the following code.

'formatters': {
    'unique_format': {
        'format': '%(unique_request_id)s %(name)s %(levelname)s %(asctime)s %(message)s '
    },
},

Create a new handler named unique_logs as shown in the code snippet below.

'handlers': {
    'newfile': {
        'level': 'DEBUG',
        'class': 'logging.FileHandler',
        'filename': '/home/ubuntu/django_loggers_example/debug_example.log',
        'formatter': 'unique_format',
        'filters': ['uniqueID_filter'],
    },
},

Now add that handler to a logger and restart the Django Application.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'uniqueID_filter': {
            '()': 'core.middleware.unique_logs.UniqueLogFilter',
        }
    },
    'formatters': {
        'verbose': {
            'format': '{levelname} {asctime} {module} {process:d} {thread:d} {message}',
            'style': '{',
        },
        'unique_format': {
            'format': '%(unique_request_id)s %(name)s %(levelname)s %(asctime)s %(message)s '
        },
    },
    'handlers': {
        'newfile': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': '/home/ubuntu/django_loggers_example/debug_example.log',
            'formatter': 'unique_format',
            'filters': ['uniqueID_filter'],
        },
    },
    'loggers': {
        '': {
            'level': 'DEBUG',
            'handlers': ['newfile'],
            'propagate': False,
        },
    },
}

Tracking an API flow in a Django application using UUID in logs