Demonstrate how to configure logging

Table of Contents

1. Introduction
2. Overview of NGINX Logging
3. Configuring NGINX Access Logs
4. Configuring NGINX Error Logs
5. Understanding Log Levels in NGINX
6. Customizing Log Locations and Virtual Host Logging
7. Advanced Logging Techniques: JSON, Conditional Logging, and Third-Party Integrations
8. Best Practices and Security Considerations
9. Visualizing the NGINX Logging Workflow
10. Differences Between Access and Error Logs
11. Conclusion

1. Introduction

NGINX is renowned for its high performance as a reverse proxy server, web server, and load balancer. A key element of maintaining and troubleshooting an NGINX installation is its logging system. This article explores the configuration of NGINX logs by focusing on two primary log types—access logs and error logs. Readers will learn how to set up logging, customize log formats and file locations, define error severity levels, and differentiate between the functions of access and error logs. Comprehensive instructions supported by code snippets, tables, and diagrams are provided to ensure that server administrators can optimize their logging configuration for better server performance and security.

2. Overview of NGINX Logging

NGINX maintains two main types of logs:

• Access Logs: These capture details about every client request to the server. Typical data includes client IP address, request date and time, requested URI, HTTP status code, size of data transferred, referrer information, and user-agent information. Such logs are crucial for analyses of traffic, user behavior, and for detecting suspicious activities.
• Error Logs: These record server-side errors and issues encountered while processing requests. They provide information on configuration problems, file access errors, and potential security warnings. Error logs also include messages classified by severity levels such as debug, info, notice, warn, error, crit, alert, and emerg. Each error message is logged at the closest configuration level where the error occurs, with settings inherited from higher contexts unless explicitly overridden.

NGINX’s logging system plays an essential role in proactive server management and troubleshooting. Analyzing both access and error logs enables administrators to pinpoint problems early, optimize system performance, and ensure security by identifying unauthorized access attempts or misconfigurations.

3. Configuring NGINX Access Logs

Access logs in NGINX record every request processed by the server. They provide vital insights into server traffic and usage patterns. By default, access logging is enabled and typically recorded in a file such as /var/log/nginx/access.log.

3.1. Enabling and Setting Up Access Logs

To enable access logging, the access_log directive is used in the main NGINX configuration file (usually nginx.conf) or within individual virtual host configuration files (server blocks). A basic configuration example is given below:

http {  
    ...  
    access_log /var/log/nginx/access.log combined;  
    ...  
}  

In this example:

• The first argument specifies the log file path (/var/log/nginx/access.log).
• The second argument, combined, is a standard log format that includes client IP, request, status, referrer, and user-agent details.

3.2. Customizing Log Format with log_format

Administrators often require additional details captured in the logs or a different arrangement of data. The log_format directive enables customization of log entries. For instance, a custom log format can include additional metrics such as request time:

http {  
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '  
                    '$status $body_bytes_sent "$http_referer" '  
                    '"$http_user_agent" "$http_x_forwarded_for" '  
                    '$request_time';  
    access_log /var/log/nginx/access.log main;  
}  

Here, $request_time is added to monitor how long each request takes, which can be vital for performance analysis.

3.3. Logging in JSON Format

For modern log analysis and integration with log management tools, structured JSON logging is increasingly popular. Although NGINX does not natively output JSON for access logs, you can use the escape=json parameter to produce logs that are easier to parse:

http {  
    log_format json_format escape=json '{ "time": "$time_local", '  
                                          '"remote_addr": "$remote_addr", '  
                                          '"request": "$request", '  
                                          '"status": "$status", '  
                                          '"body_bytes_sent": "$body_bytes_sent", '  
                                          '"http_referer": "$http_referer", '  
                                          '"http_user_agent": "$http_user_agent" }';  
    access_log /var/log/nginx/access.json json_format;  
}  

This configuration creates a JSON log file (/var/log/nginx/access.json) that can be easily integrated with log analyzers like Filebeat, Logstash, or Kibana for further visualization and analysis.

3.4. Conditional Logging

In some cases, administrators may not want to log every request—especially when dealing with high traffic volumes. NGINX allows conditional logging, which can be configured using an if statement along with variables to include or exclude certain requests. For example, you could avoid logging requests for static assets:

server {  
    listen 80;  
    server_name example.com www.example.com;  
    set $loggable 1;  # Default to logging every request  

    if ($request_uri ~* "\.(jpg|jpeg|png|gif|ico|css|js)$") {  
        set $loggable 0;  # Do not log static file requests  
    }  

    access_log /var/log/nginx/access.log combined if=$loggable;  
    root /var/www/example.com;  
}  

This logic ensures that dynamic, more relevant requests are logged, reducing log clutter and saving disk I/O resources.

4. Configuring NGINX Error Logs

Error logs capture significant issues in NGINX such as configuration errors, file access problems, and runtime warnings. They are essential for troubleshooting and maintaining server health.

4.1. Basic Error Log Setup

The error log is typically enabled by default, and errors are recorded in a file (e.g., /var/log/nginx/error.log). Setting up the error log involves using the error_log directive:

http {  
    error_log /var/log/nginx/error.log error;  
    ...  
}  

In this snippet, the log file is specified as /var/log/nginx/error.log, and it is set to capture messages at the error level and above.

4.2. Adjusting the Severity Level

NGINX recognizes several severity levels that determine which error messages are logged. The severity can be set by providing an appropriate value to the error_log directive. Common severity levels include:

• debug – Detailed debugging information (useful during development)
• info – Informational messages
• notice – Normal but significant events
• warn – Warnings about unexpected occurrences
• error – Errors that occurred during processing
• crit – Critical conditions that require immediate attention
• alert – Conditions that demand immediate action
• emerg – Emergencies that render the system unusable

For example, to log all warnings and above:

error_log /var/log/nginx/error.log warn;  

This configuration instructs NGINX to record all events at the warn, error, crit, alert, and emerg levels.

4.3. Overriding Error Log Settings by Context

Error logging in NGINX is hierarchical. Settings defined in the http block are inherited by all server and location blocks unless overridden. This allows for centralized error logging for a group of virtual hosts while still permitting custom configurations on a per-server basis. For example:

http {  
    error_log /var/log/nginx/error.log warn;  

    server {  
        listen 80;  
        server_name www.example.com;  
        # Override error log to log all errors for this specific server  
        error_log /var/log/nginx/example_error.log error;  
        location / {  
            # Inherit error log settings from server block  
            root /var/www/example;  
        }  
    }  
}  

In this case, the global error log captures warnings, but for the virtual host for www.example.com, only errors (and higher severity levels) are logged to a separate file, facilitating more focused troubleshooting.

4.4. Sending Error Logs to Syslog

In distributed systems or in environments where centralized logging is required, you can configure NGINX to send error logs to a syslog server. This is done by specifying the syslog server details in the error_log directive:

error_log syslog:server=192.168.100.1:514 warn;  

This setup sends warnings and above to a remote syslog server, where logs can be aggregated and analyzed in a centralized dashboard.

5. Understanding Log Levels in NGINX

NGINX log levels provide a clear framework for categorizing the severity of messages recorded in the error logs. The following table summarizes each level, its description, and when it might be used:

This table not only clarifies each level’s function but also helps administrators decide on the appropriate error logging configuration based on the criticality of various events.

6. Customizing Log Locations and Virtual Host Logging

6.1. Changing Default Log File Paths

By default, NGINX logs are stored in /var/log/nginx/ on most Linux distributions. However, administrators may choose to redirect log outputs to different locations based on organizational policies or storage considerations. For instance:

http {  
    access_log /custom/logs/access.log combined;  
    error_log /custom/logs/error.log warn;  
}  

Customizing the path can improve security (by restricting access to sensitive log files) and help in centralized log management.

6.2. Isolating Logs for Different Virtual Hosts

When multiple virtual hosts are served by a single NGINX instance, consolidating all logs in one file may complicate analysis. Instead, you can define separate log files for each virtual host. For example:

server {  
    listen 80;  
    server_name site1.example.com;  
    access_log /var/log/nginx/site1_access.log combined;  
    error_log /var/log/nginx/site1_error.log warn;  
    root /var/www/site1;  
}  

server {  
    listen 80;  
    server_name site2.example.com;  
    access_log /var/log/nginx/site2_access.log combined;  
    error_log /var/log/nginx/site2_error.log warn;  
    root /var/www/site2;  
}  

This separation facilitates quicker troubleshooting and makes it easier to perform traffic and issue analyses on a per-host basis.

7. Advanced Logging Techniques: JSON, Conditional Logging, and Third-Party Integrations

7.1. JSON Log Formatting for Enhanced Analysis

As mentioned earlier, producing log files in JSON format is exceedingly useful when integrating logs with advanced monitoring solutions. JSON logs simplify parsing and indexing in systems like Elastic Stack, Grafana, or Sematext. The following configuration is a practical example:

http {  
    log_format json_format escape=json '{ "time": "$time_local", '  
                                          '"remote_addr": "$remote_addr", '  
                                          '"request": "$request", '  
                                          '"status": "$status", '  
                                          '"bytes_sent": "$body_bytes_sent", '  
                                          '"referrer": "$http_referer", '  
                                          '"user_agent": "$http_user_agent" }';  
    access_log /var/log/nginx/access.json json_format;  
}  

This setup not only provides structured logs but also facilitates downstream processing by log collection agents such as Filebeat.

7.2. Implementing Conditional Logging

Conditional logging, as discussed earlier, allows customization of the logged data based on criteria such as request type, request URI, or client IP. This prevents overlogging and focuses on the important transactions. Consider a scenario where you do not need logs for static assets:

server {  
    listen 80;  
    server_name example.com;  
    set $loggable 1;  
    
    if ($request_uri ~* "\.(css|js|png|jpg|jpeg|gif)$") {  
        set $loggable 0;  
    }  
    
    access_log /var/log/nginx/access.log combined if=$loggable;  
    root /var/www/example.com;  
}  

This conditional setup illustrates how only dynamic or non-static requests are logged, which is especially beneficial for high-traffic systems.

7.3. Integration with Log Analysis Tools

Integrating NGINX logging with third-party log analysis tools like Elastic Stack, Grafana, or Sematext greatly enhances the ability to extract actionable insights. By streaming JSON formatted logs into these systems, administrators benefit from customizable dashboards, real-time alerts, and powerful search capabilities. Furthermore, some advanced setups involve using dedicated ingest pipelines (configured in Elasticsearch via Filebeat) that parse and transform log data on the fly. This integration is a crucial element in modern log management strategies, especially in cloud-based or distributed environments.

8. Best Practices and Security Considerations

Proper log management is essential not only for troubleshooting but also for maintaining the security and efficiency of your server environment. Below are some best practices:

• Regular Monitoring and Analysis:
  Consistently review both access and error logs to detect patterns that could indicate anomalies such as repeated failed attempts, unusual response times, or excessive requests from the same IP address.

• Log Rotation and Archival:
  Implement log rotation to prevent log files from growing indefinitely. Tools such as logrotate can be configured to archive older logs and compress them for future reference.

• Securing Log Files:
  Ensure that log files are stored in directories with appropriate file system permissions. Sensitive data contained in logs (like IP addresses or query parameters) should be protected from unauthorized access.

• Retention Policies:
  Define clear retention policies for logs, balanced between regulatory requirements and storage constraints. Older logs might be archived or purged to manage disk space effectively.

• Using Centralized Logging Solutions:
  For distributed systems, aggregate logs on a centralized logging server to streamline the analysis and correlation of events across multiple NGINX instances.

• Conditional and Selective Logging:
  Avoid overlogging by using conditional logging techniques. This not only reduces unnecessary disk I/O but also makes the log entries more meaningful and easier to analyze.

• Monitoring Log Integrity:
  Regularly audit your log management procedures to ensure that logs have not been tampered with and that the logging configuration is aligned with the operational requirements.

9. Visualizing the NGINX Logging Workflow

Below is a Mermaid flowchart diagram that illustrates the overall workflow of NGINX logging for both access and error events. This diagram provides a clear visualization of how a client request is processed and subsequently logged:

flowchart TD  
    A["Client Request Sent"] --> B["NGINX Receives Request"]  
    B --> C{"Request Successful?"}  
    C -- Yes --> D["Access Log: Record Request Details"]  
    C -- No  --> E["Error Log: Record Error Details"]  
    D --> F["Response Sent to Client"]  
    E --> F  
    F --> G["Administrator Reviews Logs"]  
    G --> H["Troubleshoot or Optimize"]  
    H --> I[END]

Figure 1: Flowchart of NGINX Logging Workflow, illustrating how access and error log entries are generated during request processing.

10. Differences Between Access and Error Logs

Understanding the distinct purposes of access and error logs is vital for efficient server monitoring. The table below summarizes the primary differences between the two types of logs:

Table 1: Key Differences Between NGINX Access Logs and Error Logs.

11. Conclusion

In summary, configuring NGINX logs effectively is crucial for maintaining a healthy and optimized web server environment. The key points covered in this article include:

• Access Logging:

  • Enables recording of every HTTP request processed by the server
  • Can be customized using the log_format directive, including JSON formatting for easier integration with log analysis platforms

• Error Logging:

  • Captures configuration errors, runtime warnings, and critical failures
  • Severity levels can be adjusted (debug, info, notice, warn, error, crit, alert, emerg) to suit operational needs and troubleshooting requirements

• Custom Log Locations:

  • Log file paths can be modified, and separate logs can be set for different virtual hosts to streamline analysis

• Advanced Techniques:

  • Conditional logging minimizes unnecessary log entries
  • Integration with modern log management tools such as Elastic Stack, Grafana, or Sematext enhances the overall capability for monitoring and alerting

• Best Practices:

  • Regular log review, secured storage, log rotation, and retention policies are essential for both performance and security

Administrators who adopt these practices are better equipped to maintain server reliability, optimize performance, and troubleshoot issues in a timely manner.

Main Findings:

• NGINX Access Logs: Capture detailed user requests using customizable formats, essential for traffic and performance analysis.
• NGINX Error Logs: Record critical errors and warnings, with configurable severity levels for targeted troubleshooting.
• Customization Options: Include change of file paths, conditional logging, and advanced JSON formatting for integration with modern monitoring systems.
• Best Practices: Emphasize regular monitoring, secure log storage, and using centralized logging in distributed environments.

By following these guidelines and configurations, server administrators can significantly enhance their operational oversight, leading to rapid issue resolution and a more robust, secure web server deployment.