Managing and Automatically Deleting Indices in Graylog

Managing and Automatically Deleting Indices in Graylog

Graylog is a powerful tool for log management and analysis. However, as logs accumulate over time, they can consume significant disk space and affect system performance. To address this issue, Graylog provides a feature to automatically delete old indices using a Retention Policy. This article explains how to configure Graylog to automatically delete indices and how to manually trigger the deletion process when necessary.

1. Configuring Automatic Index Deletion with Retention Policy

Graylog allows you to define policies for automatically removing indices that exceed a certain age or disk usage threshold. Follow these steps to configure this feature:

Step 1: Set Up Index Sets

1. Log in to the Graylog web interface (http://<server-address>:9000).
2. Navigate to System > Indices from the main menu.
3. Select an existing index set or create a new one.

Step 2: Define the Retention Policy

In the Retention Strategy section of the index set settings, you can choose one of the following strategies:

• Delete: Automatically removes indices that exceed a specified maximum number or age.
• Disk-Based Retention: Deletes older indices when disk usage exceeds a defined threshold.

Step 3: Configure Parameters

• Maximum number of indices: Set the maximum number of indices to retain. Older indices will be deleted once this limit is reached.
• Index Time-to-Live (TTL): Define how long indices are retained (e.g., 30 days).

Step 4: Save Changes

Once configured, save the changes. Graylog will now automatically clean up old indices according to the retention policy.

2. Triggering Manual Index Deletion

If you need to manually delete indices outside of the retention schedule, you can use the following methods:

Manual Deletion via Graylog Interface

1. In the Graylog web interface, go to System > Indices.
2. Select the index set and manually delete old indices from the list.

Manual Deletion via Elasticsearch REST API

You can also use the Elasticsearch REST API to delete indices:

1. Retrieve the list of indices:

   curl -X GET "http://<elasticsearch-ip>:9200/_cat/indices?v"

2. Delete a specific index:

   curl -X DELETE "http://<elasticsearch-ip>:9200/<index_name>"

Example:

   curl -X DELETE "http://192.168.0.104:9200/graylog_7"

Removing Index Locks

If indices are locked, you must first remove the lock:

curl -X PUT "http://<elasticsearch-ip>:9200/_all/_settings" -H 'Content-Type: application/json' -d '{"index.blocks.read_only_allow_delete": null}'

After unlocking, you can delete the indices using the above commands.

3. Managing Disk Usage with Elasticsearch Settings

Elasticsearch automatically locks indices if disk usage exceeds its thresholds. To prevent this, adjust the disk usage settings in Elasticsearch:

• Edit the /etc/elasticsearch/elasticsearch.yml file and add the following lines:

  cluster.routing.allocation.disk.watermark.low: "85%"   # Warning level
  cluster.routing.allocation.disk.watermark.high: "90%"  # Indices are locked
  cluster.routing.allocation.disk.watermark.flood_stage: "95%"  # Deletion starts

• Restart the Elasticsearch service:

  sudo systemctl restart elasticsearch

4. Automating Deletion with Cronjob

To automate the deletion process, you can set up a cronjob. For example, to delete old indices every night at 3:00 AM:

1. Open the crontab editor:

   crontab -e

2. Add the following command:

   0 3 * * * curl -X DELETE "http://192.168.0.104:9200/graylog_7"

Conclusion

Managing indices in Graylog is essential to efficiently utilize disk space and maintain system performance. By following the steps outlined in this article, you can automate log management and optimize your system resources. If you have any questions or need further assistance, feel free to reach out through the comments section!