Incident Report: Cacti Database Connection Failure Post yum upgrade

Incident Summary

After performing a system upgrade using “yum upgrade“, the Cacti monitoring system became inaccessible. The root cause was traced to a symbolic link (config.php) being renamed during the upgrade process, which disrupted the database connection configuration.

Date & Time of Incident

Start Time:
Resolution Time:

System Affected

Application: Cacti (Network Monitoring Tool)
Server OS: CentOS/RHEL-based Linux
Web Server: Apache (httpd)
PHP Module: PHP with MySQL support

Symptoms

Upon accessing the Cacti web interface, the following error message was displayed:

FATAL: Connection to Cacti database failed. Please ensure:

the PHP MySQL module is installed and enabled.
the database is running.
the credentials in config.php are valid.

Root Cause

During the yum upgrade, the symbolic link config.php (which originally pointed to /etc/cacti/db.php) was renamed to config.php.dist. This broke the link to the actual database configuration file, causing Cacti to fail when attempting to connect to its database.

The renamed file appeared as:

lrwxrwxrwx 1 root root 17 Apr 15 01:15 config.php.dist -> /etc/cacti/db.php

This indicated that the upgrade process preserved the original link but renamed it, likely due to package management behavior when detecting local modifications.

Resolution Steps

Verified the Existence of the Original Configuration File:

   # ls -l /etc/cacti/db.php

Confirmed that the file existed and contained valid database credentials.

Navigated to the Cacti Installation Directory:

   # cd /usr/share/cacti/include/

(Note: The actual path may vary, e.g., /usr/share/cacti/include/ )

Recreated the Symbolic Link:

   # ln -sf /etc/cacti/db.php config.php

This restored the expected configuration file reference.

Verified the Link:

   # ls -l config.php

Output confirmed the symbolic link was correctly re-established.

Restarted the Web Server: Depending on the web server in use:

   # systemctl restart httpd   # For Apache (httpd)

Tested the Web Interface: Accessed the Cacti web UI to confirm that the database connection was restored and the application was functioning normally.

Preventive Measures

Configuration Backup: Ensure symbolic links and configuration files are backed up before system upgrades.
Monitoring Upgrade Logs: Review yum logs post-upgrade to identify any renamed or replaced configuration files.
Automation: Consider using configuration management tools (e.g., Ansible, Puppet) to enforce and restore critical symlinks and settings.