Monitoring Database Messages and Alerts in MC

Management Console periodically checks system health and performance. When an important event occurs or system performance fluctuates beyond user- or system-defined limits, the Management Console generates a message to alert you about the state of your system. View and manage alerts in the Message Center.

Message Alert Notifications

Management Console uses multiple methods to communicate alert notifications to ensure that you are immediately aware of any changes to your database that might require attention. You receive message notifications by email, and you can view notifications using the following components:

  • Message envelope icon: This icon is located by the MC Help icon, in the top-right of any database-specific page. Select this icon display the Message Center quick view, and perform archive, read, and delete message actions. For details about message actions and alerts, see Message Center.
  • Unread Messages (This Week) widget: On the database Overview tab, this widget is located in the quick stats sidebar. It displays unread, high-priority messages. Select the number (including 0) in the widget to go to the Message Center.

Pre-configured Alerts

Management Console provides pre-configured alerts to provide system monitoring capabilities without manual setup. Each alert has a pre-configured threshold that defines the acceptable performance limit, and MC sends a message notification when the database exceeds that threshold.

By default, pre-configured alerts are not active and require minimal initial setup. For details on how to set pre-configured alert properties, see Alert Configuration.

The following table lists each pre-configured alert by domain:

Alert Domain Alert Name
Node Health

Node CPU

Node Memory

Node Disk Usage

Node Disk I/O

Node CPU I/O Wait

Node Reboot Rate

Node State Change

Node Catalog Memory

Network Health Network I/O Error
Query

Queued Query

Number Failed

Query Number

Spilled Query Number

Retried Query Number

Query Running Time

License Status License Usage

Resource Pool

Default settings apply to the GENERAL pool only.

Queries Reaching the Max Allowed Execution Time

Queries With Resource Rejections

Ended Query With Queue Time Exceeding Limit

Ended Query With Run Time Exceeding Limit

Custom Alerts

Create custom alerts to measure system performance metrics that are not monitored by the pre-configured alerts. Create a dynamic SQL query that triggers an alert when it returns any results. You can configure how often an alert is generated, the alert priority, and who receives email alerts. For example, you can create custom alerts that monitor the following:

  • Failed logins within a configurable time period
  • Idle Vertica user sessions using a configurable time limit
  • Database node is DOWN

For details about creating and managing custom alerts, including a tutorial on how to create a custom alert that monitors failed logins, see Creating a Custom Alert.

Default Notifications

Management Console generates the following messages about the database that appear only in the Message Center:

  • Low disk space
  • Read-only file system
  • Loss of K-safety
  • Current fault tolerance at critical level
  • Too many ROS containers
  • Change in node state
  • Recovery error
  • Recovery failure
  • Recovery lock error
  • Recovery projection retrieval error
  • Refresh error
  • Refresh lock error
  • Workload Analyzer operations
  • Tuple Mover error
  • Timer service task error
  • Last Good Epoch (LGE) lag
  • License size compliance
  • License term compliance

Disk Space Check and Cleanup

When the Management Console checks alerts, it generates a result set and saves it to disk. If you use aggressive alert configurations, the result set might use a large amount of disk space. By default, Vertica reserves 500MB of disk space to save result sets.

Vertica checks the free disk space 2 times each day and cleans alerts that are older than 7 days. If the available disk space is low, custom alerts are disabled. Notifications and emails are generated when an alert is disabled due to insufficient disk space.

The /opt/console/vconsole/config/console.properties file contains these settings. Edit the following values to configure the how the MC manages your disk space:

Property Description
console.diskspace.threshold

The amount of disk space Vertica reserves to save result sets.

Default: 500MB

customthreshold.alerts.toKeepInDays

The number of days that alerts are retained on disk.

Default: 7