Monitoring Database Messages and Alerts in MC

Management Console periodically checks system health and performance. When an important event occurs or system performance fluctuates beyond user- or system-defined limits, the Management Console generates a message to alert you about the state of your system. View and manage alerts in the Message Center.

Message Alert Notifications

Management Console uses multiple methods to communicate alert notifications to ensure that you are immediately aware of any changes to your database that might require attention. You receive message notifications by email, and you can view notifications using the following components:

• Message envelope icon: This icon is located by the MC Help icon, in the top-right of any database-specific page. Select this icon display the Message Center quick view, and perform archive, read, and delete message actions. For details about message actions and alerts, see Message Center.
• Unread Messages (This Week) widget: On the database Overview tab, this widget is located in the quick stats sidebar. It displays unread, high-priority messages. Select the number (including 0) in the widget to go to the Message Center.

Pre-configured Alerts

Management Console provides pre-configured alerts to provide system monitoring capabilities without manual setup. Each alert has a pre-configured threshold that defines the acceptable performance limit, and MC sends a message notification when the database exceeds that threshold.

By default, pre-configured alerts are not active and require minimal initial setup. For details on how to set pre-configured alert properties, see Alert Configuration.

Node Health

Vertica provides the following pre-configured alerts to monitor node health:

• Node CPU
• Node Memory
• Node Disk Usage
• Node Disk I/O
• Node CPU I/O Wait
• Node Reboot Rate
• Node State Change
• Node Catalog Memory

Network Health

Vertica provides the Network I/O Error pre-configured alert to monitor network health.

Query

Vertica provides the following pre-configured alerts to monitor queries:

• Queued Query
• Number Failed
• Query Number
• Spilled Query Number
• Retried Query Number
• Query Running Time

License Status

Vertica provides the License Usage pre-configured alert to monitor the status of your Vertica license.

Resource Pool

MC can send alerts when an individual resource pool reaches a specified state or usage level. For details about resource pool configuration parameters, see Built-In Resource Pools Configuration.

You can configure the MC to send the following resource pool alerts:

• Queries Reaching the Max Allowed Execution Time: Triggers an alert when the specified number of queries reach the RUNTIMECAP execution threshold for the resource pool. You cannot set this alert if the resource pool does not have a RUNTIMECAP threshold set, or if the resource pool has a secondary resource pool.
• Queries With Resource Rejections: Triggers an alert when the specified number of queries exceed a specified number of resource rejections within a set period of time.

• Minimum Starting Resource Value: Triggers an alert when the resource pool reaches the minimum amount of resources allocated for the MEMORYSIZE value.

  By default, you cannot set MEMORYSIZE for the GENERAL pool. The GENERAL pool must have at least 1GB of memory and it cannot be smaller than 25% of the entire system memory.

• Maximum Allowed Resource Value: Triggers an alert when the resource pool reaches the MAXMEMORYSIZE value.
• Ended Query With Queue Time Exceeding Limit: Triggers an alert when the specified number of completed queries were queued for a specified length of time within a timeframe.
• Ended Query With Run Time Exceeding Limit: Triggers an alert when the specified number of completed queries ran for a specified length of time within a timeframe.

Custom Alerts

Create custom alerts to measure system performance metrics that are not monitored by the pre-configured alerts. Create a dynamic SQL query that triggers an alert when it returns any results. You can configure how often an alert is generated, the alert priority, and who receives email alerts. For example, you can create custom alerts that monitor the following:

• Failed logins within a configurable time period
• Idle Vertica user sessions using a configurable time limit
• Database node is DOWN

For details about creating and managing custom alerts, including a tutorial on how to create a custom alert that monitors failed logins, see Creating a Custom Alert.

Default Notifications

Management Console generates the following messages about the database that appear only in the Message Center:

• Low disk space
• Read-only file system
• Loss of K-safety
• Current fault tolerance at critical level
• Too many ROS containers
• Change in node state
• Recovery error
• Recovery failure
• Recovery lock error
• Recovery projection retrieval error
• Refresh error
• Refresh lock error
• Workload Analyzer operations
• Tuple Mover error
• Timer service task error
• Last Good Epoch (LGE) lag
• License size compliance
• License term compliance

Disk Space Check and Cleanup

When the Management Console checks alerts, it generates a result set and saves it to disk. If you use aggressive alert configurations, the result set might use a large amount of disk space. By default, Vertica reserves 500MB of disk space to save result sets.

Vertica checks the free disk space 2 times each day and cleans alerts that are older than 7 days. If the available disk space is low, custom alerts are disabled. Notifications and emails are generated when an alert is disabled due to insufficient disk space.

The /opt/console/vconsole/config/console.properties file contains these settings. Edit the following values to configure the how the MC manages your disk space:

Property	Description
console.diskspace.threshold	The amount of disk space Vertica reserves to save result sets. Default: 500MB
customthreshold.alerts.toKeepInDays	The number of days that alerts are retained on disk. Default: 7