Event Deduplication / Aggregation
Event Deduplication (also known as Event Aggregation) is a technique for detecting that a new incoming alert (event):
- is similar to an existing event that has already been reported
- suppressing the duplicate alert
This is somewhat related to Alert Suppression
and Event Escalation, but allows finer and configurable control over when to suppress an alert.
Event Deduplication can be configured in the Advanced Services part of the Console.
Here you can select between the default Simple event deduplication (which doesn't do very much deduplication), or advanced event deduplication. Advanced Event Deduplication is what this page will describe further.
The first thing to decide is whether or not to alert when a duplicate event arrives. Usually the new event would be shown if the previous event situation has been 'reset', meaning the system will no longer consider new events duplicates of the previous event because something has changed. Usually this means the previous event was acknowledged, or the
underlying error was fixed. If this is the case, and a new event arrives, it should not be considered a duplication but rather a new situation that should
be alerted on again.
State vs Event-type events
Some events are 'state' events, meaning they are in a good or bad state (responding to ping or not responding to ping, low disk space or OK disk space, etc). Those are easy to define as 'Fixed' or not.
Other event types are stateless, or Event-type events, meaning they happened, but don't represent a good or bad state. An error listed in the event log, an error received via SNMP Trap or syslog, or a change detected in a file are such situations. These are not good or bad states, they simply occurred.
For Event-type events, you decide how to consider them 'fixed' for use in Deduplication resetting. You can consider them immediately fixed, or fixed
after a certain amount of time.
The key principle to understand is the Deduplication ID. The Deduplication ID is a text string that represents the essence of the event. If two events have the same Deduplication ID, they are considered the same event for Event Deduplication purposes. You will therefore want to define the Deduplication ID so it combines events that you want considered the same.
For example, the default fields used are:
- Computer ID - an internal unique ID assigned to each monitored computer
- Monitor ID - an internal unique ID assigned to each monitor
- Cleaned Description - the event description text with user names, paths, dates, amounts, etc removed
With these default settings, the two events below would be considered identical, and thus the second event would not fire alerts when using Advanced Event Deduplication:
21 Feb 2014 09:05:56 PM
Monitor: [Low Disk Space]
Free disk space on F: is below the threshold of 5% (Currently 4%, 11.6 GB)
21 Feb 2014 11:05:53 PM
Monitor: [Low Disk Space]
Free disk space on F: is below the threshold of 5% (Currently 3%, 8.7 GB)
These are identical because the events are for the same computer and from the same monitor, and the Cleaned Description field (after removing dates, times
amounts, etc) is the same too -- they are about low disk space on the same drive on the same computer.
Peeking at the database...
The image above shows some of the fields in the Error History table. Deduplication IDs are shown at the right side. Notice the ErrCount column -- that shows how many incoming alerts were considered duplicates of the shown alert. Instead of alerting on that incoming event, the ErrCount column was incremented and no alerts were fired. If you think you might want a reminder about events that have not reset and thus are suppressing new incoming alerts, look at the
Alert Reminders feature.
Global with Override
This setting is a global setting that applies to all monitors. Individual monitors can define their Deduplication ID in a unique way to give added
flexibility. This is done in Advanced Monitor Options (bottom of the page).