Deviation Detection

Conolly and Begg (2014), stated that there are four operations of data mining: predictive modeling, database segmentation, link analysis, and deviation detection.  Fayyad et al. (1996), classifies data mining operations by their outcomes: prediction and descriptive.

Anomaly Detection (Deviation Detection) – identifies significant changes in the data. E.g.: Statistics (outliers)

Practical Example (Advanced)

Deviation Analysis and Detection

Deviation analysis can reveal surprising facts hidden inside data. Tools can be used to detect deviations, anomalies, and outliers. Detection is needed for various reasons;

• -knowledge discovery: often such information is vital part of important business decisions and scientific discovery.
• -auditing: examining such information can reveal problems and mal-practices.
• -fraud detection: fraudulent claims often carry inconsistent information. Such information can reveal fraud cases.
• -data cleaning: such information can be from mistakes in data entry which should be corrected.

Deviation Detection in Time-Series Trend Data

Rule-based automation can be used to detect deviant trends automatically.

Predictive Modeling

Predictive Modeling, such as decision tree, rule induction and neural network, can be used to detect deviations. To detect anomalies in categorical fields, all three tools can be used. 

Hotspot Analysis

Hotspot Analysis can detect outliers. More specifically, this will detect patterns of outliers, defined in terms of profile conditions. Outliers can have extremely high or low averages, probabilities, etc. With CMSR Data Miner, you can perform as follows;

• -Search hotspot profiles.
• -Query database using the hotspot profiles and examine the result rows.

Clustering

Clustering objects based on similarity and analyzing clusters may reveal outliers. With CMSR Data Miner, you can perform as follows;

• -Cluster objects based on similarity.
• -Examine clusters using cluster visualization tools.