Abstract Claims Description Full Text

Abstract

One embodiment of the present invention provides a system for making predictions about data records from an incoming stream of data records. This system operates by discovering predictive relationships in an online manner between fields in records in the incoming stream of data records as the incoming stream of data records is received. These predictive relationships can used to predict values in fields based on other field values in the same record. This facilitates cleansing of data by allowing transaction values to be validated based upon predictions made from other field values in the same transaction record. It also allows missing field values to be predicted based upon the other field values. A variation of this embodiment provides for filtering transaction records based upon discovered predictive relationships and routing the transaction records to other servers in real-time. Another embodiment forms association rules between fields in records in the incoming stream of records, and outputs these association rules for viewing by a human decision-maker. In another embodiment, the present invention comprises a server with an online predictive memory that can be incorporated into a heterogeneous network as a server. This embodiment is scalable and can be incorporated into an existing network with minimal integration effort. Note that the underlying model for this system continuously adapts to changes in the incoming stream of records over time without the need for any human intervention.

Other References

• Avi Silberschatz, et al., "A Belief-Driven Discovery Framework Based on Data Monitoring and Triggering," Dec. 1996
• Ronald J. Brachman, et al., "The Process of Knowledge Discovery in Databases: A Human-Centered Approach," Unpublished White Paper, 1997
• Kamal Ali, et al., "Partial Classification Using Association Rules," Proceedings of the 3^rd Int'l Conf. On Knowledge Discovery in Databses, Aug. 1997
• Heikki Mannila, "Efficient Algorithms for Discovery of Association Rules," Proc. AAAI Workshop on Knowledge Discovery in Databases, Jul. 1994
• Rakesh Agrawal, et al., "Mining Association Rules Between Sets of Items in Large Databases," Proc. of the 1993 SIGMOD Conf., May 1993
• Arakesh Agrawal, et al., "Fast Algorithms for Mining Association Rules,"Proceedings of the 20^th VLDB Conf., 1994
• Tom Mitchell, et al., "Experience with a Learning Personal Assistant," Communications of the ACM, Jul. 1994
• Avrim Blum, et al., "On-Line Algorithms in Machine Learning," Proc. Dagstuhl Workshop on On-Line Algorithms, Jun. 1996
• Mika Klemettinen, et al., "Finding Interesting Rules from Large Sets of Discovered Association Rules," Third Int'l Conf. On Information and Knowledge Management, Nov. 29-Dec. 2, 1994
• Nick Littlestone, et al., Comparing Several Linear Threshold Learning Algorithms on Tasks Involving Superfluous Attributes, Proc. 12^th Int'l Conf. On Machine Learning, Jul.1995
• Ido Dagan, et al., "Mistake Driven Learning in Text Categorization," Draft Report, Jan. 1997
• Andrew Golding, et al., "Applying Winnow to Context Sensitive Spelling Correction," Machine Learning: Proc. of the 13^th Int'l Conf., 1996
• Yoav Freund, "Sifting Informative Examples From a Random Source," Proc. Workshop on Relevance, AAAI-94 Fall Symposium Series, 1994
• Yezdi Lashkari, "Collaborative Interface Agents," Proc. of AIII '94 Conf., Aug. 1994
• Avrim Blum, "Empirical Support for Winnow and Weighted-Majority Algorithms: Results on a Calendar Scheduling Domain," Machine Learning, Proc. 12^th Int'l Conf., 1995
• Jeffrey C. Schlimmer, et al., "Software Agents: Completing Patterns and Constructing User Interfaces," Journal of Artificial Intelligence Research I (1993)61-89
• Rakesh Agrawal, et al., "Active Data Mining," Proc. 1^st Int'l Conf. on Knowledge Discovery in Databases and Data Mining, 1995
• Henry Lieberman, "Attaching Interface Agent Software to Applications," Unpublished Draft, 1993
• Nicholas Littlestone, "Mistake Bounds on Logarithmic Linear-Threshold Learning Algorithms," Ph.D. Thesis, University of California at Santa Cruz, 1989
• Rakesh Agarwal, et al., "The Quest Data Mining System," Proc. of the 2^nd Int'l Conf. On Knowledge Discovery in Databases, Aug. 199