Patent 5802506 Issued on September 1, 1998. Estimated Expiration Date: September 1, 2015. Estimated Expiration Date is calculated based on simple USPTO term provisions. It does not account for terminal disclaimers, term adjustments, failure to pay maintenance fees, or other factors which might affect the term of a patent.
The invention is an autonomous adaptive agent which can learn verbal as well as nonverbal behavior. The primary object of the system is to optimize a primary value function over time through continuously learning how to behave in an environment (which may be physical or electronic). Inputs may include verbal advice or information from sources of varying reliability as well as direct or preprocessed environmental inputs. Desired agent behavior may include motor actions and verbal behavior which may constitute a system output and which may also function "internally" to guide external actions. A further aspect of the invention is an efficient "training" process by which the agent can be taught to utilize verbal advice and information along with environmental inputs.
Other References
A.G. Barto et al., "Neuronlike Adaptive Elements Tant Can Solve Difficult Learning Control Problems," IEEE Trans. on Systems, Man, and Cybernetics, vol. SMC-13 (5), pp. 834-846, Sep. 1983
W.R. Hutchison and K.R. Stephens, "Integration of Distributed and Symbolic Knowledge Representation," IEEE First Int'l. Conf. on Neural Networks, pp. II-395 to II-398, Jun. 1987
P.J. Werbos, "Backpropagation and Neurocontrol: A Review and Prospectus," IEEE Int'l. Conf. on Neural Networks, pp. I-209 to I-216, Jun. 1989
J. Schmidhuber, "An On-line Algorithm for Dynamic Reinforcement Learning and Planning in Reactive Environments," IEEE Int'l. Conf. on Neural Networks, pp. II-253 to II-258, Jun. 1990
K.R. Stephens et al., "Dynamic resource allocation using adaptive networks,"Neurocomputing, vol. 2(1), pp. 9-16, Jun. 1990
P.J. Werbos, "Consistency of HDP Applied to a Simple Reinforcement Learning Problem," Neural Networks, vol. 3(2), pp. 179-189, Dec. 1990
J.C. Hoskins and D.M. Himmelblau, "Process Control via Artificial Neural Networks and Reinforcement Learning," Computers and Chemical Engineering, vol. 16(4), pp. 241-251, Dec. 1992
B.L. Digney and M.M. Gupta, "A Distributed Adaptive Control System for a Quadruped Mobile Robot," Int'l. Conf. on Neural Networks, pp. 144-149, Mar. 1993
H.S. Del Nero and J.R.C. Piqueira, "Cognitive Science and the Failure of Behaviorism: Quantum, classical and mind indeterminacies," Intl'l. Conf. on Systems, Man, and Cybernetics, vol. 4, pp. 638-643, Oct. 1993
K. Otwell et al., "A Large-Scale Neural Network Application for Airline Seat Allocation," World Congress on Neural Networks, vol. 1, pp. I-145 to I-150, Jun. 1994
R. Maclin and J.W. Shavlik, "Incorporating Advice into Agents that Learn from Reinforcements," Proc. 12th National Conf. on Artificial Intelligence, vol. 1, pp. 694-699, Dec. 1994
D. Gachet et al., "Learning Emergent Tasks for an Autonomous Mobile Robot," Int'l. Conf. on Intelligent Robots and Systems, vol. 1, pp. 290-297, Sep. 1994
A. Newell & H.A. Simon, "GPS, A Program that Simulates Human Thought," reprinted in Computers and Thought, AAAI Press, pp. 279-293, Dec. 1995
Barto, Reinforcement Learning and Adaptive Critic Methods; D.A. White & D. Sofge (Eds.) Handbook of Intelligent Control; Van Nostrand, 1992, pp. 469-491
Brooks, Elephants Don't Play Chess; Elsevier Science Publishers B.V. (North-Holland), 1990, pp. 3-15
Caudill, Expert Networks; Byte, Oct. 1991, pp. 108-116
Cecconi & Parisi, Neural Networks With Motivational Units; Institute of Psychology, National Research Council, pp. 346-355, 1993
Donahoe & Palmer, Learning and Complex Behavior; 1994, pp. (ii-ix) 270-323
Epstein, Generativity Theory and Education; Educational Technology, Oct. 1993, pp. 40-45
Fahner and Eckmiller, Structural Adaptation of Parsimonious High-Order Neural Classifiers; Neural Networks, vol. 7, No. 2, 1994, pp. 279-289
Hinton and Becker, Using Coherence Assumptions to Discover the Underlying Causes of the Sensory Input; Connectionism Theory and Practice (Davis, Ed.), 1992, pp. 3-21
Lin, L., "Programming Robots Using Reinforcement Learning and Teaching," Proc. Ninth National Conf. on Artificial Intelligence, 1991, pp. 781-786
Redding, Kowalczyk, and Downs, Constructive Higher-Order Newwork Algorithm That Is Polynomial Time; 1993, Neural Networks, vol. 6, pp. 997-1010
St. John, Learning Language in the Service of a Task; Department of Cognitive Science, U. of C, San Diego, pp. 271-276, 1992
Skinner, Verbal Behavior; Prentice-Hall, Inc., pp. ix, 1-453, 1957
Sutton, Temporal Credit Assignment in Reinforcement Learning (Delayed Reinforcement); 1984, pp. 92-98
Whitehead, A Complexity Analysis of Cooperative Mechanisms in Reinforcement Learning, Learning and Evaluation Functions, pp. 607-613, Proc. 9th Conf on A.I., 199
September 1, 2015. Estimated Expiration Date is calculated based on simple USPTO term provisions. It does not account for terminal disclaimers, term adjustments, failure to pay maintenance fees, or other factors which might affect the term of a patent.
