| Title | General-purpose RLADP: Solving the Scaling Problem |
| Speaker | Dr. Paul Werbos |
| Chair | Dr. Marco Wiering |
Abstract
Years ago, RLADP already reported breakthrough capabilities of great practical importance in applications such as carbon-carbon parts production (crucial to the Dreamliner), missile interception, aircraft control, low-pollution engine control and turbogenerator control, demonstrating effective control of highly nonlinear plants with 5-20 state variables. But the further advance of RLADP has been held back by the problem of “value function approximation” for systems like electric power grids, large-scale logistics systems and vertebrate brains, where one must cope with thousands or millions of variables.
Over the past two years, “deep learning” groups have defied conventional wisdom by actually implementing advanced neural network designs to handle such complexity, and thereby broken world records for image processing, speech recognition, natural language processing and other widely competed benchmarks. This talk discusses how these and other neural network concepts can solve the scaling problem in RLADP. Connections to operations research and some new stability work will also be discussed as time permits.
Biography
Dr. Paul Werbos has responsibility for the following Core ECCS areas: Adaptive and Intelligent Systems (AIS), Quantum systems and device modeling (QMHP), and systems-level power grids (GRID). Use of these acronyms in a proposal title would encourage us to handle them in the panels for those areas. Also handles the EFRI 2008 topic in Cognitive Optimization and Prediction (engineering-neuroscience collaboration to reverse engineer intelligence)
Dr. Werbos has led a variety of other areas, such as fuel cell and electric vehicles, emerging technologies, cyber systems and the sustainability part of IDR since he started at NSF in 1988. He is a Fellow of IEEE and INNS, a winner of the IEEE Neural Networks Pioneer Award and winner of the Hebb Award for 2011 from the International Neural Network Society (INNS). The Hebb Award is INNS’s highest award, to honor substantive contributions to the understanding of biological learning systems. The biography going with this award reads:
Paul Werbos began training as a mathematician, taking many university courses culminating in the graduate course in logic from Alonzo Church at Princeton while in middle and high school. Realizing the limits of deductive logic, he began his quest to understand inductive logic and intelligence in the mind back in those days, inspired by the work of John Von Neumann, Donald Hebb and early AI (Feigenbaum and Feldman). He obtained two degrees in economics from Harvard and the London School of Economics, divided equally between using mathematical economics as a model for distributed intelligence and developing some broader understanding. For his Harvard M.S., he took courses in quantum field theory (QFT) from Julian Schwinger, but did not fully understand the subject until many years later, after he started an activity in quantum technology and modeling at NSF (see his papers at http://arxiv.org/.) For his 1974 Harvard PhD thesis (reprinted in The Roots of Backpropagation, Wiley 1994), he proposed the development of more powerful, more biologically plausible reinforcement learning systems by the then new idea of using neural networks to approximate dynamic programming (ADP), including the value function. In order to implement ADP in a local biologically plausible manner, he translated Freud’s theory of “psychic energy” into an algorithm later called backpropagation, and a rigorous general theorem, the chain law for ordered derivatives, which later also became known as the reverse method or adjoint method for automatic or circuit-level differentiation. He has spent many years advancing the fields of ADP and backpropagation and brain-like prediction, aimed at developing and demonstrating the kind of designs which could actually explain the kind of general intelligence we see in the brain and in subjective human experience – collaborating at times with Karl Pribram and Walter Freeman and Pellionisz among others, and proposing biological experiments to test the theory. In looking for applications which are really important to areas like energy, sustainability and space, he has also gotten deep into domain issues and organization, as reflected at http://www.werbos.com/, serving on boards of the National Space Society, the Millennium Project, the Lifeboat Foundation, and the IEEE Energy Policy Committee, and as a Fellow in the Senate in 2009. From 1980-1989, he developed official econometric forecasting models (two based on backpropagation) and was lead analyst for the long-term future at EIA in the Department of Energy.
