xi In this book, we look to a more advanced question: What is mind, that we might engineer it?

xiii We argue that high levels of intelligence require a rich dynamic world model that includes both a priori knowledge and information provided by sensors and a sensory processing system.

xiv We believe that a rich internal representation of the world is indispensable to higher levels of intelligent behavior.

p.15 The convergence of results from the neurosciences, artificial intelligence, robotics, computer-integrated manufacturing, information science, and cognitive psychology suggests that at long last we are entering a period where the study of mind can become a hard science. Theories can be formulated in terms of computational processes and tested experimentally in software and hardware.

p.18 Our thesis is that mind will emerge from intelligent systems, and intelligence will emerge from the joint functioning of four fundamental processes: behavior generation, world modeling, sensory processing, and value judgment. We will show how these four processes can work together to process sensory information, to build, maintain, and use an internal representation of the external world to select goals, react to sensory input, execute tasks, and control actions. Sensory processing focuses attention, detects and groups features, computes attributes, compares observations with expectations, recognizes objects and events, and analyzes situations. World modeling constructs and maintains an internal representation of entities, events, relationships, and situations. It generates predictions, expectations, beliefs, and estimates of the probable results of future actions. Value judgment assigns value to objects, events, and situations. It computes the cost, benefit, risk, and expected payoff of future plans. It decides what is important or trivial, what is rewarding or punishing, and what degree of confidence should be assigned to entries in the world model. Behavior generation uses value judgment results to select goals, decompose tasks, generate plans, and control action.

p.68 To cope with ambiguity and noise in the sensory environment, both biological and artificial intelligent systems have developed sensory processing methods that extract the information needed to construct reliable world models despite ambiguity and noise in the sensory input. The most successful of these is hypothesize and test.

p.77 Making wise behavioral decisions requires a rich, high-dimensional, accurate, current, and reliable representation of the world in both symbolic and image domains from which detailed analyses of complex situations can be made.

p.79 The world is infinitely rich with detail... Yet the computational resources available to any intelligent system are finite. No matter how fast and powerful computers become, the amount of computational resources that can be allocated to any practical sensory processing system will be limited.

p.80 At any point in time and space, most of the detail in the environment is irrelevant to the immediate behavioral task of the intelligent system.

p.80 the key to building practical intelligent systems lies in understanding how to focus attention on what is important and ignore what is irrelevant. Attention is a mechanism for allocating sensors and focusing computational resources on particular regions of time and space. Attention allows computational resources to be dedicated to processing input that is most relevant to behavioral goals. Focusing of attention can be accomplished by directing sensors toward regions in space classified as important and by masking, windowing, and filtering out input from sensors or regions in space classified as unimportant.

p.82 In most cases the region in space and time that is most relevant to the behavioral choices of an intelligent system centers around the "here and now." Each intelligent system always resides at the center of its own egosphere, in both space and time. The relevance of objects in the world is typically inversely proportional to their spatial distance... Objects and events that lie far away in space and time can usually be safely ignored.
  But not always. Sometimes distant objects and remote events can be very important. There are occasions where the ability to perceive distant objects and predict events far in the future has significant survival benefit. However, this capability incurs computational costs... Only likely future scenarios should be considered seriously, and only the most important ones should be analyzed in any detail.

p.82 In all cases, knowledge of what is important and the ability to focus attention on what is important are critical to the design of intelligent systems. They are fundamental to a theory of mind. To build practical cost-effective intelligent systems, we must understand how to design sensory processing systems that mask, window, and filter the sensory data to extract information that is relevant to what is important in the world. We must understand how to build value judgment systems that can distinguish between what is important and what is not. And we must understand how to design behavior generation systems that can allocate behavioral resources to accomplish what is important to achieve.

p.90 Once knowledge is acquired and skills are learned, task execution appears deceptively simple. The skilled performer appears to move without significant effort.

p.91 There are many things about intelligent behavior that remain a mystery, but one thing is perfectly clear - the concept of a goal is central. Intelligent creatures do not wander aimlessly or simply react reflexively to their immediate environment. They act proactively to accomplish their goals of survival and propagation in the most efficient manner possible under given environmental conditions. They react to perturbations and unexpected events with alternative tactics and behaviors so as to accomplish their goals despite difficulties and distractions along the way. Intelligent creatures are goal-directed in performing behaviors... Intelligent creatures from insects to humans conserve resources and focus attention on entities and events such as food and danger that are relevant to their goals. The more intelligent creatures are, the more resourceful they are in dealing with difficulties and unexpected events. The most intelligent creatures can predict the future. They can make plans that anticipate or preempt problems, and invoke tactics that accommodate difficulties and compensate for unexpected events.

p.107 A heuristic method is a rule of thumb that often works (but is not guaranteed.)

p.121 In the case of consciousness, the simplest hypothesis is that awareness emerges in any system that has an internal world model with sufficient sophistication to represent the richness and dynamics of the physical world in which it exists. Self-awareness then follows naturally in any system that includes a self-object in its representation of the world.

p.121 Any system that can plan can use its world model (including its self-object) to simulate and predict the results of hypothesized actions. The ability to simulate possible future states of the world is commonly called imagination.

p.125 The principal contribution of behaviorist architectures is that they have demonstrated how complex behaviors can be generated by simple reactive systems operating in a complex environment... Our reason for not choosing a behaviorist architecture is the lack of a rich internal model of the world.

p.196 For mind to emerge in a system, the system must have an internal representation of what it feels and experiences as it perceives entities, events, and situations in the world. It must have an internal model that captures the richness of what it knows and learns, and a mechanism for computing values and priorities that enable it to decide what it wishes to do.

p.198-199 Value judgement (VJ) is a process that returns values for the attributes of goodness, cost, risk, benefit, or worth... Value judgement provides the basis for making behavioral decisions... Value judgement is critical to perception. Value judgement computes what is important and where attention should be focused. Entities evaluated as important become targets of attention... Value judgement is crucial for learning. Value judgement computes what is rewarding and punishing. This enables learning.

p.233 Immediate experience is rich, vivid, and dynamic. Immediate experience of visual imagery is rich in detail

p.239 Imagination is a critical factor in intelligence

p.244 Sensory processing (SP) is the set of computational processes within each RCS [JLJ - Real-Time Control System] node that keeps the world model knowledge database up to date. SP extracts from sensory signals the state variables, images, entities, attributes, events, symbols, maps, ...and other data structures necessary to generate and maintain an internal representation of the world that is useful for generating successful behavior.

p.244 SP focuses attention on entities and events of importance... and masks out sensory data that are irrelevant.

p.245 Attention selects (or windows) the regions of space and the intervals of time over which SP [JLJ - Sensory processing] processes will operate on sensory inputs. The remainder of the input can be masked out or ignored. Windowing allocates the available computational resources to the entities and events that are most important for success in achieving behavioral goals... Windowing is controlled by an attention function that determines what regions and entities in the world are important and therefore worthy of attention.

p.267 A recursive estimation loop requires initialization.

p.290 The apparent ease with which the human mind perceives the world is deceiving. Most of the sensory processing in the brain goes on below the level of consciousness.

p.290 One of the biggest problems is that sensory input is almost always ambiguous.

p.291 For hypothesis and test to be successful, it is necessary to limit the number of hypotheses to be considered.

p.293 It will be many years before artificial systems can approach the capabilities of biological systems in building and maintaining a rich and dynamic internal model of the external world. [JLJ - perhaps it is just a matter of using the right heuristics.]

p.353 When our models are wrong, our predictions fail. When our models are refined, our predictive powers improve.

p.376 The human mind can understand what is possible and predict what can be achieved through purposeful actions. The mind can imagine how the world might be changed, for better or for worse, through our own actions or inaction. The mind can even imagine things that have never existed before.