(c) 2008 John L. Jerz
Abstract: The author explains his idea that experts solve problems by first acquiring diagnostic information
(often through custom diagnostic tests), which they then use to guide the search for a solution. A model is constructed
of the situation that is being faced, and the expert manipulates the model to suggest (or predict) the next few steps towards
the solution.
When an expert has a problem to solve, the first thing done (after completing an understanding of
the problem, including the desired objectives) is usually acquiring information about the problem that is diagnostic
in nature.
The expert will use this diagnostic information (often from the results of custom designed
diagnostic tests) to focus his search efforts for a solution. The expert will construct a model in his mind
of the problem, and will mentally manipulate the model, trying several possible routes to a solution and making predictions
of what will happen next. The model he will use will be a simplification of the real world - simple enough to allow thinking,
but complex enough to allow the solution to be accurate. Eventually the expert will predict that a certain sequence of events
will solve the problem or bring him closer to a solution. He will then take the first of those predicted steps.
Think about visiting your family doctor or taking your car to a mechanic. You are approaching an
expert with a problem that you want solved.
For the family doctor, you will describe the problem as best you can. What hurts? Is there anything
visible that can be described, any swelling? When does the pain occur? How severe is the pain? What objective or final result
would be best for you? Perhaps a return to your "usual" state of health, or possibly an improved state of health.
The doctor will then start to gather diagnostic information. Your pulse and blood pressure
will be taken. Perhaps he will read your medical chart and consider your previous history of problems, including previous
treatments and their resolution. Your doctor might listen to your heart and lungs with a stethoscope, which is nothing
more than a focus for his or her hearing. He or she will examine the problem area visually and by touch, to the extent
that he or she can. Perhaps he will take an x-ray or a Magnetic Resonance Imaging (MRI) or CAT scan. Perhaps a complete
blood count is ordered. This diagnostic data is reviewed, and a diagnosis is made. Your doctor will then take the
next incremental step towards solving the problem that is suggested by the diagnostic information. The diagnostic information
reduces uncertainty as to what the problem is and will suggest which alternate course of action has the best chance of solving
your problem. The doctor might then form a treatment plan, which represents the planned sequence of steps that he
foresees as being the most likely path to the objective, which is the healing of whatever problem you have. Medication or
physical therapy might be prescribed, and a follow-up visit is likely. Your doctor will make notes on a chart which he will
use to refresh himself when you come to see him again.
A dentist might begin by looking at a set of x-rays taken during an earlier examination. After
visually inspecting your mouth, he might start probing along your gum line in order to see if there is any decay below
the surface. The probe measurements will tell him possible areas of concern. Yes, I went to the dentist yesterday. Yes, I
watched him gather diagnostic information (a full set of x-rays), examine the information, perform a mental comparison
with healthy looking teeth, and suggest a course of treatment for a few dental problems I have. The x-ray was unclear in a
few areas, but that did not stop the doctor from using it to focus his attention in certain areas. I am usually amazed
when my Dentist asks me to open my mouth, and then immediately goes to the problem areas.
A car mechanic will listen to your description of the problem and will likely take your car on a
test drive to see if he can reproduce the problem you have described. He will listen for unusual sounds or vibrations,
especially at different points in the operation of the vehicle - stopped, turning, accelerating, braking. He will
take into account the make and model of your car, how old it is and whether or not there is evidence that the vehicle has
been properly maintained. If you complain about a squeaky sound from the front of the car when pressing the brake pedal,
the mechanic might immediately suspect a problem with the brakes.
He might form an initial diagnosis, then go to the predicted problem area and focus his attention
on a visual inspection. The diagnostic information will suggest a small area to be investigated - the mechanic will usually
not need to look at every part and every connection in your car. After a brief, superficial inspection under the hood,
the mechanic can reasonably guess that the rest of your car is working ok. The mechanic will likely visualize the parts of
your car in operation - perhaps moving back and forth, or doing the 'thing' they were designed to do. Backed by the diagnostic
information he has obtained, he will decide to stop searching in certain areas that his experience tells him are likely
to be working fine. The mechanic will use his knowledge of what a brake pad should look like, compared to what he sees. If
the problem is difficult to diagnose, he might continue to gather additional diagnostic information, such as performance-related
information from your car's computer. He might ask you additional questions, or consult with a senior mechanic. At some
point he will have enough information to make a more informed diagnosis, and he will take the first of several steps
that are predicted to result in fixing the squeaking sound. Along the way, he might notice that your brakes are misaligned,
and that the problem might happen again if your brake shoes are not straightened. So, part way along the predicted path
to the solution, he might change the predicted steps he takes once new diagnostic information has been acquired. The
diagnosis he makes is based on the available diagnostic information, his knowledge of good and bad brakes, knowledge of the
probabilities of the brakes going bad, and a focused, informed search through the possibilities.
In all cases, the expert listens to a description of the problem, confirms that the problem is real,
then begins to gather diagnostic information in order to determine what to do next, based on the results of diagnostic
tests. Initial steps toward the solution are proposed, and after progressing a few steps down this path (and continuing to
gather diagnostic information), he might change the predicted path to the solution.
Choosing a Move in a Chess Game
A chess expert might not be able to describe in words how he chooses a move in a game, according
to a recent essay by Stuart Rachels [The Reviled Art]
Chess cognition is mostly unconscious. In studying a position, a master may quickly understand that there
are three viable possibilities for the player on move. But how his brain has determined this, he has no idea. And even when
he is deliberating among the viable options, there is typically little inner dialogue. Chess thinking is rarely linguistic.
In the analysis of a chess position, the chess expert will attempt to form a diagnosis, but he
might first acquire information (from the types and location of the pieces) that is diagnostic in nature, much
like the examples of the doctor, dentist and mechanic mentioned above. The chess expert might begin the analysis
effort by counting the number and types of pieces on the board (commonly called the 'material'), figure out which pieces are
safe on which squares, look for immediate tactical opportunities, then look for positional weaknesses.
The expert will continue to identify candidate moves by focusing his attention
on acquiring additional diagnostic information from the position of the pieces on the board. This diagnostic information
will tell the expert what each piece is capable of doing, and whether or not the piece is fully engaged in the game. The information
will tell the expert how 'threatening' or vulnerable each piece is, and how capable each piece is of working together with
the other pieces. The expert might slide each piece forward into the future and see what it is capable of doing. The expert
will note the limiting factors (including enemy pieces or even his own pieces) that block the piece's ability to
accomplish objectives. The expert will likely take note of any strategic factors that will enable him to remove his
opponent's limiting factors, and therefore make progress towards his objectives.
The expert might be reminded at any point in this procedure of a similar position or configuration
of pieces from a previous game, and he or she might use that information in order to plan what to do next. Regardless,
the expert will look into his or her 'bag of tricks' at the ways that he or she can use positional leverage or a tactical
sacrifice to outmaneuver his or her opponent. The basic principles of chess themselves will suggest certain moves. The expert
will perform analysis to see what types of positions are likely to result in the future from the candidate moves being
played. The expert will constantly be looking at the paths that the pieces can take to objectives, and what resources the
opponent has to block those paths. The fight will be over the control of strategic squares, and of the strategic exchange
of control of a group of squares for another group of squares. Chess pieces will come under fire from the opponent's
pieces at certain times, then retreat to safer places where they are only remotely threatened. Pieces might be exchanged,
and the squares that they formerly controlled are now up for grabs.
The expert will focus his efforts on predicting what the next few moves in the game are likely to
be, based on a positional and tactical evaluation of the present position. The expert might decide that a certain sequences
of moves are not likely and he will stop searching in that particular direction. The expert will have a degree of confidence
that he will be able to counter any moves played by his opponent if his adversary chooses to enter the particular
variation that he has stopped analyzing. The expert will likely consider long-range plans which involve sequences
of moves that are difficult to predict exactly, but might have in general a predictable outcome.
After a period of time, the move that he thinks will lead to the best possible future position,
including any uncertainty, will be played, and the selection of this move represents his decision. It is an expert opinion,
based on collected diagnostic information, certain 'cues' present on the board, principles of chess learned from
either books teachers or experience, the similarity to previous positions analyzed, and a reasonably accurate prediction
or even guess of the likely sequence of moves (as well as alternate moves) in the future.
Our computer chess program must also decide on the best move to be played. But it does not have
a memory of previously played positions (other than an opening book). Pattern matching is time consuming, difficult to program,
and might not have results that improve performance or allow us to better focus our search efforts. Our computer program must
rely heavily on diagnostic information present in the position, and in programmed chess knowledge to evaluate each position
based on its winning chances or how promising it appears to be. We must also decide how we are going to focus our
search efforts through an exponentially growing search tree.
We might choose to focus our search efforts by using a fast heuristic with a weak focus.
Our aim will be to perform a thorough search of the positions that matter by examining a large number of positions.
This is how many traditional chess programs operate. This method is good enough to produce programs that have higher
performance ratings than the best human chess players. Since our search focus is weak, we unfortunately will be examining
a large number of positions that do not, in the long run, matter. That is the price you pay for using a weak search focus.
We might however choose to spend additional time acquiring a more accurate search focus
by carefully building up and then harvesting diagnostic information which accurately identifies positions that are
promising, interesting or otherwise worth searching further. This method will work only if 1) the
diagnostic information we gather is accurate in identifying these types of positions and 2) the time we spend
gathering the diagnostic information does not impact the thoroughness of the search efforts. The margin of error for this
heuristic is much smaller. We must be precise because we have fewer positions that we can now examine in our search tree.
This method will let us use diagnostic probing to focus our search, and might let us use techniques outlined
in books on positional chess for evaluating the best course of action.
Imagine a sports team searching for a good running back in the American game of football. The team
could examine 1000 candidates and just look at the times in the 40-yard dash. Or, the team could examine 100 candidates and
look at the times in the 40-yard dash, agility drills, and performance in controlled practice games. It might be that we find
a good running back with an exceptional time in the 40-yard dash. However, the candidate might have a problem fumbling the
ball when tackled, and so might not be effective in a game. We might determine this characteristic about our fast but clumsy
candidate in one of our controlled practice games. We might therefore decide instead draft a running back who is
slightly slower in the 40-yard dash, but does not fumble the ball and is good in open-field running. We decide that this candidate
will contribute more to our chances of winning an (American) football game.
The weak focus method described above is more forgiving - we will likely 'stumble' upon
the correct move sequence in the future by using simple heuristic rules such as rewarding pieces for having a large
number of moves (mobility) or for being somewhat near the center of the board.
If we decide to improve the focus of our search efforts, the question to ask ourselves,
is what sequence of steps will let our computer chess program acquire good diagnostic information for focusing our search?
Like the doctor or the mechanic, the diagnostic information will let us focus our search on the moves that are
most likely to lead us to our objectives.
The use of a diagnostic test or probe is used in many other areas of society to estimate or evaluate
the present state or capacity of people or things. The Scholastic Aptitude Test (SAT) probes the ability of the student
to solve word or math problems and is considered a good predictor of performance in college. An exterminator might use
a probe in certain areas of a house to predict the presence of termites. A home inspector will be hired by a bank before giving
a loan to a home buyer. A teenager is given a written test and a road test to determine whether or not they are ready for
a driver license. A theater group gives performers a chance to audition for a part in a musical by performing a test
in choreography, the reading of lines and in singing a short musical selection. Athletes are tested in the 40-yard dash to predict
ability as a receiver or runner in football. Running backs are judged by yards gained, quarterbacks by attempts, completions
and total passing yards. Defensive players are judged by tackles, quarterback sacks, interceptions, etc.
The tests in these cases are generally accurate and experts find them useful, along with other
information, to help make decisions about competence or value, or to predict the performance ability in an uncertain
environment. A slow 40-yard dash time might cause a running back not be drafted. A crumbling bridge might be inspected
by an expert, and the absence of structural damage in several critical locations causes a highway department to postpone needed
(but expensive) repairs for another year. A student with a high SAT score might be accepted at a college without
any interview, homework sample, or teacher recommendation. An older home, probed in several locations for the presence of
termites (and none found) is not fumigated, but the decision might be made to continue monitoring. A teacher assigns
a grade for a course based on a student's midterm, final exam, and homework assignments. In each case, a decision
was made based on the results obtained from gathering diagnostic information.
What is the system that we are trying to improve, what's the purpose of the system, and what are
its global measures? What structures should I adopt to create the results I want to create? What diagnostic test (perhaps
via relaxing constraints) can we use to estimate how winnable or promising the position is for either
side in a chess game, without doing any further searching? What structures of pieces act as a catalyst for future positions?
How can we reduce our uncertainty about which moves deserve our attention? What information can we obtain from the position
that is timely, relevant and actionable? What agile measurements can we make that are useful? How can we estimate how
effective each piece is performing now, or is likely to perform in the future? What diagnostic test can we use to
separate the promising or interesting positions (and therefore, paths to search) from those that are not promising? What are
the Key Performance Indicators, and how can we use them to tell us what action needs to take place? When playing
a game, what conditions make me more hopeful about the future? What do we have to do today to be ready for an uncertain tomorrow?
What are the critical few drivers of a promising position in a chess game, and how might we measure them? What equivalent
to an X-ray can we construct that reveals equivalent diagnostic information? What heuristic measurements will tell us What's
Going On? What operational definition/ specific procedure can we use for measuring how 'promising' a position is in a
chess game? What tests would we perform and what diagnostic information would we gather, how would we write software to do
this, and how would we actually go about gathering it, storing it, and using it in an evaluation function for a computer chess
program? I have spent some time thinking about answers to these questions, and I have written a paper on it, A Proposed Heuristic for a Computer Chess Program (John L. Jerz)
