Koomey give his opinion for what it takes to be a good problem solver:
p.xiii"Mastering the art of problem solving takes more than proficiency
with basic calculations: it requires familiarity with how people use information and requires learning about things as diverse
as exploring your ideology, telling good stories, and distinguishing facts from values."
Koomey thinks that new problems that are particularly vexing need a certain approach if solving
them is to be successful:
p.5"Most people attack a new problem by relying heavily on the tools and skills that are most familiar
to them. While this approach can work well for problems that are similar to those previously solved, it often fails, and fails
miserably, when a new problem is particularly novel or vexing. In these circumstances, it is best to assume nothing and treat
the problem as if you have never seen anything like it before... The ideal is to combine the curiosity and the non-judgmental
observation of a beginner with the experience of a senior analyst. Together, these skills can be an immensely powerful combination."
The cycle of action is a particularly useful procedure for solving difficult problems:
p.9"Donald Norman... first introduced the Cycle of Action:... The major steps in the cycle can be
summarized as follows: Decide what to do, based on your goals; Do it (execution); Assess the results (evaluation); Decide
what to do next, and then repeat the process"
When determining if progress is being made, we should be careful that what we measure is directly
related to what we care about. One should be careful when writing an evaluation function for a computer chess program - the
machine will use the results of the evaluation function to steer itself through an exponentially growing search tree. Are
you sure that your evaluation function will steer your search in the most efficient manner possible? Are you sure that what
you are measuring is what you are really looking for?
p.14,16"People often measure only those things that are easiest to measure. Like the man looking
for his keys under the street lamp (because that's where the light is) even though he lost his key chain down the street,
you are likely to be led astray if what you are measuring is only peripherally related to what you care about... The Cycle
of Action can become the Cycle of Inaction when information is diverted from its intended audience. "
The scientific method requires that we accept what we find, and that we be willing to discover that
we are wrong.
p.27"The physicist Alan Sokal points out that science is predicated on two key attitudes: begin
willing to accept what you find; and being willing to discover that you are wrong."
We must discover our own built-in bias before we attempt to make sense of numbers.
p.31,33"Anyone who states that ideology has no effect on decisions is either lying or deluded. Everyone
has an ideology whether explicit or not. Ideology provides a simplified model of the world that reflects our values, biases,
and experiences. It helps people make decisions in the face of imperfect knowledge... Ideology is one connection between the
world of ideas and the world of choices... Ideology is the filter through which you view the world. By exploring your own
body of beliefs, you can more reliably make choices that are consistent with your values, and you will also be better prepared
to use numbers responsibly and effectively in your analyses, writing, and presentations."
Just because we can count a particular thing does not mean that it has value. Also, important things
that can't be turned into a number should not be ignored.
p.62"Don't confuse things that are merely countable with those that really count. Some analysts
systematically ignore things that can't be turned into a number and assume that because these things can't be measured, they
are unimportant."
Implementing new ideas might, for a short period, generate criticism from those in the position
of authority. It seems that all new ideas at some point are rejected as absurd.
p.68"Question Authority... Recall what a Yale University management professor said about Fred Smith's
paper proposing reliable overnight delivery service: 'The concept is interesting and well-formed, but in order to earn better
than a 'C' the idea must be feasible.' Smith went on to found Federal Express Corporation"
When making choices we are also making decisions on values. Choices and values seem to be matched-pairs.
Before we write a heuristic that [ultimately] results in a choice, we should decide ahead of time what our values are.
p.87-89"Every human choice embodies certain values... if an analyst makes a choice, that analyst
has also made a value judgment... I define facts as assertions about the physical world that can be verified through experiment,
direct measurement, or observation... Values, on the other hand, are explicitly subjective and are an expression of the ideas
and feelings that are most important to us.... The purpose of analysis is to help you illuminate the facts so that you can
understand the choices of others and best use your values to make good choices of your own. Value choices are inescapable,
which is why separating demonstrable facts from value judgments is so essential. Do so, and you are sure to be on a solid
footing."
Making choices in the face of uncertainty forms the basis for much of life. We should not be afraid
to address uncertainty or to operate in environments that are not 100% predictable. Perhaps we can make small predictions,
or begin our problem solving efforts with what we do know. Certainly, we should take into account the limiting factors or
constraints which are likely to make an initial attempt at a solution a bit easier.
p.107"Life is about making choices in the face of uncertainty."
Data is simply data, and for it to have meaning we should be skeptical about how it was gathered
and what it truly represents.
p.124"[UC Berkeley professor John] Holdren's advice when dealing with data is to 'be suspicious,
skeptical, and cynical. Assume nothing.' Though it may sound paranoid to the uninitiated, such caution is an absolute necessity
for the seasoned problem solver."
Much can be learned about a problem by making a model and using it as an experimental playground
to gain understanding.
p.125" 'An explicit model is a laboratory for the imagination.' Models allow you to experiment,
test, and learn about the world in a systematic way without leaving your office... A model is an equation or set of equations
that describe how something you care about changes when other things change."
Heuristics and quick calculations seem to appear everywhere we make decisions. Perhaps we should
not fear them, but rather learn how to use them effectively.
p.131"People involved in fast-moving new ventures usually don't have time to research issues thoroughly
and don't have money to hire someone to do it for them (sometimes even the people they hire don't have the time either!).
Often, back-of-the-envelope calculations are the best that can be done [JLJ -fast and frugal heuristics?] in the time allotted.
The law of averages keeps the accuracy of such calculations from deteriorating too much because you are just as likely to
overestimate as to underestimate (assuming that you're not biased one way or another). Across all the assumptions you make,
these deviations average out... An important part of this process is balancing accuracy against the time needed to create
the calculation. Back-of-the-envelope calculations are meant to be quick, so they usually rely on relatively simple models.
Once the relationships between the key inputs and the desired output are established,
you'll need to determine which of the inputs you know and which you'll have to assume for purposes of initial calculations.
Don't get hung up on a particular number. The best thing to do is to carry out the calculations using the very roughest
of approximations for those things you don't know, and refine the inputs later."
Planning for the future requires some degree of prediction, but we will refine these predictions
as we move through time and things that are unclear now become more certain.
p.136, 141"The future is uncertain, but people keep trying to forecast it anyway... Forecasts are
an integral part of the Cycle of Action. They are most useful in developing the action sequence, but they can also be important
in the process of goal setting and in the evaluation of how actual events compare to our goals. In spite of their failings...
they are an essential part of life and work... it is important to adopt strategies that are robust in the face of inevitably
imperfect forecasts"
