Decision Quality Blog

The foundation to behavioral economics is the idea that investors are not rational actors: That they overreact to uncertainty, are influenced by immaterial information, and act for all manner of reasons not consistent with their best interests (utility). I found a paper the other day published by The Paul Woolley Centre for the Study of Capital Market Dysfunctionality called How Do Investors React Under Uncertainty? that reinforces this point.  Here’s a clip from the conclusion:

It is proposed that uncertainty, rather than risk, provides a much more realistic representation of the setting that we face when we come to pricing asset, and particularly corporate equities. We have gone a long way down the path of developing pricing models that incorporate risk (e.g. CAPM, APT, Fama and French empirical three-factor model) but comparatively little work has been done on the role (if any) that uncertainty plays in asset pricing. In order for uncertainly to affect pricing, it must have some influence on how investors incorporate information into pricing. Our contribution is to evaluate whether uncertainty influences the way by which investors respond to earnings announcements which will provide us with valuable insights as to the role that uncertainty plays in asset pricing.

In particular, we evaluate the proposition that investors will follow maxmin expected utility and so will progressively overweight bad news and underweight good news as they become more uncertain. Using VIX as a proxy for market uncertainty and earnings announcements as our information signal, we find that there is an asymmetric response to good and bad earnings news at high levels of uncertainty which is consistent with uncertainty breeding pessimism in the minds of investors. However, we do find evidence to suggest investors might have a more optimistic bent than is allowed under maxmin expected utility as indicated by how they react to earnings announcements when uncertainty is at the lower end of the scale.

A decision table is the “best practice” tool for doing the following:

• Breaking a complex decision into component parts.
• Generating a wide range of interesting choices in each of those sub categories.
• Creating multiple possible scenarios for answering the larger exam question.

Here is a brief tutorial on how to build one.

Goal of the Activity

Building a decision table is an activity that can be done with the internal team or the client.  It can be done in small or large groups. The goal of this activity is to explore the key drivers of a complex decision-like thinking through a complex deal or developing a new strategy–to generate multiple possible scenarios. These scenarios can be further refined using a variety of tools and techniques until the team finally arrives at single, fully integrated strategy. The work is based on the principles of Decision Quality.

Typically this work is driven by a standard Decision Playbook: Either one we already have, or one we pre-build as part of the customization process.  In some cases, the team will build a Decision Table from scratch.

Categories of Choices

The team starts by identifying all the relevant categories of choices. These collectively comprise the framework you’ll use to develop and test strategic scenarios. There are many ways to do this, but success looks like topic headings in some sort of logical order.  For example, if we wanted to develop a ”’human resources strategy”’, the topic headings might look like these:

• Talent model
• Sourcing
• Recruiting
• Hiring
• Onboarding
• Training
• Coaching and Mentoring
• Career Path
• Rewards and Recognition

Note: This isn’t meant to be a complete list, just an illustration. The implication here is that we want to make choices in each of those areas. The sum of the choices, one or more from each category, becomes a possible strategy.
These topics become individual column headings. MS Excel is an excellent tool to organize the work.
In an even more complex setting, we might want to identify headings in multiple areas. For example, in a large outsourcing deal, there might be a section or table for each of these areas:

• Desktop
• Servers
• Infrastructure
• Service Desk
• Human Resources
• Financial

Note: You can imagine that under each of these super headings, there would be multiple topics or columns we would want to explore.

Ranges of Choices

Once you have laid out the story line, or the topics you want to explore (we use the idea of story line because you should be able to imagine describing your preferred strategy by simply narrating across the tops of each of the columns), the next step is to think through the full range of choices in each of those columns or categories. This is an activity best done in groups, and should be a creative, brainstorming, lateral thinking exercise.

Values are what we want, choices are ‘what we can do.  A choice is something tangible, something you can buy with time or money. So while we might want customer satisfaction, that’s a value, not a choice–only the customer can choose to be satisfied. But we can choose to invest in training our people (for example).

• Choices should be mutually exclusive and collectively exhaustive.
• Choices should range from least to most, easiest to hardest, cheapest to most expensive, etc.

It is too often true that we gravitate immediately to choices that are familiar and safe. There are few better ways to kill innovative thinking. So the key thought here is to explore the full limits of each category of choice. For example, if we have a category called PRICE, here are some of the choices we could identify:

• More than anyone else
• Top of our peer group
• Par with our peer group
• Below peer group
• Lead the market
• Free
• Pay the customer

Note: Whether you would build a range of choices using those thoughts isn’t the point. All you need to notice here is that the range explores a FULL span of ideas. Why is this important? When we start to develop possible scenarios, exploring some extreme thoughts is often the key that unlocks the second and third idea that lead to a keen insight or sparkling strategy.

Strategic Lenses

The whole point of building a decision table is to build strategic scenarios. Think of a strategic scenario as a potential story line, competitive response, or strategy for cracking the problem you’re trying to solve. Common problems we’re trying to solve are the tendency to favor solutions we’ve seen work in the past, solutions that are favored by politically powerful people, or strategies that seem easy to do. Along the way, we often start down a particular path without fully considering all the interdependencies and follow on effects. So to do these things, we build strategic scenarios, using a decision table:

• Consider a wide range of alternative strategies before we land on one.
• Ensure that we’re thinking about all the dependencies (that’s why we built the table).
• Ensure that we’re getting multiple and different points of view early in the process (that’s why we do this in teams).

The process generally works like this: Once the decision table is built, we identify three to five strategic lenses, each of which highlights a different but interesting thought. In a competitive sales situation, those scenarios might have names that sound like these:

• What Will a Competitor Likely Do?
• The Solution We Think Our Customer Wants
• Win on Price
• Change the Game Using Services
• Compliant Solution
• If the problem you’re working is internal, the strategic lenses might sound like these:
• Quick and Dirty: Just Get it Done
• Maximize Customer Experience
• Minimize Roll Out Risk
• Aggressive Cost Reduction
• Compete for Talent

Scenario Development

A team is assigned to each strategic lens. The teams can be of any manageable size. The teams create strategic scenarios using their lens and the common decision table, making choices in each column that best support their lens. For example, you can easily imagine that the team most focused on how Competitor A will bid will be making different choices that the team looking through the lens of “Changing the Game.”

This activity is done simultaneously, so the teams are all working in parallel, using the same tool set, at the same time. When they’re done, the teams present their strategic scenarios using the decision table to guide the process. Teams use some sort of code or color to mark their choices on the decision table. The different strategic scenarios are overlaid, often using dots on MS Excel documents, to show the individual choices, as well as how they cluster and compare across strategies.

A wonderful article/interview in the New York Times with David Dunning, one of the rock stars of decision-making . . . you get to be called that, at least by me, if you have an entire principle named after you (Dunning-Kruger Effect).  Donald Rumsfeld said it best but we were too stunned to hear him . . .

“There are things we know we know about terrorism.  There are things we know we don’t know.  And there are things that are unknown unknowns.  We don’t know that we don’t know.”

Here’s a snip from the interview.  Well worth reading the entire thing.  Apparently there are four more parts to come.

Dunning and Kruger argued in their paper, “When people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden: Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it.  Instead, like Mr. Wheeler, they are left with the erroneous impression they are doing just fine.”

It became known as the Dunning-Kruger Effect — our incompetence masks our ability to recognize our incompetence.  But just how prevalent is this effect?  In search of more details, I called David Dunning at his offices at Cornell:

DAVID DUNNING: Well, my specialty is decision-making.  How well do people make the decisions they have to make in life?  And I became very interested in judgments about the self, simply because, well, people tend to say things, whether it be in everyday life or in the lab, that just couldn’t possibly be true.  And I became fascinated with that.  Not just that people said these positive things about themselves, but they really, really believed them.  Which led to my observation: if you’re incompetent, you can’t know you’re incompetent.

ERROL MORRIS: Why not?

DAVID DUNNING: If you knew it, you’d say, “Wait a minute.  The decision I just made does not make much sense.  I had better go and get some independent advice.”   But when you’re incompetent, the skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is.  In logical reasoning, in parenting, in management, problem solving, the skills you use to produce the right answer are exactly the same skills you use to evaluate the answer.  And so we went on to see if this could possibly be true in many other areas.  And to our astonishment, it was very, very true.

ERROL MORRIS: Many other areas?

DAVID DUNNING: If you look at our 1999 article, we measured skills where we had the right answers.  Grammar, logic.  And our test-subjects were all college students doing college student-type things.  Presumably, they also should know whether or not they’re getting the right answers.  And yet, we had these students who were doing badly in grammar, who didn’t know they were doing badly in grammar.  We believed that they should know they were doing badly, and when they didn’t, that really surprised us.

ERROL MORRIS: The students that were unaware they were doing badly — in what sense?  Were they truly oblivious? Were they self-deceived?  Were they in denial?  How would you describe it?

DAVID DUNNING: There have been many psychological studies that tell us what we see and what we hear is shaped by our preferences, our wishes, our fears, our desires and so forth.  We literally see the world the way we want to see it.  But the Dunning-Kruger effect suggests that there is a problem beyond that.  Even if you are just the most honest, impartial person that you could be, you would still have a problem — namely, when your knowledge or expertise is imperfect, you really don’t know it.  Left to your own devices, you just don’t know it.   We’re not very good at knowing what we don’t know.

In case you missed this or don’t care, tech blogger gizmodo recently came into possession of a prototype of the latest blockbuster to be next gen iphone setting off a first class 21st century brouhaha.  So why bring it up here? As an exercise in decision-making, three thoughts . . .

Thought 1: John Stewart just devoted nearly nine minutes to poking Apple and its iconic CEO in the eye on this.  Possible second order implication: Steve, you jumped the shark.  Does it really matter or is it possible this is all a cleverly thought out publicity stunt?

Thought 2: Given that the phone in question was “disguised” in the shell of the current generation phone, tell me again how this happened?  I mean really, how many iphones get left in a bar on a daily basis?

Thought 3: Hey Gizmodo, how you feeling about the decision to take this thing public? The short term spurt in readership has to be a rush. My guess is whatever inside line you had to Apple isn’t looking so good right now.

For more on the fun, here are some links . . .

• Did Apple call the cops on Gizmodo
• Police raid Gizmodo editor’s home
• Lost iPhone: The cops are on the case
• Apple wants its secret iPhone back
• Letterman’s Top 10: Excuses for the guy who lost Apple’s top secret iPhone

Decision-making is the means by which we most directly shape our lives. Some decisions we make consciously; many more are “automatic” decisions made in response to stimulus. The first type of decision-making is a distinctly human domain. Other species don’t have the same breadth of cognitive tools, and therefore can’t be described as true decision-makers.

If we want to improve our life results and get more of what we want, we must make different and higher-quality decisions. The other choice is to keep doing the same things over and over or hope that someone or something will come along and help us out.  This is called Magical Thinking.  It isn’t helpful.

Most of us are not as good at decision-making as we think we are. Why not? One of the biggest reasons is our succeptibility to influence by the people around us, something writer Laurence Gonzalez calls “groupness.”

[amtap book:isbn=0393058387]

The idea is simple: When we are in a group setting we tend to turn our brains off and take our clues from the people around us.  Think you don’t?  You do. Consider Gonzales’s story about NASA and not one but two space shuttle disasters.

After the space shuttle Columbia broke up in flight on February 1, 2003, a commission was formed to investigate the accident. It covered all the mechanical and physical facts of the explosion, but that left a very basic and vexing problem. The accident had been avoidable and had not been avoided. That meant, in effect, that the smartest guys in the world had done the dumbest thing in the world. Twice. The commission’s report sought to explain how this could happen by talking about the very sorts of mental models and scripts, along with group hostilities, that shape the lives of the Rattlers and Eagles of this world and that shaped the experiences of John Tanner and Homo erectus as well.

Managers at NASA had fashioned a psychological framework that allowed them to systematically ignore clear evidence that they were heading into trouble. NASA’s triumphant experiences in putting men on the moon during the Apollo program of the 1960s had led to the formation of mental models and behavioral scripts in the organizational culture that persisted despite drastic changes in the environment, such as greatly reduced budgets and overwhelming evidence that essential pieces of equipment were malfunctioning.

The second force influencing critical decisions at NASA was groupness. The final report of the commission on the Columbia accident said, “External criticism and doubt…reinforced the will to ‘impose the party line vision on the environment, not to reconsider it….’ This in turn led to ‘flawed decision making, self deception, introversion and diminished curiosity about the world outside the perfect place.’”

The report is quoting Garry D. Brewer, a professor of organizational behavior at Yale University, who was attempting to explain how management at NASA could have behaved the way it did. The “external criticism and doubt” came, for example, after the explosion of the space shuttle Challenger in 1986, the first time that NASA made the worst mistake it could have made. That criticism came, significantly, from outside of the in-group.13

The combination of groupness and persistent mental models made for an organization that could not take in new information when that information did not accord with its indelible concept of itself as the “perfect place,” as Brewer called it. Moreover, it could take any contradictory information and reinterpret it as confirming the existing model. This came about through two major influences that made NASA’s models unassailable and made its culture hostile to all outside groups. To begin with, there was the unprecedented investment not just of money but also of personal and emotional effort during the Apollo program. The divorce rate was high, as marriages fell apart under the strain. People literally gave their lives for the effort. Gus Grissom, Roger Chaffee, and Ed White died in a fire during a test on the launch pad in 1967. But following all that sacrifice was an astounding success, arguably the highest achievement of human technical culture, with men walking on the moon while we watched them on television. This has all the ingredients necessary to form robust models and scripts (big investment, big reward). And that experience simply hardened the shell of groupness that already characterized NASA.

NASA’s unspoken and unconscious attitude by that time was: We must be right; after all, we put a man on the moon. There had been many reinforcing steps along the way, too. During Apollo 13, for example, the concept that “failure is not an option” was developed, and the safe return of Apollo 13 served to strengthen the models and the ability of groupness to repel ideas from outside. It also promoted a dangerously wrong idea. For failure, unfortunately, is always an option.

The big problem here is that successful outcomes, broadly defined, caused people to ignore disconfirming information, out of spec performance, and dysfunctional decision processes.  Or to put it differently, if nothing bad happens, we must be doing something right. More . . .

As the investigating board put it, “Both Columbia and Challenger were lost also because of the failure of NASA’s organizational system…. Both accidents were ‘failures of foresight’ in which history played a prominent role.”

In the case of Challenger, engineers were faced with the fact that fuel in the solid rocket boosters was burning through the rubber O-rings that sealed the seams where sections of the rockets were joined. Groupness dictated that no one outside that immediate culture was fit to judge the fruits of their labors. Confirmation bias is a phenomenon in psychology by which people tend to take any information as confirmation of what they already believe. In addition, they tend to ignore or miss any information that doesn’t confirm what they already believe. This can work to gradually revise a mental model in a one-way direction. Because NASA believed that “we’re the best” and that “failure is not an option,” all information tended to support that conclusion, no matter how contrary it might have seemed to an outsider.

Each time the solid rocket fuel burned the rubber O-rings during launch without an accident happening, the engineers at NASA readjusted their models and scripts slightly to accommodate that as “normal.” Through a subtle progression, a complete failure of design was turned into an acceptable situation. Each time nothing bad happened, they did it again. This confirmed the mental model, even while groupness helped to keep conflicting information from having any effect. The weather was the spinning roulette wheel in this complex system that NASA managers had unwittingly set up for themselves. All they needed was for cold enough weather to coincide with a launch, because cold made the rubber O-rings more brittle and therefore more likely to burn through. It was just a matter of time. The ape-like hierarchy at NASA ensured that those engineers who knew or suspected the truth would not be heard.

What makes this whole story especially sad is that NASA learned nothing about decision-making, the root of the problem.

The same array of troubles bedeviled Columbia. Insulating foam blew off the main fuel tank and hit the orbiter. The engineers had seen it happen a number of times, but management kept on launching anyway. When nothing bad happened, they took that as confirmation that they were right and reset their mental models to accommodate the malfunction. As the final report on the accident clearly stated, “The initial Shuttle design predicted neither foam debris problems nor poor sealing action of the Solid Rocket Booster joints. To experience either on a mission was a violation of design specifications. The anomalies were signals of potential danger, not something to be tolerated.”

But in the culture that had evolved at NASA, each return from a successful mission was another moon landing. If the world had largely come to ignore space launches, NASA was still hearing applause that was, by the time of Columbia, more than thirty years old. So, instead of peering more deeply into the problem, they gradually revised their models until they were literally interpreting failure as success. The final report of the commission said:

Engineers and managers incorporated worsening anomalies into the engineering experience base, which functioned as an elastic waistband, expanding to hold larger deviations from the original design. Anomalies that did not lead to catastrophic failure were treated as a source of valid engineering data that justified further flights.

The moral of the story . . .

And this is precisely how a mental model can be expected to function. It operates on a simple rule: if nothing bad happens, you must be doing something right [emphasis added]. So influential were NASA’s models and scripts, and so delusional its self-confidence bred of groupness, that even after Columbia broke up, killing all on board, the space shuttle program manager told the press that he was “comfortable” with his previous assessments of risk and didn’t think the foam debris had caused the accident. But remember that a key feature of this system is that, taken one small step at a time, each decision always seems correct.

A fine video on the topic of practical wisdom by Barry Schwartz.  I was particularly taken on his point of view on “practical wisdom.”

“Practical wisdom,” Aristotle told us, “is the combination of moral will and moral skill.” A wise person knows when and how to make the exception to every rule, as the janitors knew when to ignore the job duties in the service of other objectives. A wise person knows how to improvise,as Luke did when he re-washed the floor. Real-world problems are often ambiguous and ill-defined and the context is always changing. A wise person is like a jazz musician — using the notes on the page, but dancing around them, inventing combinations that are appropriate for the situation and the people at hand. A wise person knows how to use these moral skills in the service of the right aims. To serve other people, not to manipulate other people. And finally, perhaps most important, a wise person is made, not born. Wisdom depends on experience, and not just any experience. You need the time to get to know the people that you’re serving. You need permission to be allowed to improvise, try new things, occasionally to fail and to learn from your failures. And you need to be mentored by wise teachers.”

With great care I draw your attention to an article by Lee Siegel called The Zero-Sacrifice Presidency.

Obama tells us that we can have quality, universal health care without increasing the deficit. He tells us that he intends to have the 9/11 detainees given a fair trial in a civilian court but assures us that the trials will end in convictions. He declares that he will wage war in Afghanistan, but pledges to start bringing the troops home in 18 months. And everybody nevertheless takes these contradictory, irreconcilable statements seriously, as they parse, analyze, scrutinize Obama’s every word for some kind of coherent meaning. The president is like the character Chance in the novel and movie Being There, whose every fatuous utterance was celebrated for its profundity.

Some of Obama’s defenders chastise his exasperated listeners for their inability to detect the president’s “complexity.” But a fantasy of universal popularity that panders to every conflicting interest simultaneously is not the same thing as “complexity.” It is complexity if I tell my wife that I have to move to another state where I know I can find work, but that I realize the strain it will put on our marriage, and that I know the effect it will have on our child, and that I am aware of the consequences of such an attempt if I don’t find a job, having spent so much money on moving and establishing myself in a new place. It is not complexity if I tell my wife that I have to move to another state where I know I can find work, but that I will be back next week, and with lots of money.

In the spirit of full disclosure, my caution is based on two points.  The first is that I was and largely still am an Obama supporter (though I fully admit my reasons may not be rational).  The second is that I made a promise to myself that I would stop writing political screeds. So why this?

I have been reading for the second time a book on decision making called The Paradox of Choice by Barry Schwartz.

[amtap book:isbn=0060005696]

It’s not my favorite book on decision making though it and he have certainly enjoyed a lot of attention for a “perfect for the times” message that can best me summed up by the B-Head for his book:  How the Culture of Abundance Robs Us of Satisfaction.

Schwartz quotes a considerable amount of research to make the case that:

• An over-abundance of choices ultimately leads to confusion, poor decision making, and dissatisfaction with the decision once it is made.
• The more we are forced to justify our choice, the more dissatisfied we become with our choice.
• The reasons people give for making a choice are seldom the real reaons.
• The more people are forced to make trade-offs, the lower the satisfaction with the choice ultimately made.  Worse, the stress of making difficult trade-offs leads to low quality decision making.

Or to put it another way.  Many of us fancy that we are rational decision makers, or at least we can be on-demand when we must be.  The implication here is that we can, on an as needed basis, turn off our biology, our socialization, the mental short cuts, and the internal angels and demons we haul with us and simply add up the plusses and minuses or review the numbers and make the obvious right choice.  That’s a big ask.

That’s not to say that we can’t wrangle our emotions and engineers ourselves past many decision traps becuase we can.  It is to say that we don’t stop being human through the process. So if we believe Schwartz, and I do, many of us are left looking for paths through irreconcilable differences that often don’t exist, not becuase the facts support such a notion, but becuase we believe in Santa Claus.

I have never met the current or previous president but it seems plausible from afar that one of their many differences is that 43 didn’t appear to agonize and certainly had no visible regrets.  The same can be said for Cheney and Rumsfeld (though not Powell).  Critics of the three would say that one of the reasons that’s true is that they didn’t bother to consider alternatives that conflicted with what they had already decided to do.  If that is in fact an accurate accusation, it makes them no different than most of us (hard as that is for my liberal friends to hear).

Conversely, President Obama appears to pride himself on his thoughtful, inclusive, and complete decision making processes.  As a decision engineer, I heartily applaud his instincts and his practices.  But listening to his recent address at West Point about Afghanistan I am left wondering if Barry Schwartz doesn’t have his number: That at the end of the day, Mr. Obama hates making trade-offs and therefore makes others do it for him.

If that is true, it makes Obama like the rest of us as well.  It is also leadership sin number one: Forcing people below your pay grade to make the tough trade-offs you’re paid to make.

Decision making is a distinctly human activity: it is more than an instinctive “stimulus/response.”  Decisions aren’t found under a rock.  They are the result of cognitive processes that we can control.  They are what make us human. Because we are human, and because decision making is a distinctly human activity, decision making is subject to all manner of pitfalls, errors, traps and flaws. 

Most of us are not as good at decision making as we think we are or would like to be. There are many reasons why this is true.  Here are some.

• While we are experienced decision makers, we are not necessarily skilled in the sense that we have not thought about and internalized processes that lead reliably to high quality choices.
• We are much more easily influenced than we care to admit: by people, by word choices, by events, and by our own emotions.
• We are driven by psychological forces most of us don’t really understand: We were not evolved in modern industrialized, information-overloaded, choice-rich environments.
• We are wired to take mental short cuts.  It’s how we go through without having to make decisions about everything that we need to do.  Those mental short cuts can and do work against us causing us to make low quality decisions (again, far more often than we want to believe).
• We hate having to make trade-offs.
• We are evolved to adapt to our circumstances and to revisionist thinking,  We quickly learn to live with the consequences and outcomes of our decisions and retrofit our stories so that we can be right.

Despite everything we appear to have going against us, most of us manage to be highly functional and successful in life: We can say with confidence that we must be good decision makers judging by the outcomes we’re associated with.  Either that or we just got lucky (or a bit of both).

So why try to get better?  Why spend time and energy learning how to make higher quality decisions (and what does that mean anyway)?  After all, it’s not like we wake up in the morning thinking we need to be better decision makers?

We have over the years asked thousands of executives and leaders two simple and related questions:

• Are you a good decision maker (and how do you know)?
• Are “we” (meaning the group that person is part of) good decision makers?

As you might expect, the most frequently occurring answer to the first question is, “It depends.”  The answer to the second question is most consistently, “No.”

Why all this matters can be summed up in an oft quoted statistic from a now-retired professor named Paul Nutt who after years of research concluded that more than 50% of all decisions in business are unsuccessful.  Some of us think he’s a wild-eyed optimist.

[amtap book:isbn=1576751503]

While the dynamics of making decisions by yourself and making decisions with other people are different in many ways, they are similar in many ways as well.  Particularly where the decision is complicated—meaning difficult trade-offs, lack of clarity about the real problem, lots of uncertainty, long feedback loops, significant consequences associated with the outcomes—we can increase our confidence that we’re making a high quality choice by improving the quality of the decision process we use.  Steps that lead to decision quality include the following:

• Take time to explore the problem.  Words matter.  You’ll get new insights about the problem and later see a better set of alternatives by taking care in how you frame the problem.
• Give yourself good alternatives.  As consumers we usually face way too many choices.  In that case, quality may come from limiting alternatives.  In business, often the opposite is true: We rush to solution.  So time spent thinking about alternatives THE RIGHT WAY is important.
• Be smart about how you gather information.  As is true about choices, it is possible to have too much and too little information . . . and in the end it is not possible to know everything you might like to know.  Focus your information on gathering on what is important.  Focus on what could go wrong and what that means to your decision.
• Identify clear values and not too many of them.  Too many trade-offs is as problematic as too many choices and too much information.  The opposite is also true.

As you would expect, there are many ways to go about building quality into each of those steps.  Fortunately, for most decisions, the principles, methods, and tools you need are simple, easy to understand and master, and easy to use in both individual and group decision settings.

For more on making quality decisions visit these links: