### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# INTRO PRBLMS PHILOS Philos 23

UCI

GPA 3.63

### View Full Document

## 49

## 0

## Popular in Course

## Popular in PHIL-Philosophy

This 23 page Class Notes was uploaded by Kiarra Sipes on Saturday September 12, 2015. The Class Notes belongs to Philos 23 at University of California - Irvine taught by Staff in Fall. Since its upload, it has received 49 views. For similar materials see /class/201940/philos-23-university-of-california-irvine in PHIL-Philosophy at University of California - Irvine.

## Reviews for INTRO PRBLMS PHILOS

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 09/12/15

Journal of Experimental Psychology learning emery and Cognition 1997 Vol 23 No 2 406426 Copyright 1997 by the American Psychological Association Inc 02784393375100 Tests of Consequence Monotonicity in Decision Making Under Uncertainty Detlof von Vtherfeldt and Ngar Kok Chung University of Southern California R Duncan Luce and Younghee Cho University of California Irvine Consequence monotonicity means that if 2 gambles differ only in 1 consequence the one having the better consequence is preferred This property has been sustained in direct comparisons but apparently fails for some gamble pairs when they are ordered in terms of judged monetary certainty equivalents In Experiments 1 and 3 a judgment procedure was compared with 2 variants of a choice procedure Slightly fewer nonmonotonicities were found in one of the choice procedures and overall fewer violations occurred than in previous studies Experiments 2 and 3 showed that the difference was not due to procedural or stimulus presentation differences Experiment 4 tested a noise model that suggested that the observed violations were due primarily to noise in estimating certainty equivalents and so despite some proportion of observed violations consequence monotonicity cannot be rejected in that case A fundamental principle of almost all formal theories of decision making under uncertainty or risk ie prescribed probabilities is that when one alternative dominates an other then the dominated one can be eliminated from consideration in making choices Dominance can take several distinct forms and we focus on the one called consequence monotonicity that is both the most crucial for theories of choice as well as the most controversial empiri cally It says that if a gamble is altered by replacing one consequence by a more preferred consequence then the modi ed gamble clearly dominates the original one and so it is also chosen over the original one Certainly this assump tion seems compelling Figure 1 illustrates a speci c ex ample Indeed most people when faced with their viola tions of consequence monotonicity treat them as misjudgments to be corrected Bimbaum 1992 Brothers 1990 Kahneman amp Tversky 1979 So what is the problem The validity of consequence monotonicity was rst brought into question by what is now called the Allais paradox Allais 1953 Allais amp Hagen 1979 and later by the closely related common ratio effect Kahneman amp Tversky 1979 The Allais paradox is shown in Figure 2 It has been pointed out Kahneman amp Tversky 1979 Luce 1992 that the signi cance of these paradoxes is ambiguous because in addition to monotonicity they embody an assumption that a compound gamble is seen as indifferent to its formally Detlof von Winterfeldt and NgarKok Chung Institute of Safety and Systems Management University of Southern California R Duncan Luce and Younghee Cho Institute of Mathematical Behavioral Sciences University of Califomia Irvine This work was supported by the National Science Foundation under Grant SEES8921494 and Grant SES9308915 We would like to thank Barbara Mellers and Michael Bimbaum for letting us use their stimulus materials and for many constructive discussions on the topic of this article Correspondence concerning this article should be addressed to Detlof Von Winterfeldt 2062 Business Center Drive Suite 110 Irvine California 92612 406 equivalent reduced form It is unclear which assumption has failed When direct choices are made without any reduction the number of violations is markedly reduced Brothers 1990 This result implies that a failure in reducing com pound gambles to rstorder ones rather than a failure of consequence monotonicity is the underlying source of the paradox More recently however new evidence of a differ ent character has cast doubt directly on consequence mono tonicity Rather than asking for a choice between pairs of gambles these newer studies are based on the simple observation that it is much simpler to nd for each gamble the sum of money that is considered indifferent to the gamble such a sum is called a certainty equivalent of the gamble and then to compare these numerical amounts The simplest estimation procedure is to obtain a judgment of the dollar estimate which is called a judged certainty equivalent For n garnbles this entails n judgments rather than the nn l2 binary comparisons which is a major savings for large n However for certain classes of gambles these judged certainty equivalents appear to violate conse quence monotonicity Bimbaum Coffey Mellers amp Weiss 1992 Mellers Weiss amp Bimbaum 1992 We discuss these experimental ndings in greater detail further on The question is whether these results should be interpreted as violations of consequence monotonicity or whether there is something of a procedural problem in evaluating prefer ences Assuming that decision makers have some sense of preference among alternatives what types of questions elicit that attribute There really is no a priori way to decide this The major part of the theoretical literature on decision making which is mostly written by statisticians and econo mists with a smattering of contributions from psychologists has cast matters in terms of choice If g and h are two alternatives decision makers are asked to choose between them and when g is chosen over h we infer that the decision maker prefers g to h These theories are cast in terms of a binary relation of preferred or indifferent to Of course each of us continually chooses when purchasing goods An CONSEQUENCE MONOTONICITY 407 30 96 41 596 t dominates 6 0 20 20 Figure I A chance experiment in which with probability 80 one receives 96 and with probability 20 one receives 6 in the left gamble and 0 in the right gamble Thus the left gamble dominates the right one because 6 gt 0 alternative approach favored by some psychologists is that the evaluation of alternatives in terms of for example money is a if not the basic way of evaluating alternatives This is what a Storekeeper does in setting prices although we usually suppose that the goods are actually worth less to the seller than the price he or she has set on them Our bias is that choices are the more fundamental of the two Given that bias then interpreting apparent violations of consequence monotonicity arising from judged certainty Gamble I 5 10000 a t 50 0 50 10 l 10000 L b r 50 5 0 90 0 39 5 10000 0 0 LT 0 equivalents becomes problematic If perhaps by suitable instructions we can induce decision makers to report the true worthquot of each alternative in the sense meantby choice theorists namely CE g is the certainty equivalent of g if in a choice between g and CE g the decision maker is indifferent to which is received then such judged certainty equivalents should establish the same ordering as do choices However this consistency does not appear to hold using any of the instructions so far formulated for eliciting judged certainty equivalents Bostic Hermstein amp Luce 1990 Tversky Sattah amp Slovic 1988 That being the case is there a method of estimating certainty equivalents that is order preserving and if so do these estimates satisfy consequence monotonicity Bostic et a1 1990 adapted from psychophysics a choice method that within the noise levels of choices appears to be order preserving We explore whether the estimates obtained using this method satisfy consequence monotonicity A er probing these issues we ultimately conclude that in our experiments at least there is little effect of procedure but Gamble II versus FL 55000 versus 0 90 5 0 3910 5000 versus 0 90 0 Figure 2 Illustration of the Allais paradox The notation is as in Figure 1 Many participants prefer Gamble II to Gamble I in Conditions a and b which response is consistent with consequence monotonicity but prefer Gamble I to Gamble II in Condition c The left side of b reduces under ordinary prob is known as the Allais paradox ability to the left side of c This reversal of preference between Conditions 21 and c 408 rather that much of the apparent problem arises from variability of the estimates Consequence Monotonicity in Binary Gambles By a sure consequence we mean a concrete object about which there is no uncertainty In our experiments sure consequences will be sums of money Let C denote a set of sure consequences with typical elements x and y A gamble is constructed by conducting some chance experiment such as a toss of dice or a drawing from an um of colored balls the possible outcomes of which are associated with sums of money In the simplest cases as in Figure 1 let xEy denote the binary gamble in which when the chance experiment is realized the sure consequence x is received if the event E ocaurs or the sure consequence y is received if E does not occur Further let 2 be a preference relation over all binary gambles generated from the set E of all events and the set C of consequences Suppose E is an event that is neither the null nor the universal one and x y and z are sure consequences then binary consequence monotonicity is de ned to mean x 2 y if and only if xEz Z yEz if and only if zEX 2 zE y 1 If in Equation 1 any of x y and z are gambles rather than sure consequences then because two independent chance experiments have to be carried out to decide which conse quence is received the alternatives are called secondorder compound gambles and we speak of compound consequence monotonicity Throughout the article we use the latter term only if the compound gamble is not reduced to its equivalent rstorder form that is to a gamble in which a single chance experiment determines which consequence is received and does so under the same event conditions as in the second order gamble This is most easily illustrated when the only thing that is known about E is its probability p of Occurring in which case we write xpy In that case the rstorder equivalent to the compound gamble xpyqz is xpq y1 pqzl 4 With the meaning that x arises wi probability pq y with probability 1 p q and z with probability 1 q Almost all contemporary formal models of decision making under uncertainty have a numerical representation in which the utility of a gamble is calculated as a sum over events of the utilities of the associated consequences times weights somewhat similar to probabilities that are attached to the events Included in this general framework are all versions of subjective expected utility theory Savage 1954 prospect theory Kahneman amp Tversky 1979 and its generalizations Luce 1991 Luce amp Fishbum 1991 1995 Tversky amp Kahneman 1992 Wakker amp Tversky 1993 and several rankdependent expected utility theories for ex ample Chew Kami amp Safra 1987 Gilboa 1987 Luce 1938 Quiggin 1982 1993 Schmeidler 1986Sega1 1987 1989 Wakker 1989 Yaari 1987 All of these theories imply Equation 1 VON WINTERELDT CHUNG LUCE AND CHO Previous Tests of Consequence Monotonicity Direct tests of Equation 1 entail comparisons of such pairs as xEz and yEz or zEX and zEy when x is larger than y Brothers 1990 working with money consequences and prescribed probabilities created 18 pairs of gambles of this form For example a typical pair of gambles was 5067100 vs 5067100 Both gambles were presented in the form of pie charts on one page and participants were asked to indicate their strength of prefer ence for one of the two gambles on a rating scale that ranged from 0 no preference indifference to 8 very strong preference In this situation a participant merely had to recognizethatthetwogamblesvariedonlymthe rstconse quence and his or her choices should follow clearly from which gamble had the higher rst consequence It would be very unusual to nd violations of consequence monotonicity in such direct comparisons and indeed Brothers found virtually none Only 5 out of 540 judgments showed a violation and the few participants 4 out of 30 who showed any violation classed them as mistakes in the debrie ng session Brothers 1990 also tested compound consequence mono tonicity First he established a preference or indifference between two rstorder gambles that had identical or closetoidentical expected values but otherwise were dissimi lar For example he asked his participants to compare the gambles g 72033 280 and h 25040 80 Having established a preference or indifference in this rst pair of gambles he then created two secondorder gambles in which one outcome was one of the rstorder gambles and the other outcome was a common monetary amount For example the test pair with the above rstorder gambles was 503372033 280 vs 50332503380 In these secondorder gambles 50 was the consequence with probability 33 and either g or II respectively with probability 67 According to compound consequence mono tonicity the preference between the rstorder gamble pair should persist when comparing the second order gamble pair The test stimuli were again presented side by side on a page which thus allowed participants to recognize the common elements in the two secondorder gambles About 35 of the participants violated compound monotonicity However 34 of participants also reversed their choice of preference when the same pair of rstorder gambles was presented on two occasions Thus the violations appeared to be mainly due to the participants unreliability when compar ing gambles with equal or similar expected values In fact most participants indicated that they recognized the com mon elements in the compound monotonicity tests and ignored them In contrast to Brothers39s 1990 choice procedure and compound gambles Bimbaum et a1 1992 used judged certainty equivalents and only rstorder gambles and they found a suiking pattern of violations of consequence mono tonicity to certain pairs of binary rstorder gambles Their gambles which had hypothetical consequences ranging from 0 to 96 were represented as pie diagrams and were presented in a booklet For each gamble the participants wrote down their judged certainty equivalent under one of CONSEQUENCE MONOTONICITY 409 three distinct points of view Some were asked to provide buying prices some selling prices and some neutr quot prices The neutral prices were described as meaning that the subjects are neither the buyer nor seller of the lottery but rather a neutral judge who will decide the fair price or true value of the lottery so that neither the buyer nor the seller receives any advantage from participants judgment For most stimuli consequence monotonicity held up quite well However for several special stimuli the dominated gamble on average received a higher judged median certainty equivalent than did the dominating one For example in the pair 96950 and 969524 the former gamble although dominated by the latter received a higher median certainty equiValent than did the dominating gamble This was also true when the amount of 96 was replaced by 72 and when the probability of 95 was replaced by any probability higher than about 80 Birnbaum et al found this result to hold for about 50 of the participants for all three different points of view Birnbaum et a 1992 interpreted this violation of monotonicity in terms of their con guralweighting model In contrast to the standard weightedaverage utility models the con guralweighting model multiplies the utility of each consequence by a weight that is a function not only of the probability of receiving that consequence but also of the actual value of the consequence Summed over all of the events in a gamble the weights add to unity The observed nonmonotonicity is easily accommodated in the con gural weighting model by assigning to lowprobability events smaller weights when the consequence is 0 than when it is positive Fitting weights to a large set of data Birnbaum et al found this effect for lowprobability events In this theory weights can also depend on the point of view of the contemplated transaction eg buying vs selling thus accommodating the observed differences in the preference order of the gambles arising from assuming different pricing strategies Subsequent replications and modi cations of this experi ment established that the effect is robust For example Mellers Weiss et al 1992 used pie charts and peculiar dollar amounts in an attempt to reduce participants ten dency to do numerical calculations when estimating cer tainty equivalents They also increased the hypothetical stakes to hundreds of dollars and used negative amounts In all cases similar to those of the rst study they found violations except for gambles with mixed gains and losses and except when relatively few gambles were evaluated In accord with the con guralweighting theory account Mellers Weiss et al hypothesized that participants are more likely to ignore or put very little weight on a zero outcome In contrast when both outcomes are nonzero the con gural weighting leads to some weighted average of the two utilities Further replications by Birnbaum and Sutton 1992 using judged certainty equivalents con rmed these ndings How ever they also clearly showed that the effect disappeared when participants were presented with the pairs of gambles and were asked to choose between them directly Judged and Choice Certainty Equivalents Our concern is whether the result of Birnbaum and colleagues is typical when certainty equivalents of any type are used or whether it disappears when elicited by some other procedure Because the property of consequence monotonicity is stated in terms of choices we Suspect that some form of choicebased procedure for estimating cer tainty equivalents may yield a different result One reason for entertaining this belief is the wellknown preference reversal phenomenon in which judged certainty equivalents of certain gambles exhibit an order that is the reverse of that determined by choices Grether amp Plott 1979 Lichtenstein amp Slovic 1971 Mellers Chang Birnbaum amp Ord ez 1992 Slovic amp Lichtenstein 1983 Tversky et al 1988 Tversky Slovic amp Kahneman 1990 Speci cally this reversal occurs when for two gambles with equal expected values one has a moderately large probability of winning a small amount of money called a Pbet and the other has a small probability of winning a fairly large amount of money called a bet Typically the judged certainty equivalent for the bet is larger than that for the Pbet whereas partici pants choose the Pbet over the bet when making a direct choice The fact that judged certainty equivalents reverse the order of choices makes abundantly clear that they do not always yield the same information as choices In an attempt to estimate certainty equivalents that do exhibit the same ordering as choice Bostic et al 1990 adopted a psychophysical up down method called param eter estimation by sequential testing commonly referred to as PEST in which participants engage only in choices between gambles and monetary amounts Bostic et al investigated the impact on the preferencereversal phenom enon of this choicebased method for estimating certainty equivalents as compared with judged estimates A particular gamble money pair is presented a choice is made and after many intervening trials that gamble is again presented with an amount of money This time the money amonnt presented by the experimenter is a smaller than it was the rst time if the money had been the choice in the earlier presentation or b larger if the gamble had been the choice Such presenta tions are made many times with the adjustments in the money amount gradually being reduced in size until the amounts are oscillating back and forth in a narrow region An average of these last values is taken as an estimate of the certainty equivalent The exact procedure is described in detail further on Because such certainty equivalents are derived using choices they are called choice certainty equivalents Bostic et al found that when choice certainty equivalents were used the proportion of observed reversals was about what one would expect given the estimated inconsistency or noise associated with the choice proce dure In other words they found no evidenCe against assuming that choice certainty equivalents agree with the choice de ned preference order Thus a natural question to consider is whether the violations of consequence monoto nicity also tend to vanish when a choice procedure is used to establish certainty equivalents Birnbaum 1992 examined an alternative response mode 410 in which participants compared a gamble and a monetary amount on an ordered list of 27 monetary amounts and circled all of the monetary amounts on the list that they preferred to the gamble Tversky and Kahneman 1992 used a similar approach in estimating certainty equivalents Violations of consequence monotonicity persisted in the Birnbaum study Although this procedure is formally a choice procedure we are not at all convinced that it is likely to provide the same certainty equivalents as does an up down choice procedure In particular it may be that participants follow the strategy of rst establishing a judged certainty equivalent of the gamble and then circling the monetary amounts larger than that judged value In that case the procedure although nominally one of choices is virtu ally the same as a judged certainty equivalent procedure Alternatively even if a participant compares a gamble and each monetary amount individually the estimated certainty equivalent may be a ected by the distribution of money amounts in the list Our aim was to clarify whether or not the violations of monotonicity found using judged certainty equivalents could be signi cantly reduced by using choicebased certainty equivalents In the rst experiment we used a fairly close analog to the Birnbaum et al 1992 and Mellers Weiss et al 1992 experiments Our stimuli were very similar to theirs but were displayed on a computer screen individually The certainty equivalents were estimated using three re sponse modes one direct judgment and two indirect choice procedures Although this experiment used similar stimuli and elicited judged certainty equivalents as in the previous studies we found substantially fewer violations than in the earlier experiments and indeed were left in serious doubt about whether the judgment procedure actually produces violations of consequence monotonicity To nd out whether our computer display of an individual gamble which is different from their booklet display of multiple gambles may have led to this difference we conducted a second experiment that exactly replicated the Mellers Weiss et al experiment and used both types of displays The third experiment was an attempt to reduce further the noise levels found in Experiments 1 and 2 To that end we attempted to increase the realism of the rst experiment which we hoped would increase the participants seriousness in evaluating the gambles by providing a scenario in which the gambles were described as hypothetical college stipend applications with large potential dollar awards Although the numbers of violations dropped as we had hoped considerable noise VON WINTERFELDT CHUNG LUCE AND CHO remained in the group data making the test less sensitive than one would wish The fourth and nal experiment was aimed at modeling and estimating the magnitude and nature of the noise in individuals to see if noise is suf cient to account for the observed violations of monotonicity or whether the evidence really suggests underlying violations of monotonicity Experiment 1 As was just noted the major purpose of Experiment 1 was to collect both judged and choicebased certainty equiva lents to gambles and to check consequence monotonicity for each procedure To that end we used three different response modes to estimate certainty equivalents In one participants were asked to judge directly the certainty equivalents of gambles The previous studies that used this mode had as noted earlier found substantial evidence for violations of consequence monotonicity To investigate whether conse quence monotonicity holds when certainty equivalents are derived from choice responses we used the FEST procedure much as in the study by Bostic et al 1990 in which participants were asked to choose between playing a gamble and taking a sure amount of money the certainty equivalent of a gamble was derived by systematically changing the monetary amount paired with the gamble Although we chose the PEST procedure to generate independent choice trials it is lengthy and cumbersome We therefore intro duced a second choice procedure similar to one sometimes used by decision analysts which we called QUICKINDIFF for its presumed speed Participants were asked to indicate a strength of preference for a given alternative until they reached a point of indifference between playing a given gamble and taking a sure amount of money To make our results comparable to the previous studies we adopted the same stimuli used in Birnbaum et al 1992 and Mellers Weiss et a1 1992 but we elected to display them on a computer screen individually rather than as drawings in a booklet Method Participants 39 three undergraduate students at the Univer sity of California Irvine participated in this experiment for partial credit for courses Stimuli and design The 15 stimuli are shown in Table 1 They were adopted from the study by Bimbaum et al 1992 and closely Table l Stimuli Used in Experiment I Number Gamble Number Gamble Number Gamble 1 on ns n 6 39150536 11 6 M 74 2 on 7040 7 962056 12 9 7 74 3 on so to 8 on in 13 06 so 94 4 as R0 0 9 on an a 14 on an 94 5 06 054 n 10 05 osg 15 on W 74 Note A gamble denoted as xpy means the following Obtain 1 with probability p otherwise y with probability 1 p CONSEQUENE MONO39IONIUTY 41 their dmiyt The independent variables wete ti lowest or 39 39 39 1n V hidt they would be exactly indifferent in a choice between taking that 39 39l ui 39 39 tc 524 t 30 pt 95 and the response m t noon mm ICE UICKlNDlFF PEST m highent consequence wus at Alter the mount was typed in the compute men a wit a ways 96 All of these vadahles were rmn39p in 3 within dialogue box which asked quotAre you really indi ezennquot If they 39 39 L 39 39 e were not indi etent they could go back and reenter the certainty all 15 stimuli undernll three response modes equivalent If they were satis ed with that value it was stored as S nwlus pmmiau mpam modes All gamqu and 39 39 ry equivalent for the gamble in question quotum veer in gure 3 The sum miseqnence was always on the right side and A uh we version of the gamble was also given on the left side The pin etence The display was on u I vin Ii nut with the pie segments in blue and red and with the anus showing the dollar amounts shaded in green For a CE response model the nun amount box on me right of 39 mi et all l990Ayin this displny Wm an in Figure 3 except that the my gamble I sure amount was selected from a uniform distribu tion minimum and the maximum amount of the gnmhlel Partici ants were instructed to click the 0K butmn under their Chol e either playing the glmhle er receiving e sure amourAL Once their response was entered the computer Ilgmithm I m A a 39 a 39 t r the nnge between the mlnimum an maximum cu ces of the gamble in the direction of indifferenne Far example with the stimulus in Figure 3 step is approximately 19 so if the the bulitu the bottom of Figure 3 was not disptnyed Participants judged certainty equivalent In the gnmhle in the other box The participants were told that this value was the dollar amount to iii E sum mum of 50 the next sure unaunt presented would be 31 Mounmvely if the partiu pmt indicated n pm gamble the second sure 69 50 computer stated the modi ed money amount for the next presenta 50 Ldl strong wut protoranee preturance Figure 3 Eggtpuwr tlixptay of gmnnc stimulust The human preference I QUICKIN task not in the ICE tasks QUICKINDLFF was lite quick indi I mod on sequential strengthof preietenee Judgments JCE r j ed equivalent PEST punthem estimnu on by tequcntint testing I t 39 l inatttamnca weak strong ptelerencn preference her Appeared unlx tn the 39 etence ertaiuty mtg c 412 tion of the gamble which would occur after some intervening trials consisting of other gambles 0n the second presentation of the gamble in question and the modi ed sure amount the participant again entered his or her preference and the computer calculated the next sure amount If the participant preferred either the sure amount or the gamble three times in a row the step size was doubled If the participant changed the preference either from the sure amount to the gamble or vice versa the step size was halved If a participant made a wrong choice in the sense of choosing the sure amount when it was smaller than the worst consequence of the gamble or choosing the gamble when the sure amount was larger than the best consequence of the gamble the computer alerted the participant to the mistake and asked for a revision The procedure was terminated when the step size became smaller than V50 of the range of consequences The certainty equivalent of the gamble was then estimated to be the average of the lowest accepted and the highest rejected sure amounts among the last three responses To try to keep participants from recalling previous responses to particular gambles we conducted the PEST trials as follows First there were 15 test stimuli and each was presented once before the second round of presentations was begun Second 20 ller stimuli were interspersed randomly among the test stimuli These were similar to the test gambles but had different consequences After each trial whether a test or ller stimulus there was a 50 chance that the next item would be a ller If it was one of the 20 ller stimuli was selected at random If it was a test stimulus the next one in line on the test stimulus list was selected Because of this procedure the number of ller stimuli between tests varied and more important even after most of the test stimuli were completed and so eliminated from further presentation there were still enough llers to mask the nature of the sequencing The QUICKINDIFF procedure began as did the PEST proce dure by presenting a gamble and a sure amount that was randomly chosen from the consequence range of the gamble However instead of being asked for a simple choice the participant was asked to provide a strengthofpreference judgment for the chosen alternative To assist participants in expressing their strength of preference we showed a horizontal bar below the balance beam with markers indicating varying strengths ranging from indi erence to strong preference see Figure 3 Participants simply dragged the shaded square marker in the direction of the preferred item stopping at the point that best re ected their strength of preference After each response the QUICKINDIFF algorithm calculated the next sure amount taking into account both the direction and strength of preference It did so by using a linear interpolation between the sure amount and the maximum consequence of the gamble when the preference was for the gamble and between the sure amount and the minimum of the gamble when the preference was for the sure amount The step size in the interpolation was determined by the strength of preference If that strength was large the step size was large if it was small the step size was small The algorithm also reset the ranges for the possible certainty equivalent When the sure amount was chosen that value became a new upper bound for the certainty equivalent and when the gamble was chosen the sure amount was a new lower bound for the certainty equivalent After several such trials participants tended to report indiffer ence in choosing between the gamble and the sure amount and pressed one of the 0K buttons leaving the square marker at the indifference point The program again checked by asking Are you really indifferentquot thus allowing the participant to reenter the VON WINTERFELUI CHUNG LUCE AND CHO process or exit it After exiting the nal sure amount was recorded as the estimated certainty equivalent for the gamble Procedure Participants were introduced to the experiment the display and the rst response mode Several trial items were presented and the experimenter supervised the participants re sponses to these items responded to questions and corrected obvious errors The experimenter instructed participants that they should make their judgments independently on each trial and assume that they could play the gamble but once They were encouraged to think of the amounts of money as real outcomes of real gambles To help the participant understand the stimuli presented on the computer screen the experimenter demonstrated the displayed gamble with an actual spinner device a physical disk with adjustable regions and a pointer that could spin and whose chance location in a region would determine the conse quence received All data in a response mode were collected before the next one was started The ICE response mode was presented either rst or last and the other two modes were counterbalanced The initial order of stimuli was randomized separately for each participant and each response mode To motivate the participants we placed them into a competition where the outcome was based on their responses After the experiment 10 trials were chosen randomly for each participant and scores were calculated according to the participant39s responses to these 10 trials as follows For each of the chosen trials if the participant had selected the sure amount the score was that amount otherwise the gamble was played on the spinner and its dollar outcome was the score The total score for each individual was the sum of the scores in these 10 selected trials The 3 participants who received the highest scores were paid off with a dollar amount equaling one tenth of their scores The actual payoffs determined after the experiment were 6610 6410 and 5850 for these 3 participants The experiment was run in a single session with breaks It lasted between 1 and 2 hr Results Fifteen pairs of gambles exhibited a strict consequence dominance relationship as shown in Table 2 We operation ally de ned as did Mellers Weiss et a1 1992 that a violation of monotonicity had occurred when a participant gave a higher certainty equivalent to the dominated gamble Table 2 shows the perc CONSEQUENCE MONUIONICITY Table 2 Percentages of Participants Wolating Consequence Monotonicity in Experiment I N 31 Procedure Stimulus pair PEST QUICKINDIFF JCE 23 26 16 39 52 35 58 45 39 39 42 32 39 32 3 3 6 0 3 13 16 32 32 26 42 42 39 45 39 0 0 16 10 23 10 26 19 32 35 48 39 52 35 Average 19 30 31 Note A gamble denoted as xpy means the following Obtain x with probability p otherwise y with probability 1 p PEST parameter estimation by sequential testing QUlCKlNDlFF quick indifference procedure based on sequential strengthof preference judgments JCE judged certainty equivalent the PEST and ICE data have comparable proportions of viola tions and the QUICKI39NDIFF data are somewhat worse Because the estimated certainty equivalents were quite variable among participants we chose the median rather than the mean responses to represent aggregate ndings As in Mellers Weiss et a1 1992 we plot the median certainty equivalents against the probability of winning the larger amount 96 in the gamble for the three response modes separately The results are shown in the three panels of Figure 4 Each set of connected points shows the median certainty equivalents as a function of the probability of receiving the common value of 96 The curves are identi ed by whether the complementary outcome is 0 6 or 24 According to consequence monotonicity the curve for the 24 gambles should lie wholly above that for the 6 gamble which in turn should lie wholly above that for the 0 gamble In all methods we nd a slight tendency for crossovers between the 0 and 6 curves and very few crossovers between the 24 curve and other curves We applied a twotailed Wilcoxon test to compare the certainty equivalents of each pair across participants for each value of the gamble probability p The results are shown in Table 3 None of the pairs at p 95 was signi cantly different for any of the response modes Some pairs notably those for p 05 20 or 50 showed signi cant di 39erences ie at p lt 05 or p lt 01 Because the Wilcoxon test does not indicate whether a signi cant difference is in the direction of supporting or violating consequence monotonicity we inferred this information by directly comparing the median certainty equivalents All 413 instances of signi cant Wilcoxon tests in Table 3 support consequence monotonicity Discussion Our main concern in this experiment was to compare the response mode effects in testing consequence monotonicity especially for garnbles with a high probability of receiving 96 For all response modes we found some proportions of A JCE 96px luu 80 E 395 60 g 5 4o s o I h s 6 2039 x 24 0 r 0 02 04 06 08 10 Probability to win 596 B QUICKINDIFF 3969330 100 3 GI u G s D II 2 24 o m f j 0 02 04 06 08 10 Probability to win 396 C Pasr ssspsx 199 80 quot E 395 50 C g 40 3 20 o s 6 n 524 0 a 0 02 04 06 08 10 Pmblbillty to win 596 Figure 4 Median certainty equivalents CEs for the gambles 96px 32 being one of 0 6 or 24 N 31 JCE judged uncertainty etprivalent QUICKINDIFF quick indifference proce ure on sequential strength of preference judgments PEST parameter estimation by sequential testing 414 VON WIN39I39ERFELDT CHUNG LUCE AND CHO Table 3 Z Scores for the Wilcoxon Tests of the Certainty Equivalents CEs of Gamble Pairs in Experiment 1 Stimulus pair p 05 p 20 p 50 p 80 p 95 JCE CE96p0 VS CE96p6 095 145 129 005 071 CE96P6 Vs CE96p24 377 327 l91 022 109 CES96p0 vs CE96p24 284quot39 229 29 0 1 3 077 QUICKINDIFF CE96p0 vs CE96p6 299 065 056 006 010 CE96p6 vs CE96p24 459 433 254 171 022 CE96p0 vs CE96p24 47s 456 225 L96 014 PEST CE96p0 vs CE96p6 486 263 184 112 l76 CE96P6 vs CE96p24 484 486 429 2 13 131 CE96p0 vs CE96p24 486quot 478 446 252quot 185 Note The asterisks indicate signi cant differences that are in support of the monotonicity assumption A gamble denoted as xpy means the following Obtain x with probability p otherwise y with probability 1 p JCE judged certainty equivalent QUICKINDIFF quick indifference procedure based on sequential su39engthof preference judgments PEST parameter estimation by sequential testing p lt 05 p lt 01 violations for these gambles on the whole they were slightly lower for the PEST response mode than for the JCE response mode and both of these were substantially lower than that for the QUICKINDIFF response mode This was mainly due to the clear differences in the observed propor tions of violations for the gambles of the form 96p0 and 96p6 when p was low For example for p s 50 these violations were less for the PEST procedure ranging from 0 to 35 than for the QUICKINDIFF 23 to 58 and JCE 26 to 52 procedures The smaller proportion of violations found for the PEST procedure might have arisen because the rst sure amount was chosen randomly from the range of outcomes This means that the rst sure amount for 96p24 is on average larger than that for 96p6 which in turn is on average larger than that for 96p0 Beginning the PEST sequence with a larger sure amount might lead to higher certainty equivalents as reported by Brothers 1990 which in turn would lead to a smaller proportion of violations than with the JCE procedure To check for this we reanalyzed the PEST data by eliminating all certainty equivalents whose rst sure amounts for the 96p0 were below 6 when the gamble was compared with 96p6 or below 24 when the gamble was compared with 96p24 There still was no clear pattern of violation of monotonicity The major conclusion from this experiment is that accord ing to overall violations of consequence monotonicity the methods are ordered PEST lt JCE lt QUICKINDIFF However for large values of p where the most reversals occur there is little difference between the PEST and ICE methods It may be pertinent to note that for gambles with a high proportion 80 and 95 of receiving 96 the propor tions of violations we observed under the ICE procedure 32 to 48 were substantially less than those found in the studies by Bimbaum et al 1992 60 and Mellers Weiss et al 1992 54 Although we adopted the same stimulus and response mode as in these previous studies there were some procedural differences In Experiment 2 we attempted to make a closer comparison of our procedures and those of the earlier studies The largest proportion of violations occurred under the QUICKINDIFF procedure Even though this procedure was designed to mimic an elicitation method often used by decision analysts by no means did it appear immune to violations of consequence monotonicity Perhaps this proce dure allows or even invites participants to exit the program prematurely by stating indifference between a displayed sure amount and the gamble when it is only approximately determined If a participant accepts indifference without much thought the results would show a stronger component of random error and thereby increase apparent violations of monotonicity Across all response modes most violations occur when the gambles have a high probability of winning 96 and so have very similar expected values Thus it is possible that some of these violations were due to random errors in estimating certainty equivalents The median pattern shows no crossovers for the 24 stimulus and only a minor crossover for the 6 stimulus at the 80 probability of winning 96 Furthermore the Wilcoxon tests showed that the certainty equivalents of any pair of gambles at p 95 were not signi cantly different Without further data we cannot be sure whether the obtained proportions of viola tions re ect true violations of monotonicity or result simply from random uctuations in participants responses To gain additional understanding of the relation between these methods and consequence monotonicity we needed to take several directions The rst was to try to determine CONSEQUENCE MONOTONICITY whether our somewhat lower although still high propor tion of violations than was found in earlier experiments was due to changing from a paper display of the stimuli to one on a computer monitor We did this in Experiment 2 although it was actually run after Experiment 3 A second concern was whether we could gure out a better way to capture the participants attention and thereby reduce further the noise level We did this in Experiment 3 by attempting to provide money amounts and a scenario that would be quite meaning ful in the real world And third there was the serious problem of noise in the data both between and within subjects In Experiment 4 we attempted to estimate the nature and amount of the noise we were encountering and to determine if it was suf cient to explain under the hypothesis that consequence monotonicity holds the observed viola tions Experiment 2 0f the several procedures used in Experiment 1 and also Experiment 3 see below the ICE procedure most closely resembles that uSed in earlier experiments Although this procedure produced medium to high percentages of viola tion between 32 and 48 for the most susceptible gamble pairs they are somewhat less than those found by Birnbaum et al 1992 and Mellers Weiss et al 1992 for similar stimuli and conditions neutral point of view Several procedural differences might have caused the di erence in obtained violations We displayed the stimuli on the com puter screen as opposed to in a booklet and the number of stimuli was smaller 15 stimuli than in the study by Mellers Weiss et al Although we are not sure whether the display a ected the results it is also possible that instead the smaller number of stimuli could have affected the results With a larger number of stimuli participants might have developed simplifying estimation strategies Such strategies might be expected to involve calculations that are more prone to response biases In fact Mellers Weiss et al Table 4 Stimuli Used in 2 Number Gamble Number Note 3y with probability 1 p 415 reported a smaller proportion of violations for fewer stimuli 30 or fewer In addition the nancial incentive instruc tions that 10 gambles would be played to determine the participant s score and those who won the three highest scores would become winners might have led participants to be less risk averse or even risk seeking because the payoff is the average and only the participants with the top three averages would win money Therefore in the present experiment we attempted to replicate the studies of Bim baum et a1 and Mellers Weiss et a1 both in their original booklet presentation and in our computer display Method Participants Thirtythree undergraduate students from the University of California Irvine and 10 undergraduate students from the University of Southern California participated in this experiment for partial credit for psychology courses Stimuli and design The sn39muli were those previously used and provided to us by Mellers Weiss et a1 1992 In total there were 77 gambles 45 of which were test gambles and the remaining 32 of which were llers Table 4 shows the complete list of the 45 test gambles The 45 test gambles consisted of 3 sets of 15 gambles For a gamble xpy p was either 05 20 50 80 or 95 In the first set x was 96 and y was either 0 6 or 24 In the second set x and y were 5 times their values in the rst set and in the third set they were 10 times their values in the rst set The main factor under investigation was possible differences between the booklet and computer task in testing consequence monotonicity All of the conditions were compared within subjects Each participant received all 77 gambles and performed both tasks Stimuli presentation In the replication of the Mellers Weiss et a1 1992 and Birnbaum et a 1992 experimenm the materials were presented in a sevenpage booklet in which all gambles were presented as pie charts without numerical probabilities Dollar amounts were attached to the two segments of the pie chart The booklet contained an instruction page which was followed by a page with 10 warmup stimuli The test and ller stimuli were presented on ve subsequent sheets with up to 18 stimuli on each sheet The order of the stimulus sheets in the booklet task was Gamble A gamble denoted as xpy means the following Obtain Six with probability p otherwise 416 varied according to a Latin square design Each gamble had a number associated with it The response sheet had numbered lines for the 10 warmup trials and numbered lines for all the gambles The columns were labeled Sell and Buy The participants were instructed to judge the certainty equivalents under either the seller39s point of view or the buyer39s point of view as follows Imagine that you own all the gambles in the experiment If you think the gamble is desirable write the minimum amount dollars and cents you would accept to sell the gamble in the column labeled Sell If you think the gamble is undesirable 39te the maximum amount dollars and cents you would pay to avoid playing it in the column labeled Pay Participants wrote the appropriate price for each gamble on the corresponding number on the response sheets The computer generated stimulus display was identical to that described in Experiment 1 but only the ICE response mode was used The gambles were displayed as shown in Figure 3 both with pie charts and probabilities The instructions were similar to those described in Experiment 1 except they omitted the motivation for actually playing some gambles and winning real money All 77 stimuli were presented in random order Procedure The tasks were run in separate sessions separated by at least 1 week Each session lasted for about 45 min The order of the two tasks was counterbalanced across participants Results Because we detected no notable differences between the students from the two universities we pooled their data Table 5 shows the percentages of monotonicity violations for the nine pairs of stimuli for which Mellers Weiss et al 1992 and Birnbaum et al 1992 had found the strongest violations The overall percentages of violations were about the same for the booklet task 36 as for the computer task 37 They are about the same as those in Experiment 1 and again are somewhat lower than those reported by Mellers Weiss et al and Bimbaum et al Table 5 also shows the percentages of ties which were surprisingly high in both Table 5 VON WINTERFELDT CHUNG LUCE AND CHO tasks Most ties seemed to occur because participants rounded responses for example to 95 or 90 in the 96950 gamble The panels in Figure 5 show the median certainty equivalents of the test gambles in Table 5 as a function of the probability of winning the higher amount Although there are some crossovers at the higher probability end none is striking Table 6 shows the results of Wilcoxon tests for the nine gamble pairs with p 95 for receiving the largest outcome Only one test shows a signi cant difference in the booklet task and this test supports monotonicity In the computer task two tests show signi cant differences in the direction of violating monotonicity and two show a signi cant difference in support of monotonicity All other tests have nonsigni cant p values Discussion With both the computer and booklet tasks we found some evidence of a violation of consequence monotonicity for the nine stimulus pairs for which previous experiments showed the strongest violations Apparently the introduction of the computer procedure neither decreased nor increased the number of violations The observed proportion of violations in the booklet task 36 was again somewhat smaller than that obtained by Mellers Weiss et al 1992 and Birnbaum et al 1992 The crossovers at high probability in the plot of median certainty equivalents against the probability to receive the highest outcome were less pronounced but three pairs for the computer task were signi cantly different in the direction of violations of monotonicity The percentage of ties 38 was larger than that found by Mellers Weiss et al 14 and by Bimbaum et al 28 for equivalent conditions When ties are ignored the ratio of violations to nonviolations was however similar to the ratios from the earlier studies Percentages of Participants Violating Consequence Monotonicity 0r Assigning Equal Certainty Equivalents lies in Experiment 2 N 43 J Booklet Computer Percent Percent Percent Percent Stimulus pair violation ties violation ties 96950 vs 96956 30 33 47 37 96956 vs 969524 28 44 44 35 96950 vs 969524 35 35 49 40 480950 vs 969530 26 60 28 40 48095530 vs 96953120 35 33 21 44 480950 VS 9695120 42 33 23 28 960950 vs 969560 47 33 40 35 9609560 vs 9695240 30 35 40 37 960950 vs 9695240 51 33 44 42 Average 36 38 37 38 Note A gamble denoted as xpy means the following Obtain x with probability p otherwise y with probability 1 p CONSEQUENCE MONOTONICITY 417 SEGMSX wc Computer Booklet 00 E 3 GD 5 5 4 so 0 z a 5 20 24 o 1 1 0 02 04 00 00 10 0 02 04 06 00 10 Probability to win 396 B 4aopx 500 Computer Booklet I 400 I 3 1 B 300 5 1 e 200 we r B r 3quot to I x 0 sac 30 100 km nae5120 1 390 02 04 06 00 10 d 02 04 05 00 10 Probability to win 480 96095 IUU Computer Booklet t 000 393 l 3 600 4 1 g 400 I o 30 1 i m a sso 1 3240 0 o 02 04 06 00 10 o 02 04 06 00 10 Problbllity to win 3960 Figure 5 Median jud ed certainty equivalents CBS for the computer task versus the booklet task for the gambles 96p In summary this experiment did not show clear differ ences between the computer and booklet displays in ob served proportions of violations however we did observe consistently fewer violations than in the earlier experiments These persistent differences in our experiments whatever their cause cannot be attributed to such procedural differ ences as mode of stimulus display or number of stimuli Experiment 3 As noted earlier the purpose of this experiment was to increase somewhat the realism of the situation This was achievable in at least two ways The rst way used in Experiment 1 was to play out for money some of the gambles after completion of the experiment thereby provid ing the participants an opportunity to win more or less 480px and 960pX with 11x as given in the legends N 43 money depending on their choices and on chance Our intention was to create monetary incentives but the stakes were necessarily rather modest The other way pursued in the present experiment was to create a somewhat realistic scenario for the gambles but without an actual opportunity to play The stakes can be increased under this approach but at the expense of not providing any direct actual nancial incentive For gambles and stakes we presented hypothetical stipend offers for amounts between 0 and 9600 In all other regards Experiment 3 was identical to Experiment 1 Method Participants Twentyfour undergraduate students from the University of Southern California participated in the experiment and they received a at fee of 6 per hour for their participation 418 vow WINTERFELDT CHUNG LUCE AND CHO Table 6 Z Scores for the thcoxon Tests of the Two Certainty Equivalents CEs in Experiment 2 Stimulus pair Booklet task Computer task CE96950 vs CE96956 126 202 CE96956 vs CE969524 094 2401quotl39 CE96950 vs CE969524 187 3251391 CE480950 vs CE4809530 O69 032 CE4809530 vs CE48095120 l 46 074 CE480950 vs CE48095120 075 l20 CE960950 vs CE9609560 094 l 26 CE9609560 vs CE96095240 055 094 CE960950 vs CE96095240 137 187 Note The asterisks indicate signi cant differences that are in support of the monotonicity assumption The daggers indicate si ni cant differences that are in violation of the monotonicity assumption A gamble denoted as py means the following Obtain x with probability p otherwise y with probabihty l p p lt 05 Tip lt 01 Stimulus presentation and response modes These were identi cal to those of Experiment 1 except that all dollar amounts were multiplied by 100 Design The design was identical to that of Experiment 1 Procedure Participants were introduced to the task by the experimenter They were told to imagine that they had to choose between two options for obtaining a stipend for the next semester The rst was to accept a sure o er right now for example 2400 The second was to forgo the sure offer and to apply for a second stipend that carried a larger amount 9600 however there was a chance which varied over 05 20 50 80 and 95 that the second stipend would be lowered to either 0 600 or 2400 Participants were told to imagine that the uncertainty would be resolved quickly but that once they gave up the sure stipend it would be given to someone else and they could not revert to it Table 7 For each response mode the experimenter guided the partici pants through some learning trials and then set them up to complete all of the trials of that response mode before they started the next response mode The experiment lasted from 1 V2 to 2 hr Results Table 7 shows the overall pattern of violations of conse quence monotonicity for all stimulus pairs and the three response modes The overall percentages of violations were 14 for PEST 25 for QUICKINDIFF and 17 for ICE which are smaller than those obtained in Experiment 1 and are ordered in the same way The percentages of violations were also lower for the gamble pairs with low probabilities Percentages of Participants Violating Consequence Monatonicity in Experiment 3 N 24 Stimulus pair VS VS VS V8 V8 V5 V5 V5 VS VS Average Procedure PEST QUICKINDIFF J CE 4 25 4 8 29 21 29 17 42 29 29 29 33 63 33 4 8 8 4 17 8 4 25 8 17 24 29 29 33 21 0 8 4 0 21 4 4 13 4 17 24 21 25 46 33 14 25 17 Nate with probability 1 12 PE indifference certainty equivalent A gamble denoted xpy means the following Obtain x with probability p otherwise y ST er estimation by sequential testing QUICKINDIFF quick based on sequential strengthof prefercnce judgments ICE judged CONSEQUENCE MONUI ONICITY of wimiing 9600 which were the gamble pairs with the largest expectedvalue differences For the three stimuli that most directly tested Mellers Weiss et al s 1992 violation pattern ie those that involved a 95 probability of winning 9600 the percentages were largest but again not as large as in previous studies The highest percentages of violations were found in the QUICKINDIFF procedure 33 46 and 63 followed by the closely similar JCE 21 33 and 33 and PEST 29 33 and 25 procedures The three panels of Figure 6 show the median certainty A JCE 9600px 419 equivalents as a function of the probability of receiving 9600 When p 95 we nd some crossover in the JCE mode and to a lesser degree in the QUICKINDIFF mode The crossovers do not occur at all in the PEST procedure The results of Wilcoxon tests are shown in Table 8 The certainty equivalents of any pair of gambles with p 95 were not signi cantly di erent for the JCE and QUICKIN DIFF procedures For the PEST procedure the certainty equivalents for two pairs of gambles at p 95 were signi cantly different p lt 01 but again in the direction of supporting monotonicity The other results were fairly similar to those of Experiment 1 with all signi cant results supporting rather than violating monotonicity As with the data in Experiment 1 we reanalyzed the PEST data by eliminating all rst sure things below 600 or 2400 respectively for the 0 stimulus There were no changes in the response pattern All three response modes in Experiment 3 produced fewer violations of consequence monotonicity than in Experi ment 1 For example violations in the PEST procedure were reduced from 19 to 14 in the QUICKINDIFF proce dure from 30 to 25 and in the ICE procedure from 31 to 17 The signi cant reduction of violations in the JCE mode was observed in 9600p0 vs 9600p 600 when p 05 or 20 The QUICKINDIFF procedure showed somewhat higher proportions of violations than the PEST and JCE procedures which were nearly the same The crossovers did not appear with the PEST procedure but were still present with the QUICKINDIFF and JCE procedures However none of the certainty equivalent di erences in the direction of violations of monotonicity was signi cant Overall the number of violations of monotonicity was smaller than in the studies by Mellers Weiss et a1 1992 and Birnbaum et a1 1992 except for the QUICKINDIFF procedure Because this procedure exhibited in both Experi ments 1 and 3 appreciably more violations than the ICE and PEST procedures which may have arisen from a tendency to early exitquot it was dropped from Experiment 4 Experiment 4 One general problem in studying decision making under risk is the relatively large amount of noise in the data both inconsistencies on repeated presentations within a subject and differences among subjects Although estimating cer tainty equivalents of gambles and constructing the prefer ence relation by comparing them within subjects may appear to circumvent this speci c problem the fact is that the 6000 1 E 4 i e39 Discussion E g o 50 g 2000 5600 2400 0 f o 02 04 06 08 10 Probability to win 59600 B QUICKINDIFF 9600Pix 8000 6000 3 E 4000 s 0 g 2000 to a 600 0 2400 0 02 04 06 08 10 Probability to win 35600 C PEST 9600Pix 3000 6000 1 3 8 4000 E 0 g 20001 o 80 amp 600 0 2400 o 02 04 06 03 10 Probability to win 59600 Figure 6 Median certainty equivalents CBS for the gambles 9600px B being one of 0 600 or 2400 N 24 JCE judged certainty equivalent QUICKJNDIFF nick indifference procedure based on sequential strengthofpreference judgments PEST parameter estimation by sequential testing certainty equivalents are also quite noisy and subject to similar inconsistencies These dif culties call into question the precision of inferences we can make from both J CE and PEST data In reporting the degree of violations of monotonicity we have focused as have others mainly on results from the within subject data analysis For each pair of gambles we have compared the strict numeric value of the estimated 420 you WINTERFELDT CHUNG LUCE AND CHO gaggrsesfor the Wilcoxon Tests of the Certainty Equivalents CEs of Gamble Pairs in Experiment 3 Stimulus pair p 05 p 20 p 50 p 80 p 95 329 263 042 139 064 406quot 370quot quot 36439 157 148 426 416 351 284 l 70 QUICKlNDIFF 254 I 151 280quot 168 057 380 334 3l4 272 l49 311quot 360 349 282 069 PEST 371 401 229 240 113 426quot 426 411 357 286 420 420 426 371 320 The asterisks indicate signi cant differences that are in support of the monotonicity assumption A gamble denoted xpy means the following Obtain x with probability p otherwise 3 with indi erence procedure estimation by sequential testing p lt 05 p lt 01 certainty equivalents for each individual and reported the proportion of participants who exhibited violations of mono tonicity Of course the plots of medians are across sub jects The three previous experiments showed a substantial proportion of violations 2996 4296 for the gamble pairs consisting of 96950 versus 96956 and versus 969524 However because the estimated certainty equivalents were quite noisy comparing the strict numerical value of certainty equivalents may not correctly reveal the underlying trend especially when the difference of expected values of two gambles is small as it obviously is in these cases When the median certainty equivalents of two gambles were compared as a function of probability to win 96 Figures 46 there were no consistent crossovers and the Wilcoxon tests showed very few signi cant differences of the two certainty equivalents across subjects Thus we need some means of estimating and taking into account the noise in individual certainty equivalents This can be done in two ways First one can obtain re measures of certainty equivalents for each gamble for each individual and apply some statistical analysis Second in comparing the observed certainty equivalents one can devise a noise model and estimate the proportion of viola tions to be expected under that model and compare those predictions with the observed violations Because the rst approach is fairly timeconsuming especially using the PEST procedure and is subject to other confounding effects ty 1 p ICE judged certainty equivalent QUICKlNDIFF based on sequential strengthof preference judgments PEST parameter qu1c such as the development of response strategies we at tempted the second approach Statistical Model of Noise To this end we somewhat mimicked a method used by Slovic and Lichtenstein 1968 in studying the importance of outcome variances They compared the certainty equivalents of two gambles jointly received called duplex gambles with the certainty equivalent of a single rstorder gamble that was the convolution of the two For that purpose they used the testretest certainty equivalent differences to estab lish a benchmark of statistical variability So we conducted a fourth experiment in which each certainty equivalence was estimated twice at least 1 week apart for each gamble and we used these data to construct a simple model to estimate the magnitude of noisegenerated violations of mono tonicity For any pair of test gambles g and h where h dominates g let EV g and EVh denote their expected values CE1g and CEh their certainty equivalents estimated from the rst session and 0523 and CEh those estimated from the second session Because h dominates g EVh gt EV g A violation of monotonicity occurs when the observed certainty equivalents satisfy CEh lt CE g The question is whether the proportion of these violations is to be expected CONSEQUENCE MONOTONICITY under the null hypothesis that consequence monotonicity holds but while the data are as noisy as they are The goal is to estimate the distribution corresponding to that null hypothesis In doing so we encounter three major issues First what family of distributions should we assume The distributions over the certainty equivalents are almost certainty not normal for these garnbles For example the distribution of the certainty equivalent of 9695x with x ltlt 96 probably has a mean of about 80 to 90 is truncated to the right by 96 and has a long lefthand tail However we are not interested in this distribution but in the difference distribution of two such similarly skewed distribu tions Assuming two such distributions skewed as de scribed we simulated the difference distributions and found them to be approximately symmetrical and very near to normal We therefore assume that the noise distribution of the certainty equivalent difference is normal Second what do we take to be the mean difference in certainty equivalents under the null hypothesis of conse quence monotonicity One approach is to assume that the participants are close to expectedvalue maximizers and thus that the mean certainty equivalent difference is close to the difference in expected values However for most partici pants the observed certainty equivalents were substantially less than the expected value So an alternate assumption is that they are expectedutility maximizers with a sharply riskaverse utility function In this case the certainty equiva lence difference is roughly three times the expected value difference We therefore conducted two noise analyses one using as the mean the difference of the two expected values and the other using as the mean three times that Third what do we take as the standard deviation of the difference Here we estimate it from the observed differ ences in test retest values of the same gamble A12 gg 0518 0928 and Al2hh CErUl C5201 and then use that estimate to predict the proportion of noiseinduced violations There are two complications in estimating the variance First as we shall see the CEs are systematically somewhat larger than the CEzs That being so plus some concern about memory effects within a single session we therefore pooled all of the differences A2gg Ar2 023 CElg and Amhh CE1h CE2h and calculated the usual variance estimate for these pooled observations Second the judgment data exhibited a considerable number of tied responses These ties seemed to occur because participants often restricted their responses to either the highest conse quence 96 or to nearby multiples of 5 eg 95 or 90 Because any simple continuous noise model does not allow ties to occur we need to estimate the proportions of ties and exclude them in the analysis of noise To predict the fraction q of ties we assumed that the proportion of ties between CE g and CEh would be equal to the pooled proportion of ties in the testretest certainty equivalents of each gamble that is 392ties ofCElg and calm ties ofCE1h and calm Because of differences between the ICE and PEST proce 421 dures we applied different criteria in predicting the ties in test and retest certainty equivalents For the ICE method equal was de ned to be exactly equal dollar amounts For the PEST procedure equal was de ned to be an interval based on the termination rule that we imposed in estimating the certainty equivalent of each gamble The reason for doing this is that it is impossible given the termination rule of the PEST procedure to tell the actual sign of the difference closer than within this interval Speci cally let R g and Rh denote the ranges of the test gambles g and h respectively Then we treated any di erence lCEl g CE2g lt lie50 and lCEh CE h lt RhSO as atie The obtained proportions of ties between CEg and CE h were computed as the pooled proportion of ties in the test and retest certainty equivalents Aties ofCE1g and CE2h ties ofCEh and 023 Again we necessarily applied a di emnt criterion for the two response modes As above for the ICE mode equal was de ned to be exactly equal dollar amounts For the PEST mode a tie was de ned in terms of the following interval 10518 CEzh of 10310 CEz8i lt MR RID50 Excluding the ties from consideration we assume for each gamble pair and each participant that the differences A1hg 05101 CE2g and A1 0amp0 0323 are both distributed normally with mean EVh EV g or 3 X EVlt EVg and variance cr2 estimated as the variance of the pooled across subject values of the two testretest differences A2 1gg and Al2hh The predicted proportion of violations of monotonicity namely PrCEh CE g lt 0 was calculated based on these assumptions for each participant and each gamble pair and these proportions were then averaged over the participants If we let q denote the estimated proportion of ties then the predicted proportion of monotonicity violations excluding the predicted ties from consideration is 1 qPrCEh CEg lt 0 Method Participants Thirtytwo undergraduate students from the Uni versity of Southem California participated in this experiment They were paid 750 per hour for their service Stimulus presentation and response modes The stimuli were nine of the test stimuli used in Experiment 2 see Table 9 plus 20 ller items for the PEST procedure All stimuli were presented on a computer monitor that generated stimulus displays as described in Experiment 1 see Figure 3 they were presented in random order Procedure Except for omitting the selection of 10 gambles and playing them for money we followed the same procedure as in Experiment 1 for both the ICE and PEST procedures Their order was counterbalanced across participants The experiment was run in two sessions each lasting about 45 min that were separated by at least 1 week Each session was the same except for independent randomization 422 VON WINTERFELDT CHUNG LUCE AND CHO Table 9 Descriptive Statistics of the Test Retest Certainty Equivalents CEs in Experiment 4 N 32 Mean CEs Median CES d Gamble EV Time 1 Time 2 Time 1 Time 2 M SD t31 JCE 740 728 800 800 1 13 050 703 723 775 800 2 17 065 723 740 750 800 2 19 052 361 349 400 350 13 87 082 363 343 400 350 20 94 121 369 358 390 360 11 72 088 752 720 800 750 32 175 1 03 750 695 812 700 55 167 187 750 725 800 738 25 183 076 PEST 72 71 790 805 l l 1 062 74 72 805 800 2 14 077 76 73 858 795 3 13 115 343 325 377 371 17 95 103 356 333 396 370 23 101 130 373 351 394 386 22 86 143 685 609 757 653 76 251 171 690 652 801 787 38 210 102 740 685 829 729 56 209 151 Note A gamble denoted xpy means the following Obtain x with probability p otherwise y dged with probability 1 p EV expected value if denotes test retest difference score JCE ju certainty equivalent PEST parameter estimation by sequential testing Results Table 9 summarizes statistics of the test retest assess ments expected values the means and medians of the certainty equivalents and the means and standard deviations of the differences of certainty equivalents over the two sessions Note that the estimated certainty equivalents are typically smaller than the corresponding expected values and as was noted earlier the differences in expected value for the crucial monotonicity tests are very small Overall both the mean and median certainty equivalents are slightly higher in the rst test than in the second one although none of the differences in each gamble pair is signi cant The certainty equivalents assessed using the ICE procedure are similar to those assessed by the PEST procedure The Wilcoxon test Table 10 showed no signi cant violation of consequence monotonicity for the JCE procedure and only one signi cant violation for the PEST procedure Table 11 shows the predicted proportion of ties estimated from the test retest data the predicted violations and nonviolations estimated from the noise model and the corresponding observed proportions for the JCE method In this table the mean of the noise distribution was assumed to be the difference between the expected values of the two gambles The expected and observed proportions are very close The chisquare values testing differences between observed and predicted violations nonviolarions and ties are not signi cant for any gamble Table 12 shows the same results under the assumption that the mean of the noise distribution is three times as large as the expected value difference As expected the predicted proportions of viola tions are reduced and those of nonviolations correspond ingly increased Nonetheless the results visa vis conse quence monotonicity are essentially identical to those of Table 11 Tables 13 and 14 show the same analyses for the PEST procedure The proportion of ties is somewhat larger here Table 10 Z Scores for the Wilcoxon Tests of the 7100 Certainty Equivalents CEs in Experiment 4 z score Stimulus pair I CE PEST vs CE96956 173 295 vs CE969524 021 151 vs CE969524 072 292 vs CE4809530 073 248 vs CE48095120 053 2251 vs CE48095120 103 299 vs CE9609560 024 136 vs CE96095240 080 189 vs CE96095240 059 252 Note The asterisks indicate signi th differences that are in support of the monotonicity assumption The dagger indicates a signi cant difference in violation of the monotonicity assumption A gamble denoted xpy means the following Obtain x with probability p otherwise 32 with probability 1 p JCE judged certainty equivalent PEST parameter estimation by sequential testing Tp lt 05 quotp lt 01 CONSEQUENCE MONO39I ONICITY 423 Table 11 Expected and Observed Percentages of Nonviolation Violation and Ties in Monotonicity Tests for the JCE Task Using EVh EVg as the Mean of the Noise Distribution N 32 Nonviolation Violation Tie Gamble pair 3 vs h Expected Observed Expected Observed Expemed Observed x22 vs 96956 43 as 42 48 16 14 065 vs 969524 46 50 42 36 13 14 0 48 vs 969524 46 45 41 39 13 16 0 29 vs 4809530 44 39 43 45 13 16 0 48 vs 48095120 44 47 40 38 16 16 0 13 vs 480953120 45 52 40 39 16 1 14 vs 9609560 45 38 44 53 11 9 1 11 vs 96095240 47 56 43 39 9 5 1 45 vs 96095240 47 50 42 38 11 13 0 30 Average 45 46 42 42 13 12 Note A chi square of 599 is signi cant at the 05 level A gamble denoted xpy means the following Obtain 1 with probability p otherwise y widt probability 1 p JCE judged certainty equivalent than for the ICE procedure which is due to the less stringent de nition of a tie However the overall conclusion is the same The noise model provides an excellent t to the observed proportions of nonviolations violations and ties Because we elicited certainty equivalents twice for each gamble the participants had two opportunities to violate monotonicity Table 15 tabulates the observed proportion of 0 l and 2 violations for the two procedures The most striking result of this tabulation is how rarely participants violated monotonicity twice especially with the PEST procedure Discussion Experiment 4 yielded violations of monotonicity in about the same proportions as we found in our earlier experiments For example there were 37 violations for the J CE procedure and 34 for the PEST procedure at p 95 in Experiment 1 versus 42 and 31 respectively in Experi ment 4 Again both JCE values are somewhat smaller than those reported by Mellers Weiss et a1 1992 and Bimbaum et a1 1992 We still are not sure why this is so although a Table 12 speculation is offered below A noise model based on test retest certainty equivalent estimates for the same gambles predicted 12 ties for the JCE procedure and 20 for the PEST procedure If we exclude lies from consideration the observed proportions of violations and nonviolations were not signi cantly different from those predicted on the basis of our noise model Moreover participants rarely violated monotonicity twice Our conclusion is that given the unfortu nately noisy character of the data the observed violations of consequence monotonicity are to a large extent consistent with the underlying hypothesis of consequence mono tonicity Conclusions Earlier tests of consequence monotonicity one of the most fundamental assumptions of numerous theories of decision making under uncertainty provided mixed re sults The earliest based on a conjecture of Allais 1953 was ambiguous because both monotonicity and reduction of compound gambles were involved and it was not clear which was the source of trouble When two gambles in the Expected and Observed Percentages of Nonviolation Violation and Ties in Monotonicity Tests for the JCE Task Using 3 X EVh EVg as the Mean of the Noise Distribution N 32 Nonviolation Violation Tie Gamble pair 3 vs h Expected Observed Expected Observed Expected Observed x22 963550 vs 96956 44 38 40 48 16 14 0 92 96956 vs 969524 49 50 38 36 l3 l4 0 12 1969550 vs 969524 52 45 36 39 13 16 0 58 i4809530 vs 4809S30 45 39 42 45 13 16 0 62 t48095 30 vs 48095120 48 47 37 38 16 16 0 01 gt480955 0 vs 48095120 50 52 35 39 16 9 l 00 596095gt0 vs 9609560 46 38 43 53 ll 9 1 44 060 954601 vs 96095240 51 56 4O 39 9 5 0 04 0m 05 1 vs 9609S240 52 50 37 38 ll 13 0 09 Average 4 46 39 42 13 12 Note A chisquare of 599 is signi cant at the 05 level A gamble denoted xpy means the following Obtain 1 with probability p otherwise 5y with probability 1 p JCE judged certainty equivalent EV expected value 424 vow WINTERFELDT CHUNG LUCE AND CHO Table 13 Expected and Observed Percentages of Nonviolation Wolatian and Ties in Monotonicity Tests for the PEST Task Using EVh EVg as the Mean of the Noise Distribution N 32 Nonviolation Violation Tie Gamble pair g vs h Expected Observed Expected Observed Expected Observed x22 vs 96956 37 41 36 33 27 27 0 18 vs 969524 39 42 36 28 25 30 0 84 vs 969524 40 48 34 28 27 23 1 05 vs 4809530 43 47 42 39 14 14 0 17 vs 48095120 43 55 40 34 17 11 l 99 vs 48095120 46 58 41 34 13 8 1 91 vs 9609560 36 47 36 31 28 22 1 60 vs 96095240 36 48 34 27 30 25 2 04 vs 96095240 4O 59 37 22 23 19 5 27 Average 40 49 37 31 23 20 Note A chisquare of 599 is signi cant at the 05 level PEST parameter estimation by sequential testing A gamble denoted xpy means the following Obtain x with probability p otherwise y with probability 1 p EV expected value consequence monotonicity condition were compared di rectly the monotonicity assumption was not rejected Broth ers 1990 Bimbaum amp Sutton 1992 Kahneman amp Tversky 1979 However when research participants directly judged the certainty equivalents of gambles and the preference relations of gamble pairs were constructed indirectly by comparing the estimated certainty equivalents consequence monotonicity appeared to be consistently violated espe cially for gambles involving very small probabilities of receiving zero or nearzero outcomes Bimbaum 1992 Bimbaum amp Sutton 1992 Bimbaum et al 1992 Mellers Weiss et al 1992 The present series of four experiments was conducted in an attempt to clarify the relation between the choice and the judged certainty equivalent results We conjectured that monotonicity violations may be more prevalent in judged certainty data than in choice data because judged certainty equivalents possibly involve simpli fying calculations We were motivated then to test the dependence of violations on response modes We studied three judged certainty equivalents or ICE and two types of choicebased ones PEST and QUICKINDIFF Although the Table 14 JCE response mode in Experiment 1 produced more viola tions than the other two for the gambles with low probabili ties of receiving 96 this pattern was not replicated in Experiment 3 where the stakes were increased by a factor of 100 and a somewhat more realistic scenario was used In both experiments all response modes produced some frac tion of violations for the gambles when the probability of receiving the maximum outcome was large There were no noticeable differences in the proportion of observed viola tions between the ICE and PEST response modes When the median estimated certainty equivalents of two gambles were compared as a function of probability to win 96 or 9600 there were no consistent crossovers and the results of Wilcoxon tests did not reject consequence monotonicity between the two certainty equivalents for any of the gamble pairs in Experiments 1 and 3 Indeed for the PEST procedure the Wilcoxon tests provided signi cant support for consequence monotonicity in 10 of 15 tests in Experi ment 1 and in 14 of 15 tests in Experiment 3 with all other tests not being signi cant The QUICKINDIFF response mode was an attempt to Expected and Observed Percentages afNonviolatian Violation and Ties in Monatonicity Tests for the PEST Task Using 3 X EV h EVg as the Mean of the Noise Distribution N 32 Nonviolation Violation Tie Gamble pair g vs h Expected Observed Expected Observed Expected Observed x20 vs 96956 39 41 35 33 27 27 0 05 vs 969524 43 42 32 28 25 30 0 42 vs 969524 45 48 28 28 27 23 0 19 vs 4809530 45 47 41 39 14 14 0 08 vs 48095120 46 55 37 34 17 11 1 28 vs 48095120 51 58 37 34 13 8 0 94 vs 9609560 37 47 35 31 28 22 1 40 vs 96095240 39 48 32 27 30 25 1 26 vs 96095240 43 59 34 22 23 19 3 58 Average 43 49 35 31 23 20 Note A chisquare of 599 is signi cant at the 05 level A gamble denoted xpy means the following Obtain x with probability p otherwise y with probability 1 p PEST parameter estimation by sequential testing EV expected value CONSEQUENCE MONOTONICI39I Y 425 Table 15 Observed Distribution of 0 1 and 2 Violations of the Monotonicity Assumption N 32 Observed violation count Gamble pair 0 1 2 JCE 396950 vs 96956 5 23 4 06 9 55 vs 969524 ll 19 2 06 Msm vs 969524 11 17 4 51480950 vs 4809530 7 23 2 b4809530 vs 48095120 13 15 4 54803530 vs 480953120 10 19 3 59609530 vs 9609560 5 22 5 3596035560 vs 96095240 10 20 2 39609550 vs 96095240 10 20 2 Average 9 20 3 PEST vs 96956 9 23 0 vs 969524 12 19 1 vs 969524 14 16 2 vs 4809530 4 28 0 vs 48095120 9 22 1 vs 480 95120 10 21 1 vs 960 560 10 2 1 VS 96095240 l4 l7 1 vs 96095240 15 17 0 Average 11 20 1 Note A gamble denoted xpy means Obtain 0 with probabil ity p otherwise 3y with probability 1 p JCE judged certainty equivalent PEST parameter estimation by sequential testing develop a faster procedure than that provided by PEST for estimating the choiceinduced certainty equivalents How ever QUICKINDIFF exhibited appreciably more violations of consequence monotonicity than the other two procedures Perhaps this is because it easily allows participants to exit the program before actual indifference is established We cannot recommend its use as now formulated but further investigation of procedures faster than PEST is needed Another fast technique that is sometimes used Bimbaum et al 1992 Tversky amp Kahneman 1992 to estimate certainty equivalents involves participants marking which of a list of sure consequences are better than a gamble Although this is nominally choice based we are suspicious that it is a judgment procedure in disguise Unexpectedly in both Experiments 1 and 3 under the J CE procedure a smaller proportion of violations was exhibited than was reported in the earlier studies by Birnbaum et a1 1992 and Mellers Weiss et al 1992 To investigate the possibility that our using a computer monitor presentation rather than a booklet of drawings of pie diagrams might have been the source of this difference we conducted Experiment 2 to compare the two JCE presentations We again found lower proportions of violations in b0th presentations than in the earlier studies In the booklet task the Wilcoxon tests showed no signi cant violations of monotonicity for any of the gamble pairs that were suspected to produce violations In the computer task two of the nine tests showed signi cant violations of monotonicity So the question remains Why are we getting smaller nonsigni cant proportions of viola tions One possibility which we did not investigate is that the earlier data were collected in classroom situations whereas ours were in individual laboratory settings Perhaps the personal attention given to our participants to ensure that they understood what was to be done may have made them more careful To test whether the observed proportions of violations of both ICE and PEST response modes really can be explained simply by the apparently inherent noise in estimating certainty equivalents we proposed a noise model utilizing the test retest data collected in Experiment 4 When ties were excluded from consideration the predicted proportions of violations from the noise model were not signi cantly different from those observed in both the JCE and PEST procedures In addition when participants had two opportu nities to violate consequence monotonicity with the same gamble pair they very rarely did so Our conclusion therefore is that when the noisy nature of the data are taken into account we cannot reject the hypothesis of conse quence monotonicity Although we were unable to reject consequence monoto nicity for either the judged or the PESTbased certainty equivalents using our noise model that model has at least two rather severe limitations First we used test retest data to estimate the number of ties to be added to the continuous noise model These same data were also used to estimate the variance of the underlying noise distribution across subjects The ideal way to analyze the noise underlying our estimates of certainty equivalents would be to repeatedly measure it and to apply statistical tests for each individual Although ideal it is impractical in this context especially the PEST one to obtain a suf ciently large sample of individual test retest data One likely dif culty in pursuing this ap proach lies in how to reduce memory effects without using many ller gambles In the PEST procedure it is impossible to test a large set of gambles in one or two successive sessions Second the certainty equivalents estimated in the retest session which was performed at least 1 week later than the test session were consistently less than those of the test ones It would be interesting to obtain the test retest certainty equivalents in the same session or if that is not feasible in sessions separated by less time than a week Despite a fairly large proportion of observed violations of consequence monotonicity which seems to be largely due to the level of noise in estimating them we conclude from the several analyses that consequence monotonicity cannot be rejected This is true for both judged certainty equivalents and PESTdetermined certainty equivalents The fact that there is little difference in the proportions of violations of consequence monotonicity between the two procedures and the greater ease of collecting JCE judgments seem to recommend the JCE procedure over the PEST procedure Nonetheless we urge caution in using judged certainty equivalents to test choicebased theories of decision making For one thing the median data slightly favor the PEST Elementary lntegrals Alex Sadovsky January 237 2008 1 Review If a function x is de ned on an interval 0117 the integral ab can sometimes be computed using the relevant part of the Fundamental Theorem of Calculus That part says that if is the antiderivative of x ie7 if F x then b mm Fan e M I Not every x has an antiderivative Example x 2M However7 in your second term of calculus you will be given problems where the antiderivative can be found Here are some examples of such problems Example 11 b sinx3x2dx 1 Example 12 b sin100xcosxdx 1 Example 13 b xcosxdx 1 Example 14 b em sinxdx 1 Example 15 17 2x1 d a x2x710 Copyright by Alex Sadovsky C 2000 Example 16 b tanmdz 1 We now consider several basic types of such problems and for each type give a general strategy of attack For each type we will end up using either directly or indirectly the formula 1 given in the next paragraph The above quoted part of the Fundamental Theorem of Calculus shows that if hz is a differ entiable function on la b then b Ham 111 7 ha 1 1 Recall that the expression to be integrated is called the integrand and that the variable you are integrating with respect to is called the variable of integration 2 Substitutions Suppose the integrand is of the form f gmg m where f and g are differentiable functions on la b The problem therefore is to nd 17 rltgltzgtgt ltzgtdz 1 Reading the Chain Rule backwards we have f 9mg m lf9ml 7 so the problem reduces to evaluating ifltgltzgtgtrdz a Taking hz fgm in formula 1 we get impWm f9a e f9b lt2 An alternative method is to change the variable of integration Take u 996 Then d u 7 dm 7 9 90 so du gmdm The original problem which was now takes the form u9b fudu lt3 u9a Notice that a change of the variable of integration here we changed from x to u results in changing also the limits of integration here from a b to gagb Some people call the latter method u substution We will call it simply substitution We now consider several special cases where a substitution is required 21 The integrand gx g x Where 71 7E 71 Here fm 90 Therefore 1 men1 and the problem is solved by using either 2 or m some constant 22 The Logarithmic Derivative g xgx Suppose the integrand has the form g zgz where g is a differentiable function on la b that takes only positive or only negative values on la b In this case we take WE 111990 We see that the integrand is exactly h z so the problem is to nd hxdz and formula 1 applies once again tf mfwmwmwwemmw 900 3 Partial Fraction Decomposition Suppose the integrand is of the form where P and Q are polynomials so the problem is to nd b 1 We assume that P has lower degree than Q does otherwise we reduce the problem to this case by doing long division It turns out that in this case we can express as a sum of functions each of which falls under either 21 or 22 For a concise but complete description of the actual procedure see my note on partial fractions dm 4 Integration by Parts If f and g are differentiable functions on la b and if we take this time a product as opposed to a composition then We f g aw90

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

#### "Selling my MCAT study guides and notes has been a great source of side revenue while I'm in school. Some months I'm making over $500! Plus, it makes me happy knowing that I'm helping future med students with their MCAT."

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

#### "It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.