The discussion of “raising the bar” in Steve Evans’s Friday Firestorm #24 last month at BCC got me to thinking about what the possible effects of this more stringent missionary screening policy might be.
The screening process that includes interviews with a missionary candidate’s Bishop or Branch President can result in two types of errors. A candidate can be approved to serve a full-time mission when he or she should not have been, or a candidate can be kept home when in fact he or she was qualified to serve. If the goal of the screening process is thought of as a medical test diagnosing “shouldn’t serve syndrome,” the first kind of error would be a failure to diagnose a true case (a miss), and the second kind would be diagnosing someone who isn’t a case (a false alarm).
So what does raising the bar mean for these two types of errors?
Consider Figure 1.
In this figure, I’ve made up data for 100 hypothetical missionary candidates. Each has a score on Bishop or Branch president rating (on the X-axis, running from low on the left to high on the right) as well as on actual performance as a missionary (on the Y-axis, running from low at the bottom to high at the top). The vertical line running through the plot divides the candidates qualified by their Bishop or Branch President rating from those not qualified. The similar horizontal line divides the candidates actually qualified by their performance from those actually not qualified.
These two lines also divide the figure into four quadrants. The upper left quadrant has the false alarms, the candidates who are above the line in terms of actual performance, but below (to the left of) the line by Bishop or Branch President’s rating. The lower right quadrant has the misses, the, the candidates who are below the line in terms of actual performance, but above (to the right of) the line by Bishop or Branch President’s rating. The two types of correct decisions are also labeled. The upper right quadrant has the correct negatives, candidates who are qualified by both criteria, and the lower right quadrant has the hits, candidates who are below the line by both criteria.
You may notice that the cutoff line for actual performance is more stringent than is the cutoff line for the Bishop or Branch President rating. Twenty five candidates fall below the performance cutoff, but only 10 fall below the rating cutoff. The result of this difference is that there are far more misses (18) than false alarms (3). Although the numbers are of course made up, if I understand Elder Ballard correctly, this is how things used to be, with too many unqualified candidates being sent on missions.
So what happens when the bar is raised, when the screening cutoff is moved up to more closely match the performance cutoff? This situation is shown in Figure 2.
Now there are 25 missionaries falling below each cutoff, and the number of misses equals the number of false alarms (11 each). While this perfect symmetry of errors is unlikely to be the case in real life, the result is clear. The effect of raising the bar is to decrease misses while increasing false alarms.
In setting a cutoff for a screening test like the one for being allowed to serve a full-time mission, a major consideration is how harmful the two types of errors–misses and false alarms–are compared to each other. For example, in medicine, a preliminary screening test for a disease might use a very low cutoff because false alarms are much less costly than are misses. If you tell someone they don’t have a disease and you’re wrong (a miss), they could get really sick, die, or sue you. If you tell someone they do have a disease and you’re wrong (a false alarm), it’s much less costly. Sure, the person will be irritated, but all they’ll likely have to suffer through is a more expensive and more accurate test, which will likely tell them correctly that they don’t have the disease.
By contrast, William Blackstone’s famous preference for allowing ten guilty people go free rather than punish one innocent person implies that false alarms (false convictions) are ten times as bad as misses (erroneous acquittals). Although the ten to one ratio of importance may not be exactly how they’re treated, I think criminal law still has the preference that misses are better than false alarms.
By “raising the bar,” I think the Church is making a decision similar to the one in the medical screening example, figuring that misses are more costly than false alarms. In other words, it’s better to incorrectly keep a qualified missionary home than it is to incorrectly send an unqualified missionary out.
To think about how this conclusion might have been reached, I’ve listed some possible costs of the two kinds of errors.
Possible costs of misses (unqualified missionaries sent out):
- They don’t do missionary work. They waste the resources of the Church missionary structure.
- They make the Church look bad. They behave badly in front of members, investigators, and the general public.
- They change the norms for the missionaries around them, making it appear more acceptable to not work very hard.
- They suck up the time of their companions, preventing them from doing missionary work.
- They suck up the time of mission leaders, who spend time trying to get them to work harder or more effectively.
Possible costs of false alarms (qualified missionaries kept home):
- Fewer people are converted. Even poor missionaries may help convert some people.
- Candidates kept home may feel mistreated by the Church, may become bitter, and may leave Church activity.
- Missionary service helps prepare future leaders. Candidates kept home may be less effective as leaders, or less likely to serve as leaders. The fewer missionaries who serve, the smaller the pool of returned missionaries to draw on later for leadership positions.
- The norm of having all young men serve a mission may erode, making it more difficult to get qualified missionary candidates to serve.
It appears to me that the conclusion that misses are more costly than false alarms is a reasonable one. At the very least, a false alarm’s costs are pretty much borne by a single person, while a miss has the potential to harm many people around the “missed” candidate.
But what do you think? Which possible costs are the most damaging? What are other possible costs of the two types of errors? Have I misstated any of those that I have listed?
- 12 January 2008