Refutation of Statistics
One may attack the particular statistics evidence offered by an opponent in the presentation of his case. That evidence or information may simply be false. On the other hand, it may be true information but incorrectly interpreted.
How does one correctly interpret statistical evidence? To begin with, we use statistical evidence only when we cannot get at something directly. The group of things being examined is called the population and the portion that I directly examine is called the sample. When we draw a conclusion about the population based upon the sample, we are making a statistical inference.
If the inference is to be of any help, then the sample must not be biased or loaded. Another way of putting it is that the sample must be random in the sense that it represents a cross section of the whole population.
The most devastating blow that can be delivered against statistical evidence is the claim that is not a random sample, i.e., that the sample is biased or unrepresentative.
The trick to use here is to declare, no matter what, that your opponent's statistics are not based upon a random sample. Find a factor, any factor, and claim that the factor is crucial and has been overlooked.
Some examples follow:
Is the sample representative in time? (You can always say that last year's statistics are out of date.)
Is the sample representative in space? (Did you interview people on the first floor, the second floor, basements, and so forth? This may seem irrelevant, but who is to say for sure?)
Is the sample representative economically?
Is the sample representative geographically? (People in the Bible Belt? People in Eastern urban areas?)
Is the sample representative by sex? By marital status? By race?
Is the sample representative by age?
The same technique of emphasizing a missing crucial factor may be used in attacking any graphs or charts used. Supplying the missing factor or factors, if possible, may enable you either to undermine the graphs presented or to present contrary graphs.
The second way of attacking your opponent's statistics is to present a different set of statistics, one that controverts the original set. At the very least, this has the effect of neutralizing any advantage he may have gained from his initial presentation; if you have successfully challenged his statistics in a manner outlined above, then your counter-statistical evidence will prevail.
If you cannot find a flaw in your opponent's statistics and if you cannot present counter-statistics, then you must engage in a general attack on the use of statistics itself. You can do this by first mentioning to your audience that every sophisticated person knows that statistical information may be misused. This is true you then proceed to exemplify some cases of how statistics are misused.
To begin with, statistical information is information about a class of people, items, events, things, or the like. It is not really information about any individual person, item, event, or thing. Hence it is always possible that what is true of the group is not necessarily true of any individual member of that group.
If your opponent has presented statistical evidence about an individual, you can always construct some evidence to prove the opposite.
For example:
75% of slum dwellers are criminal types.
Smith is a slum dweller.
_____________________________________________
Therefore, it is highly probable that Smith is a criminal type.
To counter this argument and to show how ridiculous it is to use statistics, you present the following statistical argument.
1% of all Jehovah's Witnesses are criminal types.
Smith is a Jehovah's Witness.
_______________________________________________
Therefore, it is highly improbable that Smith is a criminal type.
While on the topic of statistical evidence we might point out some things about the use of probability. There is, to begin with, a long history of controversy within philosophy about how to interpret probability. Is probability an attribute of events, statements, or the attribute of the agent (gambler or social perspective?)
For our purposes it should be noted that the use of the calculus or probability employs mathematics and therefore gives the specious air of science to something that is questionable. The actual mathematical of probability is based upon situations where are know all of the possible outcome, such as 52 cards in a deck.
Moreover, when we calculate the complex probabilities in terms of the value of simpler, related ones we should keep in mind that the original weighting comes from judgments made outside of mathematics. Even the use of relative frequency, which states "what has happened in the past" (e.g., the probability of precipitation in tomorrow's weather), can be questioned on the grounds that it is not a reliable guide to unique events (like tomorrow's weather) or that the notion of a random sample is not clear.
How does one correctly interpret statistical evidence? To begin with, we use statistical evidence only when we cannot get at something directly. The group of things being examined is called the population and the portion that I directly examine is called the sample. When we draw a conclusion about the population based upon the sample, we are making a statistical inference.
If the inference is to be of any help, then the sample must not be biased or loaded. Another way of putting it is that the sample must be random in the sense that it represents a cross section of the whole population.
The most devastating blow that can be delivered against statistical evidence is the claim that is not a random sample, i.e., that the sample is biased or unrepresentative.
The trick to use here is to declare, no matter what, that your opponent's statistics are not based upon a random sample. Find a factor, any factor, and claim that the factor is crucial and has been overlooked.
Some examples follow:
Is the sample representative in time? (You can always say that last year's statistics are out of date.)
Is the sample representative in space? (Did you interview people on the first floor, the second floor, basements, and so forth? This may seem irrelevant, but who is to say for sure?)
Is the sample representative economically?
Is the sample representative geographically? (People in the Bible Belt? People in Eastern urban areas?)
Is the sample representative by sex? By marital status? By race?
Is the sample representative by age?
The same technique of emphasizing a missing crucial factor may be used in attacking any graphs or charts used. Supplying the missing factor or factors, if possible, may enable you either to undermine the graphs presented or to present contrary graphs.
The second way of attacking your opponent's statistics is to present a different set of statistics, one that controverts the original set. At the very least, this has the effect of neutralizing any advantage he may have gained from his initial presentation; if you have successfully challenged his statistics in a manner outlined above, then your counter-statistical evidence will prevail.
If you cannot find a flaw in your opponent's statistics and if you cannot present counter-statistics, then you must engage in a general attack on the use of statistics itself. You can do this by first mentioning to your audience that every sophisticated person knows that statistical information may be misused. This is true you then proceed to exemplify some cases of how statistics are misused.
To begin with, statistical information is information about a class of people, items, events, things, or the like. It is not really information about any individual person, item, event, or thing. Hence it is always possible that what is true of the group is not necessarily true of any individual member of that group.
If your opponent has presented statistical evidence about an individual, you can always construct some evidence to prove the opposite.
For example:
75% of slum dwellers are criminal types.
Smith is a slum dweller.
_____________________________________________
Therefore, it is highly probable that Smith is a criminal type.
To counter this argument and to show how ridiculous it is to use statistics, you present the following statistical argument.
1% of all Jehovah's Witnesses are criminal types.
Smith is a Jehovah's Witness.
_______________________________________________
Therefore, it is highly improbable that Smith is a criminal type.
While on the topic of statistical evidence we might point out some things about the use of probability. There is, to begin with, a long history of controversy within philosophy about how to interpret probability. Is probability an attribute of events, statements, or the attribute of the agent (gambler or social perspective?)
For our purposes it should be noted that the use of the calculus or probability employs mathematics and therefore gives the specious air of science to something that is questionable. The actual mathematical of probability is based upon situations where are know all of the possible outcome, such as 52 cards in a deck.
Moreover, when we calculate the complex probabilities in terms of the value of simpler, related ones we should keep in mind that the original weighting comes from judgments made outside of mathematics. Even the use of relative frequency, which states "what has happened in the past" (e.g., the probability of precipitation in tomorrow's weather), can be questioned on the grounds that it is not a reliable guide to unique events (like tomorrow's weather) or that the notion of a random sample is not clear.
Comments