Also Known as: Biased Statistics, Loaded Sample, Prejudiced Statistics, Prejudiced Sample, Loaded Statistics, Biased Induction, Biased Generalization, Unrepresentative Sample
The fallacy is committed when the sample of A's is likely to be biased in some manner. A sample is biased or loaded when the method used to take the sample is likely to result in a sample that does not adequately represent the population from which it is drawn. It has the following form:
Sample S, which is biased, is taken from population P.
The person committing the fallacy is misusing the following type of reasoning, which is known variously as Inductive Generalization, Generalization, and Statistical Generalization:
X% of all observed A's are B''s.
Biased samples are generally not very reliable. As a blatant case, imagine that a person is taking a sample from a truckload of small colored balls, some of which are metal and some of which are plastic. If he used a magnet to select his sample, then his sample would include a disproportionate number of metal balls (after all, the sample will probably be made up entirely of the metal balls). In this case, any conclusions he might draw about the whole population of balls would be unreliable since he would have few or no plastic balls in the sample.
The general idea is that biased samples are less likely to contain numbers proportional to the whole population. For example, if a person wants to find out what most Americans thought about gun control, a poll taken at an NRA meeting would be a biased sample.
Since the Biased Sample fallacy is committed when the sample (the observed instances) is biased or loaded, it is important to have samples that are not biased making a generalization. The best way to do this is to take samples in ways that avoid bias. There are, in general, three types of samples that are aimed at avoiding bias. The general idea is that these methods (when used properly) will result in a sample that matches the whole population fairly closely. The three types of samples are as follows
Random Sample: This is a sample that is taken in such a way that nothing but chance determines which members of the population are selected for the sample. Ideally, any individual member of the population has the same chance as being selected as any other. This type of sample avoids being biased because a biased sample is one that is taken in such a way that some members of the population have a significantly greater chance of being selected for the sample than other members. Unfortunately, creating an ideal random sample is often very difficult.Stratified Sample: This is a sample that is taken by using the following steps: 1) The relevant strata (population subgroups) are identified, 2) The number of members in each stratum is determined and 3) A random sample is taken from each stratum in exact proportion to its size. This method is obviously most useful when dealing with stratified populations. For example, a person's income often influences how she votes, so when conducting a presidential poll it would be a good idea to take a stratified sample using economic classes as the basis for determining the strata. This method avoids loaded samples by (ideally) ensuring that each stratum of the population is adequately represented. Time Lapse Sample: This type of sample is taken by taking a stratified or random sample and then taking at least one more sample with a significant lapse of time between them. After the two samples are taken, they can be compared for changes. This method of sample taking is very important when making predictions. A prediction based on only one sample is likely to be a Hasty Generalization (because the sample is likely to be too small to cover past, present and future populations) or a Biased Sample (because the sample will only include instances from one time period).
People often commit Biased Sample because of bias or prejudice. For example, a person might intentionally or unintentionally seek out people or events that support his bias. As an example, a person who is pushing a particular scientific theory might tend to gather samples that are biased in favor of that theory.
People also commonly commit this fallacy because of laziness or sloppiness. It is very easy to simply take a sample from what happens to be easily available rather than taking the time and effort to generate an adequate sample and draw a justified conclusion.
It is important to keep in mind that bias is relative to the purpose of the sample. For example, if Bill wanted to know what NRA members thought about a gun control law, then taking a sample at a NRA meeting would not be biased. However, if Bill wanted to determine what Americans in general thought about the law, then a sample taken at an NRA meeting would be biased.