Surveys

An observational study is one in which data is collected on individuals in a way that doesn’t affect them. The most common observational study is the survey. Surveys are questionnaires that are presented to individuals who have been selected from a population of interest. Surveys take many different forms: paper surveys sent through the mail, questionnaires on Web sites, call-in polls conducted by TV networks, phone surveys, and so on.

If conducted properly, surveys can be very useful tools for getting information. However, if not conducted properly, surveys can result in bogus information. Some problems include improper wording of questions, which can be misleading, lack of response by people who were selected to participate, or failure to include an entire group of the population. These potential problems mean a survey has to be well thought out before it’s given.

Many researchers spend a great deal of time and money to do good surveys, and you’ll know (by the criteria I discuss in Chapter 16) that you can trust them. However, as you are besieged with so many different types of surveys found in the media, in the workplace, and in many of your classes, you need to be able to quickly examine and critique how a survey was designed and conducted and be able to point out specific problems in a well-informed way. The tools you need for sorting through surveys are found in Chapter 16.

Experiments

An experiment imposes one or more treatments on the participants in such a way that clear comparisons can be made. After the treatments are applied, the responses are recorded. For example, to study the effect of drug dosage on blood pressure, one group may take 10 mg of the drug, and another group may take 20 mg. Typically, a control group is also involved, in which subjects each receive a fake treatment (a sugar pill, for example), or a standard, nonexperimental treatment (like the existing drugs given to AIDS patients.)

Good and credible experiments are designed to minimize bias, collect lots of good data, and make appropriate comparisons (treatment group versus control group). Some potential problems that occur with experiments include researchers and/or subjects who know which treatment they got, factors not controlled for in the study that affect the outcome (such as weight of the subject when studying drug dosage), or lack of a control group (leaving no baseline to compare the results with).

But when designed correctly, an experiment can help a researcher establish a cause-and-effect relationship if the difference in responses between the treatment group and the control group is statistically significant (unlikely to have occurred just by chance).

Experiments are credited with helping to create and test drugs, determining best practices for making and preparing foods, and evaluating whether a new treatment can cure a disease, or at least reduce its impact. Our quality of life has certainly been improved through the use of well-designed experiments. However, not all experiments are well-designed, and your ability to determine which results are credible and which results are incredible (pun intended) is critical, especially when the findings are very important to you. All the info you need to know about experiments and how to evaluate them is found in Chapter 17.

Selecting a good sample

Statisticians have a saying, “Garbage in equals garbage out.” If you select your subjects (the individuals who will participate in your study) in a way that is biased — that is, favoring certain individuals or groups of individuals — then your results will also be biased. It’s that simple.

Suppose Bob wants to know the opinions of people in your city regarding a proposed casino. Bob goes to the mall with his clipboard and asks people who walk by to give their opinions. What’s wrong with that? Well, Bob is only going to get the opinions of a) people who shop at that mall; b) on that particular day; c) at that particular time; d) and who take the time to respond.

Those circumstances are too restrictive — those folks don’t represent a cross section of the city. Similarly, Bob could put up a Web site survey and ask people to use it to vote. However, only people who know about the site, have Internet access, and want to respond will give him data, and typically only those with strong opinions will go to such trouble. In the end, all Bob has is a bunch of biased data on individuals that don’t represent the city at all.

To minimize bias in a survey, the key word is random. You need to select your sample of individuals randomly — that is, with some type of “draw names out of a hat” process. Scientists use a variety of methods to select individuals at random, and you see how they do it in Chapter 16.

Note that in designing an experiment, collecting a random sample of people and asking them to participate often isn’t ethical because experiments impose a treatment on the subjects. What you do is send out requests for volunteers to come to you. Then you make sure the volunteers you select from the group represent the population of interest and that the data is well collected on those individuals so the results can be projected to a larger group. You see how that’s done in Chapter 17.

After going through Chapters 16 and 17, you’ll know how to dig down and analyze others’ methods for selecting samples and even be able to design a plan you can use to select a sample. In the end, you’ll know when to say “Garbage in equals garbage out.”

Avoiding bias in your data

Bias is the systematic favoritism of certain individuals or certain responses. Bias is the nemesis of statisticians, and they do everything they can to minimize it. Want an example of bias? Say you’re conducting a phone survey on job satisfaction of Americans; if you call people at home during the day between 9 a.m. and 5 p.m., you miss out on everyone who works during the day. Maybe day workers are more satisfied than night workers.

You have to watch for bias when collecting survey data. For instance: Some surveys are too long — what if someone stops answering questions halfway through? Or what if they give you misinformation and tell you they make $100,000 a year instead of $45,000? What if they give you answers that aren’t on your list of possible answers? A host of problems can occur when collecting survey data, and you need to be able to pinpoint those problems.

Experiments are sometimes even more challenging when it comes to bias and collecting data. Suppose you want to test blood pressure; what if the instrument you’re using breaks during the experiment? What if someone quits the experiment halfway through? What if something happens during the experiment to distract the subjects or the researchers? Or they can’t find a vein when they have to do a blood test exactly one hour after a dose of a drug is given? These problems are just some examples of what can go wrong in data collection for experiments, and you have to be ready to look for and find these problems.

After you go through Chapter 16 (on samples and surveys) and Chapter 17 (on experiments), you’ll be able to select samples and collect data in an unbiased way, being sensitive to little things that can really influence the results. And you’ll have the ability to evaluate the credibility of statistical results and to be heard, because you’ll know what you’re talking about.

Descriptive statistics

Descriptive statistics are numbers that describe a data set in terms of its important features:

• If the data are categorical (where individuals are placed into groups, such as gender or political affiliation), they are typically summarized using the number of individuals in each group (called the frequency) or the percentage of individuals in each group (called the relative frequency).
• Numerical data represent measurements or counts, where the actual numbers have meaning (such as height and weight). With numerical data, more features can be summarized besides the number or percentage in each group. Some of these features include
  • Measures of center (in other words, where is the “middle” of the data?)
  • Measures of spread (how diverse or how concentrated are the data around the center?)
  • If appropriate, numbers that measure the relationship between two variables (such as height and weight)

Some descriptive statistics are more appropriate than others in certain situations; for example, the average isn’t always the best measure of the center of a data set; the median is often a better choice. And the standard deviation isn’t the only measure of variability on the block; the interquartile range has excellent qualities too. You need to be able to discern, interpret, and evaluate the types of descriptive statistics being presented to you on a daily basis and to know when a more appropriate statistic is in order.

The descriptive statistics you see most often are calculated, interpreted, compared, and evaluated in Chapter 5. These commonly used descriptive statistics include frequencies and relative frequencies (counts and percents) for categorical data; and the mean, median, standard deviation, percentiles, and their combinations for numerical data.

Charts and graphs

Data is summarized in a visual way using charts and/or graphs. These are displays that are organized to give you a big picture of the data in a flash and/or to zoom in on a particular result that was found. In this world of quick information and mini-sound bites, graphs and charts are commonplace. Most graphs and charts make their points clearly, effectively, and fairly; however, they can leave room for too much poetic license, and as a result, can expose you to a high number of misleading and incorrect graphs and charts.

In Chapters 6 and 7, I cover the major types of graphs and charts used to summarize both categorical and numerical data (see the preceding section for more about these types of data). You see how to make them, what their purposes are, and how to interpret the results. I also show you lots of ways that graphs and charts can be made to be misleading and how you can quickly spot the problems. It’s a matter of being able to say “Wait a minute here! That’s not right!” and knowing why not. Here are some highlights:

• Some of the basic graphs used for categorical data include pie charts and bar graphs, which break down variables, such as gender or which applications are used on teens’ cellphones. A bar graph, for example, may display opinions on an issue using five bars labeled in order from “Strongly Disagree” up through “Strongly Agree.” Chapter 6 gives you all the important info on making, interpreting, and, most importantly, evaluating these charts and graphs for fairness. You may be surprised to see how much can go wrong with a simple bar chart.
• For numerical data such as height, weight, time, or amount, a different type of graph is needed. Graphs called histograms and boxplots are used to summarize numerical data, and they can be very informative, providing excellent on-the-spot information about a data set. But of course they also can be misleading, either by chance or even by design. (See Chapter 7 for the scoop.)

You’re going to run across charts and graphs every day — you can open a newspaper and probably find several graphs without even looking hard. Having a statistician’s magnifying glass to help you interpret the information is critical so that you can spot misleading graphs before you draw the wrong conclusions and possibly act on them. All the tools you need are ready for you in Chapter 6 (for categorical data) and Chapter 7 (for numerical data).