Five Things You Must Know Before Choosing a Statistical Test
Choosing the correct statistical test is vital to make sure your results are valid. I recently took a class through the University of York on five things that can help you choose the best test. You can read more about them below.
I also used a tool they provided as inspiration for creating my own tool for choosing the best statistical test. Here is the original tool and here the tool I created. The tool I made also contains additional information about the assumptions that should be met for each test.
1. Your Hypothesis
Statistical tests are generally done to determine some relationship between variables. You might be checking to see if:
- there is a difference between variables or conditions
- one or more variables has an affect on another variable
- there is a correlation between variables
To decide this, we usually write a hypothesis to check. This is often written in the form of a null hypothesis. A null hypothesis basically assumes that there is no relationship between the variables. (In other words, there is some difference, affect or correlation between the variables.) The alternative hypothesis is that there is a relationship between the variables. Your test results will tell you if you can reject the null hypothesis and accept the alternative. If you cannot reject the null hypothesis, there is no relationship between the variables you are testing.
2. What types of variables do you have?
If you are testing to see if one variable or more variables affect another variable, have dependent and independent variables. The dependent variable is the one that is affected. The independent variables are the ones that cause the effect. However, you might also just be checking to see if there is a correlation between the variables, with no assumption of one affecting the other.
3. What type of data your variables are?
Most statistical tests only work with specific data types, so you must know what type of data each of your variables is.
Your data will be one of the following types:
- Nominal: This is also called categorical data. Data divided into groups that differ by some characteristics are of this type. For example, jobs that people have would be categorical data.
- Ordinal: This data is categorical, but the categories have a ranking. One obviously comes before another. It often comes from people giving ratings on a scale and can sometimes be recorded as numbers.
- Continuous data: This is numerical data. There are two types:
- * Interval: These are numbers that measure something without an absolute zero. This means the zero does not imply the non-existence of something. Temperature is an example of this type of data. You can add and subtract this data, but multiplication and division do not make sense.
- * Ratio: These numbers measure something that does have an absolute zero. Addition, subtraction, multiplication, and division make sense with this type of measurement.
4. Does your data contain repeated measures or paired data?
Sometimes we want to see how something changes over time, so we measure the same thing for the same subjects over a period of time. This would be repeated measures. Paired data would be if the groups in your variables can be paired. For example, if you were testing the same subject before and after an event or were doing a study on twins.
5. How many groups does each of your nominal variables have?
In addition to the five items above, you might also need to know if your variables have a normal distribution, a similar variance, or outliers. Most statistical software will have tests that allow you to check for these. You might also use a graph to check these. Histograms can be a quick way to look for a normal distribution, and box plots can help you see outliers.
Once you have the answers to the five questions above, you can use one of the tools below to help you decide on the best test for your data.
Statistics Test Picker (The website I created)
University of York Statistics Test Tool
The Laerd Statistic website has a lot of good information about the test assumptions. If you will be doing a lot of statistics tests, it might be worth subscribing to. However, I found that I could get information for many tests by searching for the test name and including the word “Laerd”.
