Understanding P-values and How to Interpret Them

With an example using Stata and a publicly available data

Photo designed by the Author using canva.com

You tried to find answers through research publications but the articles you are reading seem to be written in codes that make no sense to you, you learned the term P-values in a statistics or an econometrics class, or you have trouble understanding what it is and the meaning researchers try to portray through it. Either way, you are yearning to understand this term and set your quest to fully comprehend it. I am here today to explain what P-value is, demonstrate it through an example using Stata, and explain how to interpret it.

What are P-Values?

Before we deep dive into P-values, let us first understand hypothesis testing. A hypothesis is a suggested statement made without supporting evidence as a starting point for further analysis. Hypothesis testing is where a researcher examines a supposition about a population or a sample parameter. It is a way of checking whether we can have confidence in results from a regression analysis or not. There are two types of hypothesis; the null and the alternative hypothesis. The null hypothesis is used to test the assumption that the result is not significant or not different from zero, or a model without some of the independent variables fits the data better than when those variables are included, or two variables have no effect on each other, or that the difference between an observed means of two variables in a study is in fact zero. It varies from one study to another and it is denoted as H0. The alternative hypothesis is used to test against the null hypothesis; the result is significant or different from zero, or a model with all the independent variables fits the data better than when those variables are excluded, or one variable in a study impacts the other, or the difference between an observed means of two variables is not zero and there is a real difference. It also depends on the hypothesis of interest in a study and is denoted as H1. The purpose of hypothesis testing is to avoid two errors commonly known as type I (an error committed when you reject a correct null hypothesis) and type II (an error committed when you do not reject a null hypothesis that should have been rejected).

P-value is a probability approach to hypothesis testing that states the minimum or the smallest level of significance for which the null hypothesis can be rejected. The good thing with the probability approach is that most statistical software automatically produce this P-value whenever you run a regression.

Example Using Stata

Here, I am using the SLEEP75 data (which you can download from here if you want to follow along) from Biddle and Hamermesh (1990) to study whether there is a tradeoff between the time spent sleeping per week and the time spent in paid work. Assume our hypothesis of interest, the null hypothesis, states that time spent on paid work does not affect time spent sleeping. Such that the alternative hypothesis states that time spent on paid work affects time spent sleeping per week.

• Exploring the Data

Below is the code to explore your data in Stata.

describe

The above code produces the output in the image below.

A description of the data

• Computing Summary Statistics

The sleep and totwrk are the only variables relevant to this example. Use the two lines of code below to get the summary statistics of these variables.

sum sleep
sum totwrk

The summary statistics of these variables are shown in the images below.

Summary statistics of the sleep variable in the data

Summary statistics of the totwrk variable in the data

• Running a regression analysis

To run regression analysis in Stata use the example code below.

reg Y Xs

Y stands for the dependent variable and Xs are the independent variables. This example has only one control variable; totwrk while the response variable is sleep.

Here is the regression result image from Stata:

Report of regression result from Stata

Interpretation of P-values

This data, as the regression report shows, has 706 observations. The coefficient of the totwrk variable tells us that if time spent on paid work increases by 1 minute, time spent on sleeping per week will fall by 0.15 minutes. In addition, this result is statistically significant as the p-value is zero. Therefore we can reject the null hypothesis and conclude that an increase in time spent on paid work has a significant negative impact on sleeping time per week.

Generally, a p-value of 0.000 shows that the results are statistically significant at the 1% significance level. In other words, there is less than one in a hundred chance that this result is entirely a chance finding. A p-value of 0.03, means there is a 3% chance that you can reject the null hypothesis. So, you can reject the null hypothesis at the 5% level but not at the 1% significance level. Similarly, a p-value of 0.09 tells us that there is a 9% chance that we can reject our null hypothesis. Therefore, we can reject the null hypothesis only at the 10% significance level but not below. On the other hand, a p-value of 0.12 signals a 12% chance that we can reject a null hypothesis — which is above the most commonly used values of alpha (1%, 5%, and 10%). Thus, we cannot reject the null hypothesis in such a case.

Some academic papers use stars to indicate statistically significant results and include a footnote under tables. Three stars (***) show that the observed result is statistically different from zero with a p-value less than 1%. Two stars (**) denote a result that is less than one in a twenty or 5% chance of a chance finding. One star (*) indicates that there is less than one in a ten or 10% chance that such a result is merely due to chance.

Thanks for reading! I hope it helped you. Feel free to check out some of my other related blog posts below.

• Interpreting Different Regression Models
• A Beginner’s Guide to R