Complete Cases in R (3 Programming Examples)

A complete data set (i.e. data without any missing values) is essential for many types of data analysis in the programming language R.

In order to deal with missing data, it is crucial to find missing values and to identify observations in your data without any missings.

I’ll show you in this article how to handle missing values in R with the complete.cases function. Select the specific topic you are interested in:

Example 1: Data Frame Example 2: Vector Example 3: Real Data Video Examples Questions or Comments?

 

Example 1: Find Complete Rows of a Data Frame

The complete.cases function is often used to identify complete rows of a data frame.

Consider the following example data:

data <- data.frame(x1 = c(7, 2, 1, NA, 9), # Some example data
                   x2 = c(1, 3, 1, 9, NA),
                   x3 = c(NA, 8, 8, NA, 5))
data # This is how our example data looks like

Example Data for complete.ceases Function in R

Table 1: Incomplete Example Data

 

We can use complete.cases() to print a logical vector that indicates complete and missing rows (i.e. rows without NA).

Rows 2 and 3 are complete; Rows 1, 4, and 5 have one or more missing values.

complete.cases(data)
# [1] FALSE  TRUE  TRUE FALSE FALSE

 

We can also create a complete subset of our example data by using the complete.cases function.

This process is sometimes called listwise deletion:

data[complete.cases(data), ] # Keep only the complete rows
data_complete <- data[complete.cases(data), ] # Store the complete cases subset in a new data frame

 

Note that such a complete case data set might consist of a much smaller sample size compared to our original incomplete data.

In our example, data_complete consists of only 2 rows. Rows 1, 4, and 5 were deleted.

For that reason, it might be worth to conduct some more sophisticated missing data techniques such as a missing value imputation or a simple replace of missing data by zero or a variable’s mean.

Did you have problems to understand the previous code? No problem! I have recorded a video, in which I’m explaining the previous example in more detail:

 

 

Example 2: Apply the complete.cases Function to a Vector

The complete cases function can also be applied to vectors or columns (even though the is.na function is more popular for this task).

Similar to Example 1, the function returns a logical vector (TRUE = observed; FALSE = missing value).

set.seed(10101) # Set seed in order to create a reproducible example
vec <- round(runif(20, 0, 10)) # Create example vector
vec[rbinom(20, 1, 0.2) == 1] <- NA # Insert some NA values to the vector
 
complete.cases(vec) # The R programming language uses for vectors the same procedure as for data frames
# [1] FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
# [2] TRUE  TRUE FALSE  TRUE  TRUE FALSE  TRUE FALSE FALSE  TRUE
 
# Delete missing values and store the complete vector in the new object
vec_completevec_complete <- vec[complete.cases(vec)]

 

Example 3: How to Use the Complete Cases Function for Real Data

The 2 examples above illustrate the usage of the complete cases function on the basis of synthetic data.

However, the real world is different and therefore I’m going to show you now how to find observed and missing values in a real database.

# Load and inspect data
data(airquality) # Load the data set airquality
dim(airquality) # The data has 153 rows and 6 columns
head(airquality) # Head of data; Missing values are, for instance, in column 1 & 2 in row 5
 
# Check the whole data frame for missing values
complete.cases(airquality) # TRUE indicates a complete row; FALSE indicates a row with at least
                           # one incomplete column
sum(complete.cases(airquality)) # We have 111 complete rows
airquality_complete <- airquality[complete.cases(airquality), ] # Create new data without missing values
 
# Find incomplete cases in a column
complete.cases(airquality$Ozone) # By adding $Ozone behind airquality,
                                 # we identify observed values in the column Ozone
sum(complete.cases(airquality$Ozone)) # We have 116 observations
Ozone_complete <- airquality$Ozone[complete.cases(airquality$Ozone)] # Exclude missing data
                                                                     # from our column vector

 

Example Video: Find Complete Cases in R

You need more examples? So be it!

In the following YouTube video, the speaker Dragonfly Statistics explains how to check a real data set for complete cases (he also uses the airquality data set which I used in Example 3).

He shows several examples in the R programming language.

 

 

Now it’s On You

I showed you how I’m applying the complete cases function in RStudio.

What are your thoughts? Will you identify your complete data like me or do you know a better approach?

Did you have any problems with the complete cases function that I didn’t cover in this article?

I’d love to hear about your experiences in the comments!

 

Further Reading

 



 

Appendix

The graphic of the header of this site shows a data frame with missing and observed values (indicated by TRUE and FALSE).

The graphic was created with the R programming language as follows.

set.seed(34756) # Set seed
 
data_header <- data.frame(Household_ID = runif(100), # Some dummy data
                          Sex = runif(100),
                          Age = runif(100),
                          Nationality = runif(100),
                          Year = runif(100),
                          Health = runif(100),
                          CoB = runif(100),
                          Income = runif(100),
                          Household_Size = runif(100),
                          Holidays = runif(100),
                          Marital_Status = runif(100),
                          Expenditure = runif(100))
 
data_header$Sex[rbinom(100, 1, 0.1) == 1] <- NA # Insert NA's
data_header$Age[rbinom(100, 1, 0.15) == 1] <- NA
data_header$Nationality[rbinom(100, 1, 0.1) == 1] <- NA
data_header$Year[1:7] <- NA
data_header$Health[rbinom(100, 1, 0.25) == 1] <- NA
data_header$Income[rbinom(100, 1, 0.4) == 1] <- NA
data_header$Marital_Status[rbinom(100, 1, 0.05) == 1] <- NA
data_header$Expenditure[rbinom(100, 1, 0.25) == 1] <- NA
 
data_logical <- as.data.frame(is.na(data_header) == FALSE) # Check for missing data
data_logical

 

11 Comments. Leave new

SHRINIVAS DHARMADHIKARI
February 13, 2019 2:01 pm

Very comprehensive treatment Indeed. Thanks alot

Reply
SHRINIVAS DHARMADHIKARI
February 14, 2019 1:31 pm

Hello Joachim
After using na.omit, I am still getting the following result
> dm1 table(dm1)
dm1
no yes
330 60 13

Why this NA reappears again?

Reply

    Hi Shrinivas,

    if you apply the following code, your NAs should be removed:

    dm1_updated <- dm1[complete.cases(dm1), ]

    Reply
SHRINIVAS DHARMADHIKARI
February 14, 2019 2:36 pm

Hello Jo
I am losing my confidence. Am I so dumb ? See I tried earlier what you told me and got stuck as follows
dm1 dm dm1 length(dm1)
[1] 403
> table(dm1)
dm1
no yes
330 60 13
> sum(complete.cases(dm1))
[1] 403
> dm1 table(dm1)
dm1
no yes
330 60 13

Reply
SHRINIVAS DHARMADHIKARI
February 14, 2019 2:40 pm

Not getting it at all. See as under:
dm1 <- dm[complete.cases(dm), ]
Error in `[.default`(dm, complete.cases(dm), ) :
incorrect number of dimensions

Reply

    Is dm1 a vector or a data.frame? You can check that with class(dm1). If it is a vector, you can try: dm1_updated <- dm1[complete.cases(dm1)] (without the comma at the end)

    Reply
SHRINIVAS DHARMADHIKARI
February 14, 2019 3:28 pm

This is what I got

> class(dm1)
[1] “factor”
> dm1_updated table(dm1_updated)
dm1_updated
no yes
330 60 13

Reply

    Your data seems to be a one-dimensional vector and not a two-dimensional table/data.frame. Furthermore, it seems like your missing values are stored as “” instead of NA. Try the following:

    dm1_updated <- dm1; dm1_updated[dm1_updated == ""] <- NA; dm1_updated <- dm1[complete.cases(dm1)]; sum(is.na(dm1_updated)); # The result should be 0

    Reply
SHRINIVAS DHARMADHIKARI
February 14, 2019 4:45 pm

Jo

Sorry forgot to mention. dm is a column vector in a data frame. Any change in your advise?

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.

Menu