df1: Column 1 Column 2 Column 3 Column 4 Row 1 1.0 2.0 NaN 4.0 Row 2 5.0 6.0 7.0 8.0 Row 3 9.0 NaN 11.0 12.0 Row 4 NaN 14.0 15.0 16.0 Row 5 NaN 18.0 19.0 NaN df1.isnull: Column 1 Column 2 Column 3 Column 4 Row 1 False False True False Row 2 False False False False Row 3 False True False False Row 4 True False False False Row 5 True False False True
As we can see the non-null values are indicated as False and the null values are indicated using True in the output of the DataFrame.isnull() function.
At this point, if we compute DataFrame.isnull().mean(), the True values in the DataFrame will contribute 1 to the sum and the False values will contribute 0. As a result, the mean value will indicate: (the total number of True values)/(the total number of True + False values). If we multiply this value with 100, we will get the percentage of null values in each column of the DataFrame.
import pandas import numpy list1 = [[1, 2, numpy.nan, 4], [5, 6, 7, 8], [9, numpy.nan, 11, 12], [numpy.nan, 14, 15, 16], [numpy.nan, 18, 19, numpy.nan]] df1 = pandas.DataFrame(list1, index=["Row 1", "Row 2", "Row 3", "Row 4", "Row 5"], columns=["Column 1", "Column 2", "Column 3", "Column 4"]) print("df1: \n", df1) print("df1.isnull: \n", df1.isnull()) print("Percentage of missing values: \n", df1.isnull().mean()*100)
The output of the above program will be like the following:
df1: Column 1 Column 2 Column 3 Column 4 Row 1 1.0 2.0 NaN 4.0 Row 2 5.0 6.0 7.0 8.0 Row 3 9.0 NaN 11.0 12.0 Row 4 NaN 14.0 15.0 16.0 Row 5 NaN 18.0 19.0 NaN df1.isnull: Column 1 Column 2 Column 3 Column 4 Row 1 False False True False Row 2 False False False False Row 3 False True False False Row 4 True False False False Row 5 True False False True Percentage of missing values: Column 1 40.0 Column 2 20.0 Column 3 20.0 Column 4 20.0 dtype: float64
Hence, columns 1, 2, 3, and 4 contain 40%, 20%, 20%, and 20% missing values, respectively.
Now, let’s compute the percentage of missing values in the titanic dataset. We can use the following Python code to compute the …






0 Comments