ML: Data Moments

In this article, I will explain you what data moments are and how to find them with different approaches.

Data Moments:

Moments are mainly help us to describe the distribution of the data. Also, these can provide us the useful information about the spread, shape and center of a distribution.

Mean, variance, skewness and kurtosis are the four data moments.

Mean: It mainly measures the central location of the distribution i.e., gives us an idea of where the data is centered in a distribution.

Get Leela Jagadeeswar P’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Variance: It measures the spread of values in the distribution or how far from the normal value (central point of respective data column) i.e., how the data is dispersed around the mean.

Skewness: It measures the how asymmetric the distribution is about its mean.

The Gaussian curve has 0 skewness. So, anything which is not bell-shaped (Gaussian Distributed) is skewed. The skewness happens when the data distribution is biased either to the left side or right side.

We can find the skewness value by implementing using NumPy (Fisher-Pearson Coefficient) or through the SciPy stats library.

Kurtosis: Kurtosis describes the peakedness or flatness of the distribution. It measures the amount in the tails and outliers since the peakedness is directly related to the outliers.

We can find the kurtosis value by implementing using NumPy (Karl-Pearson Coefficient) or through the SciPy stats library.

# Data Moments with Numpy Library:
def data_moments_np(df, li):
    for col in li:
        col_df = df[[col]]
        col_df["_dif1"] = col_df[col] - np.mean(col_df[col])
        col_df["_dif2"] = col_df["_dif1"] ** 2
        col_df["_dif3"] = col_df["_dif1"] ** 3
        col_df["_dif4"] = col_df["_dif1"] ** 4
        _mu2 = np.mean(col_df["_dif2"])
        _mu3 = np.mean(col_df["_dif3"])
        _mu4 = np.mean(col_df["_dif4"])

        beta1 = (_mu3 ** 2)/(_mu2 ** 3)
        skewness_np = np.sqrt(beta1)

        beta2 = (_mu4)/(_mu2 ** 2)
        kurtosis_np = beta2 - 3
        
        print("Mean of the column     " + col + " is " + str(col_df[col].mean()))
        print("Median of the column   " + col + " is " + str(col_df[col].median()))
        print("Mode of the column     " + col + " is " + str(col_df[col].mode()[0]))
        print("Variance is            " + col + " is " + str(col_df[col].var()))
        print("SD of the column       " + col + " is " + str(col_df[col].std()))
        print("Skewness of the column " + col + " is " + str(skewness_np))
        print("Kurtosis of the column " + col + " is " + str(kurtosis_np))
        print()

# Data Moments with Scipy Library:
def data_moments_sc(df, li):
    for col in li:
        print("Mean of the column     " + col + " is " + str(df[col].mean()))
        print("Median of the column   " + col + " is " + str(df[col].median()))
        print("Mode of the column     " + col + " is " + str(df[col].mode()[0]))
        print("Variance is            " + col + " is " + str(df[col].var()))
        print("SD of the column       " + col + " is " + str(df[col].std()))
        print("Skewness of the column " + col + " is " + str(skew(df[col])))
        print("Kurtosis of the column " + col + " is " + str(kurtosis(df[col])))
        print()