Databricks Certified Machine Learning - Associate
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
You are given a Spark DataFrame 'df' with a numerical column 'height'. Write a code snippet that removes outliers from the 'height' column based on the interquartile range (IQR) method, and explain the steps involved.
A
Q1 = df.height.quantile(0.25) Q3 = df.height.quantile(0.75) IQR = Q3 - Q1 lower_bound = Q1 - 1.5 * IQR upper_bound = Q3 + 1.5 * IQR df = df.filter((df.height >= lower_bound) & (df.height <= upper_bound)) print(A)
B
Q1 = df.height.approxQuantile(0.25, 0.01) Q3 = df.height.approxQuantile(0.75, 0.01) IQR = Q3 - Q1 lower_bound = Q1 - 1.5 * IQR upper_bound = Q3 + 1.5 * IQR df = df.filter((df.height > lower_bound) & (df.height < upper_bound)) print(B)
C
Q1 = df.height.percentile(0.25) Q3 = df.height.percentile(0.75) IQR = Q3 - Q1 lower_bound = Q1 - 1.5 * IQR upper_bound = Q3 + 1.5 * IQR df = df.filter((df.height > lower_bound) & (df.height < upper_bound)) print(C)
D
Q1 = df.height.approxQuantile(0.25, 0.01) Q3 = df.height.approxQuantile(0.75, 0.01) IQR = Q3 + Q1 lower_bound = Q1 - 1.5 * IQR upper_bound = Q3 + 1.5 * IQR df = df.filter((df.height >= lower_bound) & (df.height <= upper_bound)) print(D)