Answer-first summary for fast verification
Answer: The split() operation does not accomplish the requested task. The explode() operation should be used instead.
The code uses `split()` on the `productCategories` column, which is intended to split a string into an array of substrings based on a delimiter. However, the sample data shows `productCategories` as an array type (e.g., `[netus, pellentes...]`). The goal is to create one row per element in the array, which requires the `explode()` function. Using `split()` here is incorrect because it does not transform array elements into rows; instead, `explode()` is the correct operation to achieve the desired result of expanding array elements into individual rows. The other options either suggest irrelevant operations (e.g., `broadcast()`, `array_distinct()`) or misunderstand the usage of `split()` (e.g., missing alias or method invocation).
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
No comments yet.
Identify the error in the following code snippet intended to transform the productCategories column (containing arrays of strings) into individual rows with one word per row, and explain how to fix it:
storesDF.withColumn("productCategories", split(col("productCategories")))
storesDF.withColumn("productCategories", split(col("productCategories")))
Given this sample of storesDF:
storeId | productCategories
------- | -----------------
0 | [netus, pellentes...]
1 | [consequat enim,...]
2 | [massa, a, vitae,...]
3 | [aliquam, donec...]
4 | [condimentum, fer...]
5 | [viverra habitan...]
storeId | productCategories
------- | -----------------
0 | [netus, pellentes...]
1 | [consequat enim,...]
2 | [massa, a, vitae,...]
3 | [aliquam, donec...]
4 | [condimentum, fer...]
5 | [viverra habitan...]
A
The split() operation does not accomplish the requested task in the way that it is used. It should be used provided an alias.
B
The split() operation does not accomplish the requested task. The broadcast() operation should be used instead.
C
The split() operation does not accomplish the requested task in the way that it is used. It should be used as a column object method instead.
D
The split() operation does not accomplish the requested task. The explode() operation should be used instead.
E
The split() operation does not accomplish the requested task. The array_distinct() operation should be used instead.