Databricks Certified Data Engineer - Associate

Get started today

Ultimate access to all questions.

A data engineer needs to apply custom logic to identify employees with more than 5 years of experience in array column employees in table stores. The custom logic should create a new column exp_employees that is an array of all the employees with more than 5 years of experience for each row. In order to apply this custom logic at scale, the data engineer wants to use the FILTER higher-order function. Which of the following code blocks successfully completes this task?

Real Exam

Community

KKeng

Last updated: January 13, 2026 at 09:01

SELECT
    store_id,
    employees,
    FILTER (employees, i -> i.years_exp > 5) AS exp_employees
FROM stores;

SELECT
    store_id,
    employees,
    FILTER (employees, i -> i.years_exp > 5) AS exp_employees
FROM stores;

SELECT
    store_id,
    employees,
    FILTER (exp_employees, years_exp > 5) AS exp_employees
FROM stores;

SELECT
    store_id,
    employees,
    FILTER (exp_employees, years_exp > 5) AS exp_employees
FROM stores;

SELECT
    store_id,
    employees,
    FILTER (employees, years_exp > 5) AS exp_employees
FROM stores;

SELECT
    store_id,
    employees,
    FILTER (employees, years_exp > 5) AS exp_employees
FROM stores;

SELECT
    store_id,
    employees,
    CASE WHEN employees.years_exp > 5 THEN employees
         ELSE NULL
    END AS exp_employees
FROM stores;

SELECT
    store_id,
    employees,
    CASE WHEN employees.years_exp > 5 THEN employees
         ELSE NULL
    END AS exp_employees
FROM stores;

SELECT
    store_id,
    employees,
    FILTER (exp_employees, i -> i.years_exp > 5) AS exp_employees
FROM stores;

SELECT
    store_id,
    employees,
    FILTER (exp_employees, i -> i.years_exp > 5) AS exp_employees
FROM stores;

Explanation:

Explanation

Option A is correct because it properly uses the FILTER higher-order function with the correct syntax:

FILTER function syntax: FILTER(array, lambda_function)
Correct array reference: It filters the employees array column
Proper lambda expression: i -> i.years_exp > 5 where i represents each element in the array, and i.years_exp accesses the years_exp field of each employee object

Why other options are incorrect:

Option B: Incorrectly references exp_employees in the FILTER function before it's created. Also missing the lambda expression syntax.
Option C: Missing the lambda expression syntax. The FILTER function requires a lambda expression when accessing struct fields.
Option D: Uses CASE statement instead of FILTER function, which won't work correctly with arrays. CASE would return the entire array or NULL, not filter individual elements.
Option E: Similar to Option B - references exp_employees before it's created and uses incorrect syntax.

Key concepts:

The FILTER higher-order function in Spark SQL filters an array using a lambda expression
Lambda expressions use the syntax element -> condition
When accessing fields of struct elements in arrays, use dot notation within the lambda expression
The result is a new array containing only elements that satisfy the condition

Powered ByGPT-5.2

Comments

Loading comments...