Databricks Certified Data Engineer - Associate

Explanation

In Apache Spark Structured Streaming, stream-static joins have specific restrictions:

1. Stream-static joins must be inner or left outer joins when the streaming DataFrame is on the left side of the join.
2. The streaming DataFrame must be the left side for left outer joins.
3. The streaming DataFrame can be either side for inner joins.
4. Right outer joins and full outer joins are NOT supported for stream-static joins.

Let's analyze each option:

A. userLookup.join(streamingDF, ["userid"], how="inner") - VALID: This is an inner join where the static table is on the left and streaming DataFrame is on the right. Inner joins are supported with streaming DataFrame on either side.

B. streamingDF.join(userLookup, ["user_id"], how="outer") - INVALID: This is a full outer join (how="outer"). Full outer joins are NOT supported in stream-static joins.

C. streamingDF.join(userLookup, ["user_id"], how="left") - VALID: This is a left outer join with the streaming DataFrame on the left side. Left outer joins are supported when the streaming DataFrame is on the left.

D. streamingDF.join(userLookup, ["userid"], how="inner") - VALID: This is an inner join with the streaming DataFrame on the left side. Inner joins are supported with streaming DataFrame on either side.

E. userLookup.join(streamingDF, ["user_id"], how="right") - VALID: This is a right outer join where the static table is on the left and streaming DataFrame is on the right. Right outer joins are equivalent to left outer joins with the streaming DataFrame on the left side, so they are supported.

Key Points:

• Full outer joins (how="outer") are the only join type that is completely unsupported for stream-static joins.
• The column names (userid vs user_id) don't affect the validity of the join type itself.
• The question asks for the code block that attempts to perform an invalid stream-static join, which is option B.