Answer-first summary for fast verification
Answer: Utilize schema-on-read capabilities to dynamically adapt to changes in the data structure at query time.
Utilizing schema-on-read capabilities offers the flexibility needed to handle evolving data formats without manual intervention. This method involves storing data in a semi-structured format, like JSON or Parquet, without a strict initial schema. Upon querying, the schema is inferred based on the current data, enabling seamless adaptation to structural changes. This approach is efficient, avoiding the need to manage multiple schemas for different firmware versions and simplifying the handling of evolving data in IoT streams. It also facilitates easy data querying and analysis, free from the constraints of rigid schema definitions.
Author: LeetQuiz Editorial Team
Ultimate access to all questions.
In the context of IoT devices that frequently update their firmware, altering the data structure they emit, what is an effective strategy for a lakehouse schema to automatically accommodate these evolving data formats without manual adjustments?
A
Store raw data in a semi-structured format, applying transformations to a structured format during downstream processing.
B
Automatically trigger schema evolution in Delta Lake tables based on anomaly detection in incoming data streams.
C
Utilize schema-on-read capabilities to dynamically adapt to changes in the data structure at query time.
D
Implement a versioned schema system where each firmware version corresponds to a different schema, with a master schema to unify them.
No comments yet.