You are tasked with optimizing a Delta Lake table for time-series analysis to support a financial application that requires high performance and scalability. The application must comply with GDPR and handle petabytes of data efficiently. Considering these requirements, which of the following strategies would BEST ensure efficient querying and analysis while adhering to compliance and scalability constraints? (Choose one option)

Simulated

Implement a single, monolithic query that aggregates all time-series data in one operation, assuming the query engine will optimize performance automatically.

4.2%

Partition the Delta Lake table by the time dimension and apply time-based indexing, enabling targeted queries that minimize data scanned and improve performance.

90.2%

Store the time-series data in a separate, non-indexed format and rely on brute-force scanning for each query, prioritizing simplicity over performance.

3.4%

Export the time-series data to an external time-series database, bypassing Delta Lake's built-in optimizations for time-series analysis.

2.3%

Databricks Certified Data Engineer - Professional

Get started today

Comments