Databricks Certified Machine Learning - Associate
Get started today
Ultimate access to all questions.
Comments
Loading comments...
Ultimate access to all questions.
When dealing with large datasets (approximately 1GB or more) in Hyperopt with SparkTrials, what is the recommended method to efficiently manage the dataset, and why?
A
Utilize Databricks Runtime 6.4 ML or higher for optimal large dataset management.
B
Explicitly broadcast the dataset using Spark and access it via the broadcasted variable within the objective function.
C
Store the dataset in DBFS and reload it onto workers using the DBFS local file interface.
D
Directly load the dataset on the driver and reference it from the objective function.