A machine learning engineer is working on optimizing a Spark job in Databricks that involves processing a large dataset with numerous features. The goal is to ensure data is handled efficiently and memory usage during transformations is minimized. Which advanced optimization technique should they consider?