Consider a Delta Lake table that is heavily impacted by 'smalls' (tiny files, scanning overhead, over partitioning). Describe in detail how you would redesign the data processing pipeline to mitigate these issues, focusing on the use of CDF and proper partitioning strategies. Provide a code snippet illustrating the key changes.