
Ultimate access to all questions.
You are working on a data pipeline for a recommendation system that suggests products to users based on their browsing history and preferences. The pipeline processes data from user interactions, product information, and user profiles. How would you optimize the pipeline for real-time recommendations?
A
Use a batch processing approach to process the data in large batches, ignoring the real-time requirements of the recommendation system.
B
Optimize the pipeline for transactional processing by ensuring data consistency and atomicity, but do not consider the real-time requirements.
C
Optimize the pipeline for analytical processing by focusing on data aggregation, windowing, and stateful processing, but do not consider the real-time requirements.
D
Optimize the pipeline for real-time processing by focusing on low latency, high throughput, and efficient data processing techniques, such as data aggregation, windowing, and stateful processing.