You are working with a media company that needs to process and analyze large volumes of video and audio content for content recommendation systems. The data is highly unstructured and requires real-time processing. Describe how you would set up a data processing pipeline to handle this task, including the technologies you would use and how you would ensure the pipeline is scalable and efficient.