You are part of a data science team at a bank tasked with creating a machine learning model to predict loan default risk. You have gathered and meticulously cleaned a vast amount of training data, consisting of hundreds of millions of records, which are now stored in a BigQuery table. Your next step is to develop and compare multiple models on this extensive dataset using TensorFlow and Vertex AI. Given the scale of the data, you want to ensure the data ingestion phase is efficient and scalable, minimizing any potential bottlenecks. What should you do to achieve this?