Databricks Certified Generative AI Engineer - Associate

Explanation

When preparing a large legal document for use in a generative AI model that has a token limit of 2048, the most efficient way to chunk the text is to split the document into smaller, manageable sections that respect natural language boundaries, such as sentences. This ensures that each chunk is coherent and meaningful without exceeding the model's token limit.

• Option A is the best strategy because it preserves the logical flow of the content while ensuring each chunk stays within the model's constraints, avoiding fragmented or incomplete sentences.
• Option B may result in chapters that are too large to fit within the token limit, while Option C doesn't account for section lengths, which could also exceed the limit.
• Option D, although it ensures the token limit is respected, could cut off sentences, leading to incomplete or less meaningful chunks for the model to process.