Explanation:
Option C is the correct solution because it provides near real-time monitoring, hallucination detection, and cost anomaly awareness using built-in Amazon Bedrock and Amazon CloudWatch capabilities, with minimal custom development.
Near Real-time Monitoring: Amazon CloudWatch anomaly detection alarms provide immediate alerts for abnormal token consumption patterns, enabling early warnings of cost anomalies.
Built-in Hallucination Detection: Amazon Bedrock guardrails with contextual grounding checks can detect factual inconsistencies and hallucinations without requiring custom development or complex evaluation jobs.
Minimal Custom Development: The solution leverages native AWS services:
Comprehensive Data Collection: Storing model invocation logs in Amazon S3 with text output logging provides complete audit trails for compliance and retrospective analysis.
Option A: While CloudWatch alarms monitor token usage, using AWS Glue and Amazon Athena for hallucination detection requires significant custom development and is not near real-time.
Option B: Amazon Bedrock evaluation jobs are batch processes, not near real-time. Creating AWS Lambda functions adds custom development overhead.
Option D: AWS CloudTrail logs API calls but doesn't capture detailed model outputs. Amazon SageMaker Model Monitor is designed for traditional ML models, not generative AI hallucinations, and requires custom development.
Ultimate access to all questions.
No comments yet.
A healthcare company uses Amazon Bedrock to deploy an application that generates summaries of clinical documents. The application experiences inconsistent response quality with occasional factual hallucinations. Monthly costs exceed the company's projections by 40%. A GenAI developer must implement a near real-time monitoring solution to detect hallucinations, identify abnormal token consumption, and provide early warnings of cost anomalies. The solution must require minimal custom development work and maintenance overhead.
Which solution will meet these requirements?
A
Configure Amazon CloudWatch alarms to monitor InputTokenCount and OutputTokenCount metrics to detect anomalies. Store model invocation logs in an Amazon S3 bucket. Use AWS Glue and Amazon Athena to identify potential hallucinations.
B
Run Amazon Bedrock evaluation jobs that use LLM-based judgments to detect hallucinations. Configure Amazon CloudWatch to track token usage. Create an AWS Lambda function to process CloudWatch metrics. Configure the Lambda function to send usage pattern notifications.
C
Configure Amazon Bedrock to store model invocation logs in an Amazon S3 bucket. Enable text output logging. Configure Amazon Bedrock guardrails to run contextual grounding checks to detect hallucinations. Create Amazon CloudWatch anomaly detection alarms for token usage metrics.
D
Use AWS CloudTrail to log all Amazon Bedrock API calls. Create a custom dashboard in Amazon QuickSight to visualize token usage patterns. Use Amazon SageMaker Model Monitor to detect quality drift in generated summaries.