Explanation:
When using shell commands within a notebook's cell via the %sh magic command, the cell will not fail even if the command encounters an error. To ensure the cell fails on error, use the -e option with the %sh command, like so: %sh -e\ncp /tmp/raw_data.csv /FileStore/data.csv. This would cause the job to fail if the /tmp directory is unavailable, highlighting the importance of error handling in scheduled jobs.
Ultimate access to all questions.
A data engineer scheduled a daily job at 2 PM that uses a legacy shell script to copy a file from /tmp to /FileStore. Despite the /tmp directory being unavailable for a week, the job did not fail. What could explain this behavior?
A
The cp command lacks the -e option, which should be cp /tmp/raw_data.csv /FileStore/data.csv -e
B
The cp command lacks the -error option, which should be cp /tmp/raw_data.csv /FileStore/data.csv -error
C
The %sh magic command lacks the -e option, which should be %sh -e
D
The %sh magic command lacks the -error option, which should be %sh -error
E
Both the %sh magic command lacks the -error option and the cp command lacks the -e option
No comments yet.