What is the most cost-effective way to optimize the performance of a complex analytical Spark job that involves shuffling operations and uses initial data in parquet format (each file averages 200-400 MB in size), after migrating from an on-prem Hadoop cluster to Dataproc on GCS, considering the organization's cost sensitivity? The Spark job is currently running on preemptible VMs with only two non-preemptible workers.