Preview Ocean for Apache Spark

Fully automated, optimized Spark on top of fully-automated, optimized cloud infrastructure.

• Optimization engine that automatically chooses the best infrastructure to run a Spark job
• Spark-centric observability and monitoring UI that lets you track app configuration, logs, and system metrics
• Run Spark applications reliably on a mix of spot, on-demand, and reserved instances
• Spark application rightsizing that tunes configurations based on actual resource consumption to limit over-provisioning
• Container bin-packing that ensures pods are allocated to nodes efficiently, reducing underutilization of compute
• Pre-built integrations with big data tooling to meet data engineers and scientists where they are