Machine Learning with Apache Spark

What Will You Learn?

• Describe ML, explain its role in data engineering, summarize generative AI, discuss Spark's uses, and analyze ML pipelines and model persistence.
• Evaluate ML models, distinguish between regression, classification, and clustering models, and compare data engineering pipelines with ML pipelines.
• Construct the data analysis processes using Spark SQL, and perform regression, classification, and clustering using SparkML.
• Demonstrate connecting to Spark clusters, build ML pipelines, perform feature extraction and transformation, and model persistence.