A company will create multiple ML prediction models. Training data is stored in Amazon S3 and the full dataset is larger than 5 TB and contains CSV, JSON, Parquet, and text files. The data must undergo several sequential processing steps that include complex transformations and some NLP work that can take hours to run. The entire process must be automated. Which solution meets these needs?

Amazon SageMaker Pipelines is ideal for orchestrating complex, multi-step ML workflows, including data preprocessing, model training, and evaluation. It supports various data formats and integrates with SageMaker Processing for large-scale data transformations and NLP tasks, which can run for hours. Automating the pipeline with Amazon EventBridge allows for scheduled or event-driven execution, meeting the automation requirement. Using SageMaker Data Wrangler for each step is less efficient for sequential, complex transformations across diverse data types and large datasets, and its job capabilities are more focused on individual data preparation tasks rather than full pipeline orchestration. SageMaker notebooks are interactive and not designed for automated, production-grade workflows. AWS Lambda functions have execution time limits (15 minutes) and memory constraints, making them unsuitable for long-running, complex data processing and NLP tasks on large datasets.