AWS Data Pipeline

Caution

AWS is phasing out this service or has closed it to new customers. Check its current status in the official docs before using it in a new project.

Concept

AWS Data Pipeline was an older service for moving and processing data from one place to another on a schedule. A pipeline described where the data would come from, what work would be done and when it would run. To run the work, it used resources such as EC2 or EMR. If a step failed, there was also a way to retry it. AWS has closed it to new customers; the alternatives are AWS Glue, Step Functions or Amazon MWAA.

Simple example

An old van service — it delivered goods on a routine, but now there are faster and more modern options.

Why it’s needed

To automate the regular work of moving and processing data.

Where it’s used

AWS has closed it to new customers; alternatives: AWS Glue, Step Functions, MWAA.

Azure equivalent

Azure Data Factory

Remember

Data Pipeline = the old data-moving service; use Glue or MWAA for new work.

Learn more

For details and the latest information, see the official documentation:

• AWS Data Pipeline — AWS official documentation ↗