AWS Glue
#167 · Level 3 · Modern architecture · Data and analyticsConcept
Section titled “Concept”AWS Glue is a serverless ETL service — it brings in data from different places, cleans and organizes it, and stores it in a new place. ETL means Extract (bring in data), Transform (change it) and Load (store it). One of its main parts is the Data Catalog, which describes which data is where and in what structure. A Crawler can go through the data by itself and write table descriptions into this catalog. The actual transformation work happens in a Glue job, which runs behind the scenes on an engine like Apache Spark.
Simple example
Section titled “Simple example”Washing, cutting and sorting vegetables from the market into separate bowls so they are ready for cooking.
Why it’s needed
Section titled “Why it’s needed”Raw data is messy — wrong formats, missing information. It must be cleaned before analysis.
Where it’s used
Section titled “Where it’s used”Bringing data from databases and files and organizing it in S3, building a list of data (Data Catalog).
Azure equivalent
Section titled “Azure equivalent”Azure Data Factory
Learn more
Section titled “Learn more”For details and the latest information, see the official documentation: