Amazon EMR
#169 · Level 3 · Modern architecture · Data and analyticsConcept
Section titled “Concept”With Amazon EMR you can run big data tools like Apache Spark and Hadoop to process huge amounts of data very fast. It splits the work across many computers and runs it at the same time; this group of computers is called a cluster. The cluster can run on EC2, on Amazon EKS, or on EMR Serverless, where you don’t have to manage a cluster. Usually the data is in S3, and EMR reads from there and writes the results back to S3. AWS handles creating the cluster, installing software, and scaling up and down.
Simple example
Section titled “Simple example”Splitting a job across a team of a hundred people instead of one person — even a huge job finishes fast.
Why it’s needed
Section titled “Why it’s needed”Processing terabytes or petabytes of data on one computer takes days or weeks.
Where it’s used
Section titled “Where it’s used”Analyzing huge logs, preparing data for machine learning, large ETL jobs.
Azure equivalent
Section titled “Azure equivalent”Azure Databricks / HDInsight
Learn more
Section titled “Learn more”For details and the latest information, see the official documentation: