What is a Data Lake?
#165 · Level 3 · Modern architecture · Data and analyticsConcept
Section titled “Concept”A Data Lake is a huge place where all kinds of data are stored as they are, to be used later as needed. You can keep everything here — table-like structured data, semi-structured data like JSON, and unstructured data like images or text. You don’t have to decide the structure when storing the data; it is decided when reading. A Data Lake is not a single service, but an approach built from several services together. On AWS, S3 is usually the main storage, with catalog, permission and analytics services working around it.
Simple example
Section titled “Simple example”A big warehouse. Rice, lentils, cloth, paper — everything goes into the warehouse first. Later, whatever is needed is taken out, sorted and put to use.
Why it’s needed
Section titled “Why it’s needed”A company’s data comes from many places (website, app, sales, logs). Keeping it all in one place first makes analysis easier later.
Where it’s used
Section titled “Where it’s used”On AWS, S3 is usually used as the data lake; Lake Formation, Glue, Athena and EMR work around it.
Azure equivalent
Section titled “Azure equivalent”Azure Data Lake Storage Gen2
Learn more
Section titled “Learn more”For details and the latest information, see the official documentation: