AWS Athena as Query Engine
Category: Data Platform
We use AWS for many of our operational systems.
We use AWS S3 as Storage for Data Products.
How can we analyze data stored on S3?
We use AWS Athena as Query Engine.
- We can use SQL to analyze data stored in S3 buckets
- Data virtualization tools, such as Quicksight, Looker, Tableau can access data products
- Data structures need to be defined in Glue Data Catalog
- Multiple file formats supported
- Serverless pricing model
- May become expensive for regular queries, such es data quality checks and monitoring
- For scheduled queries, an external scheduler is required
- AWS Redshift Spectrum
- S3 select
- Athena databases can be created with Terraform
- A dbt-athena integration is available to leverage dbt models with athena