June 20 AAT Community - Scalable Big Data Analytics Architecture for Data Engineering & AI/ML

When:  Jun 20, 2024 from 13:00 to 14:00 (CT)

Join the Analytics Architecture & Technology (AAT) Community on Thursday, June 20, 2024 at 1:00 PM CT for a presentation with Q&A and open discussion on "Scalable Big Data Analytics Architecture for Data Engineering & AI/ML on AWS with Apache Iceberg and AWS EMR on EKS" led by Senthilkumar Sakthivel, Senior AI Cloud Architect, AI Organization at Entergy.

Session Description:
During this session you will learn about how Entergy designed Modern Cloud Big Data Analytics Architecture using tools like AWS EMR, AWS EKS (Kubernetes), Apache Iceberg, EMR Studio, Snowflake, and Power-BI

Key Takeaways:

  • Learn about the challenges Entergy faced with on-premises Cloudera data lake.
  • Discover the benefits of migrating to a cloud-native data lakehouse on AWS S3 with Apache Iceberg.
  • Explore how to leverage containers and Kubernetes for efficient data engineering and machine learning spark workloads using EMR on EKS.
  • Understand how Snowflake acts as a front-end query tool utilizing Iceberg external tables, enabling secure and scalable access to data.
  • Gain insights into using PySpark as the primary data engineering language and Airflow for scheduling workflows.
  • See how this architecture integrates with PowerBI for data visualization, achieving flexibility and avoiding vendor lock-in (data resides on AWS S3).
  • Learn how the containerized infrastructure extends to support AI workloads.
  • This presentation will provide valuable insights for anyone looking to modernize their analytics architecture and leverage the power of the cloud for data management, processing, and analysis.

Members of the AAT Community will receive a Microsoft Teams link prior to this meeting. Not a member of this community? UAI Utility Members can register for any of our communities by using our Request To Join link. 


Dial-in Instructions:
Microsoft Teams link will be provided to community members prior to this meeting.


Kevin Praet