Big Data Engineer

🔹 Key Responsibilities:
🔸 Perform end-to-end data store migration from on-prem Data Lake to AWS LakeHouse
🔸 Work closely with cross-functional teams on large-scale data transformation initiatives
🔸 Ensure high performance, scalability, and reliability of data pipelines
🔹 Skill Set 1:
✔️ Python, PySpark (Strong hands-on)
✔️ Apache Spark, Hadoop (HDFS/Hive), Sybase IQ
✔️ Shell Scripting
✔️ Apache Iceberg
✔️ AWS Big Data Stack: EMR, Redshift, Athena, Glue
✔️ AWS Lambda, Step Functions
✔️ CI/CD best practices & Kubernetes deployment
🔹 Skill Set 2:
✔️ Java, Python
✔️ Snowflake, Kafka, SnowSQL
✔️ Apache Iceberg
✔️ Shell Scripting
✔️ AWS Big Data Stack: EMR, Redshift, Athena, Glue
✔️ AWS Lambda, Step Functions
✔️ Data Modeling (SCD Type 2)
✔️ CI/CD best practices & Kubernetes