Data Engineer

eal-time data pipelines using Spark Structured Streaming, MapReduce, and other Big Data frameworks.
Ingest data from multiple sources such as message queues (Kafka), file shares, REST APIs, and relational databases.
Transform, clean, and validate data in HDFS, Hive, Impala, or Spark SQL.
Convert and manage data in formats like JSON, CSV, XML to support downstream analytics.
Perform data validation, profiling, and analysis to identify anomalies and ensure data integrity.
Troubleshoot issues in data pipelines, SQL jobs, or Spark applications, including slow-running jobs or failures.