About the Role
BigData Engineer — Position Summary
At Adroit Affine we partner with Fortune 100 enterprises globally to design and operate large-scale data platforms. As a BigData Engineer you will build and maintain distributed data pipelines and analytics infrastructure that drives real-time and batch processing for mission-critical business decisions.
Key Responsibilities
• Design, develop and maintain scalable ETL/ELT pipelines using Apache Spark, Kafka and the Hadoop ecosystem.
• Build and optimise data lake and data warehouse solutions on AWS, Azure or GCP.
• Define and enforce data quality standards; implement monitoring and alerting for pipeline health.
• Collaborate with data scientists to productionise ML models within existing data infrastructure.
• Optimise query performance across Hive, Presto/Trino, Spark SQL and distributed storage systems.
• Participate in architecture reviews and contribute to the evolution of the big data platform.
• Write thorough technical documentation and support knowledge transfer across teams.
• Mentor junior engineers and conduct code reviews to maintain engineering excellence.
Required Qualifications
• 3+ years of hands-on big data engineering experience.
• Strong proficiency in Apache Spark (PySpark or Scala) and the Hadoop ecosystem (HDFS, YARN, Hive, HBase).
• Experience building real-time streaming pipelines with Apache Kafka or Kinesis.
• Solid understanding of data lake architectures (Delta Lake, Apache Iceberg or Hudi).
• Proficiency in Python or Scala for pipeline development.
• Experience with AWS (EMR, Glue, S3), Azure (Databricks, ADLS) or GCP (Dataflow, BigQuery).
• Strong SQL skills and ability to write complex, optimised queries.
• Familiarity with CI/CD practices and Git.
Preferred Qualifications
• Experience with Databricks, Snowflake or Apache Flink.
• Knowledge of MLOps and integrating ML pipelines into the data platform.
• AWS Certified Data Analytics or equivalent cloud certification.
• Exposure to data governance tools (Apache Atlas, Collibra) and PII handling.
Compensation & Benefits
• Salary: $75,000 – $125,000 based on experience.
• Comprehensive health, dental and vision insurance.
• Remote-friendly with a modern Phoenix, AZ office.
• Professional development budget: certifications, conferences and training.
• Exposure to Fortune 100 global clients.
• Collaborative, growth-oriented engineering culture.
Requirements
3+ years of hands-on big data engineering experience.
Strong proficiency in Apache Spark (PySpark or Scala) and the Hadoop ecosystem.
Experience building real-time streaming pipelines with Apache Kafka or Kinesis.
Solid understanding of data lake architectures (Delta Lake, Iceberg or Hudi).
Proficiency in Python or Scala for pipeline development.
Experience with AWS, Azure or GCP cloud data services.
Strong SQL skills. Familiarity with CI/CD and Git.
About the Company
Adroit Affine is a premier IT solutions and staffing company headquartered in Phoenix, AZ. We partner with Fortune 100 enterprises globally, delivering cloud transformation, cybersecurity, data analytics, DevOps and IT staffing services. Our team of 100+ trained professionals brings deep technical expertise and a commitment to measurable outcomes. We offer competitive compensation, certification reimbursement, a professional development budget and a collaborative culture where every engineer has the opportunity to make a real impact.