Astek
Singapore / Global
GET
The Big Jobsite
Singapore / Global
Astek Singapore is looking for a skilled Data Engineer - Scala/Spark to support our project based in Singapore.
Data Engineer, you'll be part of a team who are passionate about learning and prototyping cutting-edge technologies and convergence initiatives with latest Big Data services on Spark, Scala, Databricks, PySpark and leverage GitHub to support things like CI/CD and integrations.
5-10 years of experience as a Data engineer who works on enterprise standard frameworks like Spark, Scala, Databricks, PySpark.
Development experience in unit and integration test cases
Intermediate level of Database (SQL) skills to develop SQL queries, functions
Good Understanding on CI/CD Pipeline i.e. GitHub Actions
Strong knowledge of version control tools, preferably GitHub
Basic Knowledge on Linux/Unix environment (basic commands, shell scripting, etc.)
Demonstrated ability to thrive in an enterprise Agile/SCRUM environment
Implement data governance practices, data lineage, and metadata management to ensure data accuracy, traceability, and compliance,
Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements,
Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure,
Experience using Collaboration Technologies: GitHub, Jira, Confluence
Experience using Object-oriented languages Java
Experience working with testing tools and Automation test needs
Singapore / Global
Singapore / Global
Singapore / Global
Singapore / Global
Singapore / Global
Kallang / Global