03.09.2024
Intermediate Data Engineer
Aesafrica
South Africa, Cape Town
The Company A leading AI consulting firm specializing in leveraging data to drive business value. Known for delivering cutting-edge solutions in Natural Language Processing, GenAI, LLMs, Predictive Analytics, and Machine Learning, they help clients optimize operations, enhance customer experiences, and gain a competitive edge. The Role Building modern database structures storing terabytes of data used to develop and maintain machine learning models in a cloud environment. Collaborate with Software Engineering stakeholders and Leads to define and implement new data-powered solutions and deploy and maintain them in production. Supporting best practices in cataloguing, characterising, classifying and communicating accessible data and initiatives relative to data creation so teams can easily access the data. Creating and owning best practices in terms of data ‘organisation’, standards and versioning including in a Cloud environment. Own data privacy and security standards according to GDPR and apply these to production systems. Embracing modern DevOps principles and working in a setting where code is expected to be shared and peer-reviewed. Experience 3 years experience with data wrangling/data engineering, implementing, and supporting robust data transformation pipelines in a production Cloud environment. Experience with RDBMS systems for OLTP workloads, such as Postgres or MySQL. Implementing methods to ensure large-scale (big data) quality and security. Knowledge of software design patterns and test-driven development. Experience working with big data technologies such as Spark, Hadoop, Presto, Hive, AWS Athena, and Terraform. Proficiency in dynamic languages, such as Python or R. Familiarity with container technology, like Docker. Experience with JavaScript frameworks like Vue.js and Node.js. #J-18808-Ljbffr
Attention! You will be redirected to another site