Yesterday
AWS Data Engineer (Expert)
Liyema Consulting
South Africa, Midrand
Other High-Paying Jobs in South Africa
ESSENTIAL SKILLS REQUIREMENTS:Above average experience/understanding (in order of importance): - Terraform - Python 3x - SQL - Oracle/PostgreSQL - PySpark - Boto3 - ETL - Docker - Linux / Unix - Big Data - PowerShell / Bash - Cloud Data Hub (CDH) - CDEC Blueprint - Experience with Enterprise Collaboration tools such as Confluence, JIRA, etc.- Experience developing technical documentation and artefacts.- Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV, etc.- Experience working with Data Quality Tools such as Great Expectations.- Knowledge of the Agile Working Model.- Any additional responsibilities assigned in the Agile Working Model (AWM) Charter. ADVANTAGEOUS SKILLS REQUIREMENTS:- Demonstrated expertise in data modelling with Oracle SQL.- Exceptional analytical skills in analyzing large and complex data sets.- Perform thorough testing and data validation to ensure the accuracy of data transformations.- Strong written and verbal communication skills, with precise documentation.- Self-driven team player with the ability to work independently and multi-task.- Experience building data pipelines using AWS Glue or Data Pipeline, or similar platforms.- Familiarity with data stores such as AWS S3, and AWS RDS or DynamoDB.- Experience and solid understanding of various software design patterns.- Experience preparing specifications from which programs will be written, designed, coded, tested, and debugged.- Strong organizational skills.- Experience developing and working with REST APIs is a bonus.- Basic experience in Networking and troubleshooting network issues. Basic experience/understanding of AWS Components (in order of importance):- Glue- CloudWatch- SNS- Athena- S3- Kinesis Streams (Kinesis, Kinesis Firehose)- Lambda- DynamoDB- Step Function- Param Store- Secrets Manager- CodeBuild/Pipeline- CloudFormation- Business Intelligence (BI) Experience- Technical data modelling and schema design ("not drag and drop")- Kafka- AWS EMR- Redshift QUALIFICATIONS/EXPERIENCE REQUIRED FOR THE ROLE:- Relevant IT / Business / Engineering Degree- Certifications: Candidates with one or more of the following certifications are preferred:- AWS Certified Cloud Practitioner- AWS Certified SysOps Associate- AWS Certified Developer Associate- AWS Certified Architect Associate- AWS Certified Architect Professional- Hashicorp Certified Terraform Associate ROLE AND RESPONSIBILITIES:Data Engineers are responsible for building and maintaining Big Data Pipelines using Cloud Data Platforms. Data Engineers are custodians of data and must ensure that data is shared in line with the information classification requirements on a need-to-know basis.
Attention! You will be redirected to another site