04.04.2024
Data Engineer JHB - Northern Suburbs
Eqplus Technologies (Pty) Ltd
South Africa, Johannesburg
These jobs were popular with other job seekers Reference: GTG007386-KV-1Responsible for maintaining the data warehouse through design and implementation of ETL/ELT methodologies and technologies, as well as providing maintenance and support to our ETL and ML environments. To ensure optimal performance, he conducted root cause analysis on production issues and provided technical leadership throughout the entire information management process of both structured and unstructured data. Duties & Responsibilities Duties:Responsible for solution design and development of Various Functionality in AWS for the project flowDevelop and maintain automated ETL pipelines (with monitoring) using scripting languages such as Python, SQL and AWS services such as S3, Glue, Lambda, SNS, Redshift, SQS, KMSDevelop Glue Jobs for batch data processing and created Glue Catalog for metadata sync upDevelop Data Pipelines using AWS lambda, Step Functions for data processingManage data coming from different sourcesExperienced with AWS services to smoothly manage application in the cloud and creating or modifying instances.Implement with Scaled Agile Framework (SAFe)Be involved with the performance and optimization of the existing algorithms in Hadoop using Spark Context, SparkBe involved in creating Hive Tables, loading with data and writing Hive queriesRequirements:Degree (Information Technology)6 -7 Experience in the same roleTalendSSIS, SSAS, and SSRSClover ETLStrong SQL BackgroundAWSJupiter NotebookData WarehousingCloud Warehousing Other Sectors & Industries These jobs were popular with other job seekers#J-18808-Ljbffr
Attention! You will be redirected to another site