Pyspark AWS Developer – PAD202305003

Date Posted: 5/10/2023, Job Code: PAD202305003

Job Responsibilities:

• Participate in Full Stack Data Engineer practice design, build, and develop rapid Proof-of-Concept (POC) solutions and services.
• Build applications using Bigdata, Spark, Pyspark, ETL, Python, SQL, Databricks, and AWS, and should take end-to-end responsibility of the datalake application life cycle.
• Develop and implement data migration from Legacy on-premises systems to the AWS Cloud Environment.
• Identify and implement opportunities to automate tasks and develop reusable frameworks.
• Adhere to standard methodologies for coding, unit testing, and designing reusable code/components.
• Participate in sprint planning meetings and provide estimations on technical implementation.
• Propose improvements to the AWS data pipeline.
• Implement the standardized CICD tools with automated operational and quality control processes in Pyspark and AWS to deliver accurate and timely data.
• Collaborate with the other engineering team members to ensure all services are reliable, maintainable, and well-integrated into existing platforms.
• Review functional and technical designs to identify areas of risk and/or missing requirements.
• Provide technical write-ups and drawings to promote the proposed solutions.

Skills Requirements:

• Must possess a Bachelors degree in Computer Science/ Computer Engineering/Software Engineering/Computer Applications/ Information Technology/ Information Systems/ Electrical or Electronics/ Mathematics or related field or equivalent degree
• Technical skills : Experience in Hadoop, Spark, pySpark, HDFS, Yarn, Hive, MapReduce, Sqoop, Tez, Ambari, AWS S3, Athena, Glue Crawlers, Glue ETL, Lambda, EMR, Redshift, Step Functions, Eventbridge, Python, Scala, HBase, Cloudera, Hortonworks, DataStage 8.1 & 8.5, Oracle, MS-SQL, IntelliJ, SQL Workbench, PyCharm. Tortoise SVN, TFS, GitHub and Bit-bucket, Control-M.

Location: Raleigh, NC

Submit resume to: usitcareers@egrovesys.com