Job Description

One of our direct client is urgently looking for a Data Engineer @ Dallas TX

TITLE: Data Engineer 

LOCATION: Dallas TX

Duration: 6 to 12+ Months.

Description:

Design, develop and build database to power Big Data analytical systems.

• Design data integration pipeline architecture and ensure successful creation of the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark, SQL, HQL and other technologies.

• Build robust and scalable applications using SQL, Scala/Python and Spark.

• Create real time data streaming and processing using Kafka and/or Spark streaming.

• Work on creating data ingestion processes to maintain Global Data lake on Google cloud or Azure

• Engage with architects and senior technical leads to create and enhance complex software components.

• Design, configure and implement systems that can scale to process terabytes of data between heterogeneous systems on premise and cloud.

• Work with business customers, product managers and engineers to design feature-based solutions and implement them in an agile fashion.

• Develop proof-of-concept prototype with fast iteration and experimentation.

• Develop and maintain design documentation, test cases, performance and monitoring and performance evaluation using Git, Crontab, Putty, Jenkins, Maven, Confluence, ETL, Automic, Zookeeper, Cluster Manager

• Perform continuous integration and deployment using Jenkins and Git