qode.world

Hadoop Data Engineer

qode.world  •  Texas (Onsite)  •  2 hours ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

Hadoop Data Engineer responsible for designing, developing, and maintaining large-scale data processing systems within a distributed Hadoop ecosystem. The role focuses on enabling data-driven decision-making across banking operations, risk management, compliance, and customer analytics.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Hadoop ecosystem tools (HDFS, Hive, Spark, Sqoop, Kafka).
  • Build and optimize ETL/ELT processes to support data ingestion from multiple banking systems.
  • Develop and manage big data solutions for structured and unstructured data.
  • Collaborate with data analysts, data scientists, and business stakeholders to deliver data solutions.
  • Ensure data quality, integrity, and governance aligned with banking and regulatory standards.
  • Perform performance tuning and optimization of Hadoop/Spark jobs.
  • Implement data security controls to comply with financial regulations (e.g., PCI, SOX).
  • Support real-time and batch data processing frameworks.
  • Troubleshoot production issues and provide continuous support for data platforms.
  • Work with cloud platforms (e.g., AWS, Azure) for modern data solutions.

Required Skills & Qualifications

Technical Skills

  • Strong experience with:
  • Hadoop ecosystem (HDFS, MapReduce, Hive, HBase)
  • Apache Spark (Scala/Python)
  • SQL & NoSQL databases
  • ETL tools (Informatica, Talend, or similar)
  • Kafka or other streaming tools
  • Proficiency in programming:
  • Python / Java / Scala
  • Experience with:
  • Data warehousing concepts
  • Workflow orchestration tools (Airflow, Oozie)
  • Unix/Linux environments
  • Knowledge of cloud data platforms (AWS EMR, Azure Data Lake) is a plus

Domain Knowledge

  • Understanding of banking and financial services data
  • Exposure to risk, compliance, or fraud analytics is preferred

Soft Skills

  • Strong problem-solving and analytical abilities
  • Excellent communication and collaboration skills
  • Ability to work in Agile/Scrum environments

Education & Experience

  • Bachelor’s or Master’s degree in:
  • Computer Science, Information Technology, or related field
  • Typically 5–10 years of experience in data engineering or big data development
qode.world

About qode.world

We revolutionize how talent finds meaningful careers by harnessing the power of data and automation. Our platform utilizes LLMs to parse resumes and reconstruct queries, transforming unstructured data into actionable insights. This enables us to build robust data moats, such as creating 'Private Talent Pools' for recruiters where autonomous agents enrich candidate profiles.

By automating high-volume recruiting workflows, we reduce the marginal cost of work to zero. Agents match profiles to job descriptions, find contact information, and send personalized messages and schedule interviews automatically, significantly decreasing the time to close. Additionally, we transcribe the interviews and make the data searchable, making hiring decisions more objective.

We drive confidence by raising the quality bar for job seekers. We automate technical exercises such as coding tests, evaluate candidates on merit, providing recruiters with pass/fail scores and qualitative feedback.

We also provide Exclusive or Retained Recruitment services, offering specialized recruitment with no upfront cost or a retained model with a partial fee, ensuring exclusivity and dedicated support throughout the hiring process.

Our Fractional Head of People and HR Advisory services offer flexible, strategic support through part-time or interim roles, as well as comprehensive advisory services to guide crucial HR decision-making.

Lastly, our HR Due Diligence process provides thorough insights into the HR frameworks of target companies, helping mitigate risks across the board.

How do you envision the future of recruiting with the integration of such advanced technologies?

Industry
IT & Software
Company Size
51-200 employees
Headquarters
Singapore , SG
Year Founded
2023
Social Media