Job Description

Team Red Dog is hiring a Data Pipeline Engineer for our client, an AI-driven technology organization and global technology leader. This role is part of an applied science team focused on combining research, innovation, and production engineering to develop and train advanced AI models supporting Windows and devices. The engineer will build Python-based data pipelines from the ground up, transforming large-scale document and text datasets into accurate, validated, training-ready data through OCR processing, data cleansing, deduplication, and automated quality controls. This is an opportunity to contribute directly to the data infrastructure powering next-generation AI while collaborating with applied scientists and model-training teams on highly innovative projects.

Top Required Skills (Must Haves):

1. Data Engineering & Dataset Development: 5+ years of data-related experience, including hands-on experience building, creating, and curating datasets from the ground up. Must demonstrate the ability to develop new datasets rather than simply maintain, analyze, or work with existing datasets.
2. Advanced Python Programming: Strong proficiency developing production-ready Python code, including reusable modules, debugging, testing, logging, error handling, and automated data-processing workflows.
3. Large-Scale Data Processing: Hands-on experience processing large structured and unstructured datasets using memory-efficient, parallel, batch, or distributed processing techniques that can handle data volumes exceeding single-machine memory.
4. Azure Data Management: Practical experience with Azure Blob Storage or Azure Data Lake Storage, including bulk data transfers, storage organization, synchronization, and data integrity verification.

Opportunity Overview:

Join an applied science team at the forefront of AI research and production, where data engineering plays a critical role in developing and training advanced AI models for Windows and devices. This position offers the opportunity to work on end-to-end AI data management and dataset curation, building the pipelines and processes that transform large volumes of raw documents and text into high-quality training datasets.

You will have ownership of the data preparation lifecycle, from initial ingestion and OCR processing through validation, normalization, deduplication, and final dataset release. The work requires a combination of software engineering, data science, Python development, and cloud storage expertise, with a strong emphasis on data quality, operational reliability, and risk mitigation. This is a highly collaborative opportunity to partner with applied scientists and AI model-training teams while contributing to confidential, next-generation technology initiatives.

How you will make an impact:

• Develop, maintain, and optimize Python-based data pipelines for ingestion, OCR processing, text extraction, cleaning, normalization, and deduplication.
• Build datasets from the ground up by transforming raw document and text sources into structured, validated, training-ready data.
• Execute and monitor large-scale batch processing jobs, troubleshoot failures, and implement reliable retry and recovery mechanisms.
• Develop automated data quality checks to identify missing or corrupted files, incomplete OCR outputs, duplicate content, formatting issues, and metadata inconsistencies.
• Prepare AI training datasets through filtering, text segmentation, tokenization, and conversion into standardized data formats.
• Manage bulk uploads, downloads, and synchronization between local processing environments and Azure storage while verifying transfer completeness and data integrity.
• Organize raw data, intermediate outputs, and released datasets using consistent naming conventions, metadata, versioning, and source traceability.
• Optimize processing throughput, memory utilization, storage consumption, and cloud costs through batch, parallel, and distributed processing approaches.
• Track dataset statistics, processing coverage, quality metrics, and exceptions while producing clear operational reports.
• Collaborate with applied scientists and model-training teams to translate data requirements into reliable, reproducible processing workflows.
• Maintain tested, documented, and version-controlled code while following established data access, security, and governance requirements.

The expertise you bring:

• 5+ years of professional data-related experience in data engineering, data pipeline development, data science, or a related discipline with substantial hands-on programming responsibilities.
• Strong proficiency in Python, including developing reusable modules, debugging, testing, logging, and handling processing errors.
• Demonstrated experience creating and building datasets from scratch, including working with raw data sources and developing processes to transform them into usable datasets.
• Hands-on experience processing large structured and unstructured datasets using memory-efficient, parallel, batch, or distributed computing techniques.
• Practical experience with Azure Blob Storage or Azure Data Lake Storage, including bulk data transfers, storage organization, and access troubleshooting.
• Familiarity with Python data-processing libraries and formats, including pandas, NumPy, JSON/JSONL, CSV, and Parquet.
• Working knowledge of SQL, Git, Linux environments, and command-line tools.
• Strong troubleshooting skills, attention to data quality, and the ability to communicate technical issues clearly.
• Bachelor's degree in Data Science, Mathematics, Computer Science, Engineering, or a related technical field, or equivalent practical experience.

What makes a candidate highly successful in this role:

A highly successful candidate brings a combination of hands-on data engineering, Python development, and practical experience building datasets from the ground up. They have personally created and curated datasets from raw structured and unstructured sources rather than exclusively consuming or maintaining existing datasets. Candidates who can demonstrate end-to-end ownership of data pipelines, including ingestion, OCR, transformation, validation, quality assurance, and preparation for AI model training, will stand out.

Prior Microsoft experience is a strong advantage, particularly for candidates familiar with enterprise data environments and the expectations of large-scale technology organizations. Experience with Azure storage, distributed data processing, and production-ready Python workflows is especially valuable. Candidates with backgrounds in data science, mathematics, computer science, or related technical disciplines who can combine coding expertise with a strong understanding of data quality and risk mitigation will be particularly competitive.

Why Work with Team Red Dog?

At Team Red Dog, people are at the heart of everything we do. Our commitment to personalized service and our deep experience in matching talented professionals with meaningful roles at some of the world’s most inspiring companies is what sets us apart. We take the time to understand your unique skills, strengths, and passions—because we believe your career should reflect who you are.

Whether you're looking to grow, pivot, or simply find a place where your work truly matters, we offer opportunities that empower you to make a positive impact. With excellent benefits, a supportive team, and a role where you can thrive while doing what you love, we’re here to help you take the next step with confidence. Join us—and discover what it means to be genuinely valued in your career.

Generous benefits package for qualified employees includes:

• Health insurance (medical, dental, vision, and life)
• Employer-matched 401K plan
• Paid Time Off
• Flexible Paid Holiday Benefit

Estimated Start Date: Immediately

Location: Redmond, WA – 100% onsite, Microsoft Building 87

Job #: TBD

Job Type and Estimated Duration: W2 contract opportunity through June 30, 2027, with the potential to extend up to 18 months total, subject to performance, budget, and client discretion.

Rate: $90–$94/hr. (Maximum rate subject to candidate qualifications and approval)

Team Red Dog is committed to providing equal opportunities to everyone, regardless of race, ethnicity, gender, age, religion, sexual orientation, disability, or any other characteristic. If you need accommodation during the recruitment process, reach out to hr@teamreddog.com, and we will work to ensure an accessible experience. We strictly adhere to federal, state, and local laws to maintain a workplace free from discrimination and harassment.

We offer competitive compensation aligned with U.S. industry standards, and our final offer will reflect the candidate’s location, job-specific skills, experience, and knowledge.

• All applicants must be authorized to work in the U.S. without the need for sponsorship.
• Team Red Dog is an E-Verify employer.
• Employment is contingent upon the successful completion of a reference and background check.
• Please no solicitations from C2C or recruiting firms.