Railroad19

Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake • Databricks)

Railroad19  •  United States (Remote)  •  9 hours ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

Railroad19, Inc is seeking a Senior Data Engineer with deep, hands‑on experience in building modern lakehouse architectures on GCP. This role focuses on designing, developing in Python & Spark, and delivering reusable data‑sharing adapters that connect BigQuery‑backed data products to Snowflake and Databricks using Iceberg UniForm and Delta Sharing.

About Railroad19:

At Railroad19, Inc, we develop customized software solutions and provide software development services. We’re a specialized team of developers and architects. As such, we only bring an “A” team to the table, through hard work and a desire to lead the industry — this is our company culture — this is what sets Railroad19 apart.

As a Railroad19 employee, you will be part of a company that values your work and gives you the tools you need to succeed. Our headquarters is in Saratoga Springs, New York, but this position is 100% remote. Railroad19 provides competitive compensation and excellent benefits, including Medical/Dental/Vision/Pet Insurance, Paid Time Off, and 401 (k).

NO 1099, C2C, Corp-to-Corp; only full-time employment.

NO Agencies.

Core Responsibilities:

    • Design and implement the UniForm write layer (Delta + Iceberg dual metadata).
    • Build GCS → BigQuery ingestion pipelines for structured operational datasets.
    • Develop and implement in Python and Spark.
    • Implement Kafka-based CDC patterns for real-time and near-real-time ingestion.
    • Develop data lineage, dependency tracking, and modular adapter code.
    • Configure Snowflake Horizon external tables for zero-copy reads.
    • All data hub tables are to be written once using Delta Lake with Iceberg UniForm enabled… readable by all target consumers without conversion.
    • Implement and certify Delta Sharing endpoints for Databricks consumers.
    • Build governed access layer components: RBAC, connector registry entries, tenant-scoped authorization.
    • Align semantic layer models with LookML and KPI catalog definitions.
    • Collaborate with cross‑functional teams to deliver end‑to‑end features.
    • Troubleshoot issues across the full stack and contribute to code quality.

Skills/Experience:

    • 6+ years of proven enterprise-level experience in Python & Spark
    • Advanced experience in GCP BigQuery
    • Strong working knowledge of Apache Iceberg, Delta Lake, Iceberg UniForm
    • Experience with Delta Sharing; Kafka / CDC pipelines
    • Specific work experience in Snowflake Horizon Catalog; Databricks Unity Catalog
    • Solid experience with Data lake architecture & ingestion pipeline design
    • Excellent Communication skills and the ability to work cohesively with multiple teams.

Preferred Experience – Nice to Have

    • Prior delivery in enterprise SaaS, media, or advertising technology.
    • Active daily use of AI-assisted development tools (Claude Code preferred).
Railroad19

About Railroad19

At Railroad19, we develop customized software solutions that make your processes smarter and your bottom line stronger. How? We work ahead of the curve. An accomplished team of national talent, including back-end, front-end and user interface developers and cloud platform engineers, we help move you ahead of the competition. Whether adding depth to a team or skills to a project, let Railroad19 turn your objectives into results.

Industry
IT & Software
Company Size
51-200 employees
Headquarters
Saratoga Springs, NY
Year Founded
1997
Social Media