Job Description

Minimum qualifications

• Bachelor's degree in Computer Science or a related technical field, or equivalent practical experience.
• 8 years of experience designing and optimizing distributed systems, storage architectures, or database internals (columnar storage formats, serialization, or query execution engines).

Preferred qualifications

• Experience with next-generation or specialized file formats like Vortex, Lance, Nimble, or FastLanes.
• Experience with vector search (ANN) and building storage layers optimized for retrieval-augmented generation (RAG) and multimodal data.
• Strong understanding of modern machine learning data pipelines, including how PyTorch/TensorFlow consume data and the I/O bottlenecks of AI training.
• History of successful contributions to major open-source data projects (e.g., Apache Arrow, Parquet, Iceberg).

About the job

As a part of BigQuery, which is at a pivotal moment in the evolution of cloud data warehouses, we are seeing a massive shift in file format momentum, with next-generation columnar formats like Vortex redefining core analytics performance, and multimodal formats like Lance pulling data gravity toward AI/ML training and vector search workloads.

In this role, as a visionary Tech Lead, you will own the goal, strategy, and architectural design for BigQuery’s file format. In this role, you will define how BigQuery adapts to these industry shifts, whether by evolving our native Capacitor format, adopting and extending open-source standards like Vortex, or bridging the gap to AI-native lakehouses.

Google Cloud accelerates every organization’s ability to digitally transform its business and industry. We deliver enterprise-grade solutions that leverage Google’s technology, and tools that help developers build more sustainably. Customers in more than 200 countries and territories turn to Google Cloud as their trusted partner to enable growth and solve their most critical business problems.Individual pay is determined by factors including job-related skills, experience, and relevant education or training.

US: $262000 - $364000 (USD) + 25% bonus target + equity + benefits

Learn more about benefits at Google.

Responsibilities

• Direct the end-to-end technical strategy for BigQuery's storage formats, and determine the roadmap for our existing proprietary formats (Capacitor) versus adopting or contributing to emerging open standards (Vortex, Lance, BtrBlocks).
• Design storage layer innovations that unlock zero-copy column updates/additions, extremely efficient random access, and seamless integration with deep learning training pipelines.
• Partner with Dremel, BigLake, and GCS teams to ensure our execution engine seamlessly adopts next-generation formats without regressing SQL concurrency.
• Position Google as a thought leader in the open data lakehouse ecosystem, steering the direction of the formats that will define the next decade of data infrastructure.
• Lead technical and Return on Investment (ROI) evaluations of open-source formats, and assess the engineering costs to integrate Arrow-native execution with BigQuery pipelines for wide tables and multimodal AI.