Sev1Tech LLC

Senior Databricks Platform Engineer

Sev1Tech LLC  •  Arlington, VA (Onsite)  •  4 days ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

Job Responsibilities

We are seeking a highly skilled Senior Databricks Platform Engineer to design, implement, and maintain our enterprise-level Databricks platform supporting a federal customer organization's unified data initiative. This role's primary focus is platform management—centered on building scalable infrastructure, governance frameworks, and operational excellence—with secondary responsibilities in data engineering to support platform optimization and best practices.

You will serve as the steward of our Databricks platform, acting as the frontline technical owner responsible for creating robust, scalable processes and frameworks that enable multiple data teams to work efficiently and securely. This position offers a unique opportunity to work with cutting-edge cloud data technologies and shape the foundational infrastructure of our federal customer's data ecosystem.

Responsibilities:

  • Maintain enterprise-scale Databricks platform, managing multiple workspaces.
  • Build and maintain platform observability through development of KPIs, executive dashboards, and monitoring solutions that provide visibility into system performance, resource consumption, cost trends, and compliance adherence.
  • Develop and implement scalable user provisioning and de-provisioning processes.
  • Design and manage RBAC (Role-Based Access Control) frameworks and security groups.
  • Integrate with enterprise identity management systems (SAML, SCIM, Active Directory).
  • Manage Unity Catalog implementation and governance structures.
  • Oversee metastore configuration, catalog hierarchies, and data organization.
  • Implement data lineage, auditing, and compliance frameworks.
  • Design and enforce compute policies, cluster configurations, and pool management strategies.
  • Implement cost optimization frameworks and resource allocation policies.
  • Monitor and optimize cluster utilization and performance.
  • Manage serverless SQL warehouses and compute resource governance.
  • Establish and manage connections to diverse source systems (databases, APIs, file systems).
  • Configure and maintain storage integrations (S3, ADLS, GCS, external locations) in addition to setting up and data access patterns.
  • Implement and monitor secure credential management and secrets handling.
  • and mount points.
  • Build and configure Databricks workspaces, clusters, notebooks, jobs, and Delta Lake storage, integrating with AWS services such as S3, IAM, and KMS.
  • Implement and maintain security controls including IAM policies, cluster security configurations, KMS-based encryption, and data masking to protect sensitive data.
  • Monitor and tune Databricks clusters and job workloads for reliability, cost optimization, and performance, leveraging autoscaling and workload management.
  • Apply data governance best practices including cataloging, metadata management, and data lineage using Databricks Unity Catalog and AWS-native capabilities.
  • Collaborate with cloud infrastructure teams to configure supporting AWS components such as S3 storage, networking, logging, monitoring, and access controls.
  • Maintain detailed technical documentation including solution designs, data flow diagrams, configuration standards, and operational procedures.
  • Stay up to date with advancements in Databricks, Delta Lake, Spark, and AWS services, integrating new features that improve automation and efficiency.
  • Partner with data engineers, analysts, and scientists to implement data models and reusable transformation patterns within Databricks and AWS.
  • Troubleshoot and resolve platform-level issues including but not limited to workspace connectivity, cluster startup failures, authentication problems, and infrastructure performance degradation.
  • Ensure compliance with data governance, privacy, and security requirements by applying secure architecture patterns and validating controls throughout the data lifecycle.
  • Support the evaluation and hands-on testing of new AWS and Databricks features or services that enhance the Data Lake environment.

Minimum Qualifications

  • Bachelor's degree in computer science, information technology, or a related field. Equivalent experience will also be considered.
  • Proven experience in building and configuring enterprise-level data lake solutions using Databricks in an AWS or Azure environment.
  • In-depth knowledge of Databricks architecture, including workspaces, clusters, storage, notebook development, and automation capabilities.
  • Deep expertise in Databricks Unity Catalog, workspace management, and security models
  • Experience with big data technologies such as Apache Spark, Apache Hive, Delta Lake, and Hadoop.
  • Solid understanding of data governance principles, data modeling, data cataloging, and metadata management.
  • Hands-on experience with cloud platforms like AWS or Azure, including relevant services like S3, EMR, Glue, Data Factory, etc.
  • Proficiency in SQL and one or more programming languages (Python, Scala, Bash, or PowerShell).
  • Knowledge of data security and privacy best practices, including data access controls, encryption, and data masking techniques.
  • Strong problem-solving and analytical skills, with the ability to identify and resolve complex data-related issues.
  • Excellent interpersonal and communication skills, with the ability to collaborate effectively with technical and non-technical stakeholders.
  • Experience in a senior role, providing technical guidance and mentorship to junior team members.
  • Relevant certifications such as Databricks Certified Developer or Databricks Certified Professional are highly desirable.
  • Eligibility/Clearance Requirements Must be eligible to obtain a Department of Homeland Security EOD clearance (Requirements 1. US Citizenship, 2. Favorable Background Investigation)

Desired Qualifications

  • Experience in designing and implementing data ingestion pipelines, data transformations, and data quality processes using Databricks.
  • Experience working with federal government customers and compliance requirements (FedRAMP, FISMA)
  • Experience with CI/CD practices and version control for infrastructure (Git, GitHub/GitLab/Bitbucket, CI/CD pipelines)

Clearance Preference

  • Active DHS/CISA suitability - 1st priority
  • Any DHS badge + DoD Top Secret - 2nd choice
  • DoD Top Secret + willingness to obtain DHS/CISA suitability - 3rd choice (it can take 10-60 days to obtain suitability – work can only begin once suitability is fully adjudicated).

About Us

Formed through the strategic union of Sev1Tech and ERT, Entarian is a premier provider of mission-critical engineering and technology solutions. Founded on a legacy of excellence dating back to 1993, Entarian is a product of an evolved and fully diversified engineering and federal technology leader. From deep space to defense and civilian missions, Entarian delivers secure, mission-aligned digital solutions that drive national resilience and operational effectiveness. We don't just support modernization; we define it.

Join the Mission and Start your Career Journey: Apply Directly via our Careers Portal Connect, Referrals & Inquiries? Email the team: careers@entarian.com

Entarian is an Equal Opportunity and Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.

Sev1Tech LLC

About Sev1Tech LLC

Sev1Tech is a leader in providing innovative technology, high-end engineering and mission solutions and services. We use the latest methods and technology to solve the most challenging problems across a wide variety of functional areas, steadily earning our customers' trust, becoming an invaluable part of their organization, and advancing them beyond expectations. We deliver mission-focused subject matter expertise combined with corporate core competencies to serve the needs of our customers.

Sev1Tech focuses on corporate responsibility and excellence in service, both critical components in our business and operating policies. We take great pride in conducting our business with integrity and honesty and hold our team members to the same high standard of performance, quality and commitment to the organization. We believe that everyone on our team should improve our customer's missions and strengthen our communities.

Better Solutions. Faster. Three simple words, but when put together, they describe Sev1Tech to its core. Through years of experience, we've developed a proven approach using industry and technology expertise and repeatable processes to create solutions for our client's most significant challenges. Our solutions and services approach pairs functional experts with technology experts. Sev1Tech works collaboratively with our customers throughout the project lifecycle through agile, iterative, state-of-the-art technologies and methodologies and CMMI-based processes. The success of our approach is recognized by numerous customer recognitions and many awards, including one of the fastest-growing companies.

Sev1Tech strives to create an exciting and challenging work environment for all employees by providing opportunities to work on our country and community's toughest challenges in the IT and engineering fields. We are committed to ensuring our employees maintain up-to-date, relevant certifications, use proven solution methods and processes.

Industry
IT & Software
Company Size
501-1,000 employees
Headquarters
Woodbridge, Virginia
Year Founded
2010
Social Media