When you join the growing BILH team, you're not just taking a job, you’re making a difference in people’s lives.
The Senior AI Data Scientist role serves as a technical leader for complex machine learning and generative AI initiatives within the organization. This role is responsible for designing, fine-tuning, evaluating, and governing large and small language models (LLMs) and advanced machine learning models used in healthcare and operational settings. The Senior AI Data Scientist owns model quality, evaluation rigor, and responsible AI outcomes, while mentoring junior team members and guiding project-level technical direction. The position emphasizes model development, experimentation strategy, evaluation frameworks, and applied generative AI. The Senior AI Data Scientist partners closely with engineering, clinical, and data teams to ensure AI solutions are accurate, safe, explainable, and fit for production use.
Essential Duties & Responsibilitiesincluding but not limited to:
1. Leadthedesign, development, and evaluationof advanced machine learning and generative AI models across healthcare use cases.
2. Serveas a subjectmatter expert onlarge and small language models (LLMs), including architecture tradeoffs, inference behavior, strengths, and limitations.
3. Designand overseeLLM fine-tuning strategies, including instruction tuning andparameterefficientfine-tuning approaches, ensuring alignment with domain needs and model governance standards.
4. Defineand executemodel evaluation frameworksfor LLMs, including quantitative metrics, qualitative review, bias analysis, hallucination assessment, and safety testing.
5. Guideand reviewprompt design and optimization strategies, ensuring robustness, consistency, and interpretability of generative AI outputs.
6. Leadand improveretrieval ofaugmentedgeneration (RAG)solutions, including data selection, grounding quality, relevance evaluation, and retrieval performance analysis.
7. Conductadvancedexploratory data analysis, feature engineering, and statistical modelingto support predictive and prescriptive analytics.
8. Translatecomplex clinical or business problems intoclear modeling strategies and evaluation plans
9. Mentorentrylevel and midlevel data scientists through model reviews,bestpractice guidance, and collaborative problem solving.
10. Partnerwith data engineering, software engineering, and clinical teams to ensure AI models are deployable, monitored, and compliant with governance requirements.
11. Defineand promoteResponsible AI best practices, including fairness, bias mitigation, transparency, documentation, andappropriate clinicaluse.
12. Communicatemodel behavior, tradeoffs, and outcomes to technical leaders, clinicians, and executive stakeholders.
Minimum Qualifications:
Education:
Bachelor’sdegree inData Science, Computer Science, Artificial Intelligence, Statistics, Mathematics, or related quantitative disciplinerequired
Master’sdegree preferred.
Licensure, Certification & Registration:
Nonerequired
Experience:
5+ yearsof experience in data science, machine learning, or applied AI roles.
Demonstratedexperience leadingmodel development and evaluationefforts, including generative AI orNLP-basedsolutions.
Experiencementoring or guiding junior data scientists.
Skills, Knowledge & Abilities:
Deepunderstanding ofmachine learning and generative AI fundamentals
Handsonexperience working withLLMs, including finetuning concepts and inference tradeoffs.
Strongexpertiseinmodel evaluation, including both traditional ML metrics and qualitative LLM assessment.
AdvancedproficiencyusingPythonfor data analysis, modeling, and experimentation.
Stronganalytical judgment and ability to assess model risks and limitations.
Excellentwritten and verbal communication skills, especially for explaining complex AI behavior.
Abilitytooperateindependently on complex problems while providing technical leadership to others.
Preferred Qualifications & Skills:
Advancedexperience withLLM evaluation frameworks, test harnesses, or human-in-the-loop review.
Experiencedesigning or governingRAG architectures
Familiaritywith healthcare data, clinical terminology, or regulated data environments.
KnowledgeofAI ethics, fairness, bias mitigation, and responsible AI governance
Experiencecontributing to AI standards, documentation, orbestpracticeframeworks.
Exposureto production monitoring or model lifecycle management in regulated environments.
DeptUnit-Specific Skills:
KeyBusinessRelationships:(Title and Purpose)
1AI Data Scientists
Technical leadership, mentoring, and model review
2Data Engineering
Data availability, quality, and pipeline alignment
3Clinical Stakeholders
Validation of model relevance and safety
4AI Governance / Compliance
Responsible AI alignment and oversight
Pay Range:
$120,000.00 USD - $165,000.00 USD
The pay range listed for this position is the annual base salary range the organization reasonably and in good faith expects to pay for this position at this time. Actual compensation is determined based on several factors, that may include seniority, education, training, relevant experience, relevant certifications, geography of work location, job responsibilities, or other applicable factors permissible by law.
Beth Israel Lahey Health is a new, integrated system providing patients with better care wherever they are. Care informed by world-class research and education. We are doctors and nurses, technicians and social workers, innovators and educators, and so many others. All with a shared vision for what health care can and should be.
