AECOM

LLM Quality Engineer

AECOM  •  Oslo, NO (Onsite)  •  11 days ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

Selskapsbeskrivelse

Work with Us. Change the World.

At AECOM, we're delivering a better world. Whether improving your commute, keeping the lights on, providing access to clean water, or transforming skylines, our work helps people and communities thrive. We are the world's trusted infrastructure consulting firm, partnering with clients to solve the world’s most complex challenges and build legacies for future generations.

There has never been a better time to be at AECOM. With accelerating infrastructure investment worldwide, our services are in great demand. We invite you to bring your bold ideas and big dreams and become part of a global team of over 50,000 planners, designers, engineers, scientists, digital innovators, program and construction managers and other professionals delivering projects that create a positive and tangible impact around the world.

We're one global team driven by our common purpose to deliver a better world. Join us.

Jobbeskrivelse

In AECOM’s AI Engineering team your code will directly shape the physical world around us. We build AI-driven technology that revolutionises how infrastructure and buildings are designed and engineered; reducing waste, cutting CO₂, and making the built environment more efficient and sustainable. This is where software has measurable, real-world impact.

With our AI Engineering team we’ve created a unique setup: a lean, highly technical team with the speed and ownership of a start up, backed by the scale, resources, and domain expertise of one of the world’s leading engineering firms. 

There has never been a better time to be at AECOM. We are leading the industry’s AI transformation, and with our people and technology we deliver excellence and innovate with impact.

We invite you to bring your bold ideas and big dreams to solve the world’s most complex challenges. We're one global team driven by our common purpose to deliver a better world. Join us.

What You’ll Do

As part of the Assurance & Insight team, you will focus on ensuring the quality and reliability of our AI-driven systems. This role sits at the intersection of testing, AI evaluation, and operational quality.

You will help build the foundations for how we test, benchmark, and monitor both software systems and LLM-driven outputs ensuring that what we deploy performs as expected in real-world scenarios.

You will take ownership of quality across our AI systems, from automated testing to LLM evaluation and production monitoring. Your work will help reduce regressions, improve reliability, and free up engineering time to focus on building.

We're not looking for years of experience. We're looking for someone detail-oriented who wants to own a domain, can write clean Python, and is genuinely curious about how AI systems behave. It's a good fit for a recent graduate who wants end-to-end ownership early in their career, is comfortable working close to AI systems, and cares about making quality measurable rather than subjective.

Kvalifikasjoner

Must-Have Qualifications

  • Degree in computer science, software engineering, or a related technical field or equivalent demonstrable experience
  • Exposure to LLMs or AI systems (even personal projects or coursework counts)
  • Familiarity with test frameworks such as pytest or similar
  • Understanding of CI/CD concepts and release pipelines
  • Comfortable working with Python and APIs
  • Experience writing or evaluating prompts for language models
  • Strong attention to detail and a methodical approach to problem-solving

Preferred Skills

  • Exposure to AI/LLM tools or projects
  • Experience with testing, QA, or automation (academic or personal projects)
  • Interest in data quality, monitoring, or analytics
  • Experience working in team-based or project-based environments

Ytterligere informasjon

Our Hiring Process

  • 25-minute screening call
  • Take-home challenge: A hands-on task to assess your problem-solving and technical skills
  • Combined technical and cultural interview (in-person)
    • Whiteboard Interview: 1-hour with 2 of our engineers to discuss your solution to the take-home challenge
    • Culture fit: 30-minute meeting with our leadership team

Why Join Us?

  • Work on real-world problems where AI creates measurable impact.
  • Be part of a team where your work matters, and your ideas become real.
  • Collaborate with sharp, driven colleagues in a culture of trust, ownership, and high standards.
  • Contribute to making the built environment smarter and more sustainable.
AECOM

About AECOM

AECOM is the global infrastructure leader, committed to delivering a better world. As a trusted professional services firm powered by deep technical abilities, we solve our clients’ complex challenges in water, environment, energy, transportation and buildings. Our teams partner with public- and private-sector clients to create innovative, sustainable and resilient solutions throughout the project lifecycle – from advisory, planning, design and engineering to program and construction management. AECOM is a Fortune 500 firm that had revenue of $16.1 billion in fiscal year 2024. Learn more at aecom.com.

Industry
Construction & Skilled Trades
Company Size
10,000+ employees
Headquarters
Dallas, Texas
Year Founded
Unknown
Website
aecom.com
Social Media