SiteGround

AI Evaluation Engineer

SiteGround  •  Sofia, BG (Onsite)  •  5 days ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

YOUR ROLE

At SiteGround, we’ve built and launched our own suite of AI products:

Coderick AI — our vibe coding platform.

AI Studio — access to flagship AI models, providers, and intelligent AI Agents.

WordPress AI Agent — one AI assistant for virtually every WordPress task.

We’re moving fast: adding new capabilities, launching new AI agents, and serving a growing number of customers who use our AI products every day.

This role sits at the heart of how we measure and improve those products. Your job is to make our LLM-powered features provably reliable — not just impressive in a demo.

Prompt engineering is a central part of the role. You’ll design, test, and iterate on prompts and agent behaviour directly. But your defining contribution will be rigour: turning “this seems to work” into measurable, repeatable, production-ready behaviour through evaluation and observability.

We work across the major model ecosystems — OpenAI, Google Gemini, and Anthropic — and you’ll regularly make pragmatic decisions about model choice, quality, latency, and cost.

You’ll work closely with backend engineers, fellow evaluation engineers, and product managers. Together, you’ll dig into real customer interactions, identify where our AI succeeds or fails, and turn those insights into better prompts, agents, tools, and product experiences.

YOUR RESPONSIBILITIES

OUR EXPECTATIONS

  • Strong prompt engineering and LLM application experience — you’ve shipped agents or LLM-powered features to production, not just prototyped them;
  • Hands-on evaluation experience and a data-driven mindset — you’ve built evaluation datasets and metrics, and you measure results before claiming something works;
  • Python experience and comfort consuming and integrating REST APIs;
  • Familiarity with modern LLM tooling and patterns, including function/tool calling, RAG, agentic workflows, observability platforms, and current model families;
  • Healthy scepticism toward AI output and a strong instinct for safety, failure modes, and guardrail design.

GREAT ADVANTAGE WILL BE

WHAT WE OFFER

  • Competitive remuneration and a performance-based bonus system;
  • Shorter workday on Fridays – we finish at 3:00 PM;
  • 5 extra days off at 5 years of service, plus 1 more each year after – up to 10;
  • 2 extra paid days off for volunteering to support the causes you care about;
  • Premium additional health insurance coverage and annual medical check-ups;
  • Modern and cozy offices across Bulgaria, plus flexible work options;
  • Chef-prepared breakfast and lunch provided daily at our HQ restaurant – all on us;
  • Free parking and a metro shuttle;
  • Fully equipped in-office gym with professional coaches;
  • On-site sports at our HQ: yoga, spinning, group conditioning, Brazilian Jiu-Jitsu, dancing, table tennis, and more;
  • Multisport or Coolfit cards fully covered by the company;
  • Free massages;
  • Memorable team events, internal gatherings, and company festivals;
  • Opportunities for professional development through trainings and conferences;
  • Knowledge-sharing culture through meetups and events;
  • Anniversary and company gifts along the way.

Come and make a difference at SiteGround with people who inspire you to grow!

Please note: Only shortlisted candidates will be contacted for further steps.

SiteGround

About SiteGround

SiteGround is a tech product and service company powering millions of online businesses worldwide.

From world-class web hosting, to SaaS platforms for site building, e-commerce, and email marketing, we develop all our products in-house to help businesses start, grow and succeed online more easily.

At the heart of our 20-year success is our people-driven company culture focused on performance, innovation, technical excellence, and early strategic adoption of AI.

Together, we are one of the most technically-advanced companies in our industry, recognized by over 90 of the most prestigious business, employer, and social responsibility awards worldwide.

Industry
IT & Software
Company Size
201-500 employees
Headquarters
Sofia, BG
Year Founded
Unknown
Social Media