Senior AI QA Engineer
Engineering
London
Full-time
About the Role
We are seeking a Senior AI QA Engineer to lead the design and execution of advanced testing strategies for Large Language Models (LLMs), Generative AI, and AI-driven systems. You will play a key role in ensuring model accuracy, reliability, safety, and consistency across real-world use cases.
This role is ideal for someone passionate about AI quality, automation, and validation at scale.
What You’ll Do
Design and implement AI testing frameworks for LLMs and GenAI systems
Validate factual accuracy, hallucination rates, and response consistency
Test model behavior across edge cases, adversarial prompts, and failure scenarios
Define and track AI quality metrics (accuracy, error rate, bias, drift)
Collaborate with AI engineers, product teams, and cloud specialists
Contribute to internal tools, documentation, and best practices for AI QA
Requirements
- Strong experience in AI, ML, or software quality assurance
- Hands-on experience testing LLMs, chatbots, or GenAI systems
- Understanding of:
Factual accuracy validation
Consistency across multiple runs
Edge case handling
Read Error rate measurement.
- Experience with automation, Python, or testing frameworks is a plus
- Strong analytical and problem-solving skills
Nice to Have
Experience with RAG systems and prompt evaluation
Knowledge of bias, safety, and ethical AI testing
Experience working with cloud platforms (AWS, GCP, Azure)
Why Join AI Test Group?
Work at the forefront of AI-powered quality assurance
Remote-first and flexible work culture
Access to premium AI tools and cloud credits
Continuous learning and certification support
Opportunity to shape the future of AI testing
Benefits & Perks
Private health and dental coverage
Flexible working hours and remote options
Annual learning and AI conference budget
Performance-based bonuses
Home office setup allowance