adzup✦GOOD THINGS START WITH CURIOSITY.
STRATEGY × CREATIVE × TECHNOLOGYSay hello on WhatsApp ↗

AI QA Engineer

Test AI applications across model behaviour, integrations and user journeys so teams can release with clear evidence of quality.

Full-timeRemote · India
Apply for this role ↗

The opportunity

Build a practical quality discipline for applications that combine models, retrieval, tools and conventional software. You will turn product expectations into repeatable test cases and explain where model variability, weak grounding or integration failures create risk.

The role covers manual investigation and automated regression checks. Work with engineers and product colleagues to define useful acceptance criteria, keep representative test data and verify fixes before release. Use synthetic or appropriately approved data when testing sensitive workflows.

What you’ll do

  • Translate AI feature requirements into measurable acceptance criteria
  • Design evaluation cases for grounding, structured outputs, tool use and failure recovery
  • Build and maintain API, UI and integration regression checks
  • Test handling of untrusted inputs and inappropriate data exposure in authorised environments
  • Investigate defects with reproducible evidence and severity based on user impact
  • Track release quality and collaborate on fixes and retesting

What you’ll bring

  • Experience testing software through APIs and user interfaces
  • Ability to write automated checks in Python or TypeScript
  • Understanding of LLM variability, retrieval and evaluation methods
  • Ability to distinguish deterministic software defects from model-quality limitations
  • Clear defect reports, test documentation and cross-team communication

Nice to have

  • Experience with evaluation datasets and test orchestration
  • Familiarity with accessibility and performance testing
  • Experience testing business assistants or retrieval applications

Your toolkit

Quality engineeringLLM evaluationAPI testingPython or TypeScriptTest automationDefect analysis

Make your next move.

Share your CV, portfolio or relevant project links through the application form. Include the role title and job reference in your response.

Salary and role-specific working arrangements will be discussed during hiring.

Apply for AI QA Engineer. Include the job reference ADZ-015 in your response.

Open application form ↗

Applications are handled separately from this public job board. Privacy policy.