Remote opportunity at
AnaplanSenior Software Development Engineer in Test - AI
Anaplan seeks a Senior Software Development Engineer in Test for AI to lead the quality assurance efforts for its scenario planning and analysis platform. The organization…
Career Tools
About This Role
Anaplan seeks a Senior Software Development Engineer in Test for AI to lead the quality assurance efforts for its scenario planning and analysis platform. The organization provides enterprise planning technology utilized by numerous global companies. In this newly created position, the professional will shape testing strategies, evaluation frameworks, and quality metrics specifically for applications powered by large language models and agentic workflows. This opportunity suits an experienced quality assurance professional…
Job Description
Anaplan seeks a Senior Software Development Engineer in Test for AI to lead the quality assurance efforts for its scenario planning and analysis platform. The organization provides enterprise planning technology utilized by numerous global companies. In this newly created position, the professional will shape testing strategies, evaluation frameworks, and quality metrics specifically for applications powered by large language models and agentic workflows. This opportunity suits an experienced quality assurance professional with strong automation skills and expertise in artificial intelligence behaviors who wants to ensure that AI features remain trustworthy, accurate, and reliable.
The role involves building comprehensive testing suites, performing adversarial red-team testing to mitigate model biases and hallucinations, and setting up automated prompt testing systems. The engineer will also create internal tools that allow software teams to validate their artificial intelligence implementations seamlessly and deploy real-time monitoring to spot production quality degradation.
Responsibilities
- Design and architect test plans, quality metrics, and evaluation frameworks for LLM-powered applications and agentic workflows
- Develop automated end-to-end test suites using Python across UI checks, evaluations, API checks, and performance tests
- Build automated regression suites and prompt testing systems to detect unintended changes in model behavior
- Establish multi-dimensional evaluation criteria to measure GenAI quality across consistency, relevance, accuracy, safety, and latency
- Conduct adversarial red-team testing to uncover and mitigate security vulnerabilities, model biases, and hallucinations
- Build internal tools and testing frameworks to help software engineering teams validate their GenAI implementations
- Deploy alerting and real-time monitoring systems to quickly catch quality drops in production AI features
- Maintain golden example test datasets and define quality assurance best practices for the workstream
Requirements
- Deep understanding of generative AI technologies including prompt engineering, LLMs, and AI application patterns
- Hands-on Python coding experience
- Experience with test automation frameworks and scripting using JavaScript, Python, Pytest, or Selenium
- Background in software testing methodologies covering functional, regression, integration, performance, and security testing
- Ability to design test cases and evaluation criteria for non-deterministic systems
- Experience with backend testing and API testing tools like REST Assured and Postman
- Experience integrating automated testing into CI/CD pipelines
Core Skills
Frequently Asked Questions
Answers are based only on the employer’s listing; where it doesn’t say, neither do we.
What is the remote work policy for this position?
The source posting states that the position is remote.
What employment type is this role?
The posting specifies that this is a full-time employment opportunity.
What compensation is provided for this role?
The salary is not stated in the source job posting.
Sample Interview Questions
AI-generated questions tailored to this specific role — a preview of the full practice set.