Remote opportunity at
ClickUpSenior AI Developer
ClickUp operates as an AI-native company building a converged productivity platform that integrates calendars, chat, documents, tasks, and enterprise search through context-driven artificial intelligence. The organization…
Career Tools
About This Role
ClickUp operates as an AI-native company building a converged productivity platform that integrates calendars, chat, documents, tasks, and enterprise search through context-driven artificial intelligence. The organization evaluates AI fluency during recruitment and expects team members to utilize AI daily. To support this mission, ClickUp seeks a Senior AI Developer to join the engineering team with a primary focus on artificial intelligence evaluations. In this remote, full-time position, the chosen candidate…
Job Description
ClickUp operates as an AI-native company building a converged productivity platform that integrates calendars, chat, documents, tasks, and enterprise search through context-driven artificial intelligence. The organization evaluates AI fluency during recruitment and expects team members to utilize AI daily. To support this mission, ClickUp seeks a Senior AI Developer to join the engineering team with a primary focus on artificial intelligence evaluations.
In this remote, full-time position, the chosen candidate will design measurement systems to track quality across AI features, build core agent harnesses, and develop backend architectures that ensure AI capabilities remain scalable, secure, and reliable. The role involves working directly with large language models to address user challenges and advance product innovation. Responsibilities include constructing golden datasets, setting up automated scorers and LLM-as-judge frameworks, managing model selection and routing across multiple providers, and implementing self-improvement loops where agents modify prompts and rerun evaluations automatically.
This opportunity is well-suited for an experienced software engineer with a strong background in production machine learning systems, backend architecture, and experiment design. Candidates must be comfortable collaborating across cross-functional product and data teams while adhering to security and privacy best practices for artificial intelligence.
Responsibilities
- Design and own the artificial intelligence evaluation system, incorporating golden datasets, automated scorers, regression detection, and quality tracking
- Construct and evolve core agent harnesses and develop the underlying AI to perform effectively on evaluations
- Build automated self-improvement loops allowing agents to alter prompts, rerun evaluations, and iterate independently
- Evaluate new models upon release to manage model selection, routing, and fallback strategies across various providers while balancing cost, quality, and latency
- Implement scalable platform services for the orchestration, deployment, and lifecycle management of machine learning models
- Develop backend APIs and infrastructure for model serving, monitoring, and logging to guarantee reliability in production
- Drive security, privacy, and compliance standards including data anonymization and regulatory adherence
- Partner with product, frontend, and data teams to deliver integrated artificial intelligence experiences
Requirements
- Minimum of 5 years of software engineering experience
- At least 2 years of experience building production machine learning or large language model systems
- Hands-on expertise constructing production evaluation systems such as golden datasets, offline and online evaluations, and judge calibration
- Demonstrated background in experiment design, statistical rigor, A/B testing, and metric separation
- Practical record of deploying artificial intelligence and large language models into customer-facing features
- Experience integrating and handling multiple large language models, including benchmarking and tradeoff analysis
- Extensive expertise in backend engineering, distributed systems, and cloud-native technologies like Kubernetes, Docker, and major cloud platforms
- Proficiency in Python, Go, TypeScript, or similar programming languages used for backend and platform development
- Familiarity with orchestration frameworks and workflow engines including LangGraph, Airflow, or Ray
- Understanding of search and retrieval technologies integrated into AI applications
Qualifications
- Background in data science, applied science, or machine learning engineering
- Familiarity with evaluation tooling such as Braintrust, LangSmith, Weave, Arize, or proprietary frameworks
- Experience involving model routing, AI gateways, or inference optimization
Core Skills
Frequently Asked Questions
Answers are based only on the employer’s listing; where it doesn’t say, neither do we.
Is this position remote?
Yes, the posting lists the location as remote.
What is the salary range for this role?
The salary ranges from USD 200,000 to 250,000 per year.
What level of experience is required?
Candidates must have at least 5 years of software engineering experience, including a minimum of 2 years building production machine learning or large language model systems.
Are visa sponsorships available?
Visa sponsorship is not available for roles outside of engineering and product at this time, and sponsorship for engineering and product roles is decided based on business needs.
Sample Interview Questions
AI-generated questions tailored to this specific role — a preview of the full practice set.