Opportunity at
CanvaPrincipal Research Scientist - Evaluations
Canva is looking for a Principal Research Scientist to set the course for how generative output is judged across Canva Research. This is a company-wide leadership…
Career Tools
About This Role
Canva is looking for a Principal Research Scientist to set the course for how generative output is judged across Canva Research. This is a company-wide leadership position in research craft rather than a single project, and it tackles evaluation questions nobody has formalised yet, such as new kinds of output and measurement that accounts for how formats, audiences and regions diverge. The scientist will define long-term measurement approaches for design,…
Job Description
Canva is looking for a Principal Research Scientist to set the course for how generative output is judged across Canva Research. This is a company-wide leadership position in research craft rather than a single project, and it tackles evaluation questions nobody has formalised yet, such as new kinds of output and measurement that accounts for how formats, audiences and regions diverge.
The scientist will define long-term measurement approaches for design, imagery, video, sound and multi-step AI agents. A central task is keeping automated graders and benchmarks trustworthy: comparing them with human preference and real product results, quantifying grader bias, and noticing early when a benchmark stops being informative. The position also expects the holder to connect measurement with how people actually experience a product, including how easily outputs can be edited, and to recognise that a good grader does not automatically make a good training reward.
The work spans research teams in Australia, Europe, the United States and China. The principal will create a shared measurement foundation so that every site, from London and Vienna to Sydney and China, produces comparable results. Mentoring senior scientists and engineers, presenting the strategy to senior leaders, and deciding what to do when scores and human judgement disagree are all part of the role.
Responsibilities
- Define long-term measurement strategy for design, imagery, video, audio and agent-driven workflows
- Check automated graders against human preference and product outcomes, including grader bias and benchmark decay
- Advise on how to split budget between compute, paid human annotation and measurement precision
- Set launch criteria and make the final call when results are unclear
- Investigate odd scores during production training to separate true regressions from pipeline faults
- Help design, inference and agent teams use evaluation to guide training
- Convert designers' taste into criteria that can be measured
- Coach senior scientists and engineers
Requirements
- A history of building assessment frameworks and standards from scratch
- Experience tying scores to business or user results and explaining the gaps between them
- Skill in turning subjective taste into reliable signals using rubrics and annotation pipelines
- Solid knowledge of multimodal generators, including diffusion models, transformers, vision-language and multimodal LLMs
- Practical work on reward modelling, preference learning and feedback-based alignment
- Track record of guiding technical direction across several groups spread over time zones
Qualifications
- Background in perception science, psychophysics, aesthetics or human-computer interaction
- Experience assessing harm, bias and safety alongside output quality
- Published work on evaluation, alignment or generative models
- Interest in or practice within illustration and graphic design
Core Skills
Benefits
- Equity
- Health coverage plans
- 401(k) plan with employer contributions
- Parental leave policy covering all parents and carers
- Yearly Vibe and Thrive allowance covering wellbeing, social activities and home-office setup
- Flexible leave options
Frequently Asked Questions
Answers are based only on the employer’s listing; where it doesn’t say, neither do we.
Where is this role located?
The listing names San Francisco and marks the position as on-site.
What is the target salary?
The posting sets a target range of $310,000 to $420,000 and says offers follow market data and each candidate's background.
What experience does the role require?
The posting asks for a history of building measurement frameworks from scratch, hands-on work with reward modelling or preference learning, and a solid grounding in multimodal generators.
How are interviews run?
Interviews take place virtually, according to the posting.
Sample Interview Questions
AI-generated questions tailored to this specific role — a preview of the full practice set.
