Remote opportunity at
ToptalML Inference Engineer – Terminal-Bench Task Designer
Toptal is building a benchmark designed to test the ability of artificial intelligence agents to manage actual professional work within a terminal. To support this initiative,…
Career Tools
About This Role
Toptal is building a benchmark designed to test the ability of artificial intelligence agents to manage actual professional work within a terminal. To support this initiative, the company seeks remote ML inference and serving experts to convert authentic, paid tasks from their professional careers into advanced Terminal-Bench challenges. Every single challenge consists of an execution environment, a functional solution, and an automated verification method confirming accuracy. This role utilizes a…
Job Description
Toptal is building a benchmark designed to test the ability of artificial intelligence agents to manage actual professional work within a terminal. To support this initiative, the company seeks remote ML inference and serving experts to convert authentic, paid tasks from their professional careers into advanced Terminal-Bench challenges. Every single challenge consists of an execution environment, a functional solution, and an automated verification method confirming accuracy.
This role utilizes a task-based compensation model where professionals are compensated per task. Each individual task is sized to require approximately 60 to 75 minutes of professional effort. Candidates will deliver their work incrementally, incorporating feedback provided by reviewers.
This opportunity is ideal for experienced machine learning practitioners who possess deep, hands-on knowledge of model inference, serving stacks, and command-line operations, and who want to contribute to the evaluation of AI agents.
Responsibilities
- Design expert-level Terminal-Bench challenges by transforming real, paid machine learning inference work from personal professional experience into scenarios that artificial intelligence agents can solve from end to end
- Specify the files, supporting systems, and operating environments for each challenge, including model serving scenarios featuring streaming, to ensure realistic, reproducible, and command-line solvable tasks
- Develop a functional reference solution for every challenge and verify that it executes properly from start to finish
- Create automated validation checks that objectively confirm result accuracy, accounting for silent machine learning output failures such as metric bugs and numerical drift
- Package challenges using Git and Docker for reproducibility while documenting the data, tools, systems, and rules so reviewers can assess task difficulty
- Refine challenges based on agent test runs and reviewer feedback by adjusting complexity and resolving ambiguities to keep tasks fair, gradable, and at an expert level
Requirements
- Accumulate at least three years of hands-on experience in machine learning inference or serving, or hold a PhD combined with active research in the field
- Perform this professional work within the past three years with the ability to name the specific rules, data, tools, and systems utilized
- Train, serve, or evaluate models at a genuine scale such as production serving or multi-GPU environments
- Possess practical familiarity with JAX or PyTorch internals as well as serving frameworks such as TensorRT-LLM, SGLang, or vLLM
- Articulate how machine learning outcomes can fail without throwing errors, including numerical drift and metric bugs, alongside methods to detect them
- Demonstrate technical proficiency with the command line, Python, databases, APIs, JSON or CSV formats, and files
- Maintain availability to commence work by early November 2026 and execute tasks requiring roughly 60 to 75 minutes each
Qualifications
- Prior experience working in artificial intelligence laboratories or with large language model infrastructure
- History of contributions to projects like vLLM, SGLang, or PyTorch
- Background in graphical processing unit profiling
- Proficiency with Git and Docker
- Experience in authoring automated tests
Core Skills
Frequently Asked Questions
Answers are based only on the employer’s listing; where it doesn’t say, neither do we.
Is this job remote?
Yes, the posting lists the location as remote.
What is the compensation model for this position?
The role uses a task-based compensation model where professionals are paid per task, with each task taking approximately 60 to 75 minutes.
What are the key technical requirements for applicants?
Candidates need at least three years of hands-on ML inference or serving experience (or a PhD with active research), practical work with frameworks like vLLM, SGLang, or TensorRT-LLM, and technical fluency in Python and the command line.
What is the interview or vetting process?
Instead of a traditional interview, candidates complete a vetting task so the client can evaluate their task design abilities.
What is the salary for this role?
The posting does not specify a salary.
Sample Interview Questions
AI-generated questions tailored to this specific role — a preview of the full practice set.