Remote opportunity at
CobaltStaff AI Engineer
Cobalt operates production AI agents designed to test live customer applications. The company is currently hiring a Staff AI Engineer to direct the technical strategy for…
Career Tools
About This Role
Cobalt operates production AI agents designed to test live customer applications. The company is currently hiring a Staff AI Engineer to direct the technical strategy for agent systems, enhance their reliability and security, and reduce operational costs. In this capacity, you will establish frameworks for evaluation, tracing, and execution controls while remaining hands-on with foundational builds. Your duties will span across research, product, security, and engineering departments, where you will…
Job Description
Cobalt operates production AI agents designed to test live customer applications. The company is currently hiring a Staff AI Engineer to direct the technical strategy for agent systems, enhance their reliability and security, and reduce operational costs. In this capacity, you will establish frameworks for evaluation, tracing, and execution controls while remaining hands-on with foundational builds. Your duties will span across research, product, security, and engineering departments, where you will guide engineers and transition experimental prototypes into robust production workflows. Additionally, you will assist internal teams in maximizing the utility of AI coding agents to boost productivity and code quality.
This position is ideal for an experienced software professional with extensive background in LLM-based agent operations, distributed systems, and technical mentorship. Cobalt provides competitive compensation alongside a comprehensive benefits package including medical, dental, vision, life insurance, a 401(k) program, wellness and home office stipends, flexible paid time off, and parental leave.
Responsibilities
- Define technical strategies for agent systems across departments, addressing orchestration, context handling, memory management, and failure recovery
- Design and operate core foundational infrastructure, encompassing evaluation suites, execution safeguards, and model routing
- Formulate evaluation metrics to gauge accuracy, task success, false positive rates, security posture, and financial cost
- Collaborate with subject matter experts to construct representative test cases and regression tests based on production incidents
- Build resilience into agent execution pipelines against timeouts, rate limits, and partial completions
- Partner with security teams to implement data isolation, scoped permissions, and secure tool execution protocols
- Set engineering benchmarks regarding CI evaluations, version control for models and prompts, cost tracking, and phased rollouts
- Lead model selection and financial optimization by benchmarking workloads against latency, quality, and expense trade-offs
- Guide internal engineering units on adopting AI coding assistants and evaluate their impact on software delivery speed
- Mentor technical leads and software engineers through design reviews, pair programming, and written documentation
Requirements
- Possess eight to ten or more years of software engineering background with documented technical leadership across multiple teams
- Demonstrate hands-on practice building and managing LLM-driven agents within a production setting
- Show a proven history of elevating agent reliability, handling non-deterministic failures, and creating evaluation frameworks
- Exemplify deep familiarity with agent architectures including tool integration, planning, orchestration, and memory handling
- Display experience utilizing managed cloud model platforms or direct provider APIs while managing rate limits and cost tracking
- Hold strong fundamentals in distributed backend systems covering concurrency, asynchronous workflows, and fault tolerance
- Understand security vulnerabilities inherent to agent setups such as prompt injection and sensitive data exposure
Qualifications
- Prior familiarity with Go, GCP, or Kubernetes
- Background in creating sandboxed execution environments for untrusted code or external interactions
- Experience in offensive security, application security, or related tooling
- Track record of deploying AI coding tools throughout engineering departments and measuring results
- Demonstrated public contributions, writing, or speaking engagements connected to agent engineering
Core Skills
Benefits
- Attractive equity plan
- 401(k) program for US employees or pension for EU staff
- Medical, dental, vision, and life insurance for US personnel, or statutory healthcare for EU personnel
- Wellness stipends
- Work-from-home equipment and internet stipends
- Learning and development stipends
- Flexible and generous paid time off
- Paid parental leave
Frequently Asked Questions
Answers are based only on the employer’s listing; where it doesn’t say, neither do we.
What is the salary range for the Staff AI Engineer position?
The salary range for this role is between $275,000 and $300,000 per year, accompanied by equity and benefits.
What remote work options are available?
The posting specifies the job location as remote.
What level of experience is required for this role?
Candidates must have significant software engineering experience, typically 8 to 10 plus years, along with a track record of technical leadership and hands-on LLM agent production experience.
What benefits are included with the employment?
Benefits include equity, a 401(k) program or pension, medical, dental, vision, and life insurance or statutory healthcare, stipends for wellness, home office equipment, and learning, plus flexible paid time off and parental leave.
Sample Interview Questions
AI-generated questions tailored to this specific role — a preview of the full practice set.