Senior Engineer, Competitive Benchmarking & Performance

Career Tools

About This Role

Elastic builds search AI platforms utilized by organizations globally for observability, security, and retrieval applications. The firm is currently hiring a Senior Software Engineer to spearhead performance evaluations and competitive benchmarking initiatives within the Elasticsearch Performance team. In this position, you will drive testing projects, build automated testing workflows, and examine how Elasticsearch functions against alternative architectures. The role blends hands-on engineering execution with public technical publishing, collaborating closely with…

Job Description

Responsibilities

• Manage prioritized competitive benchmarking initiatives spanning search, vector retrieval, hybrid search, observability, and analytical workloads
• Construct and maintain automated testing harnesses, datasets, query sets, and load profiles that mirror production environments
• Design equitable system comparisons by documenting software versions, tuning choices, resource limits, and configurations
• Analyze throughput, latency distributions, cost efficiency, and resource utilization across deployments
• Measure retrieval performance against comparable relevance or recall targets for retrieval operations
• Modify and scale relevant benchmarking utilities such as ClickBench, Elasticsearch Rally, and vector search tools
• Examine behavioral disparities using profiling, system metrics, and controlled tests alongside engineering units
• Assess managed cloud architectures and self-managed deployments, accounting for service limitations and scaling dynamics
• Collaborate with technical content creators to publish reproducible evaluation artifacts and clear technical papers
• Work with product management and engineering groups to translate benchmark conclusions into product enhancements

Requirements

• Solid software engineering background with the ability to construct maintainable analysis tools and automation using Python or alternative programming languages
• Practical experience diagnosing and assessing performance in search infrastructure, databases, or distributed systems
• Deep expertise in at least one relevant domain such as vector retrieval, search, observability systems, or analytical databases
• Experience creating representative datasets, concurrency models, ingestion patterns, and query behaviors
• Sound experimental methodology, accounting for cache impacts, warm-up phases, measurement noise, and variability analysis
• Experience setting up reproducible test environments with well-documented procedures and configurations
• Clear communication and technical writing capabilities to articulate intricate findings and limitations
• Collaboration skills to transform performance data into actionable engineering proposals

Qualifications

• Prior background working with Lucene, Elasticsearch, Rally, ClickBench, or vector search testing frameworks
• Familiarity with evaluating retrieval quality concurrently with throughput, resource consumption, and latency
• Knowledge of cloud infrastructure cost and performance analysis, including serverless architectures and autoscaling
• History of published benchmark studies, open-source code contributions, or public technical writing

Core Skills

Benefits

• Base salary ranging from CAD 128,300 to 203,000 per year
• Eligibility to participate in the company stock program
• Company-matched Registered Retirement Savings Plan matching dollar-for-dollar up to 6% of qualifying earnings
• Health coverage for employees and their families in many regions
• Flexible schedules and locations for many roles
• Generous yearly vacation days
• Financial donation matching up to $2,000 or regional currency equivalent
• Up to 40 hours annually for volunteer projects
• Minimum of 16 weeks of paid parental leave

Frequently Asked Questions

Answers are based only on the employer’s listing; where it doesn’t say, neither do we.

Sample Interview Questions

AI-generated questions tailored to this specific role — a preview of the full practice set.