Remote opportunity at
PavagoAI Operations Specialist - LLM & Conversation Systems
Pavago is hiring an AI Operations Specialist for a full-time, remote position operating during U.S. Business Hours. In this role, you will work directly with the…
Career Tools
About This Role
Pavago is hiring an AI Operations Specialist for a full-time, remote position operating during U.S. Business Hours. In this role, you will work directly with the founder to manage, monitor, troubleshoot, and optimize production AI conversation systems for an AI-powered SMS lead recovery platform that serves telehealth brands and handles millions of monthly messages. This is a hands-on technical role focused on real-world execution rather than experimentation with AI tools.…
Job Description
Pavago is hiring an AI Operations Specialist for a full-time, remote position operating during U.S. Business Hours. In this role, you will work directly with the founder to manage, monitor, troubleshoot, and optimize production AI conversation systems for an AI-powered SMS lead recovery platform that serves telehealth brands and handles millions of monthly messages. This is a hands-on technical role focused on real-world execution rather than experimentation with AI tools.
You will onboard new clients, configure conversation workflows, manage knowledge bases and webhook integrations, and review live customer interactions. Your core duties include prompt engineering, diagnosing LLM failure modes, conducting iterative testing, and analyzing how system updates impact lead conversion and engagement. The position suits an independent, detail-oriented professional with strong analytical and written communication skills who has proven experience operating production-level LLM applications.
Responsibilities
- Onboard new clients and configure client-specific AI conversation systems, workflows, knowledge bases, and webhook or API integrations
- Monitor live AI-generated customer interactions, review conversation transcripts, and diagnose errors or missed conversion opportunities
- Write, edit, and optimize system prompts and customer-facing messaging instructions to improve response quality and consistency
- Conduct structured testing and evaluate AI behavior before and after implementing prompt or workflow changes
- Analyze conversation performance and customer data to track lead recovery, engagement, and conversion rates
- Collaborate directly with the founder on daily AI operations, document system modifications, and prioritize issue resolutions
Requirements
- Proven hands-on experience operating, monitoring, troubleshooting, and improving LLM or AI systems in production
- Strong understanding of Large Language Models and their behavior in real-world customer-facing applications
- Practical experience with prompt engineering, system prompt optimization, and resolving LLM failure modes
- Familiarity with webhooks, APIs, knowledge bases, or similar technical integrations
- Strong analytical, problem-solving, documentation, and communication skills with excellent written English
- Ability to work independently during U.S. Business Hours and collaborate directly with a founder in a fast-paced environment
Qualifications
- Experience managing conversational AI systems or AI-powered messaging platforms
- Experience optimizing LLM-generated responses specifically for customer engagement or conversion
- Familiarity with SMS-based customer communication workflows and lead generation or recovery systems
- Experience supporting SaaS platforms, customer-facing AI products, or telehealth environments
- Background in conducting structured prompt evaluations, regression testing, and analyzing large volumes of conversation transcripts
Core Skills
Frequently Asked Questions
Answers are based only on the employer’s listing; where it doesn’t say, neither do we.
Is this position remote?
Yes, the source posting explicitly states that this is a remote position.
What are the working hours for this role?
The posting specifies that the working hours are during U.S. Business Hours.
What is the salary for the AI Operations Specialist role?
The salary is not stated in the job posting.
What level of AI experience is required?
Candidates must have hands-on, real-world experience operating LLM or AI systems in production rather than simple experimentation with AI tools.
Sample Interview Questions
AI-generated questions tailored to this specific role — a preview of the full practice set.