Education: PhD / Specialized Diploma
Experience: Minimum 7 Years
Location: USA
Salary: Market Competitive
Welo Global is seeking a senior-level Lead Prompt Engineer for Project Lion, a remote engineering role based in the United States. Working on a part-time basis, you will guide and manage a team through the full technical migration process of transitioning templates to LLM autoraters. Your core focus will be leveraging advanced prompt engineering techniques and internal tools to optimize model performance, drive strategy, and mentor junior engineers.
Core Responsibilities
- Utilize Automatic Prompt Generation (APG) tools to create baseline prompts for complex parent-child template clusters.
- Run and supervise Automated Prompt Optimization (APO) tools, reviewing outputs and flagging deadlocks or plateaus.
- Manually draft, test, and refine prompts to handle edge cases, complex architectures, and anti-patterns.
- Monitor shadowbot runs to track human and LLM rating disagreements.
- Measure autorater quality against human crowd baselines by running prompt versions against gold data, calculating F1 scores, precision, and recall.
- Draft technical launch readiness justifications for Launch Certification Documentation.
Requirements and Qualifications
- Must be based in the United States with native fluency in English.
- Master’s or Doctorate degree in Computer Science, Data Science, Computational Linguistics, Human-Computer Interaction (HCI), Cognitive Science, or a related analytical field.
- Minimum of 7 years of experience as a Prompt Engineer, with proven expertise tuning Large Language Models (LLMs) for strict, structured outputs and complex classification tasks.
- Strong data analysis skills, including proficiency in SQL or other analytics tools and identifying error patterns.
- Ability to quickly master proprietary tools with minimal supervision and communicate effectively.
Compensation for this remote engineering position is market competitive. All persons hired must comply with federal law regarding employment eligibility verification, including completing a live video verification and providing photo IDs within their first 3 days of employment.