AI Solutions Architect (USA)
Testlio is seeking a senior-level AI Solutions Architect for a 100% remote architecture role within the United States. In this position, you will own the technical design, packaging, definition, and delivery methodology for Testlio's AI testing offerings, combining deep practitioner expertise with a strong business perspective.
Role Responsibilities and Design Work
- Design testing approaches for complex AI-powered applications, including agentic systems.
- Advance Testlio's proprietary framework, which combines automated and human-in-the-loop evaluation.
- Define the service catalog, pricing structure, and staffing models for what Testlio sells as a service.
- Decompose complex ecosystems and architectures into structured testing criteria.
- Collaborate with sales and delivery teams to drive the AI business forward, join sales calls, and guide delivery on AI testing opportunities.
- Interface directly with data scientists, machine learning engineers, and executive sponsors at global brands.
- Define and maintain the AI service catalog offerings, tiers, packaging, and delivery methodology while tracking and presenting key metrics to leadership.
Requirements and Background
- Education: Bachelor's degree required.
- Experience: 7+ years of relevant experience.
- A visionary and hands-on approach to AI quality blueprints, evaluation systems, and validation of large language models and data pipelines.
Company Culture and Benefits
Testlio is a female-founded, growing, profitable, and cash-strong company supporting a fully remote, globally distributed team. Benefits include a culture built around asynchronous work, flexible paid time off covering national holidays, personal days, and sick days, stock options, and a $300 annual learning stipend.