Senior Analyst, Advanced Analytics: Auto Physical Damage (APD)

Description The Auto Physical Damage (APD) Data Science team builds and deploys data science products that power faster, more consistent, and more accurate claims outcomes. Our portfolio spans both traditional machine learning models and Generative AI systems (e.g., document summarization, LLM-driven decision support, and unstructured-data extraction). As our model footprint grows, ensuring these systems remain accurate, reliable, and trustworthy in production is mission-critical.We are seeking a Model Monitoring Analyst to design, build, and operate the systems that keep our production models healthy. You will be the owner of model observability across the APD portfolio - establishing how we detect performance degradation, data drift, and anomalous behavior for both classical ML and GenAI systems. This is a highly visible role that partners closely with data scientists, ML engineers, claims business partners, and model governance teams. Candidates who live within 50 miles of Boston, MA; Portsmouth, NH; Seattle, WA; Columbus, OH; or Plano, TX will follow a hybrid schedule, coming into the office two days per week. Otherwise, this role is remote with occasional travel. Key ResponsibilitiesBuild monitoring infrastructure for production models, covering both traditional ML and GenAI/LLM systems, including automated pipelines, dashboards, and alerting.Define and track model health metrics - for ML: accuracy, precision/recall, AUC, calibration, feature and prediction drift. For GenAI: output quality, hallucination/grounding checks, relevance, latency, token/cost usage, and guardrail adherence.Detect and diagnose issues such as data drift, concept drift, performance decay, and data-quality breaks, then triage and escalate to the appropriate model owners.Establish thresholds and alerting that balance early detection with alert fatigue, and document expected behavior and remediation runbooks.Partner with data scientists and ML engineers to integrate monitoring into the mode