MoodysNWC is seeking a highly analytical Campaign & A/B Test Design and Performance Measurement Analyst to evaluate and optimize the effectiveness of an AI-driven customer engagement platform. This role sits at the intersection of digital marketing, analytics, and AI performance measurement, translating complex customer interactions into actionable insights that drive conversion, revenue, and customer experience improvements.
You will partner closely with Marketing Analytics, Data Science, and Operations teams to design rigorous experimentation frameworks, measure AI impact versus traditional channels, and scale data-backed optimization strategies across products and customer segments.
Role & Responsibility
Design and implement measurement frameworks to assess AI-driven customer interactions, including conversion, close rates, and revenue impact Establish baseline KPIs and benchmarks across products, channels, and operational sites Build dashboards and reporting to track AI performance relative to human-assisted experiences Analyze customer behavior patterns, conversation flows, and conversion paths within AI-led interactions Design and execute A/B and multivariate tests comparing AI-assisted experiences with traditional approaches Develop control and treatment methodologies for phased rollouts and pilot programs Define statistical significance thresholds, sample size requirements, and test governance standards Partner with data science teams to support continuous testing and optimization of AI models Analyze offer performance across customer segments and channels, including take rates and lifetime value Evaluate offer survivability, promotional effectiveness, and margin contribution Collaborate with modeling teams to validate and improve offer ranking and decisioning logic Quantify revenue impact and performance trade-offs across different offer strategies Translate customer interaction data and call transcript insights into actionable recommendations Identify high-performing behaviors and patterns to inform AI training and optimization Deliver clear, concise performance reports to business stakeholders Develop documentation and use cases to support AI learning and future experimentation Skills
3–5 years of experience in campaign measurement, experimentation, or performance analytics Strong understanding of statistical concepts including hypothesis testing, control groups, and significance testing Proficiency in SQL, Tableau (or similar BI tools), and Excel Experience working with large datasets and building performance dashboards Proven ability to design clean, well-controlled experiments in real-world environments Understanding of customer lifecycle metrics such as churn, retention, survivability, and lifetime value Experience analyzing offer performance, promotional effectiveness, or sales outcomes Ability to identify patterns in both quantitative and qualitative data Comfortable making recommendations in environments with imperfect or evolving data Experience measuring or evaluating AI/ML-driven products or models Familiarity with conversational AI, chatbots, or voice-based platforms Exposure to call center, subscription-based, or retention-focused business models Experience working with call transcript or conversation analytics tools Basic understanding of propensity modeling, decisioning engines, or margin analysis About us
Moodys NWC is a result-driven management consulting firm headquartered in the Seattle metropolitan area. We specialize in best-fit consulting based on client needs, with a focus on all aspects of digital product management. We have an active role that we are looking to assist our premier client with and are seeking an experienced consultant with the right set of values. For us, personality fit is paramount. We look for self-starters who are experienced yet flexible in their approach and who maintain a customer-focused outlook. Our consultants go above and beyond, stay focused on the end goal, and do what it takes to deliver results. If you believe you align with the above qualities, please review the opportunity and share why you are a strong fit for Moodys NWC. Our compensation reflects the cost of labor across several US geographic markets. The salary for this position ranges from $120,000/year in our lowest geographic market up to $160,000/year in our highest geographic market. Pay is based on factors including market location and may vary depending on knowledge, skills, and experience. Additional compensation includes medical, vision, and dental benefits. Moodys NWC believes in a culture of diversity, acceptance, and belonging. We are committed to equal opportunity in hiring without discrimination based on race, color, creed, national origin, sex, marital status, age, sexual orientation, gender identity, disability, medical status, or veteran status. Moodys NWC complies with federal and state disability laws and makes reasonable accommodations for applicants and candidates with disabilities. If you require reasonable accommodation during the application or interview process, please submit a contact form here: https://moodysnwc.com/contact-us/
#J-18808-Ljbffr
Design and implement measurement frameworks to assess AI-driven customer interactions, including conversion, close rates, and revenue impact Establish baseline KPIs and benchmarks across products, channels, and operational sites Build dashboards and reporting to track AI performance relative to human-assisted experiences Analyze customer behavior patterns, conversation flows, and conversion paths within AI-led interactions Design and execute A/B and multivariate tests comparing AI-assisted experiences with traditional approaches Develop control and treatment methodologies for phased rollouts and pilot programs Define statistical significance thresholds, sample size requirements, and test governance standards Partner with data science teams to support continuous testing and optimization of AI models Analyze offer performance across customer segments and channels, including take rates and lifetime value Evaluate offer survivability, promotional effectiveness, and margin contribution Collaborate with modeling teams to validate and improve offer ranking and decisioning logic Quantify revenue impact and performance trade-offs across different offer strategies Translate customer interaction data and call transcript insights into actionable recommendations Identify high-performing behaviors and patterns to inform AI training and optimization Deliver clear, concise performance reports to business stakeholders Develop documentation and use cases to support AI learning and future experimentation Skills
3–5 years of experience in campaign measurement, experimentation, or performance analytics Strong understanding of statistical concepts including hypothesis testing, control groups, and significance testing Proficiency in SQL, Tableau (or similar BI tools), and Excel Experience working with large datasets and building performance dashboards Proven ability to design clean, well-controlled experiments in real-world environments Understanding of customer lifecycle metrics such as churn, retention, survivability, and lifetime value Experience analyzing offer performance, promotional effectiveness, or sales outcomes Ability to identify patterns in both quantitative and qualitative data Comfortable making recommendations in environments with imperfect or evolving data Experience measuring or evaluating AI/ML-driven products or models Familiarity with conversational AI, chatbots, or voice-based platforms Exposure to call center, subscription-based, or retention-focused business models Experience working with call transcript or conversation analytics tools Basic understanding of propensity modeling, decisioning engines, or margin analysis About us
Moodys NWC is a result-driven management consulting firm headquartered in the Seattle metropolitan area. We specialize in best-fit consulting based on client needs, with a focus on all aspects of digital product management. We have an active role that we are looking to assist our premier client with and are seeking an experienced consultant with the right set of values. For us, personality fit is paramount. We look for self-starters who are experienced yet flexible in their approach and who maintain a customer-focused outlook. Our consultants go above and beyond, stay focused on the end goal, and do what it takes to deliver results. If you believe you align with the above qualities, please review the opportunity and share why you are a strong fit for Moodys NWC. Our compensation reflects the cost of labor across several US geographic markets. The salary for this position ranges from $120,000/year in our lowest geographic market up to $160,000/year in our highest geographic market. Pay is based on factors including market location and may vary depending on knowledge, skills, and experience. Additional compensation includes medical, vision, and dental benefits. Moodys NWC believes in a culture of diversity, acceptance, and belonging. We are committed to equal opportunity in hiring without discrimination based on race, color, creed, national origin, sex, marital status, age, sexual orientation, gender identity, disability, medical status, or veteran status. Moodys NWC complies with federal and state disability laws and makes reasonable accommodations for applicants and candidates with disabilities. If you require reasonable accommodation during the application or interview process, please submit a contact form here: https://moodysnwc.com/contact-us/
#J-18808-Ljbffr