Product Manager, Search Evaluation

Website Jet

What you’ll do

Through the innovative fusion of retail, social and mobile, @WalmartLabs is redefining Commerce for the largest retailer worldwide. We are a group comprised of the brightest technologists and business people in the industry, excited about the limitless opportunities that this next generation of Commerce will bring to billions of people around the globe, all in an effort to help them save money and live better. As the idea incubator for the world’s largest global retailer, we don’t just build products, we create experiences. Every day is an opportunity to reshape the landscape of ecommerce while having a lasting impact on the industry. @WalmartLabs taps into the talents of online retail visionaries to design, prototype and build technology-fueled products that bridge the gap between what’s next and what’s best. @WalmartLabs brings together dozens of engineers, scientists and product experts to execute a cutting-edge vision. If you’re fluent in the language of innovation, @WalmartLabs is a place to become a leading change agent. Join us.

WalmartLabs is looking for a Product Manager who will be responsible for evaluating the quality of’s eCommerce search engine. The candidate will work closely with the search engineering, analytics, and product teams to design and implement experiments that quantify each feature’s potential impact to the site.

Position Description

  • Design and improve testing methodologies for search relevancy evaluations, including data set selection, testing strategies, and rater evaluation criteria.
  • Collaborate with engineering and analytics teams on evaluation quality metrics and criteria for determining the success and regression of new features.
  • Create actionable feedback from analyst and crowd judgments to improve relevance and user experience in search.
  • Manage testing queue and create strategies for scaling the evaluation team.
  • Manage crowdsourcing and tools tracks to compliment in house custom evaluations and analysis.
  • Demonstrates up-to-date expertise in Product Management and applies this to the development, execution, and improvement of action plans
  • Develops and champions a product vision, strategy, and roadmap for a specific product within a product area in support of corporate goals and objectives
  • Drives the product discovery process
  • Models compliance with company policies and procedures and supports company mission, values, and standards of ethics and integrity
  • Provides and supports the implementation of business solutions
  • Supports product implementation and deployment

Minimum Qualifications

  • Bachelor’s degree in Business Administration, Information Systems, or related field
  • 5 years experience in product management, business consulting, or related field.
  • 3-5 years designing and implementing test methodologies for relevance, personalization, or other predictive algorithms.
  • Excellent leadership and mentoring skills.
  • Strong analytic skills and a drive to be data driven first.
  • Unwavering commitment to user experience and search quality.
  • Strong business acumen and the ability to explain problems and solutions effectively to both technical and non-technical audience.
  • Excellence in implementation, attention to detail and strong organizational skills.
  • Ability to excel in a fast paced environment.

Additional Preferred Qualifications

Ecommerce experience Statistics, mathematics, computer science or other quantitative degree a plus


The Walmart US eCommerce team is rapidly innovating to evolve and define the future state of shopping. As the world’s largest retailer, we are on a mission to help people save money and live better. With the help of some of the brightest minds in merchandising, marketing, supply chain, talent and more, we are reimaging the intersection of digital and physical shopping to help achieve that mission.

To apply for this job please visit