Web Experiment for Recommendations

On the Sysco Shop platform, customers frequently make repeat orders of the same items in the restaurant and hospitality industry. When they want to try new items or make adjustments, they usually browse on Shop or reach out to their sales representative. However, the product discovery experience can be cumbersome for customers that may have a narrow window of time to discover new offerings.
By introducing more opportunities for customers to explore and discover products during their journey, we can improve the overall experience, encourage purchasing more unique items in their orders, and ultimately drive sales growth.
Solution
Customers are empowered to regularly repurchase their regular items through their purchase history and custom lists, accounting for >80% of all add-to-carts on Sysco Shop. 
To meet the customer where they’re at, we ran an experiment to surface product recommendations on their purchase history and custom lists on web for a more streamlined journey.
Validation
At the direct request of Sysco’s executive leadership team, a recommendation carousel was already implemented by engineers at the top of the customer’s list pages. Anecdotally, an increase in revenue from Sysco Shop is attributed to the new carousel, but at the expense of many customers that have smaller viewports and a less desirable shopping experience on the list pages.
Experimentation
Given the additional revenue generated, it’s important to various stakeholders that the recommendation carousel remain on lists. Based on higher than usual anecdotal customer feedback, however, it was apparent that there was an imbalance of user and business needs.
As the lead designer for Sysco Shop’s experimentation pod, I spear-headed this initiative and developed multiple design iterations of the list page's recommendations. 
My primary objective was to alleviate customer frustration by freeing up verticality so that customers could view more of their products on their lists. 
The next important factor was making sure these product recommendations were still easily accessible, but not invasive, to encourage discovery and increase conversions. 
Finally, there would not be exponential design changes to the UI to avoid customer reorientation with the list pages and assuage scope creep.
We deployed the limited-audience experiment through Optimizely into production, leveraging the originally implemented carousel design as the control for the experiment with my new designs: A, B, and C variants. 
Variant A
Carousel in right-hand panel.
Carousel Add to Cart:
+ 22.4%
Carousel Product Click:
+ 1%
Variant B
Carousel in a triggerable modal pop-up.
Carousel Add to Cart:
+ 19%
Carousel Product Click:
+ 8.8%
Variant C
Carousel in a bottom sticky container.
Carousel Add to Cart:
+ 20%
Carousel Product Click:
+ 6.3%
Results
Over a 30 day period, all three variants saw an increase in recommendation add to carts with Variation A performing the strongest, boasting an increased add to cart rate of +22.4% and an additional ~$10m in total revenue conversions when compared to the control state. This particular variant also doesn’t affect the verticality of the list page itself, making it an ideal location that will not interfere with potentially net new features on future roadmaps.
+22.4%
add to cart rate*
~$10m
additional revenue*
*measured over a 30 day period
Next Steps
Immediate priorities are centered on cleaning up the code while reducing latency of the list page that may be imposed by recommendations. 
As this initial experiment focused solely on presentation and placement, future experiments will target recommendation types based on customer’s lists. This will allow us to improve the relevancy of our recommendations and revenue generated.
Additionally, the UI will be refined to meet Sysco Shop’s design standards as the recommendation logic on the backend is fine-tuned. Mobile carousels already have designs standardized, but placement of recommendations in lists and purchase history pages still needs further experimentation to optimize conversions.
​​​​​​​Team Paradigm
Product Manager
Product Designer (myself)
UX Researcher
Data Analyst
Experiment Engineering Lead
Tools
Figma
Slack
Optimizely
Redash
Atlassian Jira & Confluence
Back to Top