As a starting point, please note that to comply with my non-disclosure agreement, I have omitted any confidential information in this case study. All information provided is my own and does not necessarily represent the views of Shopify and the Shop team.
Shop is a shopping destination and package tracking app from Shopify -- a place where you can discover a wide variety of products from Shopify's catalog, tailored to the users preferences and interests.
My primary responsibility for this project was to display customer reviews for the first time in the Shop app, resulting in an impressive 12% increase in purchase conversion for the initial MVP.
When I first joined the Shop team, the app was mainly focused on its utility feature: tracking packages purchased from Shopify stores and any other provided tracking numbers.
The future plan for the business was for Shop to become a comprehensive shopping assistant, from product searching to tracking the items you've purchase in-app to your house.
As Shop evolved into a shopping destination, our aim was to improve the overall in-app buying experience, providing users with the necessary information to make well-informed purchasing decisions, leading to a successful transaction in-app.
Reviews are a commerce primitive and a lever to pull in order to bridge the gap between the utility and shopping worlds inside Shop.
The Reviews team had a big responsibility: transforming the new buying experience inside Shop — a shift in the business that the majority of our current tracking focused users didn't ask for or needed — into a market leader for online shopping.
Shop Reviews should be the gold standard for trust and engagement, helping buyers purchase confidently and merchants of any size grow their business on Shop.
I was the sole designer on the Reviews team, collaborating with a cross-functional group of dedicated and amazingly talented team members: 1 Product Manager, 1 Engineering Manager, 3 Front-End Engineers and 4 Back-End Engineers, and 1 Data Scientist.
As part of a cross-functional leadership team, the lead members and I met weekly to discuss our goals, roadmap, and deliverables. We ensured alignment in our efforts to deliver the first MVP and define future milestones for the project as we learned from its iterations and development.
In this case study, I'll highlight some of these efforts and discuss the insights that guided us from discovery to the delivery of the first iteration of in-app reviews in Shop.
As this was my first project with the Reviews team, my primary focus when I joined was to gather context and understand the why and what we were building towards. Since the project was still in its early stages, I had spare time to talk with knowledgeable stakeholders, conduct user research, and explore initial ideas.
At the start of the project, I interviewed 15 users to understand how reviews influence online purchase decisions and how they use these tools. This helped us determine the essential features and nice-to-haves for our MVP.
All 15 users mentioned customer reviews during our interviews, and most of them use Amazon or Google to find trustworthy reviews online.
One interesting finding was that many users don't necessarily trust customer reviews displayed on the merchant's website. They fear merchants will tamper with the reviews to show only positive ones or pay people to leave false reviews.
"I usually look at Amazon first because I know they have true reviews, most of the time… (…) Amazon also have more reviews"
Another insight was that even though 100% of users read reviews before buying online, less than 30% actually leave reviews for products they bought.
When asked how they usually found what they were looking for before making a purchase, users mentioned looking for reviews that resonated with them and their experiences. This helped them determine if the product they were researching made sense for their specific use case.
“I want to learn what people like me are talking about this product”
Another frequently mentioned topic was how users tend to filter by 1 or 5 stars. They look for the best and worst reviews to check for consistency and determine if the product and merchant are trustworthy enough to influence their purchase decision.
Nowadays, especially with the rise of video reviews, many users look for YouTube reviews, TikTok influencers, and blog posts with detailed comments and pictures before considering specific brands. They say media adds a great sense of trust around products and often converts doubtful buyers into certain ones.
“Being able to check how the product looks in real life is always helpful. (…) (That way) I know is not a scam”
Based on these results, we could decide which features were necessary for the first version. This also helped us define our roadmap structure and the next steps for successfully improving the reviews experience in the app, guided by user expectations and preferences.
During this project and its iterations, our PM and I conducted many user testing sessions to validate our assumptions and design decisions before releasing them to all users.
Reviews were already a widely used tool online, so we were not inventing something new. By exploring what other eCommerce apps and sites focused on, based on their own research, I could identify existing patterns and use them to our advantage when building our own 0-to-1 feature.
On eCommerce platforms, reviews are always present and usually displayed early in the user journey.
As Shop was still developing its shopping assistant goal, reviews were collected only for in-app purchases. This limited the number of reviews and affected the coverage we could achieve by the time we started displaying them in the app. Considering this, we had to decide on an entry point for reviews that would make sense for the available coverage and ensure a consistent experience throughout the app.
Based on that, we introduced customer reviews on the latest user journey surface, the product page.
After users mentioned trust issues, especially with online stores not on major marketplaces like Amazon, I explored ways on how to address these concerns within the app.
One big user concern was that reviews could be manipulated by merchants. Implementing our in-house moderation system was a key solution, but I also wanted to address this from a UI perspective.
I decided to add a note saying "Verified by Shop" at the top of the page, which users could tap to learn more. Later, to avoid friction or confusion in the UI, we simplified it to a "Learn more" button.
This is a view of the complete flow, from reviews collection, which was already in production at the start of this project, to the proposal for the display of reviews in-app.
When working on a 0-to-1 solution, I prefer a designing backwards approach. While aiming for an MVP, it's important to envision the future of the feature so that all the ideal elements can come together seamlessly during iterations.
With this in mind, I listed all the features for the ideal version of the product and worked backwards to identify the essentials for a first version that could be shipped quickly and effectively. This ensures the experience is both useful and simple, meeting user expectations.
For the MVP's final solution, the team prioritized delivering essential features that provide immediate value to customers using reviews to make purchase decisions. These features include:
During the build phase, I worked closely with the front-end team to ensure my UX vision aligned with time and technical constraints. With my background in web development, I always try to leverage pairing with engineers.
As a common team practice, we made incremental releases (both internal and external) with controlled tests before the full project release. During these tests, we monitored our key KPIs and tripwires to assess the impact on the goals set for this MVP.
Some of our main metrics results were:
After launching the MVP and monitoring its performance, we worked on several initiatives to increase coverage and improve the user experience.
First, we added review indicators to products and stores without any reviews, assuming this would add trust value.
We also made multiple UI improvements, adding search and filter options, implementing a histogram on the reviews details page, and showing the average rating at the store level.
Next on our roadmap, I began exploring ways to introduce media into the product and reviews details page.
When I joined the Review team, I had the chance to work ahead of engineering. This allowed me to focus on the discovery phase longer than usual, providing valuable insights and time to get constant feedback throughout the early stages of the project.
During the development of this project, there was a company-wide rebranding effort. Because we focused on delivering the MVP, the reviews collection part of the app (which was already fully developed) was neglected and didn't align with the rest of the app.