Rent the Runway is an online e-commerce website where women can rent designer apparel and accessories, or as it describes itself: a fashion company with a technology soul. And indeed technology it has - with over 65,000 items to browse on its website and mobile app, Rent the Runway maintains several separate platforms that perfectly integrate together in order to support all the supply chain, logistics, commerce and analytics needs of its business. With serious DevOps works on all of its systems comes serious testing to make sure that thousands of transactions per day happen flawlessly and everything runs smoothly on the back-end platform, customer-facing platform and mobile app.
Today, we're speaking with Jeremiah Ivan, VP of Engineering at Rent the Runway.
Q: What types of software are you testing?
A: We primarily test the e-commerce platform that our customers use on a daily basis and a back-end system that we use internally. Our e-commerce platform integrates five systems together, giving our customers a seamless experience from order to payment to product delivery. Thousands of transactions per day go through our systems.
Q: How many projects, test cases and executions are associated with these platforms?
A: We have over 4,000 test cases and run more than 10,000 executions per month overall. On our e-commerce platform, we run 30 different test cases. Our back-end system has over 100 test cases associated with it.
Q: Are you using Atlassian products? If so, which ones?
A: We are using a bunch of Atlassian products: JIRA, JIRA Agile, Confluence, HipChat. It's great that Zephyr is integrated into JIRA because it makes everything simple. JIRA integration allows us a window into coverage for the user stories and the sprints.
Q: Why did you pick Zephyr over other vendors?
A: We picked it because of its fine support, responsive account team and great visibility for our product owners.
Q: What were the challenges you faced before deploying Zephyr?
A: It was difficult to maintain a dashboard for manual testing, and it was one person's full-time job to provide reporting. We were manually creating reports to track readiness to deploy, quality and status, and it was hard to get traceability to connect requirements to test cases to defects. We needed an easier way to figure out coverage gaps. With Zephyr, we leverage Requirement Coverage and Requirement Distribution Metrics reports to fill these areas.
Q: How does Zephyr support your Agile processes?
A: We are very much into Agile - we use Kanban, for instance. Zephyr allows us to see how our stories are tested, and product owners can review the results and provide us with their own feedback.
Q: What are the benefits you've received from using Zephyr?
A: One benefit is the ability to maintain dashboards for manual cases. It's great to have visibility and improved confidence in reporting, not to mention our increased readiness to deploy and better coverage gaps. We've eliminated a lot of administrative work, especially when it comes to combining spreadsheets - I'm estimating it's up to one-third of one person's job savings. And we really appreciated the ability to integrate to JIRA to see failure reports and gain end-to-end traceability.
Q: How do you quantify each of the above stated benefits?
A: Visibility reduces our risk as we are able to spec out testing needs when changing large areas of the business. By using these systems, we've also been able to save around 20 to 24 percent of our man-hours that were once spent in coordinating coverage gaps.