Experimentation

Overview

Developing a framework for continuous discovery and experimentation

My Contributions

I actively sponsored a focus on experimentation and secured additional resources to ensure 'test and learn' could form a greater part of the product roadmap

When I joined Sky Betting and Gaming, there was a small amount of experimentation going on within the Sky Vegas product roadmap. Working with my Product Manager for Vegas, we determined a strategy to dial-up the activity and also started a programme to better evangelise what we were doing.

The first step was to ensure our colleagues and stakeholders from across the business shared a common understanding of experimentation in the context of continuous product discovery. I brought a small group together and we developed a bespoke framework for how experiments should be defined, prioritised and executed in the Gaming Tribe. What started out as a rather messy whiteboard diagram (excuse my scribbles!) was then converted into a single visual and a number of open workshop sessions where we shared our thinking.

It was amazing how quickly colleagues from various teams engaged and wanted to support. This was especially true for the Data Analytics team, who wanted to bring their expertise in the careful set-up and measurement of each experiment. From my team's perspective, we also decided to run sessions to level-up all Product Managers and UX Designers on how to create clear hypotheses.

In a few months, we were able to create a monthly communication that clearly showed the breadth of experiments currently in-play, next in the queue and a summary analysis for those recently completed. We also started the process of Tribe-wide play-back sessions for completed experiments. These proved invaluable as teams got up-to-speed on how to interpret the results and gave us all time to debate, challenge and hone the framework as we went.

In conjunction with the framework, I created and sponsored a specific investment case to increase the number of Engineers in the two Vegas squads. This was to ensure more bandwidth and capacity to maintain the uplift in experiments, alongside the established product roadmap. After several rounds of influencing at the C-suite level, I secured this funding and the additional Engineers were recruited.

By no means did we suddenly double the number of experiments being run across the Vegas product. However, it was clear to me that we'd reignited the understanding and ambition of a true 'test-and-learn' programme across the Tribe. We also established the very important mindset that whether a hypothesis was proven or not, it is the continuous process of learning that is critical.

Experimentation

Want to work together?