Build, measure, learn... IRL at a large company

+Jonathan Bertfield and I are big believers in teaching folks to get IN and through the BML curve (credit to Eric Ries) once. All the way through, meaning every single step... in real life not just in an exercise. THROUGH is defined as starting N+1 experiment with the DIRECT LEARNINGS of experiment N factored in to the experiment.

The following is a list of EVERY SINGLE STEP REQUIRED TO GET ALL THE WAY THROUGH, common blockers, and things we have learned from experience that can help overcome the blockers.



  • External to the process... a vision or strategy must be in place. A north star must be set by someone.
  1. ACTION First assumptions must be defined // BLOCKERS Usually none... teams over the years have shown a willingness and ability to at least list basic assumptions about the project. This excercise works best in person, but even distributed teams, both synchronously and asynchronously have demonstrated an ability to list 10-15 basic assumptions without any problem. REMEDY N/A
  2. ACTION Picking the RISKIEST ASSUMPTION // BLOCKERS similar to the above, most product teams have demonstrated an ability to use heuristics like the RA matrix, or the KILL WONT KILL (credit Aubrey Smith) to come up with a designated RA on paper. The BLOCKER of course is that not everyone will agree on the RA and they will spend hours (if not days or weeks) debating why one is riskier than the other. The REMEDY in this case is to pick one. A technique we like to use here is a bit unconventional, but it has tremendous rewards when used correctly. Allow the loudest person or the surest person (not the highest paid person in the room) to pick it, unilaterally. Avoid a passive aggressive stance on this, once they pick it THE WHOLE TEAM MUST GO ALONG WITH IT AS IF THEY PICKED IT. The trade off here is that that person now has the burden of proof on them and them alone. This introduces, VERY EARLY in the BML curve, two very important elements of culture. Data, proof evidence. It also teaches another subtle lesson. It teaches the importance of fuck it ship it, not just with features but with experiments. It is favorable, especially while learning these skills to test the WRONG THING the RIGHT WAY (NOW) instead of testing the "RIGHT" thing the wrong way at some point in the future. 
  3. ACTION Converting the RA into a Testable Hypothesis - BLOCKERS the first blocker here is that the blocker in step two carries over to this step. Not until people start really aligning the variables in this hypothesis, especially the SINGLE SEGMENT and the ACTION, do they REALLY SEE THE LEVEL OF RISK in terms of priority. So the first blocker in this section is agreement. This creates a mini loop within a loop where your team will go back and forth between steps 2 and 3. Allow it. But use the REMEDY step two. Other BLOCKERS include simple structure, not adding a deadline, not adding a proper metric, not aligning the action with the SEGMENT. REMEDY (HIRE US)
  4. ACTION Testing the RA BLOCKERS listed in order of likely occurrence... to be broken down by the TYPE of experiment but for now...  legal, marketing turf wars, finding customers, scheduling interviews, risk aversion, interviewing all segments even after agreeing on one, understanding how to use the tools, understanding how to do things that do not scale REMEDY - marketing turf wars are resolved by testing on NEW clients and giving marketing the leads after you run the experiments. 




Post a Comment

Popular Posts