A startup experiment funding startup experiments.

A startup experiment funding startup experiments. 

My goal for 2015 was 10,000 interviews logged. It was a vanity metric, and it probably was not the most important thing to measure, but it DID give us a direction and an opportunity to learn. Fell short of that 10,000 goal by about 6500, but we learned a ton this year and are going to continue the experiment. My goals for 2016...

1. 25 teams who have spent less than 1 month OR $5,000us will go through the program, doing at least 25 interviews and running 3 experiments in less than 4 months -- will THEN go on to "some quantifiable outcome regarding product market fit"
2. 100 new mentors
3. 100 teams who change course quicker and more cheaply than they would have otherwise and credit our program for doing so 
4. Prove causation between custdev +///
5. Raise $1,000,000 in funding 

HERE IS OUR FINAL YEAR END FUNNEL UPDATE 

METRICS

1. Acquisition =  4667 clicks to the bitly as of midnight on December 31 (up from 4448 clicks as of 12/4 >> up from 2483 clicks as of 8/3

2. Activation = 179 entrepreneurs have accepted (up from 91 people on the last report) have accepted the challenge directly from the entry point above, submitting at least one interview. This number is up from 51 from our last report.

3. Activation / Retention

• CURRENTLY we have around 3000 interviews (the interviews themselves are not as valuable as we thought - see LEARNINGS below)
• OLD = 918 interviews have been entered in the database, up from 402 interviews at the last update.

4. Retention = 4 OUR ONE METRIC THAT MATTERS FOR THE REST OF THE YEAR IS # OF EXPERIMENTS FUNDED, # OF TEAMS FUNDED. 

• CURRENT = We funded 4 more teams over the Thanksgiving break in the United States. We have now funded a total of 10 teams and about 20 experiments - we are taking the promise of funding VERY seriously and will not hide behind the term EXPERIMENT, that said, as Bruce Springsteen said, the door is open but the ride ain't free. SEE THE RULES BELOW... but remember, you must use the funds to run an experiment on an UNKNOWN variable. When in doubt, do not rely on funds until they are in your account. We are being transparent with everything, using our best judgement, and calling on mentors and OTHER TEAMS LIKE YOU for final decisions on funding when there is a dispute.
• LAST POST "We invest not in ideas or even in teams, we invest in experimentation. We have funded 10 experiments so far. "

5. Referral (?) = You can help with this! We are not really measuring it right now. Encourage any entrepreneur to start the interview process and enter the results here: bit.ly/milvalchal (make your own bitly if you want us to track it, get 100 shares and we will give you something valuable).

LEARNINGS 

1. as someone pointed out on twitter, the 100 interviews are a great filter and a great proxy for hustle - we knew that going in - what we didn't realize is that it can also be a proxy for stubbornness... the interviews are supposed to be about learning, but some people see the offer and just put their heads down and complete it as a task - a quid pro quo if you will... that's not what we are looking for so we've pivoted :) you do not have to do 100 interviews but you do have to use the funds for an experiment with a TESTABLE HYPOTHESIS - rules below
2. learning #2 is that it is REALLY hard for teams to come up with a quantifiable hypothesis baseline... its relatively easy to say I BELIEVE THAT THIS METRIC WE ALREADY HAVE WILL INCREASE BY X IF WE DO Y BY N DATE... but it is really hard to be specific about #s in the first place and it is also hard to test the RIGHT thing 

RULES

1. you must use the money to prove something true or false 
2. you cannot give the money directly to the person on whom you are relying for an action-- we are starting to see this with loans, and focus groups... you cannot say "I am trying to learn if x person likes my product so I am paying them 50 each to do an interview" -- we are still learning ourselves out to stay one step ahead of this with the language but right now we think the rule is - not giving money directly to the subject of the experiment
3. you must have a photo and bio on your twitter account 
4. you must mentor 2 other teams if you actually get funded... 
5. EVERY decision maker on your team must be on board (we will come back to this next month)

GOOD LUCK - we think this is the fairest way for problem solvers around the world to learn about learning... we are not perfect but we are doing our best!