I was reading thru a reminder of basic principles for business decision making (95% confidence ....) and felt quite uncomfortable with marketing standards for decision making ... Probably because I have spent 20 years helping Companies making high value at stake and difficult decisions. As time and contexts are changing fast, I guess this may help a lot thinking thru the difficult decisions that most Companies will have to make in the next 5 years. SIGNIFICANCE STANDARD IS OK AS LONG AS CONTEXT IS ALWAYS THE SAME Decision making standards are good as long as we always make decisions in same context (the famous 95% confidence). However, life is not as simplistic as that. Significance calculation and the decision making threshold depend on decision you want to make, on size of opportunity and risk you have if you make a wrong decision. For example if you gamble in a casino, - would you play 1$ if you have an 5% chance to win 10000$ ? Probably yes, meaning 95% (= 100% - 5%) chance making a false decision is a good threshold if you have little to lose and a lot to win - would you play 5000$ if you have have a 5% chance to win 10000$ ? Probably not, meaning 95% chance making a false decision is a very bad threshold if you have a lot to lose and little to win DESPITE SIGNIFICANCE STANDARDS, CONTEXTS OF MARKETING DECISIONING MAY BE VERY DIFFERENT Coming back to marketing decisioning, best practice of statistical calculation rules depend on what risk you want to cover : - In some cases you want to test whether you can conclude on similarity (for example you want to replace a product or a pack and make sure you reach at least parity) - In other cases you want to conclude on difference (for example you want to make sur a new product is at least superior to previous one) However the confidence level calculation may be quite meaningless if context is not well understood. WHAT IS AT STAKE FOR THE YEARS 2020'S ? Beyond this, value at stake of projects may change a lot in years 2020's because lot of projects will help decide how to reduce cost, with a very high value of what may be lost and a much smaller opportunity for savings (for example one may risk more than 10% of total revenue to save only 2% of production cost). So my suggestion is to consider that decision making of the 2020's is going to be very different from previously established good practices. In this context, the critical success factor is going to be the capacity to model properly value at stake and chose the good statistical method accordingly. #decisionmaking #risks #measurement Thank you to Nishant Kaushal and all participants to this fascinating conversation