Our product development approach needs to change

“Golden rules” de-brief

I’m not a believer of “golden rules”: Email are better than phone number for registration; registration is needed before using app; force update is bad so we never do it; big button looks better than small buttons, churn rate is bad thing so don’t ever let it happen, etc. there are two issues with golden rules: First, it disregards the context element in which the rules should apply. Second, as the result of the first one, golden rules are never correct 100%, yet product managers are creating their own versions of rules and debating which rules are “correct”. Third, once believing a golden rule is “correct”, product managers are likely to unconsciously apply it in every kinds of products.

And I’m not alone. Linkedin, google, facebook,.. are not “golden rules” believers. They are doing few thousands of A/B testing a year to understand their own user base, rather than relying on a set of golden rules to design their products. There is a story, two companies meet the same consultant, who advises the 2 companies 2 exactly contrary advices – one company to force user to register before using the app, which results in more conversion; the other company was advised to remove such “force registration” and as the result see the sales increases while the registration metrics remain the same. If we put people of the 2 companies together and let them debate, they can debate for weeks and months on whether registration should be mandatory before consuming apps.

Fancy golden rules aren't real

Fancy golden rules aren’t real

Defined vs empirical approach

I would name the “golden rules” approach – where group of SMEs get together, brainstorm and debate to reach a consensus on whether a solution is best, and then follow it – defined approach. On the other hand, the “A/B testing and analytics” approach – where product managers analyse real metrics of their products, make decision and adjustment based on the changes in the metrics – empirical approach.

Defined approach roots back to 90s when there were hardly any other alternatives. In the early days of the Internet, analytic tools were mostly counts in server logs for web application. Analytics back then were uncommon, basic and costly. Product development, as a result, relies mostly on SMEs (and managers) in the form of brainstorming and debate. Such practices were transferred from older generation of product managers to younger generation, and was carried on till date.

In our new world of abundantly available, robust, cost-effective analytics tools, I would recommend the empirical approach to product development. The fact that user’s behavior changes fast makes “golden rules” obsolete fast; hence blindly following fixed set of golden rules will mislead the product into the past’s direction.

The real consequence

There is a hot debate in company M, whose products include product P, product A, and a platform S (for login and registration) to which P and A both use. It makes sense to cross-sell users from P and A to the other.

One day, a number came “churn rate of phone number is 15%”. Platform S team got together and agreed that this churn rate of 15% is unacceptable, hence S should address it by making email registration mandatory. Phone number is bad, hence phone number is optional. When that decision came to other products, it becomes controversial and people start arguing about whether phone or email are better.


Debate, debate, debate

Where did they go wrong

I would say “churn rate of 15% for phone number” might be bad, but S, as a platform, is not supposed to decide solution to address that issue. The real users of each product should decide, and the ones who represent the users to make decision should be the products – A and P. They should interview users, do analytics, make persona, do A/B testing to really understand how bad it is the impact of 15% phone churn rate, on their own user base. They should weight by themselves if the pros of using phone number worth taking the impact of 15% churn rate. As a platform, S is not to address that 15% churn rate issue, but to allow its partners to answer the question, by giving flexibility and customisability in the system so each partner can implement customised logic to do their own experiments.

Maybe later the products really find out 15% churn rate is indeed bad in their domain. But if S try to address it from the beginning, all partners will live their whole live wondering how their lives would be if they were having the chance to test the theory. And for any future product X of company M, they don’t have the chance to design their solution about mobile-first approach due to S’s technical constraints.

If we think of S as a startup, and customer is the one who pays, then A and P..are S’s customers. End users who use A and P are A’s and P’s customers, so A should be the expert of their customer, and so should P be to their customers, but not S.






Tung Dao

Tung is a Msc. in IT Management from University College Dublin. Tung is currently working in Astro Bhd as Scrum Master.