A tweet from Ash Winter (@NorthernTester) caught my attention this morning. Marks & Spencer made the headlines as a result of a recent upgrade to their website. The BBC reports that the changes have had a negative impact on the customer experience and as a result, have caused online sales to drop 8.1% over the past 5 months.
There’s the odd quote during the report to the effect that customer objectives are harder to achieve on the new site. It also states that 6 million customers had to re-register as a result of the upgrade.
There may have been some errors in judgement during the course of the project…
Having worked on a number of e-commerce web projects, I have some sympathy with the plight of the various team members who would have been involved in the decision making processes. Problems such as those reported in the article rarely occur as a result of a single act or decision but instead accumulate over a period of time and are then compounded by time-to-market, budgetary and political pressure to release the new system. Sometimes, as it would appear in this case, before the product is actually ready.
Without knowing the full story, it’s difficult to know exactly where the problems were, but some actions that could have prevented some of the rougher edges include:
- Migrating customer credentials from the old to the new authentication database(s) and testing that existing users can login to the new site with their old credentials ahead of release.
- User experience testing with focus groups to determine whether changes to the design, look and feel of the website have the desired effect. A/B-split testing of changes across customer groups would also have helped to understand whether changes were having a net positive or negative impact.
- Using market leader websites as oracles to drive the user experience testing and to determine whether changes to the navigation and purchasing steps were in line with established and successful ways of doing business.
No doubt there are other challenges with this particular project that warrant closer scrutiny, but the presence of a professional tester on a project team will help to deal with issues like the ones above by shining a light upon them. In this way, the people who care about the ultimate outcome of the project, i.e. the value to the business that it needs to deliver in order to be considered a success, can take the necessary decisions in order to overcome or mitigate problems they may not otherwise have been aware of.
It’s interesting to note that Laura Wade-Gery, head of online business for M&S (presumably during the delivery period) has now been promoted [to oversee their UK stores] and is tipped to “one day, take over from Mr Bolland” the chief executive. It would seem as though the project has been a success from her perspective.
So there’s a lesson for us testers here too. Remember who your customer is.
I’ll be delivering a workshop on The Evolved Tester at the next #BrummieTesterMeetup. See you there?- Simon
P.S If you're interested in learning more about performance testing, checkout my Performance Testing 101 course here.