Using Hexawise 2

Following on from my last post on using Hexawise Beta, and taking on board some of the feedback from the Hexawise developers and Justin, I continued with my test planning. One of the things I quickly discovered is that experimenting with combinations of parameters and pairwise combinations can become quite time-consuming. Over time, as your tests evolve, it’s easy to spend a lot of time tweaking and configuring variables to get them just so. I’m not yet sure whether this is a good thing! Most likely, the amount of time spent experimenting with test permutations is directly correlated with one’s experience in combinatorial testing, and to a lesser degree using the Hexawise application.

It’s also kind of addictive! My previous post refers. 🙂

After approximately an hour or so spent adjusting and tweaking, I ended up with the test plan below based on an improved understanding of the underlying business rules, and how to get more value out of the Hexawise test planning tool:

Variable Name

Values

Product(s) In basket Not in basket
Event(s) In basket
Voucher(s) In basket Not in basket
Customer Normal customer Privileged customer
Promotional code % GBP

It looks pretty simple, but it took a little while for me to realise that judicious use of the value expansion filter (as mentioned in Using Hexawise 1) would help me to expand the coverage of my tests while keeping the number of tests required relatively low. The variables listed above hide a number of additional values:

Variable Name

Values

Product(s) Product(s) in basket Products not in basket
Single product
Multiple mixed category products
Multiple same category products
Event(s) Event(s) in basket Event(s) not in basket
Single event
Multiple mixed category events
Multiple same category events
Voucher(s) In basket Not in basket
Customer Normal customer Privileged customer
Gold
Platinum
Promotional code Percentage based promo-code Fixed amount promo-code
Product promo-code Product promo-code
Event promo-code Event promo-code

I found it helpful to give the parameters more descriptive names, hence the changes in labelling between the two tables.

By this point I’m feeling reasonably confident that I’ve captured the test variables in a manner that will provide good coverage and keep the number of required tests low. Indeed, running the tests through the Create Tests tab (based on 2way interactions – comments on Using Hexawise 1 post refer) results in just 6 tests.

Should I be sceptical in regards to whether or not 2-way interactions provide sufficient coverage? Probably, but this is an area that has been studied in detail by greater minds than mine. Checking out the combinatorial testing material here introduces us to the Interaction Rule which, in summary, states that: “Most failures are induced by single factor faults or by the joint combinatorial effect (interaction) of two factors, with progressively fewer failures induced by interactions between three or more factors.” Further information can be found by following the link through to Practical Combinatorial Testing, or by reading through some of the NIST resources here.

Screen shot 2013-02-16 at 23.26.05

In any event, with my initial test plan firmed-up, I can start to ask some additional questions like what happens (to the basket total) if the customer adds some more products to their basket after they’ve input a promotional code? I decide to simulate this by adding an additional parameter: post promo-code actions.

  • post promo-code actions: products changed, events changed, vouchers changed, promo-codes changed

Adding the parameter, and the associated value expansions results in the (by no-means finalised) additional inputs below:

Post promo-code actions Product(s) changed Event(s) changed Voucher(s) changed Promo-code(s) changed
product removed event removed voucher removed promo-code removed
products removed events removed vouchers removed promo-code removed and replaced
product added event added voucher added additional promo-code applied
products added events added vouchers added  

Running this through Hexawise gives us a test plan that can be executed in just 8 tests.

Screen shot 2013-02-16 at 23.51.02

However, there’s a problem. The Post-promo codes column isn’t really giving me the variety I want. I’d like to be able to combine different combinations of changes, e.g. products and vouchers, and a promo-code change. However this is most likely a relatively trivial modification. Of more interest will be how to ensure I only have valid combinations in my test plan. E.g. in line 2, we have a remove products variable in the post promo-code actions column, but no products in the basket to remove.

I’ll look at this in more detail in my next using Hexawise post. I’d also like to take a look at the new Requirements functionality in more detail but was unable to do so while writing this post. I’ll come back to it hopefully next time.

- Simon

Leave a Reply

Your email address will not be published. Required fields are marked *

GiottoPress by Enrique Chavez