Question How to integrate the AB testing methodology into the current automation framework

[Palak Shah]

We presently use the Automation Framework with MStest and Selenium C# POM framework for the e-commerce domain in our organization.

Currently, organizations are moving toward an AB testing strategy for a few pages.

Therefore, I am unsure of the testing strategy in existing Automation Framework I should employ so that the code and pages will function properly across all design approaches.

in our situation, A/B testing will be conducted for several users; for example, one person may receive Design A for the basket page, while a different user may receive Design B.

But for the Automation Framework, the important concern is how to handle both the designs POM and how our framework can choose which code to check for if it receives Design A/B for the basket page.

I am absolutely new to AB testing, so any assistance would be greatly appreciated.

 

[Skydiver]

If the underlying IDs for the various UI elements remain the same even though design A looks different from design B, then you can use the same test suite. The problem happens when you let the designers run amuck and start changing IDs just because they can even though the controls happen to be the same thing, but just placed or styled differently.

 

[Palak Shah]

If the underlying IDs for the various UI elements remain the same even though design A looks different from design B, then you can use the same test suite. The problem happens when you let the designers run amuck and start changing IDs just because they can even though the controls happen to be the same thing, but just placed or styled differently.

@Skydiver - True.. but here problem is I am not aware if they will be using same ids for new ui or not..if they use different IDs then I need to make sure with old and new both the IDs - test should work - so for that any idea which sort of pattern can I use? - I am currently using POM pattern in our framework

 

[gulshan212]

Well, if you are thinking to integrating A/B testing with your existing framework, then it can be a powerful strategy to optimize your application and system by doing experiments with different versions. I have also seen this article, where the author published a complete guide on this.
Thanks

 

[Skydiver]

That's good to know about A/B testing, but how does that solve the OP's problem where they have an automated test framework, and the framework needs to be able to to perform tests on the UI regards of the site presenting version A or version B.