Hands On XCTest Test Plans

So recently I tried the new Test Plans for our iOS UI tests, it is quite promising and I like working with it so far.

So I thought to share with you my takeaways about it ;)

Create a plan

1. Create a new test plan: Product > Test Plans > New Test Plan
2. Create a test plan from a scheme: Product > Scheme > Convert Scheme to use Test Plans

The options start here if you want to convert your scheme to a new test plan which you will choose from three options as follows

So in my case, I just created a test plan from the scheme as I wanted to create a set of UI tests with all of the tests I already have.

Open Test Plans

The good thing about using test plans how you can deal with them with multiple of ways either through UI or source code.

If you need to deal with them through UI , just select your test plan or right click on a test plan then Open as Test Plan

If you need to deal with them from their source code, just right click on a test plan then Open as Source Code

Setup Test Plans

Select/Skip UI Tests:

Just by ticking to select or unselect (skip) tests in each test plan you have by selecting the Tests tab

FYI: if you untick the main test case file, you will not to tick or untick any tests under, you only need to enable all the test case file first and then choose which test to select.

Configure test plans:

This is the most interesting and useful part of using Test Plans on how configurable they are. You have the option to have as many configurations as you want plus a shared configurations for the tests.

This gives the opportunities to test many conditions which are:

1. Different locations and languages for localisation
2. Different options of taking screenshots
3. Many more as follows

Usage for Test Plans

We can benefit a lot from using Test plans

1. Create different set of test cases (smoke, regression, mocked, sanity, etc.)
2. Create different types of testing (functional, performance)
3. Test the iOS for localisation with simulating different locations and languages
4. Having different configurations for arguments you use at tests.
5. Having this set of shared configurations between all set of tests.

Learnings after using Test Plans

And finally here I share what were my learning after using and dealing with test plans

1. Changing the location of a test plan insider your project

First, I made a mistake and put the test plan under the main workspace folder which is not a correct thing, so I had to put it back to a folder under my UI tests group, what happened is when I tried to run them from terminal, they could not be found… why? … it was because they were still not connected to the Scheme.

This target for Variable Expansion was not correctly set

So what I did then is as follows:

• Manage Test Plans
• On the test tab, I made sure that again the test plan is connected back to the scheme and deleted the old references of the old test plan and location, Xcode will not do this for you automatically :))))

2. Handling adding new tests

If you think when you want to add new test, what will happen to a test plan you have? So there is really good option to test plan is deciding when to add new tests automatically to test plans and when not.

So default plan that will automatically include new tests and new smoke tests plan which won’t automatically do this and only use the tests selected previously.

This option can be chosen by selecting the test plan > Options > tick or untick the option Automatically include new tests

3. Which brings me to the third learning that if you have this previous option enabled

If you open the source code of the test plan and you were unselecting some tests, that group of test will be renamed as skippedTests while if it was not enabled, they will be renamed as selectedTests

4. Enable Parallel tests

From the same place as previously Option button, you can use to run test in parallel or not.

5. Run test plans from fastlane

Fatslane still not yet have an action that deal with test plans but after a research I found a workaround from this post is to add an xcargs to your lane of running the test plans

xcargs: '-testPlan SmokeUITests'

That’s it, you go and try it out ;)

HAPPY TESTING!