Speeding up Cucumber and Concurrency tests using Threads

Reading on how to make Cucumber tests faster, most literature recommends using more CI servers, or more CI nodes, and that kind of solutions. I've always found this kind of solutions very expensive, not that machines are particularly expensive nowadays, but because more machines/nodes = more maintenance and, if you're on a particularly big organisation, are very annoying to acquire.

One thing that's much more efficient and should be done before getting more machines is simply using Threads.

However, you can't simply open a new thread on every step and rejoice as your tests take 1 second.

What you can and should do, is identifying small peaces of code that could be split in different threads. How do you know that?

The easy ones to spot are iterations where you don't care about the order in which each iteration is executed.

For example, I have this test against an API, which calls an endpoint and verify that every link returned is valid. This is a very simple and obvious case that follow on what I mentioned above, since:

• I have a list of things.
• I will iterate this list and do some kind of check on each item.
• I don't care if item "B" is checked before or after item "A".

So, to speed up a test like this, I can start a thread on start of each iteration, give it the checks I want it to do, saving the thread reference and go to the next item. After sending all items to their threads, wait for all threads to finish.

Luckily, we're using Ruby, where this kind of things are very easy to make, here's a gist code example:

Earlier I set off to try this technique, but, before, I ran a scenario on my IDE that would call the original code, these are the results:

1 scenario (1 passed) 

5 steps (5 passed) 

0m23.459s

After changing the code I got:

1 scenario (1 passed) 

5 steps (5 passed) 

0m10.408s

Wow! Over 50% improvement! This is too good maybe, so I hammered some code to force it fail - I was afraid for a moment that this setup was hiding test failures, but no:

1 scenario (1 failed) 

5 steps (1 failed, 4 passed) 

0m11.366s

Pretty happy with myself I pushed my changes to my git repo, and tried to run the Jenkins job which had a few tests with this step and others without. To my amazement, the previous 3 builds had took around 10-12 minutes, the new build took 2 minutes (yes, two minutes), that's a very big performance increase.
This technique can be used for other things as well, for example, you can have a step which tests race conditions, having two users on two different threads trying to do the same thing at the same time while you make sure they don't step on each other's toes unintentionally. I deally this race conditions tests is more of a Rspec/Unit test kind of thing that falls a little out of place of Cucumber, but knowledge of how to bend your tools never hurts.
Also, this will probably be fun when I join this technique with the two simultaneous browsers one.