Things Developers say - But it works for this case

I love my developers, some of the smartest people I've worked with, but every once in a while they say something odd about why a problem isn't a problem.

(I really need to start writing these down when they happen for better posts.)

There's a feature on Skystore, where if you rent a movie but don't play it for 28 days, we'll send you an e-mail reminding you there's only two days left to watch. The subject of the e-mail is something like: "Only 2 days left to watch [movie name]".

As you can imagine waiting 30 days to test something like this is not feasible, especially for an automation enthusiast such as myself.

Luckily we have ways to trigger this e-mails before the time runs out, which leads to an e-mail with the following subject: "Only 0.0416666666666667 days left to watch".

I didn't bother to see the code on how two days turn into less than half a day if you force the service to run, but got some reassuring that this only happened if you force to service to run earlier. It's fine, we log a bug on the issue tracker the same, just not a "fix it now" kind of bug.

A couple days go by, a dev comes back to me:

- You know, that has to do with the time left calculation, if we wait 28 days the e-mail will look well.

- So you're telling me it's fine because for the current business requirement that calculation happens to give a whole number?

- Yes.

- This value is configurable, so if someone decides it's supposed to be 3 days instead of 2, we change the config value and our customers start receiving mails telling them they have 2.333333333 days to watch?

*blank stare* - Ok we'll take care of it next sprint.

Yes I did say "3" that many times out loud.

 

Watir Webdriver / Selenium HTTP Authentication - if it's stupid and it works, it ain't stupid

I've taken some time to write a small sanity test pack to run on the web app that uses the platform my team develops.

Although we have a very good and thorough automated test coverage, I'm very paranoid and want to make sure we didn't miss anything that breaks basic functionality, not that it happens often, or in recent memory, I'm just paranoid - it's a good quality for QA person, right?

So I went about the correct way of doing things, Page Object Model; I have a basic page object that has functionalities that are relevant through the whole app, like knowing when page is fully loaded, how to login, etc. Everything else inherits from this class.

First problem, http authentication, selenium and watir still (!!!) don't have a proper way of dealing with this well, so I went with the normal "https://username:password@url." method, which by the way, nobody else tells you this: use URI encoding on the username and password.

But then I got to the playback page on our app (which is kinda an important page if you're online movie retailer), for reasons not important for this post, on this page there's a https -> http redirect, this redirects cause our username:password deal to be lost and I got stuck with the authentication popup.

Googling how to get around this today still gives me about the same answers as a few years ago, which include this lovely (yet old) Watirmelon post that suggest using AutoAuth plugin for Firefox, usually I'd be against this, since it binds my tests to Firefox, but since for my particular sanity-for-other-team kind of work, I don't really mind, however AutoAuth no longer works with recent versions of Firefox, it stopped working on v39, currently we're on v47.

After much dive into documentation and stackoverflow questions, I gave up, I was ready to ask them to turn off HTTP Authentication for that page, until a co-worker casually mentioned:

Try opening the http version with username and password on the url first, at the start of your test, then go to the normal https start page and do the normal test, the browser will remember when you get redirected.

That a stupid solution... but it works, so it ain't stupid.

Here's how my code changed:

      [...]
      browser.goto inject_auth( url ) if url
      wait_for_load
      [...]

to:

    if url
      browser.goto inject_auth(url)
      wait_for_load
      browser.goto inject_auth(url).sub( 'https', 'http')+ 'playback/fcf7ce62-5695-4d80-be85-662798c9ac7c?token=undefined'
      wait_for_load
      browser.goto inject_auth(url)
    end

Hacky, I know, I have to load the normal page first, because I set up the http load purposely to hit a 404 so this wouldn't cause any unintentional side effect, but it also gives me another redirect, only after this little dance I get to go to the initial page I intended.
Hope this helps other people trying to automate apps with redirects.

The Watirmelon post I mentioned before resolves the problem with NTLM  auth, which could be applied to everything, however my solution doesn't solve that particular problem, if you find out how, please let me know.

The importance of using proper naming in your tests

The Skystore project, which I'm the lead tester for, still has a lot of unreleased features, while the reason for this are beyond the scope of what I intend to talk here, one particular feature brought down an important, albeit obvious lesson: Naming matters.

One feature that has been worked a while ago, but is still unavailable, is called "playnext"; it's a simple feature: when giving out video playback information to the client, the server will include some information about what to suggest the user to watch next, some time before the end of the video, the client will display a little box showing that information with an easily accessible play button.

This feature has several tests along with it as what should be suggested varies in accordance with different conditions which have all to be tested and made sure the server is giving out the correct info and structure.

As an automation evangelist I've automated the shit out of this feature, every normal, erroneous, corner and edge case was covered and everything was saved into GIT, available to everyone and runs everyday on every integration environment - good times.

After client applications implemented their side of this feature and found it lacking, it was decided that a new feature would be implemented server side, it would behave just like playnext, but with a different structure that would better satisfy the needs of the clients giving more complete and extendable information a "playnext 2.0". Since different teams have different priorities and implementation times, it was decided that the two features would run side-by-side for a while, to preserve those who had already implemented the first version.

When deciding which tests would be needed, I've said to my minion responsible for it: "Look at the playnext tests, copy them, adapt to the new structure, see if there's anything missing or need to be removed and you should be done quickly".

Later the feature was developed and tested, approved and delivered upon the client teams. Then they noticed that there were several missing parts and it was mostly unusable.

How did this pass the tests? It was obviously broken! My minion told me "I did what you told me, I've copied all the tests I found, there weren't many though"

I've looked for playnext in the feature files, found only a couple of tests, but I know I wrote a bunch of them, what happened? I looked into one file I knew had a few of the tests and found the problem:

play next... play next play next....

There it was, in the tests I named the feature "play next" instead of "playnext", when searching for the correct name, only a couple tests had it.

This explains why he thought there weren't many tests.

So, when naming and describing your tests and features remember:

• Describe your tests using the correct feature name.
• Make sure to use the name consistently (ie: don't change the name mid way).

That's it. After this event the tests were corrected, the missing testes added, the feature was patched and integrated with and everybody is happy, hopping it's well received when available to a wider audience.