HTTParty with cookies

On the past few months I've been doing automated testing work on an rest API that happens to rely a lot on cookies for user authentication stuff.

HTTParty is a great gem for working with rest API's, it's easy and simple, but it doesn't provide a care free way of dealing with cookies. I mean, you have a way to read the cookies from the response, you have a way to add them to the request, but you'll have to do it manually. So like any good Rubist I've made the code for it into a little gem. 

I hope it helps someone else as well. 

You can find most of the information on github.

HTTParty and non 200 response codes, cache & etags and a few other quirks

I've been playing with HTTParty to help me automate tests for an API, while its is a great gem, I found some documentation and guides lacking.

What happens when you add to your request an ETAG (for cache) and the server returns an 304 (not modified)? I'll tell you what happens, you'll get a "nil"!

No really, you get a nil!

Now, this confused the shit out me, and I've googled on and on about it, but found nothing really useful, until, for some reason I found deep into the specification my answer:

    describe 'with non-200 responses' do      context "3xx responses" do        it 'returns a valid object for 304 not modified' do          stub_response '', 304          resp = @request.perform          resp.code.should == 304          resp.body.should == ''          resp.should be_nil        end

"should be nil?" Well, that make no sense! It turns out that your nil object has some methods injected into it, because... I don't know, someone thought it was intuitive and easy - I .... disagree... 

So, how we deal with this?

Something like:

res = HTTParty.get 'someurl', headers:{'If-none-match': 'some e-tag you got from earlier'}
res.code #=> 304
res.class #=> Nilclass

This is also valid for other responses with no body that HTTParty thinks is fun to turn into nils, because if you don't have the body for it, you can't party.

Pi Facer gem version 0.0.1 alpha released

I've been playing with a Raspberry Pi and I have this project that might use the GPIO pins, I've poked around to see if any Ruby gems allowed me to use it, I've found this good one that has an event driven way of doing it.
However later I decided to use a Piface instead of the pins directly, unfortunatly there were no event driven gems for piface, which is a shame, because I figure it's the best way to interact with it. So I made one.
I've based of piper because all the threading stuff was already done, hope it helps more people with piface projects.
Oh yes, Ruby 2.0.

pi_facer on github, pi_facer on rubygem.

Using Watir-webdriver and selenium remote control to automate tests on Internet Explorer

So, you're on a Linux machine (like a boss), you have a pretty sweet battery of automated tests that you run against your web application on your CI server (which also runs Linux), but you have a few problems:

1. Internet Explorer sucks and most defects are open due to it.
2. You can't automate IE because all your systems are on Linux.
3. You need to automate on IE, because 13% of your user base are either ignorant or corporate drones who can't change their browsers.

So what you do? You can't just quit and call it a day? Nope, you keep reading.

What you need?

1. A windows machine (yeah, sorry about that, but you knew it was coming).
2. Install on it selenium remote grid server on said machine, here's a guide how to set it up.
3. Our good friend Ruby!
4. selenium, selenium-webdriver and watir-webdriver gems.

That's great, but you're wondering "How do I make watir-webdriver connect to the remote selenium hub?", that's a great question, for that all you need it's to pass it the arguments you'd use for selenium-webdriver to do that. And to be honest I only know that because I've been digging into watir-webdriver's code on github and I know that all parameters you pass it are passed along to selenium, something that's not quite clear on watir's documentation or any guide you can find on the web.

What I needed was something that would have watir-webdriver connect to Firefox locally, but if I wanted IE it would connect remotely, so, thanks to the param ":desired_capabilities" you can specify which browser you want to open on your remote connection.

I've created a sample for this on github, which you can find here.

Installing Spotify on Fedora - now with a installation kit

Edit: These instructions are now almost one year old, if you're using Fedora 17 or newer try one of the links for the community packages on the bottom of this post or in the comments section.

Me and coworker compiled instructions to install Spotify on Fedora and packed them in a nice little tar file for everyone to use.
Get the file here:

Instructions (On the README file): http://dl.dropbox.com/u/17266999/spotify-install-kit.tar
(If you rather not install stuff you got from dropbox, you can still follow these instructions since they have the steps for not using the file I'm linking.)


yum install wget rpmdevtools rpm-build yum-utils cpan
rpmdev-setuptree

#get stuff from interwebs if you don't have it, skip if using "compile kit"
wget http://leamas.fedorapeople.org/spotify/0.8.8/spotify-client.spec

#install dependencies
spectool -g spotify-client.spec
yum-builddep spotify-client.spec

#compile Alien
git clone Alien.git (or the url for the git repo if you're not using the "compile kit")
cd Alien
./Makefile.PL
make
sudo make install

#build the rpm
cd ..
cp * ~/rpmbuild/SOURCES/. #don't use -r !

env QA_RPATHS=$((0x02|0x08)) rpmbuild -bb spotify-client.spec
sudo yum install [rpm] #at the end of the build the output lists the .rpm file it wrote, copy paste that

#Launch!
spotify

On a Fedora 18 64 bits machine these instructions just worked
On Fedora 16 64 bits I had to remake the links for a few of the .so files because it was originally pointing for /usr/lib64
 but locate found them on /lib64

Links:
http://community.spotify.com/t5/Help-Desktop-Linux-Mac-and/Linux-Fedora-RPM-package-for-F17-F18/m-p/237688#M8446
http://leamas.fedorapeople.org/spotify/
http://repository.spotify.com/pool/non-free/s/spotify/
https://github.com/leamas/spotify-make

Installing a windows network printer on Fedora using SAMBA (and CUPS)

The place where I work at only supports Windows computers for the employees, the Exchange server is heavily linked in the back bone of the "administration system" - and I say administration because we produce software that is installed on Linux machines, we manage Linux machines and all clients have Linux machines. However the calendar, mail and network shares are all done with windows technology. 

If I followed the rules I'd have a windows computer and I'd run a VM just to have a Linux to do my actual work in.... which is kinda silly. So I dual boot Fedora on my laptop to be more productive on my job and "fend for myself" on the other things.

Installing the new network printers was interesting, the helpdesk provided everyone with guides on how to install on Linux and OSX, but mostly because the management gets Macbook Airs as a perk, so I guess they weren't happy when they couldn't install the printers, either way, the Linux instructions were incomplete.

With the help of another co-worker who also uses Fedora based computer we were able to configure the printers correctly.

Requirements:

I had to install the following packages (though my coworker didn't have to):

• samba-client
• samba-common
• samba
• system-config-samba
• samba-winbind-clients

These packages were already on my system:

• cups-pk-helper
• bluez-cups
• cups-libs
• cups
• cups-libs
• ghostscript-cups
• gutenprint-cups-
• python-cups

The best way we found to configure this was using cups web interface, located at http://localhost:631/admin 

We took a bunch of screen shots, they should explain better than words, note that on the network path for the printer we had "ptin/" which is the domain where our users are.

Other important thing, if you have special characters on your user name or password you'll have to uri encode them, I have a '%' on my password which became '23%' and yes, you'll have to reconfigure when you change the password :( .

(Note: For the right printers show up on my list I had to go to Lexmark's site and download the following file: openprinting-ppds-postscript-lexmark-20130226-1lsb3.2.noarch.rpm and I think it's great that they provide it, they've ranked up on my consideration.)

Fedora - Unexpected Inconsistency, moving to KDE and other Linux stuff

Oh Mondays.

Monday last week started out great and by great I mean with a broken system, I don't know why or how this happened, I blame using the "hibernate" function as it never seems to work properly for me on any OS.

The fix:

efsck /dev/disk/by-label/_Fedora-16-86_64

(That path was for the hard drive he was complaining about) 

Then I pressed "yes" to everything he asked me, I mean, he asked "would you like to fix this?", what was I supposed to say? "Maybe"?

Fixing...

I was unable to login properly as the package metacity was crashing and gnome was all sort of weird. So I did whatever a sensible person would do, I jumped in the console and installed KDE.

It's been a strange journey, not that I mind learning new things.

First thing I did was to rebind my keyboard console shortcut (which is one of the most popular posts on this blog, a testament to fedora's user friendliness or lack of thereof), cool thing though, KDE allowed me to use the Ctrl-Alt-T (like you had on Ubuntu) that for some reason Gnome didn't let me use. Second thing I did was update said blog post.

First of all, I discovered that I had to install "npmixer" to have a little volume slider on the taskbar, for some reason it was not installed by default.

Then I had to manually set the keyboard shortcut for "volume up" and "volume down" as these keys, though recognized by the system was not working.

Changing volume on the console:

amixer set Master 1%- 

amixer set Master 1%+

These two commands are the ones I used to bind my key, they allow to easily control 100 levels of volume on the keyboard. 

Since I was doing this stuff anyway, I poked around musique to find a way to bind the "play/pause".

Musique console commands:

musique --toggle-playing

musique --next

musique --previous

I've bound these commands to shift+volumeup/down and play/pause buttons on my keyboard.

I still haven't made it possible to launch the calculator with the calculator button, despite the fact I've bound the command "gcalctool"  (which opens it) the they calc key... I think this key is special, haven't looked into it yet.

Auto Mount Drives

Other thing I noticed, how easy it is to have hard drives to automount on login! I tried to follow some scary and complicated guides I found on google about it, but on KDE you just have to tick a few boxes.

Go to Settings -> Removable Devices, should be self explanatory.

Amusingly, I've also found that to take screen shots one needs to install ksnapshot (yum install ksnapshot) and I'm yet to bind it to the keyboard shortcut.

Finally, my favourite feature of all time, you can specify different desktop wallpapers to different displays! Just right click on the background of the display you want to change and go to "Desktop Settings" and settings there are display specific, might sound  basic, but I love this feature and it annoys me it's not present much of anywhere else... In fact, as far as I know, it only also this simple on OSX.

This allows me to have two different wallpaper origin for each display, this is important since one of them has better resolution (you can notice that as the screen on the right has a lot of black space bellow the memory and cpu widgets, but in reality they're at the bottom of the screen).

However for some reason so far I have not been able to make KDE remember the display settings, every time I boot I have to tell him that my VGA screen is on the left of the LCD one - and yes, I've saved it as default.

Ruby ARGF and the "<" operator

One of those things I've always should have known but somehow skipped when first read about ARGV and ARGF on Ruby.

ARGF will give you file handlers and content of all files passed to the script, but it assumes that all arguments present in ARGV are in fact files.
So if you want to be able to call your script like so:

./myrubyscript 100 file.txt

Than in the code you must have:


somvar     = ARGV.shift.to_i
filelines  = ARGF.readlines


How about, if for some reason, you want to call your script like so:

./myrubyscript 100 < file.txt



somvar     = ARGV.shift.to_i
filelines  = ARGF.readlines



It's the same code! Don't you love Ruby?

The workings:

The arguments passed to your script are stored in the ARGV Array, one argument per element. ARGF assumes that any arguments that aren't file names have been removed from ARGV.

It took me more than it should have to find this little page.

Installing Musique on Fedora

There's this music player called Musique and it looks very pretty (and macish according to the gf), it has the looks I'd like all my music players to have - artist photos, I mean, c'mon, it's 2013, can we have enough pixels, memory and bandwidth on our desktops and laptops to navigate artists by artist photo! Why can I only do that on my phone?

Anyhow, on Linux they provide the binaries and I've kinda failed to get them to work, probably dependencies, but they listed some kind folk packed it for Open Suse. so I took their src.rpm and and did a few tweaks.
First I fixed the name of the requirement for "qt-devel" as it differed from the Open Suse one.
Second I removed build requires for "update-desktop-files" since it appears to be a Suse thing to do stuff with the desktop, I didn't bother looking for a replacement as I don't want it to add desktop files.

Next I compiled, installed, and it popped up on my Applications -> Sound & Video.

Here is a link for .src.rpm and rpm file in case you want it.

PS: For a while the sound was really bad, but thanks to the author I discovered it was a problem with phonon-backend-vlc, so I added a conflicts with to the spec.

TIL About Ruby class and modules namespacing

Today's post is courtesy of Captain Obvious, but still I wasn't sure and just HAD TO experiment.
It's simple, if you have a Module A with a class A::B inside, when you include A into class C, C::B will return A::B.
C::A will return A with a warning.

Bundler is cool and all, but I like YUM better.

Managing Ruby, its different versions and gems is kinda funny thing, funny as in it has many challenges.

First of all, you might want to handle different projects that work with different Ruby versions and each have a different gem set, so RVM is born and you can change these settings with ease, one less problem - although I do have a co-worker that so far failed to install certain gems with it.

Then there's one of the most annoying problems in Ruby, say your project uses gem "a 1.1" and when you try to deploy with "gem install a" you get version 1.2 and this for some reason causes your application to fail and it fails hard! It's one of the hardest errors to pin-point. Now multiply this problem for each gem and it's dependencies you use, it can really hurt.

That's why most Ruby developers (I use this term broadly, including anyone that programs in Ruby in any way) I know use Bundler and it's great, you get a list of all the gems you have on your system and it freezes the versions you know that make your application works and when you try to deploy it simply installs exactly the same gems you have on your devel machine so you don't have any gem difference problem.

But then you get gems with extensions and they don't quite compile, sure you spend some time doing "bundler install" and then "yum/apt-get install [some lib dependency]" and eventually you're there and all is good. Except if you have some nasty gem like ruby-oci8 which can be very annoying to install and not quite  as they say.

If I still got your attention so far, perhaps it's worth noting that my biggest gripe with these solutions is that they don't solve this problem of gem's unspoken dependencies.

Then you use RVM and Bundler together and all your problems are solved and little or no worries on your side. Although I do hear on occasion that those two have a few kinks working like that, but nothing major.

Then there's what's quickly becoming my favorite way of dealing with gems dependencies and so such: YUM!

It started first as an internal rule of the company I work for, the simple version: all software developed within must be packed in a RPM (for RHEL), and all of it's dependencies, including externally developed libraries (such as gems) must also be packed and available in the company's YUM server.

The reasoning was simple, once the software is sold, one must be able to go to the client's server and do "yum install software" and it will just work. (Later I heard this was because one account of failed installation caused the company to loose millions due to some contract provisions I don't really care to understand.)

While I did find it annoying going through these hops at first, I've been growing into this system and I'm grateful that they actually gave my training on software packaging (which is kinda uncommon for companies in Portugal).

YUM allows us to do some pretty fun things, such as, on a brand new server, I can use yum install to deploy my software with just "yum install" and yes, this does solve my problem of unspoken dependencies.

Take for example, an attempt to install my team's testing library on a brand new ruby installation on RHEL 6:

Notice how the first package in the list is NOT a gem, but rather an unspoken dependency for ruby-oci8. Pretty cool uh?

That also means that I have a greater control of all little things that get installed on the system for my app to work and also that I can purge it with considerable less effort. For instance, "yum remove ruby*" will remove ruby, ruby devel, rubygems, ruby libs, irb, all the rubygems and everything that was installed a dependency of any of those.

Sure RVM let's you easily switch between gem sets, but I can easily switch between "everything sets" - of course that is if I don't mind downloading them all again.

Also, with this you can apply patches to gems that are broken and abandoned despite still being popular.

There's a price for that, like everything in that area, it's knowledge, maybe the knowledge of packing is greater than the knowledge needed for Bundler + RVM, but to fix that, there's a tool, it's called "rgem2rpm" (which I use so much that the latest version, 1.4.9, is the release of a pull request I did on Github), with it, you can simply "rgem2pm gemname.gem" and you get a nifty rpm file. You get a few more options to customize it to your needs, and in the case of gems with unspoken dependencies you can pass an easy to make spec template file (you can use the default template as base for it) and add "requires".

When I took the above screenshot I noticed something was amiss, turns out, the testing version of rubygem-headless that we had on repository was not properly packed, so with 5 minutes to make a template file I compiled a new one and voilá:

See how much stuff you truly need to run the gem headless? Well to be fair, the only true dependency is "xorg-x11-server-Xvfb", all others are dependencies of that.

I know what you might be thinking now, for this to work, you need a yum repo with a lot of gems, at least all the gems you use (and their dependencies), well, yes, right now, I believe that we have more ruby gems as rpm then Fedora's Koji and OpenSuse combined (well, combined is probably an overstatement, but I'm confident we have more than each of them, at least).

All of this is packed for RHEL, they would work on Fedora (and probably others) I'm confident, but not by design.

I have in mind setting up a YUM server and compiling all these gems to Fedora, and providing a way for those using Centos and OpenSuse to use it as well, even when though I think there's little public interest in this, it will would make the setup of my devel computers (and co-workers') much, much quicker.

Edit: I almost forgot to mention, for most people, Ruby is a tool for either Rails or Sinatra web development, with a little bit of effort you can pack your web app into an  rpm and deploy on the production machine with YUM install, then have it start restart the web server (and whatever more you need for it to work, such as running database queries and whatnot) and launch your automated tests.
The more I use it, the more I feel that YUM is a very powerful tool that many neglect in favor of other tools that each do a part of what yum does alone.

PS: I do imagine that APT, PACKMAN, PACMAN and other package managers for the many Linux families are just as capable, but since I started making RPM's for RedHat and that's what I have to deal in a daily basis, I'm bias towards YUM.