Custom RSpec matchers for testing your Ripple Documents

We're currently using Ripple for modeling our objects in Riak. Since we are test first, we needed some matchers to help make specifying our Ripple::Documents easier.

(As a side note, we are also developing a gem to wrap RiakSearch functionality. The gem is called Ripplr and has been pushed to ruby gems.)

These matchers were created by extracting specifications from within Ripple's test suite.

**Note: Matchers updated on 10-17-2012 to support embedded docs and added failure messages

Why Nginx kept Riak's secondary indexes(2i) from working

Riak recently introduced Secondary Indexes (a.k.a. 2i). 2i allows objects in Riak to be stored with additional queryable values. At Validas we have leveraged 2i for validating uniqueness instead of relying on MapReduce or RiakSearch to perform the check, among other things. This worked great for us in our local dev environment.

Unfortunately when we deployed to our staging environment any feature that leveraged 2i failed to work. We spent a good amount of time reviewing our specs and feature implementation, but everything checked out fine so we moved on to our Riak app.config, comparing line for line the staging and development configuration. This did not produce any answers either.

Eventually we landed at our load balancer, Nginx. When we bypassed Nginx 2i worked perfectly! We could add an index to an object and then find the object by searching for the indexed value. We were able to determine that Nginx was removing the index header information from our request.

Not sure why Nginx was squashing a seemingly innocent 2i header and not other Riak specific headers, we ran across this post ... How to get non standard http headers on Nginx. Nginx by default removes header that use underscores.

As you can see here, Riak depends on underscores for managing secondary indexes as it suffixes the index name with '_bin'.

That same post generously linked to Nginx: underscores in headers which shows you how to modify your Nginx config and allow underscores in headers. Once we made that change, all of our Riak 2i features worked flawlessly!

Hopefully if you have this problem you find this blog post in less time than it took us to troubleshoot!

Riak Search with Ripple in 3 easy steps

Modify your Riak app.config

By default, Riak Search is disabled. You can enable it in the app.config on each Riak server by changing the value from false to true. The config section should then look something like this:

%% Riak Search Config
 {riak_search, [
                %% To enable Search functionality set this 'true'.
                {enabled, true}
               ]},

Don't forget to restart Riak afterwards.

Enable the commit hook

From the Rails console you can enable the commit hook for the model you would like to make searchable. Keep in mind anything created prior to doing this will not be searchable. You will need to go back afterwards and essentially touch each of those objects.

MyObject.bucket.enable_index!

Just replace MyObject with class you wish to make searchable.

Search!

Ripple.client.search MyObject.bucket.name, "some_field:query"

For additional information, refer to these sites:

http://djds4rce.wordpress.com/2012/03/22/riak-fulll-text-search-with-ripple/ 

http://www.ibm.com/developerworks/opensource/library/os-riak2/os-riak2-pdf.pdf

 

How to install Ruby and Rails on Mountain Lion as of now

Since XCode no longer packages gcc, you need to do some fancy dancing to get your rubies compiled.


Install XCode 4.4
Install XCode 4.4 from the App Store
Install XCode Command Line Tools (Open XCode, Preferences, Downloads)
sudo xcodebuild -license agree

If you skip that last step, you will see the following while attempting to install apple-gcc42

You have not agreed to the Xcode license agreements, please run 'xcodebuild -license' 

Install Mac Ports
Download the Mountain Lion installer from http://www.macports.org/install.php/ and run it
sudo port selfupdate

Install GCC
sudo port install apple-gcc42
sudo port install gmake
sudo port install gpatch
sudo ln -s /opt/local/bin/gcc-apple-4.2 /opt/local/bin/gcc
sudo ln -s /opt/local/bin/gmake /opt/local/bin/make
sudo ln -s /opt/local/bin/gpatch /opt/local/bin/patch

Install RVM
curl -L https://get.rvm.io | bash -s stable --ruby
source ~/.rvm/scripts/rvm
rvm install 1.9.3-p194
rvm use 1.9.3-p194 --default

Happy Times!

Specify What, Not How

It was January 11, 2012 and I was attending Jim Weirich’s session on Vital Testing at CodeMash. Now I’ve seen his talks numerous times and continue to attend his talks any chance I have. Why? Because Jim is just that awesome! There is always something Jim says that triggers a challenge within myself that leads to improvement in how I approach crafting software. That day it was, “Specify what, not how”

I already knew that specs should not describe implementation and I prided myself in being able to abstract the implementation details away from my specs. But as Jim elaborated further on specifying behavior and what was sufficient, I realized that I could not entirely rewrite my apps from just the specs. Why? Because even though I had abstracted away implementation details, my specs still contained some implementation details.

Take this snippet for example. What’s wrong with it?


Most Ruby developers would say nothing. And I used to too. Now look again at the very beginning. What is it that is being specified? The class name. The class name is being directly stated in our specification identifying how we are implementing our behavior. This is typically an accepted approach when using RSpec, but if our goal is to achieve the writing of specifications and remove implementation details from our code, then this is not ok.

“Ok then, how should it look?”, you ask. Let’s take a step back. In our example we are trying to describe converting roman numerals to integers, so let’s write that.



Ok, easy enough and it reads nicely and clearly states our intentions. “But how do you write the actual test?” Great question. If you could write it anyway you wanted, how would it look? Using our first example I propose “1 should be converted to I”. That’s great, but it will not execute without some heavy lifting. Not to fear! With a little tweaking we could write it instead as:

1.should be_converted_to "I"

Awesome (I <3 Ruby)! That is something we can work with. And the magic sauce that brings it all together ... custom matchers! Custom matchers easily allow us to be more expressive in how we write our specifications and provide nice wrappers for our implementation details.

Here is an example of what a custom matcher would look like based on how we just wrote our test.



All of our implementation details have now been abstracted away from the specifications and wrapped up in this cool little matcher. Now when we decide to change our implementation details we can leave our specifications alone and focus solely on our matcher. Expanded out a bit further our specifications could look something like this:



I don’t know about you, but those specifications are much easier to read AND to maintain. And yes, this does translate to your project. We've been taking this approach since January and have seen big gains in reducing any brittleness in our tests, but also we’ve seen the quality of our code increase significantly! 

The custom matchers have also helped identify when an object is doing too much. So when setup starts to creep into our matchers we quickly reevaluate our approach and refactor to remove the setup from the matchers, resulting in more independent objects. (Gotta love SRP!)

I hope this helps you in your quest to write better specifications. Special thanks to Jonathan Knapp for pairing with me that day as we explored this approach together in our quest to best roman numerals!

What processes do you use?

I am often asked how we deliver software here at Dix Communications, to which I reply, “Our way”. Over the years I have been exposed to a variety of methodologies for developing and releasing software; the list includes RUP, eXtremeProgramming, Lean, Scrum, and even some Waterfall. The one aspect of all of those that sticks out above the rest is the opening sentence in Kent Beck’s book ‘Extreme Programming Explained’, “XP is about social change”.

Charles is happy! His tests are GREEN!

Everything about a software project requires social interaction. So if you are looking to improve your projects chances of success, look for ways to improve the social interaction with your project. Agile, Lean, eXtreme Programming and Scrum all introduce ways to improve social interaction and we have taken a best of bread approach that suits our needs and that is what we use here in the Digital dept.

In forthcoming posts I will describe which techniques we have found instrumental to us building and delivering successful software applications.

undefined method `version' for nil:NilClass

The last thing I expected to see while I was upgrading ruby gems was

"undefined method `version' for nil:NilClass"

Fortunately I was not the first person to encounter this and Google pointed me in the right direction. To solve this problem run

gem install rubygems-update

and then run

update_rubygems

Thanks to Tommy Ryan for posting his solution!

Also, while upgrading gems there seems to be some controversy surrounding recent updates to Ruby Gems that can cause conflicts with your bundle. Per The RailsApps Project:

As of release 1.8.0 (4 May 2011), the RubyGems system gem began a series of rapid updates (with controversy and a promised resolution). The example apps all work with version 1.8.15 but if you install a gem and find it doesn’t work, roll back to an older version of RubyGems: