There is one thing that we need to get out of the way first, a breaking change. In v0.0.10 and below, queries and some instances of calls to save_changes would return a single entity if there was only one returned. This was confusing and could lead to errors. As a consumer of the gem, you wouldn’t know if a call to the Service’s execute method was a single object or an enumerable. Instead of you having to guess, those calls now return enumerables all the time. This applies to calls to execute and save_change additions. Updates, deletes, and add_link calls (we’ll get to those in a minute), as well as batch saves all return Booleans, but single save_change add calls will return an enumerable because those results are parsed similar to execute calls.

Now that that’s out of the way, let’s look at the new features.

The Features

Partial Results

The first change is thanks to arienmalec. OData allows services to do server-side paging in Atom by defining a next link. The default behavior is to repeatedly consume partial feeds until the result set is complete, for example:

svc.Partials
results = svc.execute # => retrieves all results in the Partials collection

If desired (e.g., because the result set is too large to fit in memory), explicit traversal of partial results can be requested via options:

svc = OData::Service.new "http://example.com/Example.svc", { :eager_partial => false }
svc.Partials
results = svc.execute # => retrieves the first set of results returned by the server
if svc.partial? # => true if the last result set was a partial result set (i.e., had a next link)
  results.concat svc.next # => retrieves the next set of results
end
while svc.partial? # => to retrieve all partial result sets
  results.concat svc.next
end

Single Layer Inheritance

This feature was long overdue! Back in July 2010, Scott Allen wrote a post about ruby_odata where he mentioned a change to the Service class that would parse models with single layer inheritance. His code has finally been integrated into ruby_odata. Note, that the addition currently only support single layer inheritance, but that should handle most of the cases you will encounter.

Querying Links

There are times when you don’t necessarily want to bring down full navigation property entities. In these cases the OData protocol has support for Addressing Links Between Entities. To support this, there is a new method that can be tacked on to a single entity query (e.g. finding an entity by id) called links where you pass the name of the navigation property of the links that you want to retrieve. For example:

svc.Categories(1).links("Products")
product_links = svc.execute # => returns URIs for the products

Add Link

There are times when you need to create a linkage between a parent and a child, and you don’t want be required to perform an add or update. You can now simply call the add_link method on the OData::Service with the parent object, the name of the navigation property, and the child object. Finally, call save_changes. For example:

svc.add_link(<Parent>, <Navigation Property Name>, <Child>)
svc.save_changes

Lazy Loading

In previous versions of ruby_odata, if you wanted to perform lazy loading of navigation properties you would need to do it manually, which was extremely ugly. I’m happy to say that there is now a way to perform lazy loading in a graceful way. All you need to do is to call the load_property method on the OData::Service with the entity to fill and the name of the navigation property. For example:

# Without expanding the query
svc.Products(1)
prod1 = svc.execute.first
# Use load_property for lazy loading
svc.load_property(prod1, "Category")

Reflection / Introspection

Paul Gallagher (tardate) had a great suggestion: Why should you have to look at the EDMX metadata file directly when ruby_odata parses all that data? You can now use the OData::Service to access a list of the collections that the service exposes. There is also a classes method, which gives you a list of the classes generated by the OData::Service. A function_imports method, which will list function imports exposed by the service (we’ll get to those in a minute). On a class, you can call the class method, properties, which as you guessed will give you information about each property. Each of these methods return hashes where the keys are the name, and the values are hashes of the metadata for the member that you are introspecting.

Function Imports

Quite a while ago, I received an message from a user via this blog asking about the ability to call custom service methods. Within a WCF Data Service, you can expose your own custom methods, which get translated into the EDMX as FunctionImports. These aren’t to be confused with Entity Framework’s FunctionImports. These will be parsed and dynamically added as methods to the OData::Service. When you call one of the methods, they will return a result without you needing to make a call to execute or save_changes. There are different result types. You may get back a true if the FunctionImport doesn’t return any content (for example, a delete). You may get back a single entity, or a collection of entities. Finally, you may get back a single primitive value (e.g. Integer, String, etc), or a collection of primitive values. You can use the function_imports method on the Service to determine the type that will be returned.

Namespaces

In the same message as the request for the custom service methods (ok, it was actually a reply), the user mentioned that he received type conflicts with the generated classes if one by the same name already existed. If this is a problem for you, you now have the ability to tell the OData::Service to generate the models in a specific namespace. There is a new option that can be passed in to the constructor’s option hash, :namespace. If you use this option, there shouldn’t be any collisions. You can pass a namespace in either Ruby (with double colons, e.g. RubyOData::Rocks) or .NET style (with periods, e.g. RubyOData.Rocks) if there is more than one part to the namespace that you want to use. If you don’t provide a namespace, then ruby_odata will generate the classes without any namespace just as it has before.

The Beginnings of Something Cool?

As developers, the majority of the time we either retrieve collections or a single entity, usually by an ID. I didn’t really like the current steps to retrieve a single entity by ID:

svc.Categories(1)
category = svc.execute.first

Sort of ugly, right? So inspired by Ruby on Rails, I added a class method for each created class so that you could just do this:

category = Category.first(1)

Great, right? It’s a little nicer, but sadly it’s a one trick pony right now. There’s no filtering, eager loading, etc. My idea was to hopefully expand on the concept like Rails and ActiveRecord, but for now it’s just an easier, cleaner way to get an entity.

Testing

In addition to the new features, I’ve been busy working on the testing suite.

No More Hardcoded URIs

Having to rely on Windows for testing has made me stick to Windows and e-TextEditor for ruby_odata development. The key word there is actually testing. All that Windows is required for is hosting the WCF Data Service for the Cucumber integration tests, but Cucumber is just one part of it, RSpec is the other. It’s no denying that Ruby on Windows is slower than its *nix counterparts (not to knock the RubyInstaller.com team, if it wasn’t for them, Ruby on Windows would be non-existent). Where you really see a slowdown is when you are doing TDD/BDD testing. You need tests to run fast, and if you ran the command time rake spec, my Windows box is more than 4 times slower than my old MacBook (note, not a Pro), which has similar hardware. Now, multiply that by every spec I write and I want to throw my machine out a window. So, in an effort to migrate development, the URLs and ports required for the sample WCF service are now dynamic. Take a look at features/support/constants.rb for more information. Thanks to Sean Carpenter (scarpenter) for his fork where he changed the static values in that file to be optionally set by environment variables.

Hello New Technology

I decided to change up the technologies used within the test service. First, no more SQL Express, hello SQL Compact 4. Why the change? SQL Compact 4 is simply DLL deployed, you don’t need to install anything, cool. Also, since SQL Compact 4 databases are file based, instead of having to truncate to clean the database for testing, the file could just be blown away and rebuilt easily.

So, a new database, how about a new ORM? Entity Framework 4.1, also known as “magic unicorn edition” is now being used. EF 4.1 “Code First” as it is called, is just awesome. Ditching Entity Framework’s EDMX for pure code, now that is right up my alley.

Finally, since I switch to EF 4.1, the current released version of WCF Data Services, well, didn’t work right. You know what did? The WCF Data Services October 2011 CTP, w00t.

Everything you need should be in the new test project (now called RubyODataService, changed from SampleService, though that’s still in the test URLs) thanks to NuGet and my copying of the WCF DS CTP DLLs to the packages directory since there wasn’t a NuGet package for it).

Pickle

I love the pickle gem. Aside from being authored by a guy with my last name (no relation), pickle makes Cucumber features so much nicer to read. Throughout development of ruby_odata, I’ve written my own step definitions to do things, essentially cloning what pickle does out-of-the-box, e.g. creating models, asserting against models, etc. I had toyed with getting pickle to work around June of last year, but was having issues. As time passed and experience grew, I finally got it all working. I used it in a few places in new tests, but I hope to do a complete overhaul. My pickle adapter also needs a bit of work since I didn’t put it through all of its paces, and some things need to be added to ruby_odata before it will function as expected. All in all though, it is great to finally have pickle working.

Conclusion

I hope that you find the new version of ruby_odata to be useful. In the next version, I hope to further enhance the gem with more of the different operations that OData can perform. I’m sure I’ll perform some more housekeeping in the test suite removing my ugly Cucumber steps for nice, clean Pickle steps.

What would you like to see in the new version? Add your suggestions to the issues section on GitHub. I’d also like to know what you think of the API as it stands right now. I’ve essentially been following the OData Silverlight client API syntax (e.g. AddTo<Collection Name> methods, add_link, etc). With the addition of the :first class method, it got me thinking of a bit of a different approach. Not necessarily giving up one for the other, but I think another more Ruby-esq style could co-exist. Anyway, that’s all for now, go forth and do cool things with ruby_odata. I’d love to know how you are using the gem. I was excited to find out that ruby_odata was being utilized by the rgovdatablog gem.

I skipped an update for v0.0.9, so let’s revisit what was added/fixed back then. The major change was support for self-signed SSL certificates. That addition was courtesy of J.D Mullin. I’m sure you remember his contributions back in v0.0.8 when he added Basic HTTP Authentication. Aside from J.D.’s contributions, the library was refactored to only make one call to the service for building classes and collections. Finally, a few bug fixes wrapped up the release.

Now onto v0.0.10. A lot of changes were implemented in this release that ended up allowing for wider support for ruby_odata. Thanks to input from Michael Koegel (@konfusius), Alvaro Tejada (@Blag), Ingo Sauerzapf (@IngoS11), and Juergen Schmerder (@schmerdy) (I apologize if I missed anyone!), ruby_odata now supports (at least basic functionality) for SAP NetWeaver GateWay.

Another big change is the initial support for feed customizations. Properties may not be represented within the <m:properties> collection of an entity. Currently ruby_odata supports SyndicationTitle and SyndicationSummary, but adding the other syndication options should be easy to support. If you can supply me with the metadata and a copy of the output that includes the additional feed customizations, I will be happy to add them (or feel free to fork the project and add them yourself).

Here’s are the complete list of changes for v0.0.10:

• New Features
  • Added the ability to pass additional parameters that are appended to the query string for requests
  • Added initial support for feed customizations (SyndicationTitle and SyndicationSummary)
  • Enhanced ruby_odata’s awareness of classes based on the metadata instead of relying on results that are returned
• Bug Fixes
  • Fixed issues with nested collections (eager loading)
  • Handled ArgumentError on the Time.parse for older versions of Ruby; used DateTime.parse instead if Time.parse fails
  • Removed the camelize method call when building the root URL for collections (reported by mkoegel, issue #3 on github)
  • Handled building results (classes) where the category element is missing but there is a title element instead (reported by mkoegel, issue #3 on github in the comments)
• Other
  • Change HTTP port to 8989 since 8888 conflicts with the Intel AppStore
  • Refactored service step for HTTP calls where the service address is defined within the step making it easier to make changes in the future

For the next release, I hope to continue to fix issues, finally add support for inheritance, add the ability to call a custom WebGet methods added to the service, and hopefully namespacing of entities to limit collisions. Do you have other requests? Add them to the issues list on GitHub. The previous Lighthouse site for issues has been closed in favor of using GitHub.

require 'lib/ruby_odata'
svc = OData::Service.new "http://127.0.0.1:8888/SampleService/Entities.svc", { :username => "bob", :password=> "12345" }

In addition to the big change by @jdmullin, the following changes are apart of the new version:

• Support for nullable primitive types being returned from an OData service.
• A change in ActiveSupport 3.0.x broke additions in ruby_odata when you associate a previously saved entity.  This has been corrected, and ActiveSupport 2.3.x (tested 2.3.11) and 3.0.x (tested 3.0.5) are now supported. 
• Ruby 1.9.2 is now supported.

Other “backend” type enhancements were added:

• Removed Jeweler for releasing gems
• Moved to bundler
• Updated README to reflect new changes
• Updated DB management so that testers aren’t required to move files around
• Fixed gemspec to list the development gems
• Developers can just do a “bundle install” to get all the required gems

One interesting thing that I found while testing is that Ruby 1.9.2 is much faster when running Cucumber tests on Windows.  I was using Ruby 1.8.7-p330 vs 1.9.2-p180 on Windows (RubyInstaller rocks!).  Here’s a snapshot:

Note there is a bit of a problem using Ruby 1.9.2 w/ Cucumber on Windows.  The problem is that when using bundler or “gem install,” the json gem gets installed as the x86-mingw32 version.  Running cucumber on Windows with Ruby 1.9.2 (using the incorrectly compiled json gem) yields a msvcrt-ruby18.dll error.  The way I fixed the error was to install the RubyInstaller DevKit, and then reinstall the gem “correctly” using:

gem uninstall json
gem install json --platform=ruby -v 1.4.6

What we are doing is uninstalling the incorrect copy of the json gem and then installing v1.4.6 of json using the –platform=ruby flag which will involve the DevKit to compile the gem for us.  Note that the json gem version required by Cucumber 0.10.0 is ~> 1.4.6, and there is a new version out, so the -v 1.4.6 forces the correct one.  Found the solution on StackOverflow,  but it seems that there should be a better way to solve this, perhaps with bundle-config?

• <Collection> dynamic methods are available after you instantiate the service.  You can chain query operations like order_by, filter, expand, etc. on to the dynamic methods.
   
• For querying, $orderby, $top, $skip, $filter, and $expand are supported.  These are represented by the methods order_by, top, skip, filter, and expand (respectively) on the OData::QueryBuilder class (which is returned from the dynamic collection methods), for example: svc.Products.top(10).
   
• The ability to add entities.  AddTo<Collection> dynamic methods are available after you instantiate the service, allowing you to pass new entities to the methods.
   
• The ability to update entities though the update_object method on the OData::Service.
   
• The ability to delete entities though the delete_object method on the OData::Service.
   
• The ability to batch saves, which allows you to pass multiple objects to be added, updated, or deleted and have them persisted as a batch when you call the save_changes method on the OData::Service.  Note that with batch saves, the changes either complete or fail as a batch.
   
• Explicit data type support for the following: (Everything outside of this list currently is represented as a string.)
  • Edm.Int16
  • Edm.Int32
  • Edm.Int64
  • Edm.Decimal
  • Edm.DateTime – Note that DateTime is a bit odd.  There’s no time zone data passed from OData services (at least not with WCF Data Services), so it is assumed that the time zone is UTC for all DateTime types.
     
• Support for Complex Types – From OData.org: “Complex Types are structured types also consisting of a list of properties but with no key, and thus can only exist as a property of a containing entity or as a temporary value.”  You’ll typically see these to logically group properties together.  For example, with the Netflix service, they expose a BoxArt complex type on a Title.  The BoxArt type consists of 3 properties, SmallUrl, MediumUrl, LargeUrl.  This makes accessing these clearer and cleaner, for example movie.BoxArt.MediumUrl.
   
• Works with Ruby 1.8.7 and Ruby 1.9.1.  It also works with ActiveSupport 3.0.0.beta4.  These were tested on Windows due to using WCF Data Services for the test suite, although, if something Ruby works in Windows there is a good chance everything should be fine on Mac or Linux as well.  I could always change the test URLs to point at a remote machine to do my testing on a Mac, but I wish there was an OData producer like WCF Data Services that would work with ActiveRecord (or similar).

OData is so powerful because it’s REST based and you can access the services from just about everywhere, including just a simple URL.  For example, using the Netflix OData service to access the best movie ever made by title: http://odata.netflix.com/Catalog/Titles?$filter=Name eq ‘Office Space’.  Pretty simple.  Of course accessing things solely based on URLs in code isn’t the best, and hence the reason for my post.

Client Libraries

When using OData, it’s convenient to utilize an SDK to access the services.  For example, in Silverlight you can access OData services using LINQ.  There’s also a fantastic AJAX Library (which I used in a previous post).  There are SDKs for PHP, Objective-C, and many others, but there was one missing that I wanted to use… Ruby.  I am assuming this comes as no surprise given my new found love for Ruby.

Introducing ruby_odata

I’ve been messing around with Ruby enough to start working on my first Ruby open-source project.  It’s always good to contribute to the open-source community.  While I’m not a seasoned Ruby developer, I figure this is a good project to start on.  Now I can code stuff in Ruby and consume WCF services I have written in the past and will continue to write in the future (that is until there is “OData on Rails,” perhaps an idea for a new project?).

The source code, like all good Ruby code, is hosted on GitHub.  The README will give you a good idea of how to use the library.  Currently (as of 06/12/2010) there are limitations, such as not being able to batch changes (like multiple Adds), but no worries, I’m working on adding them (actually, I cut three releases today, adding extra querying abilities).  That said, it is functional enough to use for most operations right now.  Once it’s been better tested in the real world and I’ve been able to incorporate most if not all of the OData functionality, I will be versioning more responsibly.  Right now, because I’m adding functionality quickly and I want others to be able to use it, you’ll be seeing lots of updates so the version number will be a bit jumpy for a while.

The Genesis of ruby_odata

I started writing ruby_odata in order to interface with my database via services so I could do testing in Ruby and create objects from within the tests that I could use in my ASP.NET MVC application.  I wrote about BDD testing in an earlier post where I fixed the speed of running the tests using Cucumber with Mechanize.  Another area missing in .NET is the ability to automatically generate objects to test against like you can with Ian White’s Pickle.   Pickle interacts nicely with factories and your database to create objects for tests.  For example, a step like: Given a user exists with name: "Damien" will automatically create a new user in the test database.  This works dandy in the Ruby world (with ActiveRecord for example), but against something like Entity Framework, not so much.  With ruby_odata however, this is something that could work in a similar way. (Yet another idea for a project, a Pickle adapter that utilizes ruby_odata.  Man, so many ideas, so little time.)

Usage Overview

The library is pretty straightforward to use.  At the heart is the OData::Service.  This is the root object that drives all of your operations against the services.   Once you instantiate the OData::Service, dynamic methods based on the OData service’s collections are available to call.  For example, using the Netflix OData service, we can see there is a collection called “Titles.”  Within ruby_odata, the OData::Service will respond to a “Titles” method:

svc = OData::Service.new "http://odata.netflix.com/Catalog/"
puts "Responds to Titles? #{svc.respond_to? :Titles}" # => Responds to Titles? true

So, just by looking at the collections on the main service page, or metadata page, you can see what you can query.  Here’s an example of querying for a movie (like we did earlier with a straight up URL):

svc.Titles.filter("Name eq 'Office Space'")
movie = svc.execute
# Now we can access properties of the Title (defined in the service metadata)
puts "Average Rating: #{movie.AverageRating}"
puts "Release Year: #{movie.ReleaseYear}"
puts "URL: #{movie.Url}"
# => Average Rating: 4.2
# => Release Year: 1999
# => URL: http://www.netflix.com/Movie/Office_Space/20358351

Pretty cool, huh?

The Basics

There is a simple pattern for using the library.  For queries, you call a collection on the service (like you saw with Titles in the Netflix example above), and that returns an OData::QueryBuilder object to you in which you can fluently add options like filter or expand.  If you use multiple calls to filter, it will perform an “AND” query against the service, and multiple expand calls will yield multiple items being expanded.  You can also mix the calls, like .filter(…).expand(…).  Once you are done building a query, you call the execute method on the service.  The result of the execute is either a single entity or a collection of the entities based on your query.

In order to add, update, or delete, you need to call the appropriate method on the OData::Service.  To add, you would use the dynamic method AddTo<Collection Name>, where the <Collection Name> is the name of the collection on the service (as defined by the metadata).  To update, you would use update_object, passing in the updated version of the object.  Finally, to delete, you would use delete_object, passing the object to delete.  Once you do one of these operations, you need to persist the change using the save_changes method on the OData::Service.  Currently, you need to call save_changes after each operation in order to persist the data.  In the future you will be able to batch operations instead of persisting them one at a time.

More Examples

For more examples, download the source code from GitHub and setup the testing environment (instructions for setting up the test environment are located in the README).  When you setup the test environment, you can follow along with the samples as shown in the README.  There are examples for all the CRUD operations.

Conclusion

There you have it, an OData library for Ruby.  I am very excited by the potential of OData, and having the ability to consume the services easily within Ruby is extremely useful.  I hope you find the library helpful.  The current version is a bit of a work in progress since everything isn’t fully supported yet, but I will continue to work on the project.  You can get involved by checking out the resources for the project:

• Source Code (hosted on GitHub): http://github.com/visoft/ruby_odata
• Documentation (hosted on rdoc.info): http://rdoc.info/projects/visoft/ruby_odata
• Issue tracking (hosted on GitHub): https://github.com/visoft/ruby_odata/issues
• Wiki (hosted on GitHub): http://wiki.github.com/visoft/ruby_odata/

If you just want to start using it without the source code, you can install ruby_odata as a gem from RubyGems.org.