Not So Stupid

I’ve created a Google Group for ActiveDocument, since it seems there are 4 or 5 different projects going on and we should try to coordinate them into a single one, or in the worst case, we can at least exchange ideas and concerns.

I’ll probably merge my efforts into Rick’s code base, just because he has a better public GIT server setup than mine, and his code already has specs.

Since Matt Knox talked about ThruDB on last tuesday’s meeting of NYC.rb, my brain has been thinking about document-oriented databases, about how tired I am of SQL, about how tired I am of trying to scale database servers, about how tempting is to have more flexible models and data structures, and about how tempting it is to have a clear and simple scalability path.

The samples included in the ThruDB tutorial are, to be honest, ugly. But they are designed to show how thrift provides language-agnostic data types and how ThruDB can be accessed from different languages.

However, I have several ideas in my head about how to implement something I’m calling, for the time being, ActiveDocument. It won’t be a direct replacement for ActiveRecord, but it will have similar features (i.e. validations and callback hooks) and it will allow for very simple usage of ThruDB. I might later add support for CouchDB, SimpleDB and other similar technologies, but just like Rails doesn’t try to be a full database server abstraction, your ActiveDocument code will not work on different servers unless it’s limited to very simple operations. The world of document-oriented databases is even less standardized than relational database servers.

Here’s a little look at how it might look:

class User < ActiveDocument::Model
  attribute :login, :string, :indexed, :sortable
  attribute :email, :string, :indexed
  attribute :created_on, :datetime
  attribute :password, :string
  has_many :bookmarks
end

class Bookmark < ActiveDocument::Model
  attribute :title, :string, :indexed
  attribute :url, :string, :indexed
  belongs_to :user
end

User.find_by_login("sd")
User.find(:all, :conditions => “login:’s*’ AND created_at :[20071201 TO 20080115]”)

As you can see, the two biggest differences from plain old ActiveRecord is that the model will have to define it’s own schema, and that queries will use the Lucene Syntax

Relationships would be defined using fields with lists of IDs, and queried using Lucene’s fast indexes. This might make models too big when they have a large number of related objects, but that’s a problem to be solved later.

Since document-oriented databases have no concept of joins, some queries will be definitely slower than their SQL counterparts, having to make multiple calls to the server to retrieve individual objects. However, each one of those calls would be simpler and easier to cache, which I hope will reduce the performance impact. And as long as it’s not 100 times slower, I’m willing to trade off some performance for the promise of infinite scalability.

And since the models will be more flexible, you can probably skip a lot of traditional SQL tables and store the data directly into the model itself. For example, users can have preference arrays or hashes, which would have been separate tables in SQL but that are just additional attributes in ThruDB.

Speaking of attributes. ThruDB uses thrift for its own API, and the tutorials suggest using it to encode the documents themselves, but the API doesn’t require that. I’ve been trying to figure out how to encode a thrift object along with it’s own class name, to make it easier to decode afterwards, specially when performing polymorfic queries. Perhaps I’ll have to use double encoding, with an envelope thrift object containing the class name and the encoded string. Or perhaps I’ll use YAML to encode an attribute hash. YAML is tempting because it will allow for more complex objects and for dynamic schemas (i.e. an attribute that’s a hash of hashes containing values of different types).

Anyway, I’m starting to write the code, and it looks like it might be possible to have some working prototype a lot sooner than I though possible at first.

If you’re interested, just drop me a note, leave a comment, send me an email or look for me as ’sd’ on Freenode’s #nyc.rb.

I’ve just released an update to my Embedded Actions plugin for Rails.

With significant contributions from Jerret Taylor and Ryan Barber, we’ve removed a bunch of little bugs and added support for caching actions in namespaced controllers, and for passing options to the fragment caching mechanism, such as ttl settings.

Install it with

script/plugin install http://dev.notso.net/svn/rails/plugins/embedded_actions/current

In case you have been sleeping in the same cave as Osama Bin Laden, Apple’s new OS X Leopard includes Ruby as a first-class language.

But Apple’s effort to make the language and all it’s extensions universal binaries can cause you some trouble when installing gems that require compilation.

If you’re installing on an Intel machine and see an error like “ld: symbol(s) not found for architecture ppc”, you probably are installing a gem that requires an external library, for which you only have the i386 version. This is a typical situation when installing mysql (as noted in the troubleshooting page of the MacOSForge wiki for Ruby).

After trying several variations, I came out with this solution:

If your installation command was

sudo gem install mysql

you need to run it as

sudo bash -c "ARCHFLAGS='-arch i386' gem install mysql"

sudo env ARCHFLAGS="-arch i386" gem install mysql

There you go… that should be all you need to install the mysql gem on Leopard against MySQL’s prepackaged binaries.

UPDATE: The troubleshooting page has been updated to include an alternative: using “sudo -s” to start a root shell. I still like my one-liner better :-)

UPDATE 2: Using env instead of bash is slightly cleaner.

UPDATE 3: MySQL still has some problems, because the library is pointing to the wrong direction. The quick solution is to create a link to the right place:

sudo ln -s /usr/local/bin/mysql/lib /usr/local/bin/mysql/lib/mysql

This is an extraction of some things we’ve been using at StreetEasy for about two years now. Here’s the README:

Just like the traditional render :partial, embedded actions allow you to
refactor your views and extract presentation logic and templates into separate
files.

Unlike partials, embedded actions also let you define business logic to be
performed before the partial is included. That logic is encapsulated in the
already well understood metaphor of an action inside a controller.

So a simple call like

<%= embed_action :controller => “songs”, :action => “top10″ %>

lets you include an html fragment containing the top 10 songs into any of your
pages, regardless of which controller, action or view wants to do the including.

Additionally, embedded actions can provide caching of their results (allowing
for different parameters) just like page caching, but at the level of html
fragments. So your dynamic pages can still be rendered dynamically, but some of
the embedded actions can be cached (and expired) independently.

Just declare an action as ‘cacheable’ in a way similar to page caching,
by invoking “caches_embedded” with the name of the action to cache.

class TestController < ApplicationController
caches_embedded :user_list

def user_list
...
end
end

Cached fragments can be invalidated with calls to expires_embedded, but you must
remember to use the same set of parameters used to embed the cached action in
the first place.

The code is available at

http://dev.notso.net/svn/rails/plugins/embedded_actions/

and can be easily installed by running

script/plugin install http://dev.notso.net/svn/rails/plugins/embedded_actions/current

.

Enjoy, and please comment.

UPDATE: Version 1.1 has been released.

(I just submitted this story to Slashdot, but I didn’t want to see this masterpiece get lost in the bowels of their submission queue, so I’m also posting it here. Update: it got accepted)

The developers of Rubinius, an experimental Ruby interpreter inspired by SmallTalk, have been discussing the possibility of adding a Lisp dialect to their VM. Pat Eyler collected some ideas and opinions from the people involved and it makes for some interesting reading.

For many, Ruby already is an acceptable Lisp, and the language itself started as a perlification of Lisp (even Matz says so) so it is perhaps fitting and might help explain why the whole idea feels right.

Now, if someone added support for VB and gave it the respect it deserves, the world would be a better place.

Ruby has a very convenient method to inspect objects: “p”. It just prints the result of “inspect”. And it’s exactly what irb uses to show the result of each expression.

Anyway, the cool guys at projectionist just posted a little method of theirs called “m”, which provides easy access to an object’s methods.

That made me remember my old (well, not that old) “pm” method for irb, which even if I haven’t talked about here, I’ve made public at dotfiles as part of my irbirc (it’s the last method).

Anyway, looking at their implementation, I decided to polish mine and release it here:

ANSI_RESET        = "33[0m"
ANSI_BOLD         = "33[1m"
ANSI_GRAY         = "33[1;30m"
ANSI_LGRAY        = "33[0;37m"

def pm(obj, *options) # Print methods
  methods = obj.methods - (options.include?(:more) ? [] : Object.methods)
  filter = options.select {|opt| opt.kind_of? Regexp}.first
  methods = methods.select {|name| name =~ filter} if filter

  data = methods.sort.collect do |name|
    method = obj.method(name)
    args = "(" + case method.arity <=> 0
    when 1
      (”a”..(?a + method.arity - 1).chr).to_a.join(”, “)
    when -1
      (”a”..(?a - method.arity - 1).chr).to_a.join(”, “)
    else
      “”
    end + “)”
    klass = $1 if method.inspect =~ /Method: (.*?)#/
    klass = $1 if klass =~ /((.*?))/
    [name, args, klass]
  end
  max_name_length = data.collect {|item| item[0].size}.max
  max_args_length = data.collect {|item| item[1].size}.max
  data.each do |item|
    print ” #{ANSI_BOLD}#{item[0].rjust(max_name_length)}#{ANSI_RESET}”
    print “#{ANSI_GRAY}#{item[1].ljust(max_args_length)}#{ANSI_RESET}”
    print ”   #{ANSI_LGRAY}#{item[2]}#{ANSI_RESET} n”
  end
  data.size
end

Don’t try to understand it unless you can understand it :-)… just copy it to your .irbrc (you do have an irbrc file, don’t you?). And use it like this:

Here’s a little hack for those of you that run Rails’ script/server on its own window or tab.

By inserting a couple of lines into the server script, you can have it change the title of the window or tab it’s running on, making it a lot easier to look for it when you have lots of windows open.

All you need is to change “script/server” so it looks like this:

#!/usr/bin/ruby
print “33]2;Rails Server07“ # xterm window title
print “33]1;Rails Server07“ # screen/iterm tab title

require File.dirname(__FILE__) + ‘/../config/boot‘
require ‘commands/server‘

print “33]1; 07“ # screen/iterm tab title
print “33]2; 07“ # xterm window title

The second set of prints will clear the title after the server terminates. You might want to adjust it to suit your needs.

If you did not attend RubyConf 2006, then please see this movie first.

If you did attend the conference, or you have seen “Erlang, The Movie” before, then this needs no other explanation:

Erlang, The Ringtone.mp3

Joel is wrong when he says that you should pick a safe language (Java, C#, PHP or maybe python) if “someone is going to get fired”.

He almost got it right: You should go with a safe language if you’re afraid of being fired for picking the wrong language.

And in fact, in that case, I can’t understand why would you pick PHP or python. VisualBasic, sure, but using any other scripting language is probably a career-killing.