Evaluating Redcar

Intro and Background

It has been about 2 weeks since I started developing RoR and Ruby apps in general at Favorite Medium. When I got started I had an OS X environment that I was new to and Textmate as my editor. I choose Textmate because (let's be honest...) it seems to be very popular with rails developers. But I had already used and developed in ruby using IDEs such as NetBeans and Rubymine. And I still tend to prefer an IDE over a text editor.

However, now I feel more comfortable with Textmate as I have used it for over two weeks. I will be evaluating Redcar and will be comparing it (well yes, my attitude towards Redcar is that it is really inspired by Textmate) with the experience I had with Textmate. To be fair, I haven't used Textmate extensively and I am for sure missing out on some great features it may have. So forgive me if I make any inaccurate assumptions in this post.

I am doing this evaluation as a ruby developer. So developers of other languages might look at this editor under a different light.

What is Redcar

Redcar is a free and open source text editor written in ruby (but runs on jruby. You don't need to have jruby installed in order to install Redcar though. It will auto-magically download a version of jruby and other needed libs for its use). It has a Textmate-like feel to it. It also supports Textmate bundles (one of the strong points of Textmate)! Right now it has an active, healthy community around it. But it's a very young project that is still under heavy development (ex:- it doesn't have a save all function :( ).

Let's break it down to pros and cons.

Pros

• It's Open Source (thus free as in free beer and freedom!)

• Has a good momentum behind it (at the time of this writing).

• Supports Textmate bundles (This means a lot. If you have used Textmate you would know that a lot of nice features of Textmate come through bundles) and themes.

• Written in pure ruby and runs on jruby (this is good news if you are a ruby developer - it will be easy to extend and write plugins in ruby).

• Cross platform (I think I know more than 10 people who wanted to find 'the Textmate for Linux/Windows').

• Extensible (no editor in this world will ever behave and do everything just like you would want it to. You will have to customize it at some point. And the author can't implement every feature under the sun. A good plugin system is very important).

• Easy learning curve if you are coming from Textmate (Redcar supports most Textmate shortcuts although it's not a one to one mapping. It also has a lot of functionality that can be found in Textmate, where they are identified by the same names).

• Last but not least, Redcar comes with the 'cool-latest-greatest' feeling attached. So you can always feel good using and hacking it.

Cons

• Uncertainty about future support and longevity of the project (Redcar is great. And it's very promising. But we don't know where it will stand in another two years. Editors are things that people choose, learn, and live with. If I am to commit my self to an editor I need to be sure that it will be supported and actively developed for the near future. If the development stalls I can't afford to take it over and carry it on. But then Redcar is so similar to Textmate, so in the event of Redcar suffering a early death one could always go back to Textmate or to one of its many clones out there.)

• Lack of features – For its age Redcar definitely supports a ton of features! But if you want a full and complete product Redcar is not that (at least at the time of writing it's not. But I hope it will be able to come to that level in the near future). But with the active development community and the passion behind it. I hope that it can reach a level where it has a healthy set of complete features pretty soon.

• Lack of maturity – You will see it in the UI sometimes, and everywhere else. This is perfectly natural for a new project.

  Requirement of Java - I am not sure whether this is even a con. But some people might not feel very comfortable having to install java if they don't already have it (you already have Java pre-installed in OS X for example...).

All in all I feel I will give this a try and see how things go. I have just gone through the documentation and installed and used Redcar for like 2 hours. You can't tell what's working and what's not during such a small amount of time. Issues with stability and performance might pop up during prolonged usage. But I have a positive outlook about this. There is an upcoming rails project and I am going to use Redcar for it. Let's see how things go.

If you are excited by new tech and if you are in to ruby/rails and Textmate definitely do check out this new editor. Its worth the while.

RabbitMQ: PubSub Messaging with Groovy

RabbitMQ typically send "direct" message between the queue and the listeners -- if you have 5 messages put on the queue with 3 listeners, then each of the listener will get one of these messages. However if you need to have all the listeners get the same message, you need to set the exchange to "fanout".

The explanation of using rabbitmq's fanout feature may be well described in the FAQ, but it's not clear in code. So I hope someone will find the below groovy code helpful to see how to use rabbitmq fanout for pubsub topology messaging.

The below code is a basic message publisher to send message to rabbitmq queue. Note the exchange declared as "fanout" exchange. The queue binding may not be useful in fanout use case as all the messages will be send out to all listening consumers.



import com.rabbitmq.client.*


def factory = new ConnectionFactory();
factory.setUsername(username);
factory.setPassword(password);
factory.setVirtualHost("/");
factory.setHost(rabbitmqhost)
factory.setPort(rabbitmqport)
Connection conn = factory.newConnection();
def channel = conn.createChannel()


String exchangeName = "foExchange"
String routingKey = "foo"
String queueName = "fooQueue"


channel.exchangeDeclare(exchangeName, "fanout", true)
channel.queueDeclare(queueName, true, false, false, null);


(1..3).each { n ->
    channel.basicPublish(exchangeName, routingKey,  MessageProperties.PERSISTENT_TEXT_PLAIN, 
                        " (${n}) test testing test [${new Date()}]".getBytes() );
}


channel.close()
conn.close()
System.exit(0)
println " Fin "




Now lets look at the consumer side to consume messages in fanout use case.In a fanout setup, the consumer doesn't have to do anything special. If you don't specify any particular queue name, it will get all messages from the exchange. You can however continue to received message from any particular queue if you bind to it.





String exchangeName = "fooExchange"
String routingKey = "foo"
//String queueName = "fooQueue"


String queueName = channel.queueDeclare().getQueue()
channel.queueBind(queueName, exchangeName, "#");
println " Queue: ${queueName} "


boolean noAck = false;
def consumer = new QueueingConsumer(channel);
channel.basicConsume(queueName, noAck, consumer);
boolean running = true


while(running) {


    QueueingConsumer.Delivery delivery;
    try {
        delivery = consumer.nextDelivery();
        println new String(delivery.body)
    } catch (InterruptedException ie) {
        continue;
    }
    channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
    
}

Initial Thoughts on Padrino, a Fast Ruby Web Framework Based On Sinatra

In search of a Rails alternative in the Ruby world, I found a popular and simple web framework, Sinatra, that was introduced to me by a friend. It is very easy to get started with Sinatra, especially for a simple Hello World kind of web app.

But based on my experience with Rails, Grails, and CakePHP, all web frameworks that help us develop easier and faster with their features, helpers, and also lots of plugins, I admit that I was not that interested in using Sinatra in a real project.

I like Sinatra's simple and minimal concepts, but in the real world we sometimes need more than that. So, although I believe I can build any kind of web application with Sinatra, I feel that sooner or later I will look for helpers, utilities, or plugins that can help me in development.

Then I found Padrino, also introduced to me by my friend, which is a bigger and richer framework than Sinatra in term of features and helpers. It seems like a complete framework but still sticks to Sinatra's core concepts and philosophy, since the developer built this framework as a layer on top of Sinatra while trying to keep it fun and easy. You can find Padrino here: www.padrinorb.com

The things I like best about Padrino:

1. Code Generator - easily generates beautiful admin interfaces with built in authentication.
2. Agnostic Way - gives us options to choose what module or library we want to use, like letting us choose between Active Record, Datamapper, or Sequel for ORM.
3. Fast Performance - Padrino's performance benchmark shows us it is faster than Sinatra itself, which sounds impossible right?

Benchmarks for Padrino vs Merb vs Rails vs Sinatra vs Ramaze:

# Rendering a string inline

Merb 1.1.0 => 1749.97 rps

Padrino 0.9.10 => 1629.15 rps

Sinatra 1.0.0 => 1537.78 rps

Rails 3.beta3 => 381.76 rps

Ramaze 2010.04.04 => 270.08 rps

# Rendering a basic erb template

Merb 1.1.0 => 1490.8 rps

Padrino 0.9.10 => 1416.84 rps

Sinatra 1.0.0 => 1157.89 rps

Rails 3.0.beta3 => 330.58 rps

Ramaze 2010.04.04 => 254.23 rps

# Rendering a simulated simple app

Padrino 0.9.10 => 675.79 rps

Sinatra 1.0.0 => 652.0 rps

Merb 1.1.0 => 642.29 rps

Rails 3.0.beta3 => 201.86 rps

Ramaze 2010.04.04 => 130.62 rps

(copied from http://www.padrinorb.com/blog/padrino-0-9-10-released-built-for-speed)
Get the source code for benchmark from http://github.com/DAddYE/web-frameworks-benchmark

4. Comprehensive Features - Padrino provides us with a lot of features, helpers and libraries that aid us in web development

5. Great Support for Shiny Things like HAML, SAAS, MongoDB

Along with these advantages, I also found some things that need to be improved:

1. Small Community - I hope it will get better in the future (Padrino's Google group only has 122 members). Perhaps it needs more “marketing” to attract newcomers. Open source projects gain trust more easily if they have large and active communities.

2. User Guides - Padrino needs more comprehensive and complete tutorials. For example, if a new user comes to the Padrino site without existing Active Record, Datamapper, or other compatible ORM experience, I believe they will be confused because the documentation does not include those ORMs. I only found a simple active record example in a blog tutorial. I admit there are many external examples covering documentation of these ORMs, but the information is not directly linked/integrated with Padrino. From my experience, I ran into difficulty when trying to build an HTML form with HABTM relation, and it was difficult to find any relevant information about the problem as related to Padrino. Fortunately I had experience with Active Record so finally after struggling for a few hours I found a solution but this might not be the case with someone new to web development and experiencing it for the first time using Padrino.

3. Sample Project, Real Live Projects still rare
In learning, it is easier for me as newcomer to learn from examples so it will be great to see a few sample projects, something more than just a simple blog app. Also I still can't find a list of real live production sites or projects using Padrino. I found only 1-2 sites that use Padrino (1 shopping project and watched.it). There may be many more than that number who already use Padrino, but this information is not well documented. If you are interested, here is a discussion about who is already using Padrino:
http://groups.google.com/group/padrino/browse_thread/thread/d439332aa95227e3/4265f4a2cad796c7

Conclusion:
I believe that if the people behind Padrino keep doing a good job with developing and promoting Padrino, and also take care their community, Padrino will become a great web framework, maybe a better alternative than Rails. If you have experience with Rails and Sinatra, it will a be friendly framework for you to work with.

I would confidently use Padrino for simple applications but personally I feel it will be risky to build a serious application using an open source project without knowing there is more significant community support behind it (please note: I am not saying the Padrino community is not welcoming or responsive).

Spatial Search with Grails and Solr Plugin

Grails has a good Solr plugin (http://www.grails.org/plugin/solr) that works well and also provides spatial search that works out of the box. However, the documentation to use this feature is a bit skimpy so here are the basic steps to enable spatial search with Solr for your Grails app.

Install the solr plugin for Grails using 'grails install-plugin solr' and enable Solr indexing for your domain class by:

    static enableSolrSearch = true

    static solrAutoIndex = true

Next, we need to add a Solr index field annotation to the domain members.

    //import org.grails.solr.Solr

    @Solr(field="latitude_rad_d")

    Double latitude

    @Solr(field="longitude_rad_d")

    Double longitude

Once we have those in place, we can use SolrService to query:

    getSpatialQuery(query, lat, lng, range)

Android: Twitter4j with xauth

The latest Twitter4j 2.1.3 supports xauth but it's not well documented at the moment. However it's really simple to use, check it out!



// Specify the consumer key and consumer secret 
// via system properties or configurationbuilder

System.setProperty("twitter4j.oauth.consumerKey",TWITTER_CONSUMER);
System.setProperty("twitter4j.oauth.consumerSecret",TWITTER_SECRET); 



// Then authenticate as normal
Twitter twitter = new TwitterFactory().getInstance(username,
password);


// Next the token will be available to you
AccessToken token = twitter.getOAuthAccessToken();


// You must store the access token and reuse this token
twitter.setOAuthAccessToken(token);

Using Google FriendConnect for Authentication

The idea of this article is how to let user sign in to your site by using Google Friend Connect. This is a summarize steps to integrate the JS API to a Grails app and based on instruction for "Give users an option of logging in with Friend Connect".




1. Register your site with Google Friend Connect . You must register your site with Google Friend Connect to get your site ID. We will need the site ID to use with the Friend Connect API.


2. Include Friend Connect library
Refer to this page for the full outlines of the Javascript API.
  google.load('friendconnect', '0.8');

4. Render Login Button.
google.friendconnect.renderSignInButton({ 'id': 'target-id', 'text' : 'Click here to join ', 'style': 'standard' });

5. Using OpenSocialApi to detect if user already login.

google.friendconnect.container.loadOpenSocialApi({
  site: 'XXXXXXXXXXXXXXXXXXXX',
  onload: function() {
    if (!window.timesloaded) {
      window.timesloaded = 1;
    } else {
      window.timesloaded++;
    }
    if (window.timesloaded > 1) {

      // no need to handle this for now
      //window.top.location.href = "/authenticate.html";
    }
  }
});

6. Get user ID.

To obtain the login user, first initiate a data request for "VIEWER".

   var req = opensocial.newDataRequest();
    req.add(req.newFetchPersonRequest("VIEWER"), "viewer_data");
    req.send(onData);

The request will run the callback "onData" when it receive data back from the server.

Next we need to handle the callback from the request for the "VIEWER" information.

   function onData(data) {
      if (!data.get("viewer_data").hadError()) {
         var owner_data = data.get("viewer_data").getData();
         var name = owner_data.getDisplayName();
         var id = owner_data.getId();
         window.location.href = "/auth/checkfriendconnect?id=" + id;
       }
    }

The owner ID is unique to the person and will be used for us to tied in to the user in our system.

The next line, "window.location.href = "/auth/checkfriendconnect?id=" + id;" we are actually redirecting the browser to another page for the Friend Connect ID check by using the ID as a params.

7. Back in the server, query the user for Friend Connect ID with the ID specified and perform the usual login processing once the ID is matched to a user.

Setting up mail for your Ubuntu server

Need your bare-bones Ubuntu server to send out email notification and the like? Well, first you need to check what your hostname is and modify accordingly:

hostname -f

So my server's hostname is microdude, you might also get it as microdude.localdomain. You'll want to convert it into an FQDN (Fully Qualified Domain Name). That means I need to change my microdude to microdude.favoritemedium.com.

sudo vi /etc/hostname
sudo vi /etc/hosts

The output of hosts:
127.0.0.1 localhost
127.0.1.1 microdude.localdomain microdude

Would now be:
127.0.0.1 localhost
127.0.1.1 microdude.favoritemedium.com microdude

Reboot and check that the hostname has been changed to microdude.favoritemedium.com:

sudo reboot
hostname -f

After that, you're supposed to set up RDNS (Reverse Domain Name System), this helps avoid your mails falling prey to the all-powerful spam filter. You can check on this RDNS with dig which can be installed from this package:

sudo apt-get install dnsutils

This is optional, though if your mails start going into the spam folder, you'd best revisit this in the links provided.

Now to install the actual mail agent onto the server with a one-liner operation:

sudo tasksel install mail-server

The process stops at two points for user input. The first is to pick the postfix configuration for the server, select the default Internet site. The second is to set the main domain name, this should be already be filled with microdude.favoritemedium.com and all you need to do is accept.

Once finished, the postfix daemon should be running now. You need to know about two files, the main configuration file and the mail log. You don't need to mess around with the config file, since that should be set up correctly. The mail log is useful to check on whether mails have been sent or other information. Their locations:

/etc/postfix/main.cf
/var/log/mail.log

Now you'll want to test whether mails can be sent out. An email can be typed out in the terminal using the mail command:

mail testing@mailaddress.com
Subject: Do not panic, this is a test
Panic panic panic panic panic
.
Cc:

Check the email account you sent the mail to and you should see that it is present.

Congratulations, you can now spam to your heart's content =^_^=.

From:

• http://articles.slicehost.com/email
• http://articles.slicehost.com/2008/7/28/email-preparing-the-slice
• http://articles.slicehost.com/2008/7/29/postfix-installation
• http://articles.slicehost.com/2010/3/1/barebones-postfix-install-for-ubuntu

Moving Drupal from one host to another

You've got a Drupal site running and now want to move it to another server which is bare-bones. This server is luckily running Ubuntu, which makes installing the rest of the LAMP (Linux, Apache, MySQL and PHP) stack a one-line operation:

sudo tasksel install lamp-server

The install process will stop at one point to ask for the password of the root user for MySQL. Once the install completes, use the MySQL command-line client to add in the same database (an empty one) and user that Drupal uses.

Use mysqldump to dump out the Drupal database data into an sql file. Load it in the new database, populating it. You'll want to tar up the whole directory where the Drupal files reside (not forgetting the .htaccess) and expand it out in the same location within the new machine. You may (not) need to edit sites/default/settings.php or .htaccess for any host information changes. settings.php is also where you edit the database login information, if you decide to use different ones.

With Apache running, you should be able to hit the new Drupal site. It looks to be running but there are a few gotchas. If you are (most likely) running Clean URLs, the links don't work anymore. You'll need to disable it to get navigation working at http://hostname/?q=admin/settings/clean-urls, maybe going through http://hostname/?q=user to log in first. To set up Clean URLs again, you need to run this line:

sudo a2enmod rewrite

And edit /etc/apache2/sites-available/default, changing the AllowOverride None to AllowOverride All inside the directory with the path /var/www, or where the Drupal files live. Restart the server and you find that you can enable Clean URLs now.

The other gotcha is that the status report is complaining it can't find the PHP GD library. Apparently PHP library is installed but not configured so you run these lines:

sudo apt-get install php5 (optional?)
sudo apt-get install php5-gd

Lastly, cron needs to be set up to update the Drupal site. I use curl so the lines are:

sudo apt-get install curl
crontab -e
0 * * * * curl --silent --compressed http://localhost/cron.php

The last line is done inside the vi editor so after that, save and quit.

Congratulations! Your Drupal site is now running fine on the new machine.

Google Analytics not showing Firefox and Internet Explorer hits, only Safari and Chrome

If you find your Google Analytics is not showing any results for Firefox and Internet Explorer and everything is correct with no problems on the Javascript side, you might want to take a closer look at the GA code. It might be that all the code is in one script block like so:


<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape('%3Cscript src="'+gaJsHost+'google-analytics.com/ga.js" type="text/javascript"%3E%3C/script%3E'));

try {
var pageTracker = _gat._getTracker('UA-XXXXXXX-X');
pageTracker._trackPageview();
}
catch (err) {}
</script>


The code that GA gives you to use is no mistake, you need to separate them out into two blocks:


<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape('%3Cscript src="'+gaJsHost+'google-analytics.com/ga.js" type="text/javascript"%3E%3C/script%3E'));
</script>
<script type="text/javascript">
try {
var pageTracker = _gat._getTracker('UA-XXXXXXX-X');
pageTracker._trackPageview();
}
catch (err) {}
</script>


The first code block is responsible for downloading the ga.js. That file needs to be loaded up before the second code block can execute properly. If they are in the same block, the try code will fail and no exception or alert is given.

Below are three separate posts covering the same question:

• GA not tracking Mozilla or Firefox. Only shows Chrome and safari Clicks


• Reports not showing some browsers


• Analytics not showing visitors using Internet Explorer

The last comment here lets you know what to look for in troubleshooting Google Analytics. The GA cookies are named __utma, __utmb, __utmc, __utmz, and __utmv and if you do not have any of them, GA is not working.

Full-text search in Rails with Sunspot

There comes a time in every app when doing a “SQL LIKE” query just doesn’t cut it. I’m going to show you how easy it is to add proper full-text search to your Rails app using the Sunspot::Rails plugin.

Sunspot

Sunspot is a standalone Ruby library that makes integrating with a Solr search engine a cinch. It wraps all the nitty gritty of indexing and querying in a declarative DSL which you can use to expose virtually any Ruby object to be searched, not just ActiveRecord models. The sunspot gem bundles a standalone Solr search engine (mostly stock, served by Jetty, although also contains support for geolocational ordering).

Sunspot::Rails is Rails plugin which is basically Sunspot the library plus some hooks into ActiveRecord to update indexes on creates and updates as well as the Rails request lifecycle commit the index at the end of every request. It adds the DSL as class methods into ActiveRecord to allow you to configure the index much like in the style of configuring association or named_scopes. The gem also bundles a set of rake tasks to manage starting, stopping and restarting the Solr service.

Installation

1. Install the gem
   $ gem install sunspot_rails
2. Edit your config/environment.rb to include
   config.gem 'sunspot', :lib => 'sunspot'
   config.gem 'sunspot_rails', :lib => 'sunspot/rails'
3. Generate the sunspot configuration file in config/sunspot.yml
   $ ./script/generate sunspot
4. Run the Solr service
   $ rake sunspot:solr:start
   (if Rake complains that it couldn’t find this task, add require 'sunspot/rails/tasks' to the top of your Rakefile).

Defining an Index

The first thing you need to do before anything can be searched is creating an index. There is two parts to this.

The first part is defining an index. For Rails models, you can define it using the searchable class method. Suppose we have an Article that belongs to an Author.

class Article < ActiveRecord::Base
belongs_to :author

searchable do
text :title, :boost => 2.0
text :body
text :author_name do
 author.name
end
time :updated_at       # for sorting by recent
string :sort_title do  # for sorting by title, ignoring leading A/An/The
 title.downcase.gsub(/^(an?|the)/, '')
end
boolean :published, :using => :published?
end

def published?
state == :published
end
end

Sunspot supports text, string, time, boolean, integer and float fields. When planning what to index, note that only text fields are exposed as full-text search while the other field types are used for restricting, sorting and faceting.

What I like about the DSL is the flexibility. You can directly index an ActiveRecord attribute (:title, :body) or virtual attributes by giving it a block (:sort_title) or a symbol to a method (:published). Even indexing associations is really a matter of calling methods on it.

The second part is indexing. Sunspot provides a utility method to reindex all records for a particular class. In our example, we can call

Article.reindex!

and have the entire Article index rebuilt. For finer grained indexing, you can call Article#index! on a particular instance. As mentioned above, if you are creating and updating models via controllers as in a typical Rails app, this should all be transparent to you.

Querying

Sunspot provides a flexible DSL for querying. A SearchController might look something like this

class SearchController < ApplicationController
def show
@search = search(params)
end

protected
def search(options)
Sunspot.search(Article) do
 keywords options[:query]
 with(:published, true)
 order_by :updated_at, :desc
 paginate :page => options[:page]
end
end
end

keywords will be applied to all text fields. The remaining non-text fields can be defined to restrict the query (in the example, we want restrict it to published Articles) and ordering (in the example, we ordered by updated_at). If you don’t define an ordering, the results will be returned sorted by relevance based on occurence and location of the keywords in the document and the index as a whole. You can tweak the relevance score by defining boosts — in this example, Article titles that match the keywords are given a boost over other Articles that may match the keyword elsewhere.

You can define multiple restrictions and they don’t always have to be for equality. It supports restricting by a value being less-than, greater-than, between, any or all (when comparing for an indexed array). The restrictor with(:published, true) is simply a short-hand for with(:published).equal_to(true). You can also test the absense of a value using the without operator.

Finally, Sunspot plays nice with the WillPaginate plugin. In your view, you can paginate easily by doing

<%= will_paginate @search.results %>

and expect it to work seamlessly.

Conclusion

That’s all there is to it to get up and running with Sunspot. My take-home point is Sunspot exposes extremely flexible DSLs that allow you to scale from simple to pretty complicated queries with ease.

If this interested you, you may want to check out the wiki for other features not covered by this article including highlighting of keywords, facets and stored fields.

GORM query by association (query properties of child object)

Let's say we have a Book object and it has an associated Author object. To query Books based on certain properties of the Author we have 2 common ways to it.

1. Using HQL query

def results = Book.findAll(" from Book as b where :title = b.author.name) ", [title:"Mr Author"])

2. Using criteria with createAlias()

def results = Book.withCriteria {

createAlias("author", "a")

eq("a.name", "Mr Author")

}

I hope this helps some of you out there. For some reason "createAlias" is not well documented in the Grails documentation, unless you cross reference the Hibernate docs.