Robust Internationalization on Rails

Background

The standard procedure for building an app with a selectable language option is to have a set of files (a pack) for each language where the target string in the target language is referenced by a key. The keys are unique within each language pack and the same between packs.

These language packs could, for example, be in YAML files:

en.yml

hi: Hello, world!
bye: Goodbye, world.

es.yml

hi: ¡Hola, mundo!
bye: Adiós, mundo.

Ruby on Rails includes the i18n gem by default, giving us a very easy way to produce any snippet of language we need. For example, I18n.t "hi" would return the string "Hello, world!" if our environment is English, and "¡Hola, mundo!" if our environment is Spanish.

If we ask for something that is not defined in our language pack, we get a message to that effect, e.g. "translation missing: en.seeyoulater".  Note that the message includes the missing key ("seeyoulater"), so we can easily go in and add the necessary content.

The Story

In a recent project, I needed to create an admin page which would allow us to identify content that exists in our main language (English) but not in our secondary language (Spanish).  To do this, I needed a way to prevent i18n from generating the translation missing message and instead return something obvious, such as nil.

Fortunately, I18n#t provides an option to raise an exception when content is not found:

I18n.t "seeyoulater", :raise => true

which can then be caught to give us our desired nil.

I18n.t "seeyoulater", :raise => true rescue nil

Problem solved, job done.

Oops

Obviously I wouldn't be writing this post of that were the whole story.

My admin page worked perfectly on the development server, but on production it failed.  The two systems are identical; what could possibly be the cause?

A little bit of detective work uncovered two points:

1.  There's an i18n option, turned on by default in a production environment, to not generate "translation missing" messages.  Instead, when looking for a translation of something that exists in the primary language, i18n will automatically and silently use the original string (e.g. English) in place of the missing translation.

2.  This automatic and silent substitution has the unwanted side effect of pre-empting the :raise => true option above.  Now our missing-translation detector is broken.

The symptom of the failure on production was that no translations were reported missing, and much of the content that claimed to be in Spanish was actually in English.

Conclusion

In the end, all is well.  In the end, we didn't need this functionality.

But I'm bothered by the fact that the default behavior of i18n is to treat exceptions as things to be avoided at all cost.  Obviously we don't want uncaught exceptions making their way into our error log (or worse yet, to the browser), but there are valid reasons for throwing and catching exceptions, even on a production server, and this is one of them.

I'm bothered that I've explicitly written into my code that I want an exception, and i18n is pulling rank on me and saying no, you can't have one.

In the end, though, the logical solution is that any translations that need to be managed from the web should ultimately be in stored in a database and managed along with the other dynamic content. Translation data that is stored locally, e.g. in YAML files, should be managed/tested on the development system.

git bisect finds code that introduced regressions

From time to time while developing software, especially on a large project, you find a regression some time after it was introduced. Maybe introduced by someone else on the team. Maybe 50 or 100 commits after it was introduced. For example, perhaps Internet Explorer is buggy and one of our stylesheet changes broke it.  We just don't know when the regression was introduced, or exactly what code is causing it. In an ideal world we would have had adequate tests in place to reach that coveted '100% test coverage.' But back in the real world we often have to work with software that doesn't have tests in place, or is minimally tested, or just can't be easily tested.  Like the browser example.

One thing we could do is use git blame. Git blame basically allows us to know who and which commit added what to which file.

$ git blame Gemfile
^c4edf42 (Paul  2013-02-13 10:31:24 +0900  1) source :rubygems
1517dfc4 (Heath 2013-02-13 11:43:05 +0800  2) 
1517dfc4 (Heath 2013-02-13 11:43:05 +0800  3) gem "sinatra", :require => "sinatra/base"
1517dfc4 (Heath 2013-02-13 11:43:05 +0800  4) gem "sinatra-contrib"

git blame shows you which commits contributed to the final outcome of a file.

This is indeed useful in many situations, but in the course of development we find that sometimes we can observe the behavior changing, but we don't know what code caused the behavior change. In this case git blame will do us little good.

In more advanced cases, specifically when these two conditions are met:

1. You can identify the functionality that is broken.
2. You can not identify the code that was changed in the past that broke the functionality.

What we could do is blindly revert back to a previous commit, not knowing how far back it goes, one by one, until we uncover where the commit is that causes the regression to disappear.
Though consider this, in large projects where commits are small you will have many. Not a handful. You may have hundreds or even thousands of commits to sift through! Your database has to be migrated to match the new commit each time. There may be other setup required, like old services and externals that your software relied on. You want to reduce your O(n) search down. The best we're going to get here is a binary search. O(log n)

So if we implemented the search ourselves we would need to divide the search space in half and put one half on a different branch from the other half. Then continue to divide each branch into smaller branches and repeat until there's a single commit at the end of the tree. Then we'd follow the tree either down the left or right side depending on if we can see the functionality changes. For example, we go to commit #50 of 100. We run our app and we see that the functionality is fixed. So now we know the commit is greater than #50. We continue to #75 and test again. We see the functionality is buggy again at this point. We know now it's between #50 and #75. We could repeat until we find the commit which will show us exactly the causative link. Continue reading for a better solution.

Because this is exactly what git bisect is for. It automates the process by checking out specific commits and performing a binary search on your commit history. To use git bisect first use git bisect start

$ git bisect start

Next we should specify the most recent commit where the functionality was bad with git bisect bad

$ git bisect bad

git bisect bad should generally be used on the head of the branch the issue is on

Now mark the earliest commit known to not have the software defect with git bisect good [SHA]

$ git bisect good 5d1f0ddc72b791b391bce2df1fa59f55a7a25129
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[3a278003eade29adf10f3dc8357b70f51cd6c9cf] Merge remote-tracking branch 'origin/master'

You can also use tags and if you've been tagging your software correctly could specify the version number, or milestone without needing to look up the SHA

git bisect gives us information on how many more times it estimates it'll need to test using the binary search, and which commit is currently being tested. Each step along the way you'll need to either issue a git bisect good or git bisect bad to specify that the problem either is resolved at this commit or it's still there. After issuing this command for the last time you'll see this:

$ git bisect good
d0fd1a64162b4bcc49f5f8d0e81f73a4da3b736f is the first bad commit

That's all there is to it. We now know exactly which commit caused the issue. We can examine the code there and we know the causative change that caused the defect in our software. Oh, and you'll need to issue git bisect reset after you're finished.

State of Android Development 2013

Ruby Block vs Proc.new vs Lambda vs method(:func) Summarized

There are many ways to pass code around in Ruby, so today I'm going to make a comparison between the 4 different ways. The first section, I will show the syntax of using each of them:
1. Block
2. Proc
3. Lambda
4. method(:func)
On the second section, I will compare the subtle differences they have.

A quick syntax comparison:
* note that the result is at the last section of each code *
1. Block
2. Proc.new
3. Lambda
4. method(:some_method)

---

Discussion on the subtle differences:
The table below summarizes what each of them can and cannot do, under the table, there are a more in depth explanation on the things shown in the table.

Description	Block	Proc	Lambda	Method	Proof
Storing into a variable	No	Yes	Yes	Yes	See (a)
How they work	N/A	Code Replacement	Method Call	Method Call	See (b)
What class they belong to	Proc	Proc	Proc	Method	See (c)
Check for correct number of argument	No	No	Yes	Yes	See (d)

(a) Storing into a variable
Only block is not able to be stored into a variable, the rest is possible, refer to the quick syntax section above.

(b) How they work
Referring to the code right above this sentence, we can see that for proc_return, the line return "proc2 I AM HERE!" is never executed, this is because, Proc.new{return "proc1}.call works like code replacement, we can imagine proc_return to be like this:

(c) What class they belong to
Run this code: (d) Check for correct number of argument
Block and Proc don't check for the correct number of arguments, they discard extra parameters silently.(http://www.ruby-doc.org/core-1.9.3/Proc.html#method-i-call). For lambda and method(:func), they give error right away. Here's the code
For block:
For proc:
For lambda:
For method(:func):

Breaking ARC Retain Cycle in Objective-C Blocks

In a recent client project, we noticed its iOS app often received low memory warnings. The iOS app is developed with ARC-enabled (Automatic Reference Counting).
When profiling the app using Instruments > Allocations, we found that a lot of unused ViewController objects were not being released from memory.

Retain Cycle

The codebase uses a lot of Objective-C blocks as shown below, and self is often being called within the blocks:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

@interface DetailPageViewController : UIViewController @property(nonatomic, strong) UIButton *backButton; ... @end @implementation DetailPageViewController @synthesize backButton; - (void)loadView { ... [self.doneButton onTouch:^(id sender) { [self doSomething]; self.isDone = YES; }]; } ... @end

In this example code, the controller object holds a strong reference to a doneButton object. But because the doneButton onTouch: block is referencing self, now the button object holds a strong reference back to the controller object.
When object A points strongly to object B, and B points strongly to A, a retain cycle is created, and both objects cannot be released from memory.

Use Lifetime Qualifiers

Apple Developer’s guide on ARC transition suggested a few ways to break the retain cycle using different lifetime qualifiers. It is definitely a must read for iOS development.
If your app is targeted for iOS 5, you can use the __weak lifetime qualifier to break the cycle:

1 2 3 4 5 6 7 8 9 10

@interface DetailPageViewController : UIViewController - (void)loadView { ... __weak DetailPageViewController *controller = self; [self.doneButton onTouch:^(id sender) { [controller doSomething]; controller.isDone = YES; }]; } @end

Because we are using the __weak qualifier here, the doneButton onTouch: block only has a weak reference to the controller object. Now, the controller object will be released from memory when its reference count drops to 0.

Message Queue Implementations Reviewed

Message Queue Implementations Reviewed

After Googling around a while, I decided to look at the following message queues. This list was selected based on factors such as support, community size/activity, and feature set. It is by no means an exhaustive list.

• Resque 
• RabbitMQ 
• ActiveMQ 
• ZeroMQ 
• Starling 
• DelayedJob

Terms and Definitions

Most of the message queue implementations above make use of the following terminology.

Jobs

Tasks that are queued for processing. For example, if you want to send an email you would enqueue the email for future processing by a worker. This is a job. It has all the information needed by the worker to process it. In this case, we're talking about things such as recipient address, subject, body, etc.

Workers

Modules of code that handle the processing of jobs. A worker will fetch jobs from a queue and process them. In the email example above, a worker would grab the job from the queue and process it by sending off the email.

Delayed::Job

• https://github.com/collectiveidea/delayed_job
• This is a quite simple solution that will address the needs of 80% of developers out there
• Delayed::Job was initially extracted from the Shopify.com codebase and was used by Github before they moved on to Resque
• Delayed::Job stores its jobs in a table in your database
• You can write jobs by extending a class provided by delayed_job
• I find it to be a good solution if you need something very simple
• Delayed::Job polls your database periodically for new jobs which is is not ideal but should not be an issue for most use cases
• You can prioritise jobs and set the number of maximum retries per job. 

Starling

• https://github.com/starling/starling
• Starling was extracted out of the early Twitter codebase
• Uses the memcached protocol
• Jobs persisted to filesystem, so no database dependency
• Lightweight
• You need to write your own workers to perform polling and job execution
• Polling means that Starling will be a bit slower than other queues such as RabbitMQ 
• Last update 1 year ago

ActiveMQ

• http://activemq.apache.org/
• Mature, stable product that has been used by a lot of folks for some time
• A top-level Apache project 
• Can use the STOMP gem if you want to implement your own thing
• Or use ActiveMessaging from ThoughtWorks 
• Evented
• Comes with a basic monitoring interface

RabbitMQ

• http://www.rabbitmq.com/
• Uses AMQP (advanced message queueing protocol) 
• Written in Erlang 
• Mature and one of the more well known message queues around 
• Has lots of features, including file streaming 
• Tons of clients/libraries for most of the major languages: http://www.rabbitmq.com/devtools.html
• Fast 
• Evented

ZeroMQ

• http://www.zeromq.org/
• Minimal, lightweight messaging framework 
• No central server 
• Ultra fast
• Focused religiously on transporting messages between n points and nothing else 
• Quite different from other MQs to the point where it's questionable as to whether ZeroMQ is a message queue or not :) 
• If used to solve the email example mentioned above, the programmer will most likely have to handle the serialization/deserialization of the email, implementing of a daemon that runs in the background to receive email jobs, process them, handle retry logic, etc. 
• Basically if you use ZeroMQ, you will have to reinvent some things that are provided out of the box in some of the solutions mentioned above
• ZeroMQ is not meant to be RabbitMQ, Resque, etc. 
• It does one job and it does it perfectly

Resque

• https://github.com/defunkt/resque
• Developed by Github for their internal use 
• Uses Redis internally and Redis is super fast 
• Feature rich 
• Persists jobs to redis as json 
• Comes with a rich Sinatra app that provides insight into what's happening with any queue
• Not evented 
• Popular within the Ruby community 
• Well supported, stable, and well tested 

Conclusion

After going through the material I found online about the message queues listed above, I would use Delayed::Job for anything trivial, mainly because I have experience using it and it just works for most day-to-day tasks I've come across.

For use cases with high load and a need for stability, I would feel more comfortable going with Resque. It seems to be fairly straightforward to use and is quite popular so it enjoys good support from the community. Github uses it! And to seal the deal it seems to have the best management interface for monitoring queue internals. This is especially relevant since many message queue servers don't provide a query DSL, as databases typically do.

For instances where high performance and throughput are paramount, I would explore ZeroMQ. It almost certainly requires more work to get up and running (e.g. you'll need to implement most things needed for handling a message queue), but it offers great flexibility and speed.

If supporting multiple languages is a concern, RabbitMQ, ActiveMQ, and ZeroMQ get the nod due to their library availability for most popular languages and platforms.

1st Singapore Clojure Meetup

Very interesting meetup last here at the Favorite Medium Singapore office. So much was discussed and I can say everyone left the meetup the wiser (in terms of Clojure knowledge). Here are some pictures from the meetup last night. If you are hacking Clojure in Singapore, you should definitely join our next meetup. Find out more about Clojure Singapore here.