Formatting dates and times in Rails with I18n.localize

irb(main):001:0> Date.today.strftime("%d-%m-%y")
=> "03-01-15"

The typical way of formatting a date in Rails apps is to use the standard #strftime. This works great to start with, but repeating the formats again and again throughout your code leads to issues with maintainability. If you want to change the format you’ll need to go through many different files, and you’ll probably end up with inconsistent formats used throughout your application.

Rails of course comes with a way to avoid this though the use of the Rails internationalisation (I18n) API. The I18n module has a #localize method which you can pass a Date, DateTime or Time object to.

irb(main):002:0> I18n.localize Date.today
=> "2015-01-03"

By default that just does the same as calling .to_s on the object, but you can also pass a format name:

irb(main):003:0> I18n.localize Date.today, format: :short
=> "Jan 03"
irb(main):004:0> I18n.localize Date.today, format: :long
=> "January 03, 2015"

Under the hood it’s just I18n, so you can define your own date formats using the standard formats. If you have this in config/locales/en.yml:

en:
  dates:
    formats:
      date_month_year_concise: '%d-%m-%y'

Then you can use the custom format like this:

irb(main):005:0> I18n.localize Date.today, format: :date_month_year_concise
=> "03-01-15"

(I prefer to give the formats most descriptive names than ‘short’ or ‘long’ as it’s easier to refer back to which is which later)

This not only makes your code more maintainable and consistent, if you ever want to internationalise your app you don’t have to worry about dates. You’ve already done the hard work, so you just need to define the new locale specific format:

irb(main):006:0> I18n.locale = :en_US
=> :en_US
irb(main):007:0> I18n.localize Date.today, format: :date_month_year_concise
=> "01-03-15"

In views this method is aliased as #l, so you can use it like this (for Slim):

= l Date.today, format: :date_month_year_concise

How RubyGems fetches Gems – Part 1

RubyGems and Bundler are often quite slow when I run them. I want to find a way to fix it. It can probably easily be done with a caching proxy, but where’s the fun in that? I want to figure out what exactly happens when I install a gem and why it is slow.

In case you want to follow along I’m using:

The index format

Let’s start by installing a gem. RubyGems has a verbose flag so you can get a bit more info about what it’s doing. This gives us the following:

$ gem install mysql2 --verbose
GET https://api.rubygems.org/latest_specs.4.8.gz
302 Moved Temporarily
GET https://s3.amazonaws.com/production.s3.rubygems.org/latest_specs.4.8.gz
304 Not Modified
HEAD https://api.rubygems.org/api/v1/dependencies
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=mysql2
200 OK
...

The first line fetches the index of Gems held by Rubygems. This is currently hosted on Amazon S3, so a redirect is performed. Let’s take a look at this file:

$ wget https://s3.amazonaws.com/production.s3.rubygems.org/latest_specs.4.8.gz
$ gzip -d latest_specs.4.8.gz
$ head -2 latest_specs.4.8
[�gI"_:ETU:Gem::Version[I1.2;TI"	ruby;TI"-;TU;[I"1;T@
                                                            I0mq;TU;[I"
0.5.3;T@
0xffffff;TU;[I"
0.1.0;T@
        I"10to1-crack;TU;[I"
0.1.3;T@
        I"1234567890_;TU;[I1.1;T@
                                 I"12_hour_time;TU;[I"
0.0.4;T@
        I"16watts-fluently;TU;[I"

Well that isn't what I expected. I assumed it would have been YAML or JSON. To the source!

Searching for "latest_specs" turns up the following in lib/rubygems/source.rb:14:

  FILES = { # :nodoc:
    :released   => 'specs',
    :latest     => 'latest_specs',
    :prerelease => 'prerelease_specs',
  }

And on line 163:

  ##
  # Loads +type+ kind of specs fetching from +@uri+ if the on-disk cache is
  # out of date.
  #
  # +type+ is one of the following:
  #
  # :released   => Return the list of all released specs
  # :latest     => Return the list of only the highest version of each gem
  # :prerelease => Return the list of all prerelease only specs
  #

  def load_specs(type)
    file       = FILES[type]
    fetcher    = Gem::RemoteFetcher.fetcher
    file_name  = "#{file}.#{Gem.marshal_version}"
    spec_path  = api_uri + "#{file_name}.gz"
    cache_dir  = cache_dir spec_path
    local_file = File.join(cache_dir, file_name)
    retried    = false

    FileUtils.mkdir_p cache_dir if update_cache?

    spec_dump = fetcher.cache_update_path spec_path, local_file, update_cache?

    begin
      Gem::NameTuple.from_list Marshal.load(spec_dump)

Ok, so it’s marshalled using Ruby’s internal marshaller. 4.8 is the version number of the marshalling format that has been used since Ruby 1.8.7.

We can find the format of the marshalled data in lib/rubygems/name_tuple.rb:26:

  def self.from_list list
    list.map { |t| new(*t) }
  end

Which says its an array of arrays. And on line 8:

class Gem::NameTuple
  def initialize(name, version, platform="ruby")
    @name = name
    @version = version

    unless platform.kind_of? Gem::Platform
      platform = "ruby" if !platform or platform.empty?
    end

    @platform = platform
  end

We find it contains the following:

irb(main):003:0> Marshal.load(File.open('latest_specs.4.8')).first
=> ["_", #<Gem::Version "1.2">, "ruby"]

So the name of this gem is _, it is version 1.2 and is generic. If we look for the mysql2 gem we can find there is a generic version, and two versions for Windows:

irb(main):011:0> Marshal.load(File.open('latest_specs.4.8')).select { |spec| spec[0] == "mysql2" }
=> [
  ["mysql2", #<Gem::Version "0.3.11">, "x86-mingw32"],
  ["mysql2", #<Gem::Version "0.3.11">, "x86-mswin32-60"],
  ["mysql2", #<Gem::Version "0.3.17">, "ruby"]
]

Even though the data is basically text, the Gem::Version object is included for backwards compatibility. This seems a bit unoptimal given it adds quite a bit of data when an individual spec is marshalled:

irb(main):002:0> Marshal.dump([ "mysql2", Gem::Version.new("0.3.17"), "ruby" ]).length
=> 59
irb(main):003:0> Marshal.dump([ "mysql2", "0.3.17", "ruby" ]).length
=> 42

However the Ruby marshal format is rather efficient, so it only adds about 3kb (less than 1%) of overhead to the complete compressed spec file. It also compresses duplicate objects well:

irb(main):001:0> Marshal.dump([ "abc", "abc", "abc", "abc" ]).length
=> 45
irb(main):002:0> a = "abc"
=> "abc"
irb(main):003:0> Marshal.dump([ a, a, a, a ]).length
=> 21

In the next part I’ll look at how RubyGems figures out dependencies.

WordPress multisite with multiple domains

One of the reasons why I decided to migrate to WordPress rather than another blogging product, is because WordPress is a platform you can use to build more complicated sites on. One feature (that is now built in) is multisite – it allows you to setup multiple sites sharing a single Wordpress installation. The idea originally was to create a “network” of related sites, but you can use it to create sites that are completely separate from each other – each site can have it’s own plugins, themes and even users. Setting it up wasn’t quite as simple as I expected, so here is what I did to get it to work.

The first part of setting up multisite is to edit your wp-config.php adding a flag to enable WordPress multisite:

define('WP_ALLOW_MULTISITE', true);

Add that before the “That’s all, stop editing! Happy blogging.” line. Once that’s done in the admin panel you will find a new option Settings -> Network setup. This sets up your database (remember to take a backup first, I use Updraft Plus to put weekly backups on Amazon S3) and alters the config to enable the network.

www vs non-www

This gave me a warning though because my domains were prefixed with www. I usually setup sites to be hosted on the www subdomain and redirect the root domain to that through Nginx (or whatever web sever I’m using). This ensures search engines don’t see the different subdomains as different sites - having duplicated content on different sites impacts SEO. The warning said that removing the www prefix would still mean the content would be available and not break anything, so I proceeded to remove it but forgot about the redirect on Nginx resulting in an inaccessible site. I removed the redirect, but then everything was only accessible under stackednotion.com, going to www.stackednotion.com redirected there, but I wanted it the other way around.

I restored the latest backup and took a look again. The warning said the site url” needed to be changed, this is what is in the WordPress config:

Screen Shot 2014-07-14 at 14.06.00

So which one is the “site url”? I thought the second one, so changed that and tried again… nope. By “site url” it really means WordPress Address (URL), so I changed that to http://stackednotion.com.

When setting up the network it will give you some settings to put in your wp-config.php and .htaccess files. As I’m running Nginx I ignored the later (it worked out of the box after removing the redirect), and the wp-config.php file had been changed automatically, so there was nothing to do there. After that Stacked Notion was available under www.stackednotion.com, with stackednotion.com redirecting there. Weirdly permalinks still referred to stackednotion.com, redirecting to www when going to them – I’m not sure how to change that, so if anyone knows let me know!

Multidomain

By default multisite is designed to be setup for multiple subdomains under a single root domain. As I already have www.stackednotion.com I could add jobs.stackednotion.com. I didn’t want that though, I wanted a new site under a separate domain remotetechjobs.eu.

The first part is to add a new site in WordPress, I created jobs.stackednotion.com (My Sites -> Network Admin -> Sites then Sites -> Add New) and verified this worked as expected (at first I got a blank screen, by default it uses the twentyfourteen theme which I had uninstalled, so check the theme this site is using if you get the same). Next I needed to set it up to run from a separate domain.

The way to go about this is through the WordPress MU Domain Mapping plugin. Once I installed that plugin (you need to download and install it manually rather than through WordPress) and activated it for the network (My Sites -> Network Admin -> Plugins), you setup everything through Settings -> Domain Mapping. Again this isn’t as simple as it should be. I left the IP address field blank to ensure that the installation remains portable, and entered stackednotion.com into the Server CNAME domain field. I left the options as the default – although if you want separate users you should disable Remote Login.

If you haven’t already, setup the DNS for your new domain to point at the server WordPress is hosted on and let it propagate. To then add the domain you go to Settings -> Domains. You don’t need to setup the original domain here (stackednotion.com in my case), so I just added remotetechjobs.eu with Site ID 2. You can get this from the wp_blogs table, but confusingly it is the blog_id value not site_id. After that going to remotetechjobs.eu mapped to the new WordPress site / blog.

Tidy up

To assist testing I setup Nginx to redirect any subdomains on stackednotion.com and remotetechjobs.eu to WordPress. Going to these where the site doesn’t exist gave an error saying registration was disabled. To avoid that I just setup a redirect in Nginx so that anything other than www redirected to the root domain.

That seemed to work, until I tried logging into remotetechjobs.eu. This failed as WordPress still thought it was hosted under http://jobs.stackednotion.com/, you can’t change the domain through the interface once multisite is enabled, so I did it in the database. The options for this site were stored in wp_2_options, so I replaced http://jobs.stackednotion.com/ with http://remotetechjobs.eu/ in there:

mysql> select * from wp_2_options where option_value like '%jobs%';
+-----------+-------------+-------------------------------+----------+
| option_id | option_name | option_value                  | autoload |
+-----------+-------------+-------------------------------+----------+
|         1 | siteurl     | http://jobs.stackednotion.com | yes      |
|         2 | blogname    | Remote Tech Jobs              | yes      |
|        33 | home        | http://jobs.stackednotion.com | yes      |
+-----------+-------------+-------------------------------+----------+
3 rows in set (0.00 sec)

mysql> update wp_2_options set option_value = 'http://remotetechjobs.eu' where option_id in (1, 33);

After that everything worked as expected!