WordPress permission issues

If you try and install a WordPress plugin or upgrade to a newer version through the admin console you might get this error message and a prompt to enter FTP details:

To perform the requested action, WordPress needs to access your web server. Please enter your FTP credentials to proceed.

Screen Shot 2015-05-26 at 18.22.58

Contrary to what it says, you don’t actually need to enter your FTP credentials, you just need to set the correct permissions for WordPress to be able to write to it’s own directory. How this is done depends on your hosting environment, which is why it prompts for FTP credentials as that it definitely the easiest way, but if like me you don’t have FTP you’ll need to change things yourself.

Option 1: Set the correct user

The first thing to try is setting the correct user, check the configurations for whatever web server you are using, and chown the files in the WordPress directory to be owned by that user:

$ sudo chown -R www-data:www-data /var/www/wordpress

Option 2: Set more writable permissions

If that doesn’t work (for example if like me you have WordPress running in Docker), the easiest option is to just set the directory to be writeable by everyone. This is less secure though, so only do this if you are sure about what you are doing:

$ sudo chmod -R 777 /var/www/wordpress

After that you’ll be able to update plugins and WordPress itself directly from the admin console!

Still not working?

If it’s still not working and you are sure you have permissions set correctly (try uploading attaching an image to a post), you can force WordPress into skipping the permissions check. Add the following to wp-config.php:

define('FS_METHOD','direct');

If it’s still not working, sorry, you’ll just have to update WordPress manually :(

Formatting dates and times in Rails with I18n.localize

irb(main):001:0> Date.today.strftime("%d-%m-%y")
=> "03-01-15"

The typical way of formatting a date in Rails apps is to use the standard #strftime. This works great to start with, but repeating the formats again and again throughout your code leads to issues with maintainability. If you want to change the format you’ll need to go through many different files, and you’ll probably end up with inconsistent formats used throughout your application.

Rails of course comes with a way to avoid this though the use of the Rails internationalisation (I18n) API. The I18n module has a #localize method which you can pass a Date, DateTime or Time object to.

irb(main):002:0> I18n.localize Date.today
=> "2015-01-03"

By default that just does the same as calling .to_s on the object, but you can also pass a format name:

irb(main):003:0> I18n.localize Date.today, format: :short
=> "Jan 03"
irb(main):004:0> I18n.localize Date.today, format: :long
=> "January 03, 2015"

Under the hood it’s just I18n, so you can define your own date formats using the standard formats. If you have this in config/locales/en.yml:

en:
  dates:
    formats:
      date_month_year_concise: '%d-%m-%y'

Then you can use the custom format like this:

irb(main):005:0> I18n.localize Date.today, format: :date_month_year_concise
=> "03-01-15"

(I prefer to give the formats most descriptive names than ‘short’ or ‘long’ as it’s easier to refer back to which is which later)

This not only makes your code more maintainable and consistent, if you ever want to internationalise your app you don’t have to worry about dates. You’ve already done the hard work, so you just need to define the new locale specific format:

irb(main):006:0> I18n.locale = :en_US
=> :en_US
irb(main):007:0> I18n.localize Date.today, format: :date_month_year_concise
=> "01-03-15"

In views this method is aliased as #l, so you can use it like this (for Slim):

= l Date.today, format: :date_month_year_concise

How RubyGems fetches Gems – Part 1

RubyGems and Bundler are often quite slow when I run them. I want to find a way to fix it. It can probably easily be done with a caching proxy, but where’s the fun in that? I want to figure out what exactly happens when I install a gem and why it is slow.

In case you want to follow along I’m using:

The index format

Let’s start by installing a gem. RubyGems has a verbose flag so you can get a bit more info about what it’s doing. This gives us the following:

$ gem install mysql2 --verbose
GET https://api.rubygems.org/latest_specs.4.8.gz
302 Moved Temporarily
GET https://s3.amazonaws.com/production.s3.rubygems.org/latest_specs.4.8.gz
304 Not Modified
HEAD https://api.rubygems.org/api/v1/dependencies
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=mysql2
200 OK
...

The first line fetches the index of Gems held by Rubygems. This is currently hosted on Amazon S3, so a redirect is performed. Let’s take a look at this file:

$ wget https://s3.amazonaws.com/production.s3.rubygems.org/latest_specs.4.8.gz
$ gzip -d latest_specs.4.8.gz
$ head -2 latest_specs.4.8
[�gI"_:ETU:Gem::Version[I1.2;TI"	ruby;TI"-;TU;[I"1;T@
                                                            I0mq;TU;[I"
0.5.3;T@
0xffffff;TU;[I"
0.1.0;T@
        I"10to1-crack;TU;[I"
0.1.3;T@
        I"1234567890_;TU;[I1.1;T@
                                 I"12_hour_time;TU;[I"
0.0.4;T@
        I"16watts-fluently;TU;[I"

Well that isn’t what I expected. I assumed it would have been YAML or JSON. To the source!

Searching for “latest_specs” turns up the following in lib/rubygems/source.rb:14:

  FILES = { # :nodoc:
    :released   => 'specs',
    :latest     => 'latest_specs',
    :prerelease => 'prerelease_specs',
  }

And on line 163:

  ##
  # Loads +type+ kind of specs fetching from +@uri+ if the on-disk cache is
  # out of date.
  #
  # +type+ is one of the following:
  #
  # :released   => Return the list of all released specs
  # :latest     => Return the list of only the highest version of each gem
  # :prerelease => Return the list of all prerelease only specs
  #

  def load_specs(type)
    file       = FILES[type]
    fetcher    = Gem::RemoteFetcher.fetcher
    file_name  = "#{file}.#{Gem.marshal_version}"
    spec_path  = api_uri + "#{file_name}.gz"
    cache_dir  = cache_dir spec_path
    local_file = File.join(cache_dir, file_name)
    retried    = false

    FileUtils.mkdir_p cache_dir if update_cache?

    spec_dump = fetcher.cache_update_path spec_path, local_file, update_cache?

    begin
      Gem::NameTuple.from_list Marshal.load(spec_dump)

Ok, so it’s marshalled using Ruby’s internal marshaller. 4.8 is the version number of the marshalling format that has been used since Ruby 1.8.7.

We can find the format of the marshalled data in lib/rubygems/name_tuple.rb:26:

  def self.from_list list
    list.map { |t| new(*t) }
  end

Which says its an array of arrays. And on line 8:

class Gem::NameTuple
  def initialize(name, version, platform="ruby")
    @name = name
    @version = version

    unless platform.kind_of? Gem::Platform
      platform = "ruby" if !platform or platform.empty?
    end

    @platform = platform
  end

We find it contains the following:

irb(main):003:0> Marshal.load(File.open('latest_specs.4.8')).first
=> ["_", #<Gem::Version "1.2">, "ruby"]

So the name of this gem is _, it is version 1.2 and is generic. If we look for the mysql2 gem we can find there is a generic version, and two versions for Windows:

irb(main):011:0> Marshal.load(File.open('latest_specs.4.8')).select { |spec| spec[0] == "mysql2" }
=> [
  ["mysql2", #<Gem::Version "0.3.11">, "x86-mingw32"],
  ["mysql2", #<Gem::Version "0.3.11">, "x86-mswin32-60"],
  ["mysql2", #<Gem::Version "0.3.17">, "ruby"]
]

Even though the data is basically text, the Gem::Version object is included for backwards compatibility. This seems a bit unoptimal given it adds quite a bit of data when an individual spec is marshalled:

irb(main):002:0> Marshal.dump([ "mysql2", Gem::Version.new("0.3.17"), "ruby" ]).length
=> 59
irb(main):003:0> Marshal.dump([ "mysql2", "0.3.17", "ruby" ]).length
=> 42

However the Ruby marshal format is rather efficient, so it only adds about 3kb (less than 1%) of overhead to the complete compressed spec file. It also compresses duplicate objects well:

irb(main):001:0> Marshal.dump([ "abc", "abc", "abc", "abc" ]).length
=> 45
irb(main):002:0> a = "abc"
=> "abc"
irb(main):003:0> Marshal.dump([ a, a, a, a ]).length
=> 21

In the next part I’ll look at how RubyGems figures out dependencies.