Out with Dream Host, in with Rails Playground

Well, well, well, almost a year after signing up with Dream Host, after going through many turbulent times, trying to fix things up myself, and again, surviving an unannounced, and not so welcome upgrade, after hoping for mongrel support, it really came down to a severe case of overselling. Performance had become abysmal, database connections hard to come by, site was down for longer and longer periods of time. Sometimes several hours without much I could do to revive it.

Don’t get me wrong, not all overselling is bad as long as it is managed properly. In my case, it came down to cpu and resource usage (database, memory). I mean, a rails application like typo is no small app. It routinely had 3 fcgi processes, each 70Mb plus, which on a 4 Gb machine is over 5% of memory, ouch! A good thing that not everyone there runs typo! But I don’t think that was the problem though. On average, the load of the machine was between 10 to 20, which is high but could be sustainable if it wasn’t for the fact that cpu usage seems to be above 90% all the time. This does not give much for a little boost when needed. And on that front, Dream Host needs to make some improvements quickly!

To be honest, this does not seem to be a big issue to run something like wordpress and my other sites on the same machine seem to be doing just fine and I don’t plan to move then anywhere anytime soon. I mean, it is hard to beat the amazing storage space and bandwidth allotment.

So with all that in mind, it was time for a new host! I knew what I wanted:

  • mongrel support
  • ssh support
  • no need for extravagant bandwidth
  • no setup fee, reasonable month to month cost

To find the best contenders, I used Google and the Hosting page on rubyonrails.org.

The best fit: Rails Playground. This post is hosted there, and so far, so good. For $12/month, that looks like a very competitive plan for a rails app. Granted I can only run one app with 2 mongrels, but that will do great for now.

The transition went really smoothly, all things considered.

First thing was to export the database, and I hit a first snag trying to get the data. I could not get a database connection to dump the data, fortunately, I was able to do the export using phpMyAdmin.

On the Rails Playground side, it came down to setting up a database, importing the data, setting up Capistrano, and putting out a request for mongrel setup. Less than a couple hours works and the request for mongrel setup was completed overnight.

Then the only thing left was to switch dns (so the old site was still up, at least as much up as it can be while the dns change propagates), and that’s where the only hitch happened. I ran into a perl error on the Dream Host control panel. Nothing their support couldn’t solve, and an hour later the whois database was updated, and dns servers all over started to pick up the change, and Voila! Rails Playground, here I come!

I now have the distinct (ahem… cough) advantage of having the 2 biggest processes running on this new machine but shush, don’t tell anyone…

And what a difference this makes. Average cpu usage seems to be below 5% and so far, not dreaded 500 error page :)

Note: if you read this via RSS, this may get delayed because as I write this, FeedBurner is still polling the old site. Hopefully this will update by tomorrow.

Bookmark It! Rails plugin for adding bookmarklets.

A while back I wrote BookmarkIt! , a social bookmarllet plugin for Typo, and it worked for a while, but the Typo plugin API changed, and changed again, and again, and keeping up was no fun. So since then, I’ve already changed my approach for my TextLinkAds plugin and instead wrote a bona fide rails plugin, and since Typo 4.0.3 broke the plugin one more time, it was time to take the same route.
Morevover, the previous version did not allow for putting the Bookmkarlets inline with the post and this new version addresses that also. It is now a real Rails plugin, useable in any Rails app.

How to Get the plugin

script/plugin install http://svn.nanorails.com/plugins/bookmarkit/

or to use svn:externals to get updates

script/plugin install -x http://svn.nanorails.com/plugins/bookmarkit/

The install will automatically copy the config file, the images, the css and the partial in their final location. If you delete these files, you can reinstall with:

script/plugin install --force http://svn.nanorails.com/plugins/bookmarkit/

How to configure the plugin

To configure, edit the file config/bookmarkit.yml to change the title and select which bookmarklets you want displayed.

title: Bookmark It!
delicious: true(default)/false
digg: true(default)/false
reddit: true(default)/false
blinklist: true(default)/false
dzone: true(default)/false
newsvine: true(default)/false
furl: true(default)/false
spurl: true(default)/false
simpy: true(default)/false
fark: true(default)/false
blogmarks: true(default)/false
myweb2: true(default)/false
wists: true(default)/false

How to use the plugin

To use, inside any view (.rhtml), use:

<%= render_BMI(title, url) %>

For example, in Typo, you would include the following in app/views/articles/read.rhtml


<%= render_BMI(@article.title, @article.location) %>

For finer control of the rendering, it also copies a css (bookmarkit.css) to public/stylesheets. To use, include in your main layout with:

<%= stylesheet_link_tag "/stylesheets/bookmarkit" %>

Adding more bookmarklets

All the bookmarklet html is in the _bookmarkit.rhtml partial.

For example, for BlinkList, I’ve used:

      <a style="font-size:22px;" title="Blink '<%= title %>'" href="http://www.blinklist.com/index.php?Action=Blink/addblink.php&Url=<%= url %>&Title=<%= title %>">
            <img style="border:0;" hspace="3" class="bookrmarkit" alt="Blink" title="Blink '<%= title %>'" src="/images/bookmarkit/blinklist.gif" />
      </a>

You should also add an entry in bookmarkit.yml

If you want more bookmakrlets included, submit your patches in the comments or send to psq_at_no_spam_nanorails_dot_no_spam_com.

Installing Rails on Mac OS X Tiger (10.4.8)

Just a few days ago, I had the opportunity to upgrade my iMac from a 20” to a 24”. Not only does it have more real estate on the screen, not only is the CPU slightly faster, but it also can take up to 3 Gb of RAM, from 2Gb, which is quite an improvement, especially if you like to run multiple VMs (using Parallel).

So it was quite a relief to us my notes to install Rails on Mac OS Tiger. Except that a lot has changed in the past 3 months since I last wrote that article, and before I update the page to make more sense to newcomers, I’ll just write down a few notes on what I had to do differently.

These reflect the installation on MacOS 10.4.8 and the latest version of MacPorts (as of 10/14/06).

First of all, Darwin Ports is no more: long live MacPorts!

Which means that here is a new location for darwin ports installer.

Then I uncovered a problem in the order of the steps:

Set the PATH before running “sudo port -d selfupdate”, not after as suggested.

One thing that I did not realize, is that you need to actually install a C compiler before you can get started! The easiest is to install XCode 2.4. There is a free registration you need to fill in.

I missed a sudo:

cp httpd.conf.sample httpd.conf => sudo cp httpd.conf.sample httpd.conf

The path to the launchctl file for apache2 has changed:

sudo launchctl load -w /Library/LaunchDaemons/org.macports.apache2.plist instead of sudo launchctl load -w /Library/LaunchDaemons/org.darwinports.apache2.plist

To initialize the database, you need to add:

add "sudo -u mysql mysql_install_db5"

The path to the launchctl file for mysql5 has changed as well:

sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist instead of sudo launchctl load -w /Library/LaunchDaemons/org.darwinports.mysql5.plist

replace rb-mysql5 with rb-mysql since rb-mysql supports both flavors

And that’s where it got a bit tricky. With that combination of MacOS 10.4.8 and the current version of MacPorts, rb-mysql fails to compile.

gcc -I. -I. -I/opt/local/lib/ruby/1.8/i686-darwin8.8.1 -I. -DHAVE_MYSQL_SSL_SET -DHAVE_MYSQL_H -I/opt/local/include/mysql5/mysql/ -I/usr/local/include -O -pipe -I/opt/local/include -fno-common -O -pipe -I/opt/local/include  -fno-common -pipe -fno-common  -c mysql.c
mysql.c: In function 'Init_mysql':
mysql.c:2015: error: 'ulong' undeclared (first use in this function)
mysql.c:2015: error: (Each undeclared identifier is reported only once
mysql.c:2015: error: for each function it appears in.)
mysql.c:2015: error: parse error before numeric constant
mysql.c:2018: error: parse error before numeric constant
{standard input}:6318:non-relocatable subtraction expression, "_cMysqlTime" minus "L00000000065$pb"
{standard input}:6318:symbol: "_cMysqlTime" can't be undefined in a subtraction expression
{standard input}:5382:non-relocatable subtraction expression, "_cMysqlRowOffset" minus "L00000000045$pb"
{standard input}:5382:symbol: "_cMysqlRowOffset" can't be undefined in a subtraction expression
{standard input}:5355:non-relocatable subtraction expression, "_cMysqlRowOffset" minus "L00000000044$pb"
{standard input}:5355:symbol: "_cMysqlRowOffset" can't be undefined in a subtraction expression
{standard input}:5333:non-relocatable subtraction expression, "_cMysqlRowOffset" minus "L00000000044$pb"
{standard input}:5333:symbol: "_cMysqlRowOffset" can't be undefined in a subtraction expression
{standard input}:4861:non-relocatable subtraction expression, "_cMysqlTime" minus "L00000000042$pb"
{standard input}:4861:symbol: "_cMysqlTime" can't be undefined in a subtraction expression
{standard input}:4359:non-relocatable subtraction expression, "_cMysqlTime" minus "L00000000041$pb"
{standard input}:4359:symbol: "_cMysqlTime" can't be undefined in a subtraction expression
{standard input}:4106:non-relocatable subtraction expression, "_eMysql" minus "L00000000041$pb"
{standard input}:4106:symbol: "_eMysql" can't be undefined in a subtraction expression
{standard input}:3957:non-relocatable subtraction expression, "_cMysqlTime" minus "L00000000040$pb"
{standard input}:3957:symbol: "_cMysqlTime" can't be undefined in a subtraction expression
{standard input}:3895:non-relocatable subtraction expression, "_eMysql" minus "L00000000040$pb"
{standard input}:3895:symbol: "_eMysql" can't be undefined in a subtraction expression
{standard input}:3824:non-relocatable subtraction expression, "_eMysql" minus "L00000000039$pb"
{standard input}:3824:symbol: "_eMysql" can't be undefined in a subtraction expression
{standard input}:3803:non-relocatable subtraction expression, "_eMysql" minus "L00000000038$pb"
{standard input}:3803:symbol: "_eMysql" can't be undefined in a subtraction expression
{standard input}:3256:non-relocatable subtraction expression, "_cMysqlRowOffset" minus "L00000000023$pb"
{standard input}:3256:symbol: "_cMysqlRowOffset" can't be undefined in a subtraction expression
{standard input}:3226:non-relocatable subtraction expression, "_cMysqlRowOffset" minus "L00000000022$pb"
{standard input}:3226:symbol: "_cMysqlRowOffset" can't be undefined in a subtraction expression
{standard input}:3200:non-relocatable subtraction expression, "_cMysqlRowOffset" minus "L00000000022$pb"
{standard input}:3200:symbol: "_cMysqlRowOffset" can't be undefined in a subtraction expression
{standard input}:2749:non-relocatable subtraction expression, "_eMysql" minus "L00000000019$pb"
{standard input}:2749:symbol: "_eMysql" can't be undefined in a subtraction expression
{standard input}:2595:non-relocatable subtraction expression, "_eMysql" minus "L00000000018$pb"
{standard input}:2595:symbol: "_eMysql" can't be undefined in a subtraction expression
{standard input}:2412:non-relocatable subtraction expression, "_cMysqlStmt" minus "L00000000016$pb"
{standard input}:2412:symbol: "_cMysqlStmt" can't be undefined in a subtraction expression
{standard input}:973:non-relocatable subtraction expression, "_eMysql" minus "L00000000008$pb"
{standard input}:973:symbol: "_eMysql" can't be undefined in a subtraction expression
{standard input}:290:non-relocatable subtraction expression, "_cMysqlField" minus "L00000000004$pb"
{standard input}:290:symbol: "_cMysqlField" can't be undefined in a subtraction expression
{standard input}:232:non-relocatable subtraction expression, "_cMysqlRes" minus "L00000000003$pb"
{standard input}:232:symbol: "_cMysqlRes" can't be undefined in a subtraction expression
{standard input}:190:non-relocatable subtraction expression, "_eMysql" minus "L00000000002$pb"
{standard input}:190:symbol: "_eMysql" can't be undefined in a subtraction expression
make: *** [mysql.o] Error 1

At this point your are either stuck waiting for an update version, or if you are not afraid to dip into the code, you can arm-twist it into working by following these steps:

rb-mysql patch

These steps should be fairly simple to follow:

cd /opt/local/var/db/dports/build/_opt_local_var_db_dports_sources_rsync.rsync.darwinports.org_dpupdate_dports_ruby_rb-mysql/work/mysql-ruby-2.7.1
sudo ruby extconf.rb --with-mysql-include=/opt/local/include/mysql5/mysql/ --with-mysql-lib=/opt/local/lib/mysql5/mysql/
add '#include "my_global.h"' to mysql.c (as the first #include)
sudo make clean
sudo make
sudo make install

And voila. Running the tests will show 3 failing tests, mostly due to imprecisions, but I think you can live with them. Further testing with rails apps did not reveal any problems with MySQL access.

Note: I’ve contacted the maintainer of that port but I have not heard back yet. I will update this post when I do.

A few bonus gems I’ve found helpful, or can’t live without

And since you are in setup mode, you might as well add them to your collection.

sudo gem install acts_as_versioned ajax_scaffold_generator BlueCloth builder fastercsv ferret gettext  htmltools mocha payment radiant  radius rcov rdig RedCloth ruby-debug rubyful_soup selenium shipping
sudo gem install map_by_method what_methods

A new Rails plugin for TextLinkAds (including support for Feedvertising)

Change of Strategy

Instead of updating my constantly breaking Typo sidebar plugin, and to implement Feedvertising from TextLinkAds, I’ve changed gears and chosen to implement just a regular plugin (very close to the new way of doing sidebar plugins in typo 4.1). This approach should work in all versions of typo, as well as any other Ruby on Rails’s application.

The support for feedvertising is slightly different than the one from the WordPress plugin that is the only option offered so far, but should be fairly close.

Installation

Get the plugin from subversion:

script/plugin install http://svn.nanorails.com/plugins/textlinkads/

or to use svn:externals and get future updates via svn update

script/plugin install -x http://svn.nanorails.com/plugins/textlinkads/

The installation will copy a file textlinkads.yml into your config directory.

The file looks like this:

key: TLA_KEY
affiliateid: 0
title: Sponsors
advertisehere: Advertise here!
testing: false
caching: true

Replace TLA_KEY with the one provided by TextLinkAds, set your affiliateid if you’d like to have a link to TextLinkAds with your affiliate id (so you can get credit if someone signs up for a TextLinkAds account). Change the title and advertisehere messages if you don’t like the defaults.

Set testing to false once you’ve verified it works (while testing=true, the plugin will use a special page provided by TextLinkAds that displays to links. However, that page does not contain any RSS links)

Finally, if the caching done by Rails is not enough, the plugin can cache the calls to retrieve the links. See the caching section for explanations on how to setup caching.

Integrating with Typo

Adding the display of regular links

To add the regular TextLinkAds links, you need to add a call to render_TLA anywhere in the rendering code. In typo, the most likely place is in your template’s default.rhtml

Here’s what my template looks like


<div id="sidebar">
  ...
  <div class="sidebar-node"><%= render_TLA %></div>
  <% response.lifetime = 6.hour %>
  <%= render_sidebars %>
</div>

to replace the original:


<div id="sidebar">
  <%= render_sidebars %>
</div>

Add the RSS links

For RSS2.0 for example, edit the file app/views/xml/_rss20_item_article.rxml to add a call to render_TLA_RSS(post_id)

here’s the original file:


  xm.item do
    xm.title post_title(item)
    if this_blog.show_extended_on_rss
      content = item.full_html
    else
      content = item.body_html
    end
    xm.description content
    xm.pubDate pub_date(item.published_at)
    ...

xm.item do
  xm.title post_title(item)
  if this_blog.show_extended_on_rss
    content = item.full_html
  else
    content = item.body_html
  end
  content += render_TLA_RSS(item.id)
  response.lifetime = 6.hour
  xm.description content
  xm.pubDate pub_date(item.published_at)

Depending of which format you need, you may need to edit a different file.

That’s it.

Integrating with other apps

For other apps, just take a similar approach and add calls to render_TLA and render_TLA_RSS where most appropriate. Both calls are accessible from any Controller or Helper class.

Caching

Set “caching: true” in textlinkads.yml. Make sure Rails is configured with some caching. Typically, you need to have


config.action_controller.perform_caching = true
config.action_controller.fragment_cache_store = :file_store, "#{RAILS_ROOT}/tmp/cache"

either set from config/environment.rb or config/environment/[CURRENT ENVIRONMENT].rb

This will cache the pages where the ads appear, and cache the feed pages. But to make sure these pages will update properly if the TextLinkAds ads inventory is updated, you must ensure the cache will expire.

The best way to do that is to use the expiring_cache_action plugin. To install:

script/plugin install http://typosphere.org/trac/browser/trunk/vendor/plugins/expiring_action_cache

Then where you added calls to render_TLA, just add:

response.lifetime = 6.hour

This way, the cached page will expire 6 hours later so that would be the lapse of time to wait to see the updated ads.

No need to expire the cache for RSS, it should get expired automatically every time an article is added.

If you use this plugin in other applications, add a comment or send me a not (psq_0×40_nanorails_0×2e_com) and I’ll add a link to your instructions or code.

Adding CodeHighlighter to Typo

Recently, I ran across CodeHighlighter by Dan Webb. This looked like a pretty cool javascript, and I thought a better way than to do it on the server side.

Adding it to typo is fairly straightforward, baring a small patch.

Get the code

Get it from Dan’s site via subversion: http://svn.danwebb.net/external/CodeHighlighter/trunk

svn co http://svn.danwebb.net/external/CodeHighlighter/trunk

Adding CodeHightlighter to your template

Add to your template (themes/[YOUR THEME]/layouts/default.rhtml)

<%= javascript_include_tag "code_highlighter" %>
<%= javascript_include_tag "javascript" %>
<%= javascript_include_tag "css" %>
<%= javascript_include_tag "html" %>
<%= javascript_include_tag "ruby" %>

The code as provided by Dan has a small incompatibility with prototype used by Ruby on Rails. The following line

for (var i in this.styleSets) highlightCode(this.styleSets[i]);

behaves quite strangely and will pretty much kill your browser (Firefox runs out of memory and displays a bunch of “undefined” after several minutes) because it ends up calling hightlightCode way too many time because of all the methods added to all objects.

To make codehighlighter.js play nicely with prototype.js, replace that last line with:

if (this.styleSets.each) {
  this.styleSets.each(highlightCode)
} else {
    for (var i in this.styleSets) {
      highlightCode(this.styleSets[i]);
    }
}

Essentially, if prototype is present, we can use each to iterate on the styles, otherwise it is safe to used the old code.

Using Codehightlighter

Now, addng styling is easy. Using Textile, you can just put your code around a <pre> and a <code> block. If you have a javascript snippet, use a class of “javascript”.

For example:


<pre><code class="javascript">
if (this.styleSets.each) {
  this.styleSets.each(highlightCode)
} else {
    for (var i in this.styleSets) {
      highlightCode(this.styleSets[i]);
    }
</code></pre>

The other available styles are “html”, “css” and “ruby”.

For example, here’s how ruby would look like:

  def display_article(article = nil)
    begin
      @article      = block_given? ? yield : article
      @comment      = Comment.new
      @page_title   = @article.title
      auto_discovery_feed :type => 'article', :id => @article.id
      render :action => 'read'
    rescue ActiveRecord::RecordNotFound, NoMethodError => e
      error("Post not found...")
    end
  end

And it looks easy to create other styles. See the stylesetguide.html file for details.

Update: Turns out Dan already had a fix, so if you grab the trunk, you will be fine.

Here’s his version of the patch:

    for (var i=0; i < this.styleSets.length; i++) {
        highlightCode(this.styleSets[i]);
    }

Install your own ruby on a shared host

Since I upgraded to Typo 4.0, and in the process rails 1.1.6 I have had a few occurrences where nanoRAILS would hang, several bloated processes would be sitting there and not respond, and the only option at that point was to kill all ruby processes once I realized what was happening, which could be several hours. Suffice it to say, this is not a good option.

So after struggling during last rails upgrade to 1.1 on my host, the next logical step is to also use my own version of ruby so I can have better control on its environment, and even apply patches if necessary.

The following steps apply on a lot of systems. More specifically, my host is DreamHost (aff), and as best I can tell, I’m on a host with Debian Sarge.

Build your own Ruby

Download ruby from http://www.ruby-lang.org. The latest version is currently ftp://ftp.ruby-lang.org/pub/ruby/ruby-1.8.4.tar.gz

Create the makefile using

 ./configure prefix=[YOUR_OWN_RUBY_PREFIX]

Since you most likely don’t have root access, you need to override where ruby think it resides, and the way to do that is to set the prefix to somewhere into your home directory. Something like /home/USERNAME/ruby for example. From that point on, libraries, other builtin ruby files, gems will automatically install into your own ruby repository so you never have to worry about getting in trouble with an unforeseen upgrade.

Optionally, you can apply the patch used by Railsbench, with hardcoded default values because I haven’t figured how to set the environment variables for the dispatch.fcgi process (since apache in my case determines that). Download my version of rubygc.patch .

 patch gc.c rubygc.patch

Build and install ruby

 make
 make install

Additionally, so that the command line uses the same version of ruby, add this to your .bashrc or equivalent for your shel.

export PATH=[YOUR_OWN_RUBY_PREFIX]/bin:$PATH

Install your own gems

Now you are ready to install your own gems. Here’s the bare minimum you need.

First, install rubygems

 wget http://rubyforge.org/frs/download.php/11289/rubygems-0.9.0.tgz
 tar xzvf rubygems-0.9.0.tgz
 cd rubygems-0.9.0
 ruby setup.rb

Then install the minimum set of gems:

 gem install mysql
 gem install fcgi
 gem install rails --include-dependencies

Now, the only thing you need is to change the path to ruby in your dispatch file (dispatch.rb for mod_cgi, dispatch.cgi for regular cgi, and dispatch.fcgi for FastCGI/fcgid)

Typically, replace

#!/usr/bin/env ruby

by

#![YOUR_OWN_RUBY_PREFIX]/bin/ruby

Replace [YOUR_OWN_RUBY_PREFIX] by your own value you used earlier.

Upgraded to Typo 4.0

Well, the upgrade to Typo 4.0 didn’t go so well. No data loss though, so everything’s cool.

At least most pages seem to be functional, so this is not so bad. Combined with having custom sidebar plugins that no longer work, the fact that the merge between the new code and my old one did not quite go as planned, it was not a too pleasant (most of it my own fault I guess).

Anyway, it is getting really late and I’ll finish the rest of the migration tomorrow.

If you tried to access the site while this was going on and you were inconvenienced, pleas accept my most sincere apologies!

If you notice anything weird, please send me a mail at psq _at_ nanorails _dot_ com.

Update, about 20h later, some sleep…: It seems that everything is back up, I migrated my custom plugins to typo 4.0 (about removing 1 file, removing half of the code linked to configuration and adding a few lines) and it seems that everything has been working smoothly :)

And the best of all of that: no more trackback spam! Well, not that it went away, it caught about 25 since last night, but they don’t get published anymore. Just for that, it was all worth it. Thank you everyone in the typo team!

Update 2: Well rails 1.1.5 came out, and it was not enough, so rails 1.1.6 came out and seems to be strong enough to fill in the security issues

And in the process, I also upgraded to Typo 4.0.2

That last upgrade went very smoothly!

However, I’ve had a few annoying cases of nanoRAILS hanging and not responding for hours on hand till I killed the processes. I don’t know yet at this point whether it is due to the new version of rails, the new version of typo, or pehaps some settings that changes on dreamhost. In any case, I’ve installed my own version of ruby and the full set of gems, so we’ll see if that helps!

Rails Documentation Drive

Court3nay from ~:caboose has started a fundraising to be able to pay professional tech writers to beef up the Rails documentation. It is true that it is not advancing quite at the same pace that rails is and Court3nay’s initiative is a worthy one.

He’s aiming to raise $5,000 and after just a few hours he has reached half his goal.

If you enjoy Rails, please consider making a small donation, $10, $20, or whatever you can spare. That’s an investments that is sure to provide a return! So please help.

Fund the Rails Documentation Drive.

Update: the total is now $13,310, from 94 contributors, in less than 24 hours. Thank you everyone that contributed! This is going to help improve the doc a great deal!

RadRails is switching to Ruby-debug

According to Kyle, one of the tireless developer of Rad Rails, he is working on incorporating ruby-debug.

For more details about ruby-debug, see my write up about it.

The command line is great, and there are times when you are working on a remote server where nothing else will do. But having the ability to set breakpoints at the click of the mouse, see your variables in a separate pane, see a lot more of your source code is something quite convenient.

And with the added speed of ruby-debug, this should make debugging in Rad Rails something you can’t afford not to use.

This is very welcome news and I look forward to use it!