The smallest git tutorial you’ll ever need

Let me preface with this: this is over-simplified for someone just getting started with git, who does not have the time to learn all the intricacies of git, and covers the most common workflow I’ve encountered, that is:
– download code from a remote repository
– make changes
– commit and send the changes back
– get the latest from that remote repository

So, here goes:

First you’ll need to install git is it is not aleady installed.

Checkout a project

git clone http://github.com/psq/spider.git

Commit

Edit files as you see fit, then when time comes to commit your changes back in:

git status

will show you what files you’ve modified. Or added. If you added any files, add them to the files controlled by git (this also works with a new directory and will recursively add everything)

git add README.txt

Then commit all

git commit -a -m "added readme"

-m option adds a messages, -a adds all modified files to the commit

Upload your changes

Then push your changes back to the original repo

git push origin master

Download the latest version

And finally, to get any updates from the master repo (you may need to do this before doing the push above)

git pull origin master

This is overly simplified on purpose, but will give any beginner time to dive in deeper when time permits.

See this previous post for some helpful git shortcuts

Multiple targets with iscisitarget (ubuntu 9.04)

After trying for a (long) while to have multiple targets defined on the same host, and having my initiator to see both, the solution what in fact simple: Use different values for the LUN!

Here’s a snippet of /etc/ietd.conf

Target iqn.2010-01.com.test:storage.disk1.sys1.000
        Lun 0 Path=/media/usb/test0.img,Type=fileio,IOMode=rw
Target iqn.2010-01. com.test:storage.disk1.sys1.001
        Lun 1 Path=/home/psq/test1.img,Type=fileio,IOMode=rw

Then don’t forget to restart: “service iscsitarget restart”

rebuild the journal on an ext3 partition for CentOS 5.3

Heltech does not have it quite right for centos. The options are named slightly differently on newer versions.

Here’s the correct sequence for CentOS 5.3

  • Boot in single mode
  • umount the main partition
  • check the partition for errors
  • remove the journal
  • remount
  • recreate the journal
  • for good measure, check the partition for errors

or, in other words:

# fsck /dev/VolGroup00/LogVol00
# umount /dev/VolGroup00/LogVol00
# tune2fs -O ^has_journal /dev/VolGroup00/LogVol00
# mount -n -o remount,rw /dev/VolGroup00/LogVol00
# tune2fs -j /dev/VolGroup00/LogVol00
# umount /dev/VolGroup00/LogVol00
# fsck -f /dev/VolGroup00/LogVol00

Note: this should also work on CentOS 5.4

My Ruby on Rails Gentoo Setup

I had been hearing about Gentoo for a long time, and finally discovered it in depth about 6 months ago. Gentoo is really a different beast as Linuxes go. You get to build everything from sources. No binaries. That’s right. That means that you get to build everything with compatible flags, get to remove a lot of the unnecessary bloat (Do I hear X windows for example?). It also means that the initial setup could take you several days if your machine is not so powerful. Suffice it to say, I really, really like the approach and that is now what I’m using to run this blog.

Of course, if I need a server setup in a hurry, I’ll still go for a Debian or Ubuntu. But if I want a very fined tune system, Gentoo is it.

Of course, there is an ebuild for rails (ebuild is gentoo-speak for describing a package your can grab and build), but it did bring a lot more that what I cared for, so here’s the step by step instructions for getting Rails+Mongrel+nginx going on Gentoo.

This assumes that you are starting from a Gentoo 2006.1 stage 3 without much else.

A word of warning

Even though I’ve used these notes to successfully setup 3 machines (both on x86 and amd64), your experience may still be different, or you may need slightly different options or settings as the packages are bound to evolve, or I’ve made some assumptions that do not apply in your case, or something was obvious to me that should not have been, or, perish the thought, I just plain forgot something. If that happens, remember, Google is your friend, or don’t be shy, ask a question here, someone might know the answer.

So with that out the way, let’s get started.

Preamble

auto start ssh:

rc-update add sshd default

update portage tree

emerge --sync
emerge portage
emerge -av --update --deep world

At this point, you are going to have a lot of configuration files that out of date. Since we are doing a new setup, we just want to silence the warnings and making sure we use the latest version. So to find each file:

find /etc -iname '._cfg????_*'


But do not overwrite inittab if it is only putting back more terminal login than you need.

Now it is time to add a few very useful tools like sudo, stuff to help you figure out what’s installed, and to find things (namely sudo, gentoolkit and sys-apps/slocate)

emerge -av app-admin/sudo app-portage/gentoolkit sys-apps/slocate

Now it is time to create your user. And set its password.

useradd user
passwd user

Add user to /etc/sudoers with ALL

Add vim (or whatever else you fancy)

emerge -av vim

Change default EDITOR

vi /etc/rc.conf


There, comment out /bin/nano, and uncomment /bin/vim

Install subversion

emerge -av subversion

Even though not strictly necessary for Rails, a few network utilities always come in handy (traceroute, dig, nslookup):

emerge -av traceroute bind-tools

If you want to send/receive emails:

emerge -av mail-client/mailx
emerge --unmerge mail-mta/ssmtp
emerge -av syslog-ng mail-mta/exim
rc-update add exim default

There are lots of options to do SMTP, but I find exim4 the quickest and simplest to setup:

create /etc/exim/exim.conf from /etc/exim/exim.conf.dist

To lockdown mail to localhost (you just want your Rails app to send emails), add this exim.conf:

local_interfaces = 127.0.0.1

And to allow non users to send mails (comment out this line)

##require verify        = sender

And finally, start your mail server:

/etc/init.d/exim start

Rails Setup

My preferred database remains MySQL. I know, some people prefer PostgrSQL.

emerge -av dev-db/mysql
rc-update add mysql default

Then, to configure MySQL and create a first database, first you need to figure out which version you just installed. If you were not paying attention as the logs were flying by (I know I wasn’t), use

equery list mysql

And then

emerge --config =dev-db/mysql-5.0.26-r2

To get rubygems 0.9.x, you need to unmask it. For that, add a new line to /etc/portage/package.keywords with:

dev-ruby/rubygems ~amd64

Replace with ~x86 if you are not on an AMD chip, then run:

emerge -av dev-lang/ruby dev-ruby/mysql-ruby dev-ruby/rubygems  dev-ruby/ruby-termios

For any kind of image manipulation (resizing and such), install Image Magick. First, it it does not already exist create /etc/portage/package.use

dev-ruby/rmagick lcms gif imagemagick jbig jpeg jpeg2k pdf png svg tiff truetype unicode wmf xml xpm pcre
media-gfx/imagemagick lcms gif imagemagick jbig jpeg jpeg2k pdf png svg tiff truetype unicode wmf xml xpm pcre
emerge -av media-gfx/imagemagick dev-ruby/rmagick


The above line represents the options for that ebuild.

Time to get the latest nginx

  1. add a new line to /etc/portage/package.keywords with (Again, on Intel, use ~x86)

"www-servers/nginx ~amd64

Then run emerge again:

emerge -av www-servers/nginx

Ok, now time to install what we actually wanted to install, Ruby on Rails and a few useful gems:

gem install -y rails rake capistrano daemons gem_plugin mongrel mongrel_cluster rmagick BlueCloth RedCloth ruby-debug termios ruby-openid ruby-yadis

Start mysql, create a database for rails test app

/etc/init.d/mysql start
mysqladmin -u root -p create test1_development
mysqladmin -u root -p create test1_production

And finally, install a rails app (we’ll use beast as it is pretty simple to install)

svn checkout http://svn.techno-weenie.net/projects/beast/trunk beast
cd beast/
rake rails:freeze:edge
#create config/database.yml from config/database.example.yml
rake db:schema:load RAILS_ENV=production
./script/server -e production

Test that everything’s fine fine by pointing your browser to port 3000.

Now, let’s configure mongrel (We are almost there, I promise).

cd [app]
sudo useradd -n mongrel
sudo chown -R mongrel:mongrel [app]
mongrel_rails cluster::configure -e production -p 8000 -N 3 -c [app] -a 127.0.0.1 --user mongrel --group mongrel

Later, a few useful commands to use

#start cluster
mongrel_rails cluster::start

#restart cluster
mongrel_rails cluster::restart

#stop cluster
mongrel_rails cluster::stop

To setup nginx, edit /etc/nginx/nginx.conf to match your setup

To setup mongrel_cluster for autostart

mkdir /etc/mongrel_cluster
ln -s [app]/config/mongrel_cluster.yml /etc/mongrel_cluster/[app].yml

#copy from wherever the gem is installed (use locate to figure out the location)

ln -s /usr/lib64/ruby/gems/1.8/gems/mongrel_cluster-0.2.1/resources/mongrel_cluster /etc/init.d/mongrel_cluster
chmod +x /etc/init.d/mongrel_cluster

Add nginx and mongrel_cluster to startup

rc-update add nginx default
rc-update add mongrel_cluster default

For smooth reboot, you may need to delete the pid files
Apply this patch (from http://textsnippets.com)

create file “mongrel_cluster.patch” with:

--- bin/mongrel_rails-orig      2007-05-16 14:41:51.000000000 -0400
+++ bin/mongrel_rails   2007-05-16 14:42:50.000000000 -0400
@@ -83,9 +83,17 @@
       config = Mongrel::Rails::RailsConfigurator.new(settings) do
         if defaults[:daemon]
           if File.exist? defaults[:pid_file]
-            log "!!! PID file #{defaults[:pid_file]} already exists.  Mongrel could be running already.  Check your #{defaults[:log_file]} for errors."
-            log "!!! Exiting with error.  You must stop mongrel and clear the .pid before I'll attempt a start."
-            exit 1
+            # mongrels that crash can leave stale PID files behind, and these
+            # should not stop mongrel from being restarted by monitors…
+            pid = File.new(defaults[:pid_file]).readline
+            unless `ps -ef | grep #{pid} | grep -v grep`.length > 0
+            # use "ps ax" for freebsd
+                log "!!! PID file #{defaults[:pid_file]} exists, but is stale, and will be deleted so that this mongrel can run."
+                File.delete(defaults[:pid_file])
+            else
+                log "!!! PID file #{defaults[:pid_file]} already exists and the process id referred to in it is running.  This mongrel is probably already running.  #{defaults[:log_file]} for errors.  EXITING."
+                exit 1
+            end

           end

                 daemonize

then apply the patch

patch -p0 /usr/lib64/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails < mongrel_cluster.patch

And there you have it. A nice, humming Ruby on Rails+Mongrel+nginx on Gentoo.

Install your own ruby on a shared host

Since I upgraded to Typo 4.0, and in the process rails 1.1.6 I have had a few occurrences where nanoRAILS would hang, several bloated processes would be sitting there and not respond, and the only option at that point was to kill all ruby processes once I realized what was happening, which could be several hours. Suffice it to say, this is not a good option.

So after struggling during last rails upgrade to 1.1 on my host, the next logical step is to also use my own version of ruby so I can have better control on its environment, and even apply patches if necessary.

The following steps apply on a lot of systems. More specifically, my host is DreamHost (aff), and as best I can tell, I’m on a host with Debian Sarge.

Build your own Ruby

Download ruby from http://www.ruby-lang.org. The latest version is currently ftp://ftp.ruby-lang.org/pub/ruby/ruby-1.8.4.tar.gz

Create the makefile using

 ./configure prefix=[YOUR_OWN_RUBY_PREFIX]

Since you most likely don’t have root access, you need to override where ruby think it resides, and the way to do that is to set the prefix to somewhere into your home directory. Something like /home/USERNAME/ruby for example. From that point on, libraries, other builtin ruby files, gems will automatically install into your own ruby repository so you never have to worry about getting in trouble with an unforeseen upgrade.

Optionally, you can apply the patch used by Railsbench, with hardcoded default values because I haven’t figured how to set the environment variables for the dispatch.fcgi process (since apache in my case determines that). Download my version of rubygc.patch .

 patch gc.c rubygc.patch

Build and install ruby

 make
 make install

Additionally, so that the command line uses the same version of ruby, add this to your .bashrc or equivalent for your shel.

export PATH=[YOUR_OWN_RUBY_PREFIX]/bin:$PATH

Install your own gems

Now you are ready to install your own gems. Here’s the bare minimum you need.

First, install rubygems

 wget http://rubyforge.org/frs/download.php/11289/rubygems-0.9.0.tgz
 tar xzvf rubygems-0.9.0.tgz
 cd rubygems-0.9.0
 ruby setup.rb

Then install the minimum set of gems:

 gem install mysql
 gem install fcgi
 gem install rails --include-dependencies

Now, the only thing you need is to change the path to ruby in your dispatch file (dispatch.rb for mod_cgi, dispatch.cgi for regular cgi, and dispatch.fcgi for FastCGI/fcgid)

Typically, replace

#!/usr/bin/env ruby

by

#![YOUR_OWN_RUBY_PREFIX]/bin/ruby

Replace [YOUR_OWN_RUBY_PREFIX] by your own value you used earlier.

Remote Ubuntu Dapper Drake Install

Don’t try this at home!

Well, actually, there are 2 ways you can look at it. Only attempt a remote install if it is absolutely necessary and there are no other practical ways of doing the OS install. It may be fun, and you’ll be impressed with yourself if you succeed, but the downside is that it takes a lot more time to do it remotely, it can be pretty tough on your nerves while you wait for the machine to come back up, eventually, and that’s it ever does. I can imagine a lot of things going wrong…

On the other hand, that’s exactly what you want to do. Do it from home instead of driving to the office, or better yet, to some hosting facility way out somewhere.

Anyway, now you’ve been warned and if you are still reading, here’s the challenge I was facing.

I had a machine (not the fastest, but a machine) with an old version of Linux. So old the distribution is no longer updated, and I’m not really that familiar with it anyway. So I had been itching to reinstall either a version of Debian or a version of Ubuntu Dapper as my choice of late. Plan B is having to drive to the data center, extract the machine, take it back, do the install, drive back to the data center, reinstall the machine, … and Plan B was not something I was looking forward to.

So I did some research, and using Google, it is fairly obvious that the article by Erik Jacobson that you should read first if you want to attempt any remote install of debian is the authority on the subject. Read the HOW-TO.

I could not find anything on Ubuntu, however. And I even found a few reports of people upgrading from debian to Ubuntu having troubles, but mostly from XWindows stuff, but for a server, that’s not something I was interested in. And worst case, I’d be on my driving to that data center…

So I read the HOW-TO carefully, a few times to let it sink in, and got started on my Ubuntu Dapper Drake (6.06) remote install.

Here’s what I did.

I first followed the procedure in the HOW-TO to install debian. At some points, there are a few differences that I’ll highlight. Here are some notes I took while doing the install.

I used the rpm from [http://azhrarn.underhanded.org/debootstrap-0.2.23-1.i386.rpm]()

I used the swap partition as a new boot partition for debian.

The old distribution did not support journaling, so I only used (this is only the boot partition anyway, so journaling is not as important, and that one will be temporary anyway):

mke2fs /dev/hda6

NOTE: identify your volume names at the beginning, create a table to help you translate between what the HOWTO uses, what I use and your setup otherwise you might be in for some serious trouble.

debootstrap does not support sarge, so using woody was fine. The only thing we’ll be using the bootstrap is to get apt working.

Follow the steps till you reach the part where you get apt working. Then no need to bother with netselect (didn’t work for me), now is the time to do a bit of tinkering.

Edit /etc/apt/sources.list and replace it with:

vi /etc/apt/sources.list
deb http://us.archive.ubuntu.com/ubuntu/  dapper main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/  dapper main restricted

deb http://us.archive.ubuntu.com/ubuntu  dapper universe
deb-src http://us.archive.ubuntu.com/ubuntu  dapper universe

#deb http://mirrors.kernel.org/debian/ stable main non-free contrib
#deb-src http://mirrors.kernel.org/debian/ stable main non-free contrib
#
#deb http://debian.yorku.ca/debian/ stable main non-free contrib
#deb-src http://debian.yorku.ca/debian/ stable main non-free contrib
#deb http://non-us.debian.org/debian-non-US stable/non-US main contrib non-free
#deb-src http://non-us.debian.org/debian-non-US stable/non-US main contrib non-free
#
## deb http://security.debian.org/ stable/updates main contrib non-free
#deb http://security.debian.org stable/updates main contrib non-free

Next, proceed with the update and dist-upgrade, then install a kernel.

apt-get update
apt-get dist-upgrade
apt-cache search kernel-image

Find a 2.6 kernel for your system (unless you insist on using 2.4).

Then proceed back with the rest of the script, starting with the install of ssh.

To make sure ssh was configured properly, I configured it on a different port so I could try it before the first reboot. Having everything working, but not ssh would be real bad, so spending some extra time here is worth it.

And one thing you want to take the time to do is to setup a cron job that will reboot the new system after some time (say every 2 hours), as an extra insurance, in case something wrong happens. Here’s a possible line to add to your crontab to reboot every 2 hours:

00  */2 * * * /sbin/shutdown -r now

I did mess up once, and I was really glad to see the machine come back after a very long 2 hours or so (I had almost given up on it and was getting ready to plan B). The second insurance you can take is to setup the reboot on the new kernel as a temporary reboot (as explained in the HOW-TO), you might be glad you did.

Once your reboot is successful, it is a matter of replacing the main partition with ubuntu, do the second reboot, and voila! Phewww! Not something I’d like to do everyday, but I’m glad I did it once.

Now you have a very barebone version of Ubuntu Dapper and you can customize it any which you need.

Good luck on your remote install!

Installing Ubuntu Dapper on Parallels running on Tiger

After my success with installing Windows XP on my shiny new iMac, I tried to install Ubuntu Dapper (a.k.a. Dapper Drake). Why would I want to install Linux on a Mac? Because that’s what I’m running on my server, so I need a sandbox where I won’t risk breaking real stuff.

Anyway, I created a new VM, chose Linux as the Guest OS Type, and Debian Linux as the Guest OS Version (since Ubuntu is derived from Debian and was most likely the closest).

Next, I put a CD of the latest Ubuntu install (Server Version: 6.06). Using mostly the defaults all along, the installation went smoothly. The only part were I went fancy was to use LVM (Logical Volume Manager). Everything went smoothly till it was time to reboot the machine…

The reboot went fine till after it was done uncompressing the kernel. Then it froze! Ouch.

From there, it was a lot of testing of various options, versions of Linux, even redid a full install without LVM. Nothing made any difference! So, I went ahead and tried to install Sarge (Ubuntu 3.1). This went like a breeze and worked the first time around (after all, Parallels officially supports Debian). So I was feeling a bit better…

… and went on to do some more research. And read that some people had had some luck with the Live CD (I had used the server CD since I wanted a permanent installation). So after a little while the download was over, and I booted the LiveCD, and it worked! Hurray! Bring on the champaign… Hold on, we are not quite there, but keep it in a nice cool place… Just in case…

And after playing for a while, just starring at me on the desktop was an “install” icon, so I figured, just one more try would not hurt. I went through the installation, and was bracing myself for the freeze right after the kernel gets uncompressed, but lo and behold! It did not freeze. Wow! Home free at last.

Ok, yes, you can bring on the champaign now.

Not sure what’s different about the 2 installers, but I don’t think I’ll try to figure it out. If anyone knows, would you let me know, please?

Note: this is my first post done using TextMate Blogging Bundle, so this may have a few hiccups…
In case you were wondering what the xmlrpc URL is for typo, it is /backend/xmlrpc (Thank you Damien)

Installing readline on Kubuntu

Running script/console provides the delightful answer:

/usr/local/lib/ruby/1.8/irb/completion.rb:10:in `require&#8217;:
no such file to load &#8211; readline (LoadError)

Turns out that Kubuntu doesn’t have the curses libraries.

Here’s the begining of a solution

apt-get install libncurses5-dev libreadline5-dev (Ubuntu/Debian)

Then recompile the readline extension and go from there.

cd ext/readline
ruby extconf.rb
make
sudo make install

Read more