Using Git to fork and contribute to a Rubyforge project

Posted by Brian in Howto, News, Rails, tips (January 13th, 2009)

If you’ve never comtributed to an open-source project before, there’s no better time to start. This article will walk you through forking an open-source project on RubyForge and making changes to it using Git. At the end, you’ll create a unified diff that can be sent back to the original author.

My current project FeelMySkills helps creative professionals promote themselves by creating an online portfolio that shows what they can do rather than who they know. I wanted the site to let a user export their profile page as a PDF, and so I found the amazing HTMLDoc gem which takes simple HTML pages and converts them to PDF documents.

While developing the site, I discovered that there is a bug in PDF::HTMLDoc that pops up when you embed images into the PDF. It turns out that it has to do with extra whitespace getting into the content which causes HTMLDoc to choke. The fix is really simple, and after I fixed it I found that there is already a patch for this posted to Rubyforge but it hasn’t been applied. I assume that the reason it’s not applied is that there were no tests supplied with the patch.

Let’s get the code, write the test, and submit the patch!

GITing the code

The HTMLDoc project (http://htmldoc.rubyforge.org) has a Subversion repository that we can use as our master branch. Now, if you’ve never used Git before, don’t worry about it because we’re only going to use it here as a really easy way to create patches.

Installing Git

Mac users with XCode and Macports installed can do it with

sudo port install git-core +svn

Windows users can install Msysgit.

Linux users should install Git using their package manager or from source.

Forking the code from Subversion

The git svn command lets you pull and push to a Subversion repository and is perfect for watching projects that don’t use Github yet.

Visit the project page at http://rubyforge.org/projects/htmldoc/ and click the link on the bottom for SCM Repository. That page lists the repo as http://htmldoc.rubyforge.org/svn.

Grab the trunk for HTMLDoc from RubyForge.

git svn clone http://htmldoc.rubyforge.org/svn/trunk htmldoc

Test before you start

If you want to start things off on the wrong foot, just start hacking away at the code. A much better approach is to see what tests are broken before you start working. Good Ruby projects should have a test suite that completely passes.

In the htmldoc folder, run rake which will run the test suite for this app.
Unfortunately the test suite shows an error:

Loaded suite -e
Started
..E........
Finished in 3.500649 seconds.

  1) Error:
test_get_command_pages(BasicTest):
NoMethodError: undefined method `path' for nil:NilClass
    ./test/basic_test.rb:91:in `test_get_command_pages'

11 tests, 361 assertions, 0 failures, 1 errors

However, on further inspection, the error is because of a path issue. The tests pass when you run them individually:

cd test
ruby basic_test.rb && ruby generation_test.rb
cd ..

See? Everything works!

Loaded suite basic_test
Started
......
Finished in 0.010653 seconds.

6 tests, 324 assertions, 0 failures, 0 errors
Loaded suite generation_test
Started
.....
Finished in 3.403335 seconds.

5 tests, 38 assertions, 0 failures, 0 errors

Creating a new branch for your changes

We’ll want to keep our work in a new branch. This makes it easy for us to create the patch later, as we can make Git give us the difference between the master branch, which we forked from Rubyforge, with our new branch which will contain our fixes.

$ git checkout -b fix_images

Writing a new test

Before you start hacking away on a new feature, you should write a test to prove that things are really broken. In this case, a PDF that contains a reference to an image causes things to break. A test that tries to put an image reference into the PDF data should break. Add this test to test/generation_test.rb

  def test_generation_results_with_image
    pdf = PDF::HTMLDoc.new
    pdf.set_option :webpage, true
    pdf.set_option :toc, false
    pdf << "

Random title

something you have to follow either.

Run this test

cd test
ruby generation_test.rb -n test_generation_results_with_image
cd ..

and you'll see that things don't work as expected:

Loaded suite generation_test
Started
F
Finished in 0.349939 seconds.

  1) Failure:
test_generation_results_with_image(GenerationTest) [generation_test.rb:43]:

expected to be kind_of?
 but was
.

So we can reproduce the bug, and now we just have to fix the problem.

A simple fix (this time)

The reason for the error is because HTMLDoc hates extra whitespace created by the inclusion of the image. Open up lib/htmldoc.rb and change line 186 from

          case line

to

          case line.strip

Save the file. Now run the entire test suite again to make sure that nothing else broke.

Loaded suite basic_test
Started
......
Finished in 0.007855 seconds.

6 tests, 324 assertions, 0 failures, 0 errors
Loaded suite generation_test
Started
......
Finished in 3.434714 seconds.

6 tests, 43 assertions, 0 failures, 0 errors

Hurray! We fixed it! Now we just have to make a patch!

Commit your changes!

You should commit your changes to your branch so you don't lose them.

  git commit -a -m "Fixed problem when embedding images in PDFs"

Making a patch

If you're quick, you can probably just create the patch right now, but let's assume the project is a fast-moving one, like Rails. Youl want to pull down the latest version of the project and fix any conflicts.

git checkout master
git svn rebase
git checkout fix_images
git rebase master

There won't be any collisions you have to fix now, so you can just make the patch file.

git format-patch master --stdout > htmldoc_fix_images.diff

This creates the file htmldoc_fix_images.diff which is a unified diff containing both the fix and the test. You can now send the patch file to the maintainer who will happily apply it because it has tests!

Wrapping up

Working with open-source projects gets easier every day, and you can really take ownership of the tools you use if you get involved. Hopefully this article gets you started on the path to contributing to projects. Good luck, and leave comments if something needs more clarification!

“Meet Rails” presentation materials from Chippewa Valley Code Camp

Posted by Brian in Howto, Rails, tips, web (November 24th, 2008)

I gave a talk at the Chippewa Valley Code Camp earlier this month where I built a Rails pastebin application using test-first development principles. Since there were no slides for this, I’ve prepared a small PDF that will walk you through the installation instructions and the coding of the application I built.

Download Meet Rails to get started.

You can grab the source code for the finished project at Github.

Watch this site, as this project may get turned into a full-scale book.

Fixing TextMate for Rails 2.0

Posted by Brian in Rails, Testing, tips (November 7th, 2008)

Rails 2.0 and above have some changes that break Textmate, a very popular development environment for Rails developers.

Running Tests

Rails 2.0 projects generate unit and functional tests with a relative require to test_helper. This change breaks
the Command-R (Run) and Command-Shift-R (Run Focused Unit Test) commands to fail since they can’t
include the necessary helpers. I finally spent a little time figuring this one out and the
answer is to modify the bundle commands themselves.

Open up the bundle editor, locate the Ruby bundle, and choose to edit the &lquot;Run&rquot; command.

Change it from

  export RUBYLIB="$TM_BUNDLE_SUPPORT/RubyMate${RUBYLIB:+:$RUBYLIB}"

to this:

  export RUBYLIB="..:$TM_BUNDLE_SUPPORT/RubyMate${RUBYLIB:+:$RUBYLIB}"

Now edit the &lquot;Run Focused Unit Test&rquot; command and make the same change.

Fixing incompatibilities with Builder

Rails 2.0 contains its own version of builder.rb. We have to take out the original one that TextMate provided
so our stuff starts working again.

mv /Applications/TextMate.app/Contents/SharedSupport/Support/lib/Builder.rb /Applications/TextMate.app/Contents/SharedSupport/Support/lib/Builder.rb.bak 

New Rails bundle

Open a new terminal and type this:

  cd "~/Library/Application Support/TextMate/Bundles"
  git clone git://github.com/drnic/ruby-on-rails-tmbundle.git "Ruby on Rails.tmbundle"

Working with Docbook on Windows

Posted by Brian in Howto, snacks, tips (March 3rd, 2008)

Setting up a toolchain for working with Docbook on Windows often requires setting things up using Cygwin. Many people are just simply not willing to do that. This tutorial will show you how to set up a native environment to work with Docbook, and show you how to make CHM and PDF files on Windows.

Thanks to http://supportweb.cs.bham.ac.uk/documentation/tutorials/docsystem/build/tutorials/docbooksys/segmentedhtml/ch03s03.html#DocBookSys-Chapter3-XML-Install-libxml-Windows for much of this information.

Getting the tools

The tools you need to work with Docbook XML and XSTL are all available on Windows. The first thing you need to do is visit http://www.zlatkovic.com/pub/libxml/ (new window) and grab the latest versions of

  • libxml2
  • libxslt
  • iconv

Download each and unzip the contents of the folder to c:\windows or another location on your path. For reference, these files are the ones you’re looking for:

iconv.exe
libexslt.dll
libxml2.dll
libxslt.dll
xmlcatalog.exe
xmllint.exe
xsltproc.exe

If you feel better about putting these in their own folder, that’s fine as long as you add the new folder to your path.

Getting the Stylesheets

In order to build a book, you need to have the XSLT stylesheets so you can transform your XML into a pretty-looking book with a table of contents and nicely formatted text.
Download the docbook-xml-ns files from sourceforge: http://sourceforge.net/project/showfiles.php?group_id=21935

Unpack to your c:\ drive and then rename the extracted folder to c:\docbook-xsl

Generating PDFs

In order to create a PDF, you have to first convert to the FO format and then use a Java library to convert the FO to a PDF. Apache FOP does this for you. You’ll need to have a JRE (Java Runtime) installed though. Visit http://java.sun.com/ for that.

Get FOP to build PDFs

Download FOP at http://www.uniontransit.com/apache/xmlgraphics/fop/fop-0.94-bin-jdk1.4.zip and unzip it to a temp location. Copy all .jar files in build/ and lib/ to your Java installation’s lib/ext folder. On my system it’s C:\Program Files\Java\jre1.5.0_11\lib\ext. Your system will differ depending on your installed version of Java.

Next, download OFFO-hyphenation from http://offo.sourceforge.net/index.html and grab the offo-hyphenation-fop-stable.zip file from the downloads page and put the jar files in the same folder as the FOP files.

Building your first book

Create a project folder called “my_book” and create a new file called “book.xml” in this folder.




  
  
  My Simple Book
  
 
  


Then create a chapter for your book. Create the file chapter1.xml in your project folder with this content:





  Introduction
  This is just a simple book.



Notice that the chapter and book each have their own doctype, This is really important. Each chapter file needs to have this structure in order to work properly.

Generating HTML from the document

The easiest way to use Docbook is to export to HTML. Execute this command to create an HTML version of your book:

   xsltproc --xinclude --output book.html c:/docbook/xsl/html/docbook.xsl book.xml

Creating a makefile to build the PDF

The PDF creation process is similar to the HTML process but it does require two steps. You need to first convert the document to the FO file format. Then you use FOP to convert it to the PDF. We can automate this by using Ruby.

Create a Ruby file in your project folder called “make”. You’ll use this file to build the PDF of your book.

file = ARGV[0]
cmd1 = "xsltproc --xinclude --output #{file}.fo c:/docbook-xsl/fo/docbook.xsl #{file}.xml "
cmd2 = "java org.apache.fop.cli.Main -fo #{file}.fo -pdf #{file}.pdf"

puts "Building FO file"
`#{cmd1}`

puts "Building PDF"
`#{cmd2}`

puts "Cleaning up"
`del #{file}.fo`

puts "Done"


Now, build your book:

ruby make book

Creating a Help File

Generating a Windows HTML Help file (CHM) is pretty similar to the way you make a PDF. You first need to make the HLP file using xsltproc, and then you use a commandline tool to build the CHM.

Grab a copy of Microsoft’s HTML Help Workshop here and install it. Open a command prompt and copy the hhc file to the c:\windows directory so that the file is on your path.

copy "c:\Program Files\HTML Help Workshop"\hhc.exe c:\windows

Next, we can use Ruby to make a file to create another build file. Create a file called “make_chm” in your project folder.

file = ARGV[0]
cmd1 = "xsltproc --xinclude c:/docbook-xsl/htmlhelp/htmlhelp.xsl #{file}.xml"
cmd2 = "hhc htmlhelp.hhp"

puts "Building HLP temporary files"
`#{cmd1}`

puts "Building CHM"
`#{cmd2}`

puts "Cleaning up"
`rename htmlhelp.chm #{file}.chm`
`del *.hhp`
`del *.hhc`
`del *.html`

puts "Done"

Summary

Docbook is a really great way to create books, tutorials, and documentation in a format that can be transformed into various other formats. It’s extremly easy to work with in Windows too!

Working with Docbook on the Mac

Posted by Brian in Howto, snacks, tips (March 3rd, 2008)

Docbook allows you to prepare documentation by using XML markup. You can create PDFs or HTML exports of your work, and it’s really nice for collaborating with others, as you can work in a code repository easily. In this article, you’ll learn how to build a PDF using Docbook XSLT.

Getting the Stylesheets

In order to build a book, you need to have the XSLT stylesheets so you can transform your XML into a pretty-looking book with a table of contents and nicely formatted text.
Download the docbook-xml-ns files from sourceforge: http://sourceforge.net/project/showfiles.php?group_id=21935

Unpack to your home directory and rename the folder to ~/docbook-xsl

Generating PDFs with Apache FOP

In order to create a PDF, you have to first convert to the FO format and then use a Java library to convert the FO to a PDF. Apache FOP does this for you.
Get FOP. – http://www.uniontransit.com/apache/xmlgraphics/fop/fop-0.94-bin-jdk1.4.zip

Unzip to temp location and copy all .jar files in the build/ and lib/ folders to ~/Library/Java/Extensions. Create that folder if it isn’t there for you already.

Finally, download OFFO from http://offo.sourceforge.net/index.html and grab the offo-hyphenation-fop-stable.zip file from the downloads page and put the jar files in ~/Library/Java/Extensions. This enables hyphenation support.

Building your first book

Create a project folder called “my_book” and create a new file called “book.xml” in this folder.




  
  
  My Simple Book
  
 
  


Then creaet a chapter for your book. Create the file chapter1.xml in your project folder with this content:





  Introduction
  This is just a simple book.



Notice that the chapter and book each have their own doctype, This is really important. Each chapter file needs to have this structure in order to work properly.

Generating HTML from the document

The easiest way to use Docbook is to export to HTML. Execute this command to create an HTML version of your book:

   xsltproc --xinclude --output book.html c:/docbook/xsl/html/docbook.xsl book.xml

Creating a makefile to build a PDF

The PDF creation process is similar to the HTML process but it does require two steps. You need to first convert the document to the FO file format. Then you use FOP to convert it to the PDF. We can automate this by using Ruby.

Create a Ruby makefile in your project folder called “make”. You’ll use this file to build the PDF of your book.

file = ARGV[0]
cmd1 = "xsltproc --xinclude --output #{file}.fo ~/docbook-xsl/fo/docbook.xsl #{file}.xml "
cmd2 = "java org.apache.fop.cli.Main -fo #{file}.fo -pdf #{file}.pdf"

puts "Building FO file"
`#{cmd1}`

puts "Building PDF"
`#{cmd2}`

puts "Cleaning up"
`rm #{file}.fo`

puts "Done"
`open #{file}.pdf`

Now, build your book. In your project folder, type

ruby make book

Summary

Now you have a good introduction to how to work with the Docbook format. You may want to use a Textmate bundle to make editing the XML a little easier, but the syntax really isn’t that hard.

Up next… creating CHM files using Docbook on Windows.

Mini-CMS for your Rails application

Posted by Brian in Rails, snacks, tips (March 3rd, 2008)

I’ve worked on several applications where an end user may need to change text on various sections of a web site. I’ve used this technique a few times and it’s worked out well so I thought I’d share.

We’re going to create a model called Content which will be backed by a database. When a request comes in, we can look at the requested controller and action name to look up all of the content sections for a page. Some pages may have more than one content section.

This solution is intended to be implemented and modified by developers. An end user would not be able to add new content regions to a system.

Create the model and table

ruby script/generate model Content

Next, modify the migration:

class CreateContents < ActiveRecord::Migration
  def self.up
    create_table :contents do |t|
      t.string :controller, :action, :name
      t.text :body
      t.timestamps
    end
  end			

  def self.down
    drop_table :contents
  end
end

The name field is what our end user will use to locate and edit a section. They don't need to know the controller and action name. Instead, they'll probably look for a content region like "Home Page main content". In order to make this work, the user will only be able to edit the body field. The other fields will be handled by the system.

Now, open up the content model. We're going to add a few methods here to easily grab the content out so we can use it in views. Out of the box, ActiveRecord allows us to find all the sections for a controller and action name by doing

@contents = Content.find_all_by_controller_and_action("public", "index")

At first glance, that looks pretty cool, but how do you identify the individual regions? You'd have to loop through that collection every time you want to output something. If you only have one content area per page, this could be good enough. However, I like things to be a little more spelled-out.

Add this method to the content class:

  def self.get_content_for(controller, action)
    c = Content.find_all_by_controller_and_action(controller, action)
    content = Hash.new
    c.each do |record|
      content[record.name] = record.body
    end
    content
  end'

Now, to fetch all of the sections for the page, we only need to call

@contents = Content.get_content_for "public", "index"

This class method fetches all of the content sections and then puts then into a hash which you can afccess via the name.

Trying it out

Open up the console and put two content regions in:

   ruby script/console
Content.create :controller=>"public", :action=>"index", :name=>"intro", :body => "Hello world"
Content.create :controller=>"public", :action=>"index", :name=>"main_content", :body => "Main content goes here"

Now quickly generate a new controller and action

ruby script/generate controller public index

Add this to the public controller

before_filter :get_content

Now open application.rb and add this method:

def get_content
  @content = Content.get_content_for(params[:controller], params[:action])
end

Finally, open up app/views/public/index.html.erb and change its contents.


Home page

<%=@contents["intro"] %>

About us

<%=@contents["main_content"] %>

Summary

I've used this approach in a few sites and it's served me well. I'd love to hear how you've done the same. Post ideas and suggestions in the comments.

Parsetreee gem available for Windows!

Posted by Brian in Rails, tips (January 1st, 2008)

Luis Laverna compiled and released the parsetree gem for Windows. This means that libraries like ruby2ruby, heckle, flog, and even the merb framework are now available to Windows Rubyists.

You need to look at flog. It can show you how bad your code really is.

How To Follow Along with Agile Web Development with Rails Second Edition

Posted by Brian in Rails, tips (January 1st, 2008)

This is merely for reference for anyone out there new to Rails or Ruby and trying to work with the Agile book. Version 2.0.2 of Rails is out now and it’s just way too difficult to follow along with the great AWDWR book from Pragmatic. To be successful with that book, you need to use Rails 1.2.3. Here’s how you do it:

Mac

If you have a brand new macbook pro, you’re done. It comes equipped with Rails 1.2.3. Start following along with the first Hello World program.

If you’ve accidentally installed Rails 2.0, you can start a Rails 1.2.3 project like this:

sudo gem update --system
sudo gem install rails -v=1.2.3
rails _1.2.3_ depot

where depot is the name of your project.

Windows

If you’ve got a Windows machine, start from scratch. Grab The one click ruby installer (Windows installer) and install it.

Next, open up a command prompt and install Rails 1.2.3, and Mongrel

gem update --system
gem install rails -v '1.2.3'
gem install mongrel
gem install mysql

Now just go ahead and follow along with the book. When you’re done, you can update to Rails 2.0 by doing

gem install rails

which will install the latest version of Rails. Your current Rails apps are tied to the version of Rails they are created on. You change the version of Rails the app uses by “freezing gems” (placing a copy of Rails in the app’s vendor/rails folder, or by editing the app’s config/environment.rb file and changing the version specified there. Do a web search for more info on this.

You can specify the version of Rails you want to use by passing it along on the commandline

rails _1.2.3_ old_rails_app
rails _1.2.6_ ready_to_update_app
rails _1.1.6_ legacy app
rails _2.0.2_ new_rails_app

Just make sure you have all of the Rails versions that you want to use on your machine.

Deploying Multiple apps on one IP with Apache’s ProxyPass

Posted by Brian in Rails, tips (October 24th, 2007)

Say you’re poor and only have one IP address. If you have the ability to control your own DNS, there’s an easy fix.

  • Install Apache on one of your servers on Port 80
  • Add this to the bottom of the Apache httpd.conf file
  • NameVirtualHost 192.168.1.5
    
         ServerName myapp.mydomain.com
         ProxyPass / http://internal.mydomain.com/
         #TransferLog /net/www/logs/sun.docs.access
         #ErrorLog    /net/www/logs/sun.docs.errror
    
    
  • Point the myapp.mydomain.com DNS entry to your IP.
  • Start up Apache and you should be able to rock and roll.

This will work to forward requests to many backend servers. Just keep in mind that the IP addresses from the outside won’t show up in your logs. There are ways around that though.

Adding RSS to your Rails application

Posted by Brian in Rails, snacks, tips (September 8th, 2007)

Rails makes it really simple to expose your data as XML. It also makes it pretty easy to support any other type of format.

While you can do this in Rails 1.2, this writeup will focus on doing it with Edge Rails.

Assume I have a Project model, a Projects controller, and I’m using a RESTful design…

  rails my_projects_rss --database=sqlite3
  cd my_projects_rss
  ruby script\generate scaffold Project title:string description:string
  rake db:migrate
  ruby script/server

Create a new file at apps/views/projects called index.rss.builder.

Place this code in that file:

xml.rss('version' => '2.0') do
  xml.channel do 
    xml.title(@page_title)
    xml.link(projects_url)
    xml.description(@page_title)
    @projects.each { |project|
      xml.item do 
        xml.title(project.title)
        xml.link(project_url(project))
        xml.description(project.description)
        xml.pubDate(project.updated_at.rfc822)
      end
    }
  end
end

Now… take the index action in your projects controller and change it from this:

  # GET /projects
  # GET /projects.xml
  def index
 
    @projects = Project.find :all

    respond_to do |format|
      format.html # index.html.erb
      format.xml  { render :xml => @projects}
    end

  end

to this:

  # GET /projects
  # GET /projects.xml
  # get /projects.rss
  def index
  
    @projects = Project.find :all, :order => "update_at desc"

    respond_to do |format|
      format.html # index.html.erb
      format.xml  { render :xml => @projects}
      format.rss do 
        @page_title = "Journal entries"
        render :layout=>false
      end

    end

  end

Start up your server and add some projects. You can then navigate to http://localhost:3000/projects.rss and see the feed.

Now we’re not going to talk about how to cache the feed, but that’s easy enough to figure out.

How do I do this right now, without using Edge rails

The steps are simple.

  1. Use this to generate your resource scaffold:
          ruby script\generate scaffold_resource Project title:string description:string
      
  2. Name the above mentioned RSS template file projects_rss.rxml
  3. Use this for your controller action.
      # GET /projects
      # GET /projects.xml
      # get /projects.rss
      def index
      
        @projects = Project.find :all, :order => "update_at desc"
    
        respond_to do |format|
          format.html # index.html.erb
          format.xml  { render :text => @projects.to_xml}
          format.rss do 
            @page_title = "Projects"
            render :action=>"projects_rss", :layout=>false
          end
    
        end
    
      end
    
« Previous PageNext Page »