ruby

Creating a simple Sinatra app on Heroku /  by Paulo Fierro

I am a big fan of Sinatra and have been writing a few apps using this lightweight Ruby framework. Deploying them on Heroku is a breeze and painless but I always seem to forget a step so this is more of a note for future me.

I wrote a template that sets up a very basic Sinatra app that outputs JSON — perfect for when I need to set up a basic API for an iOS app. There's no database, so DataMapper is nowhere to be found and there's no tests either, just extreme simplicity.

Assumptions:

  1. you already have a Heroku account and have the Heroku Toolbelt installed
  2. you are on a Mac, though I'm sure its not too dissimilar for Windows folk
  3. You have the Bundler and Shotgun gems already installed

So with that out of the way,

  1. Download the template, and unzip it into your new apps' folder
  2. Navigate to the folder in Terminal and then run bundle install
  3. Check that everything is working by running shotgun. Then fire up a browser and navigate to http://127.0.0.1:9393
  4. Next we need to create the local git repository: git init
  5. Now we add all of the files: git add .
  6. Then we commit them: `git commit -m 'first commit'

Now we're ready to create the app on Heroku. For this example lets call it "myapp".

  1. In the Terminal create the app (the default stack nowadays is Cedar which is what we want): heroku apps:create myapp
  2. Because the git repository already exists, the Heroku toolbelt should have set up a remote for us, but lets double check: git remote -v
  3. Double check that the remotes look like heroku git@heroku.com:myapp.git for both fetch and push
  4. Deploy! git push heroku master

Now if you navigate to http://myapp.herokuapp.com you should see some successful JSON output.

Now go build your app!

Setting a photo's creation date to the date it was taken /  by Paulo Fierro

First a little back story.

The other day my mom called me up saying that her 11" MacBook Air was out of space and running slow wondering what we could do about it. Knowing that OS X runs pretty terribly without some free space I figured I'd run some of the Onyx cleanup scripts, check out where the space was being used up with Daisy Disk and get rid of some of the crap.

Onyx cleared a few gigs, and Daisy Disk said that iPhoto was the main culprit — the library being nearly 80GB,  67% of the total space on the disk.

Firing up iPhoto I created a Smart Album to see how much space videos were taking up.

The Smart Album conditions

Turns out there was nearly 10GB in videos, so I selected all of the items in the Movies album and exported them to an external disk. After removing these from the library, emptying the iPhoto trash and the actual Trash the disk had about 15GB free.

Success!

Not so fast

Of course not. After handing the laptop back my mom complained that iPhoto was running really sluggish. I got it back, checked it out and it was unusable. Scrolling had a 5-6 second lag and the UI responsiveness was generally pretty terrible.

My guess is that removing those videos screwed up the library somehow, so I opened up iPhoto holding down CMD+OPT to get the repair menu up and ran each item on the list (repairing permissions, rebuilding thumbnails, reindexing photos, repairing the library). Many, many hours later everything was still terrible so I decided to start from scratch.

My mom's iPhoto library is fairly large — nearly 15,000 photos. Initially I thought her 11" may not be able to handle that many photos so I figured exporting all of the photos and only importing the ones from the last two years would be a way of seeing if that was the case.

I left the laptop overnight exporting the 15,000 photos, backed up the library and deleted the original. Now I just had to find the photos taken within the last two years.

What should be easy...

Once again foiled. All of the exported photos had date of creation and modification set to yesterday. The sub-folders were all album names and not really indicative of when the photos were taken. I peeked inside the iPhoto library package contents to the Original and Modified folders where the photos live but found duplicates, terrible names and general cruft so I figured the actual exported photos had to be salvaged somehow.

Then I remembered EXIF data. Every photo you take contains metadata about what camera was used to take it, the model, ISO, exposure time, etc. One of those fields is the date it was taken. So if I could figure out how to read the date taken I could write a script to "fix" the photos.

ExifTool to the rescue!

With ExifTool in hand I went off to Automator and wrote some Ruby that would loop through a set of folders and change the date modified to the date the photo was taken as dictated by its EXIF data. It looked like so (download it here):

I tested this on a folder and it worked! I could follow along what was going on in Console.app and all was good.

Then I ran it on the main folder and encountered an interesting bug:

Run Shell Script failed - too many arguments.

Automator was passing in the path names to all of the photos to Ruby as arguments, but someone could only handle 4096 of these. Well below the 15,000 items I wanted to process.

Having spent enough time on this already I decided to be lazy and just run it a few times. Due to the logging to the Console I could see how far it had got and simply resume it from there with a next batch. There's also a bug where photos with single quotes or ampersands had to be renamed to be processed, which I could have fixed but again, lazy.

Now only the modified date was supposed to be changed, but I guess OS X doesn't allow a creation date to be later than its modified date so they both changed as a bonus.

Now for some Spotlight magic...

Now that the photos finally had their dates back I could make a simple search for files that were JPEGs and created in the last two years.

Importing these into a new iPhoto library took a while but once it was done scrolling was fluid and the app was generally responsive. Great! So I backed up this library and decided to try importing the rest of the photos.

A few hours later all of the photos were back in iPhoto and the app was still scrolling smoothly. What the fuck?

My guess is that removing the videos screwed up the original library in a way that iPhoto itself couldn't fix. Maybe deleting the video files themselves from the disk would have been better — leaving orphans in the iPhoto library. I have no idea.

In any case it was a learning experience and yet again proof that Automator is truly a hidden gem that I don't use enough.

You can download the Automator app (open it in Automator to edit it if you like), but remember you must have ExifTool installed first.

Setting up a fresh Lion for Ruby development /  by Paulo Fierro

Update (July 30, 2012):

If you've upgraded to Mountain Lion the below instructions still work. Once complete you should check out this guide. It contains a tip on installing gcc via brew in order to avoid using CLANG which can result in occasional errors when installing/updating gems.

--- End of update ---

A few weeks back I decided to take the plunge and wipe my machine and set it up from scratch. The main reason being I was running out of space on my SSD and I just wanted to see what a fresh Lion install would be like.

Let me tell you, its FAST! Totally worth it. It did take me some time to find the right resources and shave some yaks so this is more of a guide for future-me to save some time. If you do carry on reading from this point I assume you know what you're doing :)

So now that everything is wiped and fresh and new (backed up of course) the main task is setting up ye old dev environment. Lets start at the beginning.

Xcode

Head over to the Mac App Store and download Xcode. Once you install it you'll find that it no longer lives in /Developer. Its been moved to /Applications and its command line tools are no longer in the $PATH on the command line.

We don't have to add them ourselves, simply start up Xcode, go to Preferences and then Downloads > Components. Click on the "Install" button next to "Command Line Tools".

Heroku

If you're deploying stuff on Heroku, grab the toolbelt here.

Homebrew

Previously I was using MacPorts, but after hearing so many great things about Homebrew I took the plunge. Its simply a great package manager and you can download it here. For example, OS X doesn't ship with wget, so to install it you can simply run the following on the command line:

brew install wget

Done. Moving on.

RVM

RVM is a command-line tool which allows you to easily install, manage, and work with multiple ruby environments from interpreters to sets of gems.

What it says on the tin. It allows you to easily install multiple versions of Ruby on your system. It also allows you create gem sets and swap between them. To install:

bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)

Then:

source ~/.rvm/scripts/rvm

Now we're ready to install Ruby! The only problem is that with Xcode 4 the provided compiler (gcc) is LLVM based and not yet fully supported by Ruby and gems. Now before you go hunting for an older gcc (which I did) its not necessary.

To install Ruby 1.9.2 (which is required for Heroku) enter:

rvm install 1.9.2 --with-gcc=clang

When that is done, lets check what version of Ruby we have set as the system default:

ruby --version

Mine said Ruby 1.8.7. Lets set 1.9.2 to be default:

rvm use 1.9.2 --default

If you run ruby --version it should now say 1.9.2. If you ever want to go back to the system default simply enter rvm system. So now we have an easy way of swapping Ruby versions and you can read more about it here.

Sublime Text

Sublime Text is my favourite editor of all time. Get it here. Read some great tips and tricks here.

MySQL

Previously I was using MAMP. However I had an issue installing the do_mysql gem which is required for dm-mysql-adapter I was using with Datamapper.

Long story short I found the easiest thing to do was install mysql via brew instead of using MAMP. So to start off:

brew install mysql

Next to configure the installation (all one line):

mysql_install_db --verbose --basedir="$(brew --prefix mysql)" --datadir=/usr/local/var/mysql --tmpdir=/tmp

To get it to run on startup, first we have to create the directory if it doesn't exist:

mkdir -p ~/Library/LaunchAgents

Then we copy the mysql.plist over (the version number may have changed when you're reading this):

cp /usr/local/Cellar/mysql/5.5.20/homebrew.mxcl.mysql.plist ~/Library/LaunchAgents/

Then we tell launchctl to load it on startup:

launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist

Now to set the password:

mysqladmin -u root password {new-password}

Phew! Ok, now mysql is installed and running on port 3306. To create your first database simply enter:

mysqladmin -u root -p create {your-database-name}

And some gem love...

Finally I recommend using bundler for gem management, foreman to run your app (especially if you're on Heroku), rerun to rerun your app when something changes, datamapper for ORM and sinatra for happiness.

Good luck!

Automatically generating an HTML5 cache manifest /  by Paulo Fierro

When we were developing fotb.me we ensured the app would run offline using the fancy, new HTML5 app cache API. It can be a pain to create the cache manifest file by hand, this is obviously something that should be automated.

I looked around and found a plugin for Coda and a bundle for TextMate and TextWrangler which are good, but they don't allow me to ignore certain files and only add certain filetypes which were two things I needed. Not wanting to have to remember to edit the generated files manually I rolled our own.

CacheManifestGenerator is a simple Ruby script that traverses any folder recursively and makes a note of each file. You can add specific files to be ignored and the specific file extensions to look for. Very, very simple yet effective.

It also adds a comment with the time of creation. This is enough to trigger browsers to check the manifest contents as the file itself is different.

Oh and btw, Ruby is fun!

Update:

John Allsopp reminded me of manifestR, an online tool that creates a manifest for any page you are visiting via a bookmarklet. Very cool stuff, but it didn't work for us as the site has to be online, not be password protected (something we were avoiding pre-launch) and also the ignore files bit I mentioned above. However if you don't need those things then its definitely the easiest way of generating a manifest automatically.