Work around Rails 3 unit test transaction difficulties

What hath thou done to me, O Rails 3?

During a recent upgrade of one of our applications to Rails 3 I encountered some interesting (frustrating) errors while running the unit tests.

For reference, application was moving from 2.3.4 to 3.0.4, the database was PostgreSQL using the pg gem, and the unit test suite included shoulda + factory_girl. The tests were run with `rake test` and `autotest`.

Here is an example of some of the error messages I beheld:

4) Error:
test: A PatentSafe error email with suspect headers should create a ticket with the correct title. (EmailInTest):
ActiveRecord::StatementInvalid: PGError: ERROR: current transaction is aborted, commands ignored until end of transaction block
: SELECT a.attname, format_type(a.atttypid, a.atttypmod), d.adsrc, a.attnotnull
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"original_emails"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum

5) Error:
test: A received email note from a user should be assigned to the user if found. (EmailInTest):
ActiveRecord::StatementInvalid: PGError: ERROR: current transaction is aborted, commands ignored until end of transaction block
: SELECT "users"."id" FROM "users" WHERE ("users"."login_id" = '') AND ("users".id <> 9) LIMIT 1

31) Error:
test: An email re-sent to the notes address should return a single note. (MailReceiverTest):
ActiveRecord::StatementInvalid: PGError: ERROR: current transaction is aborted, commands ignored until end of transaction block
: SELECT "roles".* FROM "roles" WHERE (is_active = 't' and is_default = 't') LIMIT 1

As you can see, the errors aren’t really that similar except for the annoying transaction message and the traceback showed the errors weren’t coming from the same parts of the application. This happened so many times that it pains me to copy the messages here. What a weekend …

Amongst my attempted solutions were:

  1. commenting out transactions in the application
  2. turning off transactional_features in test_helper.rb with self.use_transactional_fixtures = true
  3. running the tests with rake test
  4. running the tests with autotest (not that I expected this to change my previous experience)
  5. switching to minitest with minishoulda
  6. starting a new rails 3 application with just a few tests

The most promising of which was starting a new application, but as soon as I had added most of the models, the errors returned. In all cases, the transaction errors were there, all 75 of them. Oh yes. I remember the count.

It occurred to me that I had read something about transaction and tests in the ActiveRecord documentation. From the doc:

Warning: one should not catch ActiveRecord::StatementInvalid exceptions inside a transaction block. StatementInvalid exceptions indicate that an error occurred at the database level, for example when a unique constraint is violated. On some database systems, such as PostgreSQL, database errors inside a transaction causes the entire transaction to become unusable until it’s restarted from the beginning.

Could the database be the problem? At least, in the short-term?

A quick edit to config/database.yml wasn’t too much to ask given all that I had already tried.

adapter: sqlite3
database: db/test.sqlite3
pool: 5
timeout: 5000

Lo! The true errors showed themselves unto me!

No, really.

As soon as I was on sqlite3 I could see the actual application errors that were causing me such trouble. My assumption is that the way sqlite handles transactions, without nesting, allows my application to keep on trucking when it encounters an error while wrapped in a unit test transaction.

If, you’re in a time crunch as I was, you can use this method to get your tests running and get the application upgraded but this isn’t a long-term solution. It leaves you vulnerable to differences in Sqlite and your production database, PostgreSQL in my case.

This solution raises more questions than it answers, however.

Why wasn’t I having this problem in Rails 2.3.4?

I’ve taken a quick poke around Rails and a lot has changed. It’s highly likely that my tests were passing in 2.3.4 as a fluke and not by design. Would that I could answer this question definitively.

Is the problem in the Rails test framework/runner or in ActiveRecord?

As this project needs to be upgraded now, I don’t have the luxury of working through this one. My initial attempts to use minitest for testing show that the transaction problem still existed so this makes me thing something bigger is at play.

Am I following the prescribed path with regards to Rails testing?

I copied the test setup over after setting up the new Rails 3 app and am using this as my template (mainly just test_helper.rb). It looks like I’m following the Rails Way™. I can’t shake the feeling that I’m doing something wrong in my tests.

Got any thoughts for me? Share them in the comments and help me get this figured out once and for all.