“Clean in Business” Google+ Community

On January 22, 2014, in Company stuff, by Simon Coles

Following on from the interest in the Clean Language presentation, We’ve started a Google+ group for Clean Language in Business which people might be interested in https://plus.google.com/communities/100438077387632793745

Notably I’ve got some of my colleagues to join, so if you’ve got specific questions about what we do, it should get perspectives from people other than just me. Which I think is important.

In case you are wondering why a new group, and why Google+? I’m no Google fan, but I do know people dislike using Facebook for work stuff – and indeed some people are very anti-Facebook and won’t use it. But most people have Google+ either personally or through work.

In addition, most of our people view LinkedIn as primarily a Sales tool or a job hunting tool. Some of us are at the limit of the number of Groups we can be a member of.

So Google+ seemed a good option. I’ve no idea what will happen but I figured if we were going to be more public about things Google+ was the place to try.

Tagged with:  

Clean Questioning in Software Companies

On January 20, 2014, in Company stuff, by Simon Coles

A case study on Amphora’s use of Clean Questions and Metaphor, working with Caitlin Walker of Training Attention.

 

On a new Rails app, we tried to install both ActiveScaffold and ActiveAdmin.

In both cases we got:

couldn’t find file ‘jquery-ui

When trying to access the ActiveScaffold or ActiveAdmin pages.

Turns out the later versions of JQuery remove some things which are still required in Rails 3.2. So locking the version in the Gemfile does the trick e.g. change the gem line to be

gem 'jquery-rails', '2.1.4'

 

I’m sure this will get addressed in time… but at the time of writing, that line (followed by “bundle”) allows Rails 3.2 to work with ActiveScaffold and ActiveAdmin.

 

Path issues when using Capistrano

On October 14, 2011, in Ruby on Rails, Ubuntu, by Simon Coles

I recently started using Capistrano on a personal project, which is hosted on a relatively plain Ubuntu box.

Capistrano wouldn’t do much, I kept getting errors about “bundle” not being found. Seems the newer versions of Rails and Capistrano prefix everything with “bundle” so the right Gems etc. are used.

Turns out there’s a problem with the path that my Capistrano scripts get when they ssh into the Linux machine. Seems there’s a different environment for interactive SSH sessions, and the non-interactive SSH sessions that Capistrano uses.

To diagnose this, I added this to my deploy.rb:


desc "Echo environment vars"
namespace :env do
task :echo do
run "echo printing out cap info on remote server"
run "echo $PATH"
run "printenv"
end
end

Which when I do cap env:echo tells me the path that my Capistrano scripts see – which crucially doesn’t include /var/lib/gems/1.8/bin.

Turns out you set the environment in ~/.ssh/environment, adding a PATH line in the normal way.

But then you need to edit /etc/ssh/sshd_config and add a line

PermitUserEnvironment yes

And restart SSH.

After that your Cap deployments will be smooth…

 

Breakfast of champions

Have you ever had a really fun day? A day that starts like this:

Feb 24, 2011 10:50:42 PM org.apache.catalina.core.ApplicationContext log
SEVERE: unable to create shared application instance
org.jruby.rack.RackInitializationException: stack level too deep
from c:/console/ROOT/WEB-INF/gems/gems/bundler-1.0.10/lib/bundler/resolver.rb:348:in `catch'
from c:/console/ROOT/WEB-INF/gems/gems/bundler-1.0.10/lib/bundler/resolver.rb:348:in `resolve_requirement'
from c:/console/ROOT/WEB-INF/gems/gems/bundler-1.0.10/lib/bundler/resolver.rb:299:in `resolve'
from c:/console/ROOT/WEB-INF/gems/gems/bundler-1.0.10/lib/bundler/resolver.rb:298:in `reverse_each'
from c:/console/ROOT/WEB-INF/gems/gems/bundler-1.0.10/lib/bundler/resolver.rb:298:in `resolve'
from c:/console/ROOT/WEB-INF/gems/gems/bundler-1.0.10/lib/bundler/resolver.rb:349:in `resolve_requirement'
from c:/console/ROOT/WEB-INF/gems/gems/bundler-1.0.10/lib/bundler/resolver.rb:348:in `catch'
from c:/console/ROOT/WEB-INF/gems/gems/bundler-1.0.10/lib/bundler/resolver.rb:348:in `resolve_requirement'
... 213 levels...
from file:/C:/console/ROOT/WEB-INF/lib/jruby-rack-1.0.3.jar!/vendor/rack-1.2.1/rack/builder.rb:46:in `initialize'
from <script>:2:in `new'
from <script>:2

This was how my morning looked a week or two ago when I was starting up an installation of our Custodian’s Console on a Windows server. Take another look at that stack trace. (full trace here) It looks like bundler is causing some trouble in this app, right? In fact, it reads like bundler is in some sort of recursive blackhole and JRuby just gives up – as it should. Let’s see what Google has to say about this.

First search: JRuby, bundler and this error message

The results for my first search include a Warbler issue. WARBLER-21. In the comments, Nick Sieger recommends increasing the stack size:

“We’ve heard increased reports of “stack level too deep” with Bundler lately. In fact, we bumped up the default stack size to -Xss2048k in JRuby master as of ea20ac17. Can you try running Warbler as follows:

jruby -J-Xss2048k -S warble ...

Sounds perfect. This could be the answer… annnnnd nope. I try making these changes with no effect. In fact, I try passing in stack size changes in just about every conceivable way. (In the Tomcat docs they point out that you can also pass in an increased stack size with the -JvmSs option to the service itself.)

Hrm. Case of bad Google-Fu?

What about JRuby itself? Or Bundler?

I’m able to find another mention of this problem pretty quickly. This JRuby issue speaks about the *exact* same error. No suggested solution but to upgrade JRuby to 1.6 RC1 and bundler to latest (1.0.10)

Okay. I’ll bite.

I upgrade bundler. Same error.

Then I try using JRuby 1.6 RC1. No love.

I’m going to have to dig deeper – it could be 3 or 4 pages of Google search results before I find my answer!

Liferay?

After several pages of results and no luck I stumble across some postings on a message board (yuck!) for something called Liferay. Never heard of it. Turns out it has something to do with “market leading”, “portals” and “enterprises”. Oh, and message boards.

One such post talks about the same error I’m having and a working solution. Sort of.

I found out some further information. I am running Tomcat 6 as a service through the 64-bit Tomcatw.exe (procrun implementation). If I use the “jvm” startmode I get the above error. If I use the “java” startmode I don’t. Wanting to know why still.

That poor guy made the last post in that thread. I really dislike message boards.

Startmode, what’s that all about? Turns out that Tomcat Tomcat uses “procrun” to allow it to run as a service on Windows. Ahhhh, a mention of Windows. I like where this is going.

“Procrun is a set of applications that allow Windows users to wrap (mostly) Java applications (e.g. Tomcat) as a Windows service. The service can be set to automatically start when the machine boots and will continue to run with no user logged onto the machine.”

From the procrun docs:

“startmode” is here:

One of jvm, Java or exe. The modes are:
jvm – start Java in-process. Depends on jvm.dll, see –Jvm.
Java – same as exe, but automatically uses the default Java executable, i.e. %JAVA_HOME%\bin\java.exe. Make sure JAVA_HOME is set correctly, or use –JavaHome to provide the correct location. If neither is set, procrun will try to find the default JDK (not JRE) from the Windows registry.
exe – run the image as a separate process

Confirmed. Starting in ‘java’ startmode allows the console to start up but shutdown via the service console is now broken and the process runs as java.exe instead of tomcat6.exe. That’s a start but not really a long-term solution. Why would this be working when running as a separate process?

That question leads me to spend several hours trying to figure out if my parameters are actually being passed into the JVM. If you are trying to do the same let me save you some time. Edit the line in service.bat that looks like this:"%EXECUTABLE%" //IS//%SERVICE_NAME% to include --LogLevel Debug at the end. This little change will dump lots of useful information to the catalina or commons-daemon log.

If I set the startmode to java I can get things to run, just not the way we need or want. Time to try something else.

Back to the Liferay messageboards! Hooray.

Here is a post that actually proposes a solution to the problem:

The last post says:

“Tomcat 6 ships with v1.0.2 of Commons Daemon (in the form of tomcat6.exe and tomcat6w.exe). It seems this version is not compatible on certain combinations of 64-bit Windows and JDK6.

To resolve this, simply download v1.0.4 from http://commons.apache.org/daemon/download_daemon.cgi, replace tomcat6.exe and tomcat6w.exe with prunsrv.exe and prunmgr.exe respectively, rename to tomcat6.exe and tomcat6w.exe and run service.bat.”

THAT’S what I’m talking about! I follow the proposal and it works. Neat-o.

Onward to a permanent solution

The newest procrun (v1.0.5) works just fine on Windows Server 2008 64bit and 32bit. Although I can’t find the specific bug fix in daemons (procrun) JIRA, I can confirm that it works. I see some vague references to a reorg and some other moves happening in the code but nothing concrete. I’d rather not have to maintain versions of procrun along with Tomcat so I go hunting for Tomcat updates … Tomcat has since released 6.0.32 which includes commons daemon (procrun) v1.0.5 since version 6.0.30 – the console is currently on 6.0.29 so an upgrade fixes the problem.

If you are fighting with these kinds of issues with JRuby and Tomcat (or just Tomcat) it would behoove you to try a newer version of Tomcat first, they update regularly and pull in upstream changes for other components. You can check the Tomcat 6.0 changelog for details. For instance, the procrun update message went something like this:

Update to Commons Daemon 1.0.5. (mturk)

Good hunting!

 

What hath thou done to me, O Rails 3?

During a recent upgrade of one of our applications to Rails 3 I encountered some interesting (frustrating) errors while running the unit tests.

For reference, application was moving from 2.3.4 to 3.0.4, the database was PostgreSQL using the pg gem, and the unit test suite included shoulda + factory_girl. The tests were run with `rake test` and `autotest`.

Here is an example of some of the error messages I beheld:

4) Error:
test: A PatentSafe error email with suspect headers should create a ticket with the correct title. (EmailInTest):
ActiveRecord::StatementInvalid: PGError: ERROR: current transaction is aborted, commands ignored until end of transaction block
: SELECT a.attname, format_type(a.atttypid, a.atttypmod), d.adsrc, a.attnotnull
FROM pg_attribute a LEFT JOIN pg_attrdef d
ON a.attrelid = d.adrelid AND a.attnum = d.adnum
WHERE a.attrelid = '"original_emails"'::regclass
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum


5) Error:
test: A received email note from a user should be assigned to the user if found. (EmailInTest):
ActiveRecord::StatementInvalid: PGError: ERROR: current transaction is aborted, commands ignored until end of transaction block
: SELECT "users"."id" FROM "users" WHERE ("users"."login_id" = 'some-email@amphora-research.com') AND ("users".id <> 9) LIMIT 1


31) Error:
test: An email re-sent to the notes address should return a single note. (MailReceiverTest):
ActiveRecord::StatementInvalid: PGError: ERROR: current transaction is aborted, commands ignored until end of transaction block
: SELECT "roles".* FROM "roles" WHERE (is_active = 't' and is_default = 't') LIMIT 1

As you can see, the errors aren’t really that similar except for the annoying transaction message and the traceback showed the errors weren’t coming from the same parts of the application. This happened so many times that it pains me to copy the messages here. What a weekend …

Amongst my attempted solutions were:

  1. commenting out transactions in the application
  2. turning off transactional_features in test_helper.rb with self.use_transactional_fixtures = true
  3. running the tests with rake test
  4. running the tests with autotest (not that I expected this to change my previous experience)
  5. switching to minitest with minishoulda
  6. starting a new rails 3 application with just a few tests

The most promising of which was starting a new application, but as soon as I had added most of the models, the errors returned. In all cases, the transaction errors were there, all 75 of them. Oh yes. I remember the count.

It occurred to me that I had read something about transaction and tests in the ActiveRecord documentation. From the doc:

Warning: one should not catch ActiveRecord::StatementInvalid exceptions inside a transaction block. StatementInvalid exceptions indicate that an error occurred at the database level, for example when a unique constraint is violated. On some database systems, such as PostgreSQL, database errors inside a transaction causes the entire transaction to become unusable until it’s restarted from the beginning.

Could the database be the problem? At least, in the short-term?

A quick edit to config/database.yml wasn’t too much to ask given all that I had already tried.


test:
adapter: sqlite3
database: db/test.sqlite3
pool: 5
timeout: 5000

Lo! The true errors showed themselves unto me!

No, really.

As soon as I was on sqlite3 I could see the actual application errors that were causing me such trouble. My assumption is that the way sqlite handles transactions, without nesting, allows my application to keep on trucking when it encounters an error while wrapped in a unit test transaction.

If, you’re in a time crunch as I was, you can use this method to get your tests running and get the application upgraded but this isn’t a long-term solution. It leaves you vulnerable to differences in Sqlite and your production database, PostgreSQL in my case.

This solution raises more questions than it answers, however.

Why wasn’t I having this problem in Rails 2.3.4?

I’ve taken a quick poke around Rails and a lot has changed. It’s highly likely that my tests were passing in 2.3.4 as a fluke and not by design. Would that I could answer this question definitively.

Is the problem in the Rails test framework/runner or in ActiveRecord?

As this project needs to be upgraded now, I don’t have the luxury of working through this one. My initial attempts to use minitest for testing show that the transaction problem still existed so this makes me thing something bigger is at play.

Am I following the prescribed path with regards to Rails testing?

I copied the test setup over after setting up the new Rails 3 app and am using this as my template (mainly just test_helper.rb). It looks like I’m following the Rails Way™. I can’t shake the feeling that I’m doing something wrong in my tests.

Got any thoughts for me? Share them in the comments and help me get this figured out once and for all.

 

A Ruby pre-defined variable and you. (and JRuby)

On December 16, 2010, in Development, JRuby, by Don Morrison

Strangeness

I was recently wrestling with an obscure JDBC bug in Sequel. I was seeing an error connecting to the PostgreSQL database but it was being rescued by the Sequel JDBC adapter and hidden a little bit. (If you’re curious to see the fix it’s over on Github). Anyway, the original error led me to believe that the JDBC adapter for postgres wasn’t being loaded at all.

What to do?

What else? Get on the #jruby IRC channel on Freenode. Seriously, when you’re running JRuby and you come across a problem you go to there. Charles, Nick, and Tom (JRuby core) are always lurking around and very willing to help. Charles told me to look at $” and see what was loaded.

$”

A magic-pre-defined variable in Ruby (and JRuby) – see what I did there with the title? According to zenspider’s QuickRef:

$" The array contains the module names loaded by require.

Huh. Okay, cool. As a bonus JRuby throws the loaded jars in there as well. ( I really should learn all of these, come to think it, so should you. )

So what

You take a look at that array and you’ll see all of the modules names currently loaded. Good stuff. Here is the part from the project that helped me resolve my issue:

jdbc/postgres.rb
postgresql-8.4-702.jdbc4.jar
arjdbc/postgresql/connection_methods.rb
active_record/connection_adapters/postgresql_adapter.rb
arjdbc/postgresql/adapter.rb

I was able to see that the postgresql jar was loaded along with the supporting cast of characters. This led me to the #sequel IRC channel where Jeremy Evans and I were able to figure this little guy out. Trying to troubleshoot a potential jar loading problem in JRuby? $” is your friend. (and so is IRC it seems!)

Tagged with:  

A word to the wise: pg 0.10.0 doesn’t want to compile on OpenSolaris.

The Stack

One of our production servers is running OpenSolaris version OpenSolaris 2008.11 snv_101b_rc2 X86, Ruby 1.8.7 (2010-08-16 patchlevel 302) [i386-solaris2.11], and PostgreSQL 8.3.4. We use Capistrano for deployment with some bundler goodness thrown in the mix.

The Problem

Here’s what we see when bundler tries to build the gem:

ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)

/usr/local/rubies/ruby-1.8.7-p302/bin/ruby extconf.rb
checking for pg_config... yes
checking for libpq-fe.h... yes
checking for libpq/libpq-fs.h... yes
checking for PQconnectdb() in -lpq... no
checking for PQconnectdb() in -llibpq... no
checking for PQconnectdb() in -lms/libpq... no
Can't find the PostgreSQL client library (libpq)
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/usr/local/rubies/ruby-1.8.7-p302/bin/ruby
        --with-pg
        --without-pg
        --with-pg-config
        --without-pg-config
        --with-pg-dir
        --without-pg-dir
        --with-pg-include
        --without-pg-include=${pg-dir}/include
        --with-pg-lib
        --without-pg-lib=${pg-dir}/lib
        --with-pqlib
        --without-pqlib
        --with-libpqlib
        --without-libpqlib
        --with-ms/libpqlib
        --without-ms/libpqlib

Oh yeah! Good stuff right there.

The Investigation

At first, I thought the problems were to do with the development libraries being missing. This is frequently the case, and the error message pretty much says just that. Wrong – libpq was (and is) present. In fact, prior to the pg gem, we had the postgres gem running and it had the same dependencies. Next up: must be a mixup in the deployment situation. Not so. Capistrano and bundler both checked out okay.

After my searches on The Google provided ample proof that compilation on a Solaris box can be tricky, my next suspect was a wonky compiler toolchain. Nope, toolchain was just fine. Heck, I even checked the rbconfig.rb to make sure. That’s right, I’m thorough.

The (Temporary) Solution

On a lark, and after a day of trying all kinds of weird little tweaks … I figured, what about 0.9 ??? A quick look through the commits on 0.10.0 showed some recent work on the build system. Why not? Quick, change the Gemfile:

source 'http://rubygems.org'

gem 'rails', '3.0.3'
gem 'pg', '~> 0.9.0'

# That '~>' operator tells Bundler/Rubygems to allow gem updates to
# the least significant portion of the version number for that gem. 

# In the Gemfile snippet above, I am signifying that I will allow any pg version
# in the 0.9.x branch. Had I specified '~> 0.9', bundler would install
# the newest pg gem with a major version of '0.'.

Then

  > bundle install
  > git add .
  > git commit -m "Attempt to use pg 0.9"
  > git push
  > cap deploy

SUCCESS!

I just opened this issue on the pg project and will update this space when we come up with a solution. For now, just install the pg gem from the 0.9.x version.

Epilogue

You might be wondering why I would care if I’m running this particular gem for Postgres connectivity. There are three postgres gems out there that I know of: pg, postgres, and postgres-pr. Let’s take a look at some stats from rubygems.org:

gem                 version             date            downloads
----------------    -----------------   -------------   ----------
postgres (native)   0.7.9.2008.01.28    Jan 28, 2008          49K
postgres-pr (ruby)  0.6.3               Dec 14, 2009          35K
pg (native)         0.10.0              Dec 01, 2010         210K


Even taking the subjective versioning and download numbers, I want us to use a library that is actively maintained and well used. Michael Granger is all over pg and having the most downloads by far, gives me the warm fuzzy that errors will have been encountered and (hopefully) fixed. Since we run on a platform that can use the native gems, pg is the obvious choice – for us. Of course, as they say, your mileage may vary.

UPDATE Dec. 15, 2010:

The issue has been updated – looks like this isn’t an isolated problem.

Tagged with:  

Apple iPad print server in Pure Java

On November 23, 2010, in iPad, by Simon Coles

Apple have just released iOS 4.2 which includes the ability to print. As part of PatentSafe we have a IPP print server written in Java which can accept print jobs from the iPad and other iOS devices. We also have some Java code which advertises the printer to iOS devices either locally or via wide-area Bonjour.

The net result of this is we have a pure Java print server which you can print to from an iOS device. You can print from your iPad to the server, which will produce a PDF as a result.

We’re aware this might be of interest to other people; if you have an application where you feel it might be of use, drop us a line and we can discuss… and if you need help writing/perfecting your own iPad print server I am sure we can help.

Tagged with:  

Pound 2.5 and the Include directive

On June 10, 2010, in Uncategorized, by Simon Coles

We’re very happy with Pound as a front-end Proxy to our hosted PatentSafe servers. In the version 2.5 release notes there is mention of an Include directive which is clearly of interest because our configuration file is getting rather large! Turns out there isn’t much documentation around so here’s what we understand having read the source…

In our main pound.cfg file we have a line like this:

Include "/etc/pound/morescience.config"

The morescience.config file contains just what we’d normally put in the pound.cfg file.

Seems to work fine, we haven’t tried with anything other than a full pathname in quotes.

Hope that helps someone who also can’t find the Include directive in Pound’s documentation…

 
This website uses a Hackadelic PlugIn, Hackadelic Sliding Notes 1.6.5.