Marty Andrews

artful code

Tuesday, October 25, 2011

A ruby spec profiling story

I've always used ruby-prof in the past to profile my Ruby code, but a colleague yesterday put me on to the Google perftools for Ruby code. A bit of research turned up some interesting things.

According to http://blog.pluron.com/2008/01/ruby-on-rails-i.html,
ruby-prof attributes full cost of garbage collection to the method where GC gets triggered, not the methods that allocate the memory

That was news to me. I tried running the profiler over an app I'm working on, and got this:

Total: 23530 samples
13244 56.3% 56.3% 13244 56.3% garbage_collector
4223 17.9% 74.2% 4331 18.4% ActiveRecord::ConnectionAdapters::Mysql2Adapter#execute
678 2.9% 77.1% 678 2.9% ActiveRecord::ConnectionAdapters::Mysql2Adapter#active?
402 1.7% 78.8% 403 1.7% MonitorMixin::ConditionVariable#signal
258 1.1% 79.9% 294 1.2% Method#call

Wow! That's a massive amount of time in GC! What can I do about that?

That lead me to an article by Jamis Buck at http://37signals.com/svn/posts/2742-the-road-to-faster-tests. It describes a way to address GC issues for Test::Unit, but there's a nice rspec version at https://gist.github.com/790118. I applied that to my code and ran it again. The new profile was:

Total: 11331 samples
3730 32.9% 32.9% 3804 33.6% ActiveRecord::ConnectionAdapters::Mysql2Adapter#execute
2021 17.8% 50.8% 2021 17.8% garbage_collector
720 6.4% 57.1% 720 6.4% ActiveRecord::ConnectionAdapters::Mysql2Adapter#active?
376 3.3% 60.4% 376 3.3% MonitorMixin::ConditionVariable#signal
352 3.1% 63.5% 353 3.1% Object#reconsider_gc_deferment

My spec build time also cam down from 375 seconds to 238 seconds. That's a massive improvement.

The post form Jamis also mentions cleaning up some instance variables, which I'll try amongst some further tweaking, but that improvement was significant enough that I figured it might point someone else in the right direction too.

Thursday, January 6, 2011

Collaboration Day

Many IT companies have embraced the idea of giving their staff some autonomy in working on something of their own without any constraints. Perhaps most famous are Atlassians FedEx day and Googles 20% time. Those companies, and others who are trying the idea, are beginning to reap the rewards in terms of innovations that have evolved into more concrete solutions.

At Cogent, we're trying to embrace that idea too, with a spin on it that matches the style of work we do and the values we have internally. Firstly, we're a consulting company. Most of the work we do is spent on a customer site, and they pay us for the hours we're there. We need to balance the customers needs with any internal activities we have. Secondly, our staff all want to work with each other. For most of us, that's a big factor for why we joined Cogent in the first place. Finally, we're a family friendly company that tries very hard to support a balance between home life and time at work.

The implementation we have settled on is an idea we call Collaboration Day. One day a month, everyone in the company takes the day off from consulting and comes to spend it in the office. It's booked well in advance, so our customers are aware of what is going on. The day is a normal working day, so nobody is expected to take any time off or work after hours to be there.

On the morning of the day, everyone brainstorms ideas for things to work on and we put them together on a planning wall. They might be expected to take anywhere from an hour up to the whole day. People then sign up to work on things and get started. You can switch between ideas as many times as you like during the day.

There's only one rule: you cannot work alone.

At the end of the day, we all get together and have a short presentation about what people worked on and do a demo of what was built.

For Cogent, Collaboration Day is working really well so far. Many demos spawn new ideas for the next Collaboration Day, and some inspire people to do some more in their own time. A few are coming up for discussion about Cogent sponsoring more time to evolve them further. Those outcomes are exactly what we're hoping for.