warning: Creating default object from empty value in /home/gwolf/drupal6/modules/taxonomy/ on line 33.

Back with Ruby on Debian polemics

Submitted by gwolf on Thu, 01/06/2011 - 09:38

Once again, a polemic regarding how to properly integrate the Ruby language and libraries with the Debian distribution has been ignited. Similar arguments were presented in November and December 2008 and September 2010 — And excuse me for just refering to my own blog, but there are links to some other posts from there... and there will surely be many others I just missed. There were even nice collaboration attempts (such as DebGem, announced in January 2009... But which apparently never stuck on, as it is still listed as in its public Beta period and lists only support for Debian 4.0 and Ubuntu 8.04 and 8.10).

Some days ago, while I was on vacation, I received a mail from Lucas Nussbaum expressing the burnout he had been suffering from this situation. Some days after that, he posted a corresponding blog post - Giving up on Ruby packaging. The comments following his post are most interesting — The one comment I'd like to highlight (noting I skimmed a whole deal of them) is by Paul Brannan, one of the original RubyGems authors. Yes, possibly the design criteria for Gems included some mutually-conflicting goals, and they cover some of Debian's goals. Unfortunately, this conflicting criteria... Was resolved to the opposite way we would have wanted (specifically, that a sysadmin should be able to install software with the same tools he was already familiar with). And of course, there are deeper disagreements, which are rightly stemmed from different priorities: Debian (+derivatives) is meant to make user's and sysadmin's lifes simpler. Gems are meant to make developers' lifes simpler. And while all developers are some sort of users, and sysadminsthe reverse is (obviously) not so.

A developer maintaining several different codebases will surely suffer (well, this particular developer has surely suffered) if he has to maintain them all using coherent versions of system libraries. When a system is programmed, its developers want to use the greatest, latest tools to offer the best experience and functionality. And it's natural for libraries to evolve over time. However, any sysadmin will grind their teeth at the prospect of having many different versions of libraries, as the Gem model proposes. Why? Well, I was bitten by a minor example of it — Bug fixes do not get backported. Of course, if simple bugs are not backported, security and stability of the system as a whole suffers. Often, new software versions will not only add functionality, but improve on how things are done. Not only bugs get corrected, but we get better response times, better memory handling, etc.

It might sound harsh to say this, and even more, as a developer it feels I am talking against myself — But while development time should be minimized during the system design/implementation, once systems are in production, it should be used to allow for friendlier sysadmin relations.

I have to wear both hats at my real-life job. So, what is my take on this? Of course, blaming myself for choosing the wrong version is a no-go. When evaluating a library before using it in my projects, I try to look at its history and API-stability. APIs that change often speak about immature libraries which are still trying to get the right way to implement their functionality; just-started projects might be great and revolutionary, but they do not yet show any kind of long-term committment... And can lead (and have often led me) to painful rewrites when the Next Big Thing is reached.

As for the users: Even if your favorite language has the best and friendliest distribution method, users just do not care about what language was a particular piece of software implemented in. Users want to be able to install and uninstall with their system's usual tools. Directly using Gems, CPAN, PEAR and whatnot is just unnatural for users, however convenient they are for developers. Distributions offer a non-technical advantage in this regard as well: A human filter. As an example, as I write this there are 19392 Ruby Gems and 19092 Perl modules in CPAN (note that CPAN stores some older versions but discourages authors from keeping too many - So no, they are still very different in size). Debian has around 30,000 packages. Why is that? Because all Debian packages _must_ be human-generated, human-reviewed, human-submitted. This means, a person must think each packaged piece of code is worth packaging, is stable enough and provides value to users as it is, and is fit for being part of a stable release. I am not saying with this that 90% of CPAN and Gems are crap — I am saying that they are probably early implementations, to be installed, tested and improved by developers, and still not apt for general public use. Or maybe not interesting enough to be packaged as a service for the non-techie public at large.

Oh, and one last point: Ruby is not a new language anymore. It is a mature, powerful language with different implementations, every time more stable... But it is a language deeply affected by the (not so new anymore either) the appearance of Rails. Why do I say this? Because although the language is in no way tied to Web development, many of its strongest uses are Web-oriented. How does this affect the current discussion? Well, because many people argue that users are no longer needed to install the software. Web systems are installed by sysadmins and used via a Web browser, and sysadmins are expected be more skilled than casual users. Still, in Debian (and in other distributions, surely) we try to make sysadmin's lives simpler — I have (again, talking out of personal experience) installed several webapps (and system tools, and whatnot) for which I never followed any instructions besides aptitude install foo — Using different languages, frameworks, and so on. Can I troubleshoot their installs? Probably, as there is a common logic for how the distribution I have chosen and specialized in works. Can I find causes for bugs in them? Possibly, although there are some languages and frameworks I dare not touch. Can I get help on getting them out of a tight spot? Surely, as there is a central bug tracking system for my distribution — And one of the maintainer's tasks is to separate the problems related to the distribution (packaging, installing, simple user questions and misconceptions) to those derived from real bugs upstream.

Anyway — I am not saying that our way is the best way. No, by a long shot. Again, developers should have an easy, convenient way for installing whatever they want to play with. And to publish it without jumping through hoops. With this post, I'm only trying to express –again– why Debian works the way it does. And hope for better cooperation in the future.

And as for several comments of what I read in Lucas' post, I think that there is interest for this synergy among some of the most committed Ruby people.

( categories: )

Ruby dissonance with Debian, again

Submitted by gwolf on Wed, 09/29/2010 - 11:48

Lucas has written two long, insightful posts on the frustration about the social aspects of integrating Ruby stuff Debian – The first one, on September 12 and the second one, on September 29.

I cannot really blame (thoroughly) the Ruby guys for their position. After all, they have a vibrant community, and they are advancing great pieces of work. And they know who that code is meant for — Fellow programmers. And yes, although it is a pain to follow their API changes (and several of the Gems I regularly use do often get refactorings and functionality enhancements which break compatibility but introduce very nice new features), they say that's solved with one of Gems' main features being the simultaneous installability of different versions.

The key difference in Debian's worldview with Ruby's is they cater to Fellow programmers. Even leaving aside heaps of different positions and worldview/mindset, we have a fundamental difference: Debian cares about its users, whatever that means. So, our users should not even care what language a given application is implemented in – They should only care that it works. We, as packagers, should take care of all the infrastructural stuff.

And yes, that's where we find the conflicting spot: We don't want to ship many versions of a system library (that in this case would be a Gem). Specially if later versions fix known bugs in earlier versions and backports are not available or supported. Specially if upstream authors' only response to a bug in an older release will be "upgrade and rewrite whatever breaks in your application".

As an example of this, I am not currently updating the gems I maintain, as Debian is on a freeze to get out the next stable release. Or if at all, I am targetting those uploads to our Experimental branch, in order not to create a huge backlog for me when the freeze is over (just a series of rebuilds targetted at unstable). And yes, I will have to be responsible for any bugs that will most likely not be supported by most of my upstreams during the next ~2 years.

That's the role of a Linux distribution. And yes, as Lucas writes in the comments he got as responses to the first post – This dissonance comes in no small part because the Ruby developer community is mostly made from non-linuxers. People coming from a background where (mostly propietary) applications bundle up everything they need, where static linking is more popular than dynamic libraries, where there is no coordination between parts of the system are much less likely to understand our work.

And yes, the Perl community is a joy to work with in this regard. And that's the same I understand from the Python one. Because of their origins and where their main strength was grown and remains.

PS - And yes, I will join the flock of people saying that... The specific person that attacked your work is a great programmer, but well known as intolerant and obnoxious. Fortunately, even if our respective cultures fail to mix so much, most of our interactions just end with a "sigh" of lack of understanding, and not with the flames you got targetted with :-/

( categories: )

RubyCamp Mexico

Submitted by gwolf on Fri, 06/05/2009 - 18:07

RubyCamp UNAM logo Today I attended RubyCamp (schedule available), organized by my friends at Instituto de Física, at my University. I presented three plugins I made and regularly use.

The organizers followed a short, more informal scheme than most conferences I am used to — Talk slots range between 5 and 15 minutes, so we attended a whole day of semi-lightning talks. Of course, many people have run late, and although there was quite a bit of free space in the schedule, it has been practically non-stop — I was thinking on also giving a talk on encodings (as many people really still don't understand what is UTF-8, what is Latin1, why all that mess — People, please learn at least The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)), but the schedule is full by now. Interesting and successful experiment, I'd say. The talks are being taped, and the organizers say they will be soon made available online.

There are many cultural details to note here. First of all, yes, the Rails Fanboys _are_ a cult/sect. I think we have ~90% of MacBooks... I still fail to understand why a coder feels at ease on MacOS. I deeply despise it! ☻ Also, most of this community have bitten the Twitter plague. This is also a community very much into businessspeak, speaking a word in English for each two words in Spanish (I try to be consistent, not mixing languages at least). Some conferences have been quite business- and enterpeneur-oriented. Although I should not complain too much about this, as it is an important aspect for many people — But I still don't feel at ease having talks on how to run a business if we were asked for presentations on technical aspects!

Anyway - I was quite happy to be here. This is the first real technical, code-oriented conference I have attended in a long time in Mexico. And we need more like this! We have too many entry-level, evangelization-oriented conferences, but very few like this one.

[update]: Group photo!

( categories: )

Hyperdimensional strings

Submitted by gwolf on Wed, 01/14/2009 - 17:41

I am stunned no more people have been bitten by this. Or at least, the Intarweb has not heard about it. Censorship perhaps? I haven't researched more into the causes, but anyway...
I was pushing a project I have had lingering for some time from Rails 2.0.x to 2.1.x (yes, 2.2 is already out there, but 2.1 is the version that will ship with Lenny) - The changes should not be too invasive, as it is a minor release, but there are some quite noticeable changes.
Anyway... What was the problem? Take this very simple migration:

  1. class CreatePeople < ActiveRecord::Migration
  2. def self.up
  3. create_table :people do |t|
  4. t.column :login, :string, :null => false
  5. t.column :passwd, :string, :null => false
  6. t.column :firstname, :string, :null => false
  7. t.column :famname, :string, :null => false
  8. t.column :email, :string
  10. t.column :pw_salt, :string
  11. t.column :created_at, :timestamp
  12. t.column :last_login_at, :timestamp
  13. end
  14. end
  16. def self.down
  17. drop_table :people
  18. end
  19. end

The problem is that PostgreSQL refuses to create a hyperdimensional string field. I offer this here to you, line-wrapped by me for your convenience.
  1. PGError: ERROR: syntax error at OR near "("
  2. LINE 1: ...serial PRIMARY KEY, "login" character varying(255)(255) NOT ...
  3. ^
  4. : CREATE TABLE "people" ("id" serial PRIMARY KEY,
  5. "login" character varying(255)(255) NOT NULL,
  6. "passwd" character varying(255)(255)(255) NOT NULL,
  7. "firstname" character varying(255)(255)(255)(255) NOT NULL,
  8. "famname" character varying(255)(255)(255)(255)(255) NOT NULL,
  9. "email" character varying(255)(255)(255)(255)(255)(255) DEFAULT NULL NULL,
  10. "pw_salt" character varying(255)(255)(255)(255)(255)(255)(255) DEFAULT NULL NULL,
  11. "created_at" timestamp DEFAULT NULL NULL, "last_login_at" timestamp DEFAULT NULL NULL)

Beautiful. Now I can store strings not only as character vectors, but as planes, cubes, hypercubes, and any other hyperdimensional construct! Are we approaching quantum computers?
What is really striking is that... I found only one occurrence on tha net of this bug - one and a half years ago, in Ola Bini's blog. No stunned users looking for the culprit, no further reports... Strange.
Still, the bug was fixed in Rails 2.2 about half a year ago, although not in revisions of earlier versions. I will request the patch to be applied to earlier versions as well. Sigh.

( categories: )

DebGem is on its way

Submitted by gwolf on Mon, 01/05/2009 - 18:15

This morning, I got a mail that made me very happy. I followed it up a bit, and some hours later, echoed it over the pkg-ruby-extras list. And, yes, a blog posting won't hurt :)
I have done some rantings on why it is so painful to integrate cultures such as Debian and Rails. Those rants were part of quite a large rant-net and attracted a fair share of traffic/comments over here. Flamefesting over your blog is fun! :-) but anyway, I am delighted to say that at least some people worth their weight in code were watching, interested.
The mail I got this morning (yes, follow the links above!) was from Hongli Lai, one of the very nice people at Phusion - The people behind Phusion Passenger (a.k.a. mod_rails) and Ruby Enterprise Edition. Yes, people with a very different mindset to mine (specially when it comes to being a 100% Free Software person).
Hongli invited me to try their new DebGem service (still in Beta, although quite usable as it is). They are offering an auto-built full repository of Rubyforge, translated to Debian packages. They are currently supporting Debian 4.0 (Etch) and Ubuntu 8.04 and 8.10 (Intrepid and Hardy). And, yes, installing any arbitrary Ruby module is now just as easy as aptitude install libsomething-ruby. For over 20,000 Gems.
There is a catch, yes. The service is currently free, while they finish the public beta period. Their pricing is available.
Best luck to you guys. And... Shall you enjoy fierce competition from Debian proper! ;-)
[Update]: They just posted the official Beta announcement for DebGem

( categories: )

Apt-get and gems: Different planets, right. But it must not be the war of the worlds!

Submitted by gwolf on Mon, 12/08/2008 - 23:57

Thanks to some unexplained comments on some oldish entries on my blog, I found -with a couple of days of delay- Rubigem is from Mars, Apt-get is from Venus, in Pelle's weblog. And no, I have not yet read the huge amount of comments generated from it... Still, I replied with the following text - And I am leaving this blog post in place to remind me to further extend my opinions later on.
Wow... Quite a bit of comments. And yes, given that the author wrote a (very well phrased and balanced) post, I feel obliged to reply. But given that he refered to me first, I'll just skip the chatter for later - I'm tired this time of day ;-)
Pelle, I agree with you - This problem is because we are from two very different mindsets. I have already said so - is a witness to that point.
But I do not think the divide is between sysadmins and developers. I am a developer that grew from the sysadmin stance, but that's not AFAICT that much the fact in Debian.
Thing is, in a distribution, we try to cater for common users. I have a couple of Rails apps under development that I expect to be able to package for Debian, and I think can be very useful for the general public.
Now, how is the user experience when you install a desktop application, in whatever language/framework it is written? You don't care what the platform is - you care that it integrates nicely with your environment. Yes, the webapp arena is a bit more difficult - but we have achieved quite a bit of advance in that way. Feel like using a PHP webapp? Just install it, and it's there. A Python webapp? Same thing. A Perl webapp? As long as you don't do some black magic (and that's one of the main factors that motivated me away from mod_perl), the same: Just ask apt-get to install it and you are set.
But... What about installing a Rails application? From a package manager? For a user who does not really care about what design philosophy you followed, who might not even know what a MVC pattern is?
Thing is, distributions aim at _users_. And yes, I have gradually adopted a user's point of view. I very seldom install anything not available as a .deb - and if I do, I try to keep it clean enough so I can package it for my personal use later on.
Anyway... I will post a copy of this message in my blog (, partly as a reminder to come back here and read the rest of the buzz. And to go to the other post referenced here. And, of course, I invite other people involved in Ruby and Debian to continue sharing this - I am sure I am not the only person (or, in more fairness, that Debian's pkg-ruby-extras team is not the only team) interested in bridging this huge divide and get to a point we can interact better - And I am sure that among the Rubyists many people will also value having their code usable by non-developers as well.

( categories: )

It's just a different mindset. Not necessarily a _sane_ one, though...

Submitted by gwolf on Mon, 11/24/2008 - 14:18

Wouter insists that Ruby Gems are enough of an argument to keep Rails at a distance. Even though I agree with the basic claim and think that Gems are basically insane and sick, this should be taken a bit more under perspective.
We are blessed. We are blessed to have Debian, such a rich OS with such a great package management system, and with superb integration between so many packages. Blessed are also the users of most Free Software based distributions, as they share the advantages of systems growing with full consciousness of their interaction's benefits. However, integration with the rest of the world is not seamless.
Most scripting languages have their own infrastructure for managing the modules/libraries/pacakges/whatevers dependencies. Perl has CPAN, PHP has PEAR, Ruby has Gems... I do see Gems as the most obnoxious of them all, but the basics are the same.
The Rails crowd started being Unix-centric, but the Windows (and MacOS - they are no better, believe me, at least in this regard) world has exerted its pressure. Gems caters very well to their needs, but we do suffer the integration at the distro side.
The only sane way the propietary-minded people have managed to stay clear of the well known "DLL hell" is to ship everything a given program requires bundled together - that's the main reason for the bloat of lots of applications, and for the sloppiness of security support. Every application packager is responsible for shipping updated versions to any library it bundles in, except for the very basic core that the OS itself provides. That seems so annoyingly backwards to us that... it is unbelievable.
So, yes, Rails application trees often include Rails itself. For $DEITY sake, even I have grown used to working that way, as things tend to break under your nose otherwise. My proposal (which we talked over at DebConf, but have not pushed so far) is to support simultaneous versions of Rails installed in a Debian system (of course, via different packages), more or less in the way that simultaneous versions of Ruby, PHP or Python (and, in some limited fashion, Perl - Although Perl does not suffer from this incompatible bumps. Yay for Perl!) can be installed.
...And, yes, together with the pkg-ruby-extras team, I have been trying to -slowly, yes- package whatever modules we often use so they don't have to be included in Rails applications.
So far, the best way (although by far not optimal) I have found to limit this explosion of trees is to include most libraries in my Rails application trees as git submodule trees - If they are not explicitly downloaded, the systemwide libraries will be used.
Yes, the "ship the whole thing as a bundle" approach is quite annoying. However, at least I must acknowledge that it works better than the approach I took with previous (mod_perl-based) webapps I wrote... As relatively few people grok mod_perl, I ended up with quite some apps which were not installed by anybody but me. Rails is obnoxious... but seems to be parsable by the humanity at large.

( categories: )
Syndicate content