Back with Ruby on Debian polemics
Once again, a polemic regarding how to properly integrate the Ruby language and libraries with the Debian distribution has been ignited. Similar arguments were presented in November and December 2008 and September 2010 — And excuse me for just refering to my own blog, but there are links to some other posts from there… and there will surely be many others I just missed. There were even nice collaboration attempts (such as DebGem, announced in January 2009… But which apparently never stuck on, as it is still listed as in its public Beta period and lists only support for Debian 4.0 and Ubuntu 8.04 and 8.10).
Some days ago, while I was on vacation, I received a mail from Lucas Nussbaum expressing the burnout he had been suffering from this situation. Some days after that, he posted a corresponding blog post - Giving up on Ruby packaging. The comments following his post are most interesting — The one comment I’d like to highlight (noting I skimmed a whole deal of them) is by Paul Brannan, one of the original RubyGems authors. Yes, possibly the design criteria for Gems included some mutually-conflicting goals, and they cover some of Debian’s goals. Unfortunately, this conflicting criteria… Was resolved to the opposite way we would have wanted (specifically, that a sysadmin should be able to install software with the same tools he was already familiar with). And of course, there are deeper disagreements, which are rightly stemmed from different priorities: Debian (+derivatives) is meant to make user’s and sysadmin’s lifes simpler. Gems are meant to make developers’ lifes simpler. And while all developers are some sort of users, and sysadminsthe reverse is (obviously) not so.
A developer maintaining several different codebases will surely suffer (well, this particular developer has surely suffered) if he has to maintain them all using coherent versions of system libraries. When a system is programmed, its developers want to use the greatest, latest tools to offer the best experience and functionality. And it’s natural for libraries to evolve over time. However, any sysadmin will grind their teeth at the prospect of having many different versions of libraries, as the Gem model proposes. Why? Well, I was bitten by a minor example of it — Bug fixes do not get backported. Of course, if simple bugs are not backported, security and stability of the system as a whole suffers. Often, new software versions will not only add functionality, but improve on how things are done. Not only bugs get corrected, but we get better response times, better memory handling, etc.
It might sound harsh to say this, and even more, as a developer it feels I am talking against myself — But while development time should be minimized during the system design/implementation, once systems are in production, it should be used to allow for friendlier sysadmin relations.
I have to wear both hats at my real-life job. So, what is my take on this? Of course, blaming myself for choosing the wrong version is a no-go. When evaluating a library before using it in my projects, I try to look at its history and API-stability. APIs that change often speak about immature libraries which are still trying to get the right way to implement their functionality; just-started projects might be great and revolutionary, but they do not yet show any kind of long-term committment… And can lead (and have often led me) to painful rewrites when the Next Big Thing is reached.
As for the users: Even if your favorite language has the best and friendliest distribution method, users just do not care about what language was a particular piece of software implemented in. Users want to be able to install and uninstall with their system’s usual tools. Directly using Gems, CPAN, PEAR and whatnot is just unnatural for users, however convenient they are for developers. Distributions offer a non-technical advantage in this regard as well: A human filter. As an example, as I write this there are 19392 Ruby Gems and 19092 Perl modules in CPAN (note that CPAN stores some older versions but discourages authors from keeping too many - So no, they are still very different in size). Debian has around 30,000 packages. Why is that? Because all Debian packages must be human-generated, human-reviewed, human-submitted. This means, a person must think each packaged piece of code is worth packaging, is stable enough and provides value to users as it is, and is fit for being part of a stable release. I am not saying with this that 90% of CPAN and Gems are crap — I am saying that they are probably early implementations, to be installed, tested and improved by developers, and still not apt for general public use. Or maybe not interesting enough to be packaged as a service for the non-techie public at large.
Oh, and one last point: Ruby is not a new language anymore. It is a mature, powerful language with different implementations, every time more stable… But it is a language deeply affected by the (not so new anymore either) the appearance of Rails. Why do I say this? Because although the language is in no way tied to Web development, many of its strongest uses are Web-oriented. How does this affect the current discussion? Well, because many people argue that users are no longer needed to install the software. Web systems are installed by sysadmins and used via a Web browser, and sysadmins are expected be more skilled than casual users. Still, in Debian (and in other distributions, surely) we try to make sysadmin’s lives simpler — I have (again, talking out of personal experience) installed several webapps (and system tools, and whatnot) for which I never followed any instructions besides aptitude install foo — Using different languages, frameworks, and so on. Can I troubleshoot their installs? Probably, as there is a common logic for how the distribution I have chosen and specialized in works. Can I find causes for bugs in them? Possibly, although there are some languages and frameworks I dare not touch. Can I get help on getting them out of a tight spot? Surely, as there is a central bug tracking system for my distribution — And one of the maintainer’s tasks is to separate the problems related to the distribution (packaging, installing, simple user questions and misconceptions) to those derived from real bugs upstream.
Anyway — I am not saying that our way is the best way. No, by a long shot. Again, developers should have an easy, convenient way for installing whatever they want to play with. And to publish it without jumping through hoops. With this post, I’m only trying to express –again– why Debian works the way it does. And hope for better cooperation in the future.
And as for several comments of what I read in Lucas’ post, I think that there is interest for this synergy among some of the most committed Ruby people.
Comments
Christoph Egger 2011-01-06 07:51:01
I can’t agree more. I’m
I can’t agree more. I’m running web applications and stuff using dozends of libraries and I can’t imagine anything worse than having to keep track of security fixes of each and every of them.
Just having apticron mail me when there are some fixes to install – regardless of the programminglanguage used, whether it’s a webap, a mailserver or something else and being able to fix it with just one command – always the same is just great.
I’ve wanted to write something along these lines myself after reading Lucas’ blogpost.
dominikh 2011-01-07 01:38:12
Changing the title of Lucas’s
Changing the title of Lucas’s article from “Giving up on Ruby packaging” to “Giving up on Ruby programming” just isn’t okay. It might be an honest mistake for all I know, but please, fix that. It conveys a totally distorted message for anyone who doesn’t click the link (or doesn’t read the headline a second time).
gwolf 2011-01-07 18:51:04
Oops!
A honest mea culpa. Fixed it. Thanks!
gwolf 2011-01-07 18:54:38
CPAN module count wrong
I might have got the numbers from the wrong place — http://www.cpan.org says: 2011-01-08 online since 1995-10-26 8331 MB 228 mirrors 8667 authors 19100 modules … So it is ~2500 modules short.
Anyway, thanks for mentioning the distributions/versions precision. I do not know about what is Gems’ site reporting, but I believe it is individual versions — http://rubygems.org reads 106,754,676 downloads of 19,457 gems cut since july 2009 .
gwolf 2011-01-07 18:55:23
Oops++
Thanks, fixed.
Konstantin Khomoutov 2011-01-06 21:08:52
A typo (?)
«Directly using Gems, CPAN, PEAR and whatnot is just unnatural for users, however convenient they are for users.» – supposedly the last word there should be the “developers” instead.
Paul Tötterman 2011-01-06 07:30:22
Packaging adds value
I was amazed by how slick it was to get a working redmine installation (which allowed for several instances to be created!) in debian with little more than aptitude install redmine
ruby, rails, dbconfig, passenger config for apache etc.
I really appreciate the work that went into it.
Ruby just died for me 2011-01-06 10:48:00
As far as I’m concerned,
As far as I’m concerned, Lucas’ debian packages were the only ones that did the job on on a productive system.
I never really cared about it’s development, I just want my existing aplications to work without the need of yet an other package system (and the extra maintenance).
It saddens me to see how many projects are being destroyed because of upstream ego trippers.
Jens.
Steffen Mueller 2011-01-07 08:58:33
CPAN module count wrong
Hi Gunnar,
your module count is wrong. We have 21674 distinct distributions and 88996 modules on CPAN. Not counting multiple versions of distributions.
Are you certain that your ruby gems count is distinct distributions?
–Steffen
Steffen Mueller 2011-01-08 01:45:38
If you click the link to view
If you click the link to view all gems, you get this:
“Displaying gems 1 - 30 of 1301 in total”
So, well, if we were to count the number of distributions (including all versions) ever uploaded to CPAN, we’d report crazy numbers, too.