LCA attendees rank Gentoo #4 distro

Everybody who went to LCA entered their distro, editor, and shell upon registration. Peter Lieverdink posted graphs of the results.

Gentoo made an excellent showing, coming in 4th after Ubuntu, Debian and Fedora. This is particularly neat because LCA attendees fit Gentoo’s target audience really well: developers and power users.

Thanks to Daniel Black for the link to that graph.

Improving Gentoo’s PR

This won’t be a long post, because I’m tired. Sorry for the dearth of posts on here, but I’ve been busy writing other things—see below.

For anyone who hasn’t heard, I took over as lead of Gentoo’s public relations efforts a little over two weeks ago. Three days earlier, I wrote an LWN article concluding that Gentoo isn’t falling apart, but it’s totally failing to communicate. After writing that article, I realized that somebody had to step up to deal with this problem—who better than me?

My focus right now is showing people that Gentoo development is just as alive as it’s ever been. I’m doing this by opening windows into development through more frequent news postings, with links to discussion forums to respond to the posts. Doing this, combined with writing to people (“You will”) rather than about them (saying “Users will…”), will help build better relationships with our users.

Another part of improving the perception of a lively, active community is updating the look of our website. The old website redesign never made it to fruition, so a few of us have begun taking a look at how far it got, what happened, and what to do now. At a minimum, I’d like to make some slight changes to give our site a face lift. The design hasn’t changed for 6 years now, and it shows.

One major, easily fixable problem with our website is that there’s no obvious place to go for users who want to contribute. There should be a big “Get involved!” or “Help Gentoo!” link right up at the top of the page, next to “Get Gentoo!” All this requires is a little webpage that describes all the ways people can help. In fact, the whole website isn’t task-oriented enough. This needs to change.

In the future, I’m going to begin improving the “press” aspect of PR, based on my notes from an excellent talk by Josh Berkus at OSCON 2006 on public relations for OSS projects. The main ideas here are providing a press kit for reporters with all the basic info they want, building relationships with local reporters by using local Gentoo contacts, putting together some case studies of people and businesses using Gentoo in interesting ways, and improving our process for creating and posting news and press releases.

Finally, any Gentoo users can help improve Gentoo by simply advocating it to Linux users you know, giving demos and talks at Linux user group meetings or conferences, promoting it in articles, or writing in your blog about something Gentoo does really well.

New xorg-server for testing in ~arch

I just committed xorg-server 1.4.0.90, which is a prerelease for 1.4.1. Here’s the Gentoo ChangeLog:

Bump to 1.4.1 release candidate. It’s gotta be an improvement over 1.4, so i’m letting it go into ~arch.

(#192221) ‘xorg-server-1.4 – keyboard LEDs do not work’ fixed upstream.

(#201047) ‘xorg-server 1.4 no longer loads xmodmap via xinitrc properly’ fixed upstream.

(#197104) ‘xorg-server-1.3 and 1.4 consumes 100% CPU, locking the keyboard, apparently triggered by opening an OpenOffice pulldown menu’ fixed with patch from master branch.

(#196019) ‘xorg-server creates unnecessary file /etc/X11/X11/Xsession.d/92xprint-xpserverlist’ fixed by not installing the same file twice to 2 different places (Andy Crook).

(#195886) ‘xorg-server-1.4.0-r2 built with hal USE flag crashes on shutdown if dbus service is not running’ fixed upstream.

(#195551) ‘xorg-server-1.4 fails to build w/kdrive on amd64’ fixed with Makefile.am patch designed for easier sed but unsuitable for upstream because the line gets too long (Michael Gorse).

(#194503) Don’t spit versions when showing drivers to rebuild via qlist, and also provide a command for people to do it themselves later.

The upstream X.Org change list is available here.

New GCC hotness

I’ve been talking with Lance lately about setting up a good development machine, and GCC 4.3 (still unreleased) came up so I checked out its changes. Here’s a few I found particularly cool:

  • A new forward propagation pass on RTL was added. The new pass replaces several slower transformations, resulting in compile-time improvements as well as better code generation in some cases.
  • A new command-line switch -frecord-gcc-switches has been added to GCC, although it is only enabled for some targets. The switch causes the command line that was used to invoke the compiler to be recorded into the object file that is being created. The exact format of this recording is target and binary file format dependent, but it usually takes the form of a note section containing ASCII text. The switch is related to the -fverbose-asm switch, but that one only records the information in the assembler output file as comments, so the information never reaches the object file.
  • A new internal representation for GIMPLE statements has been contributed, resulting in compile-time memory savings.
  • A new command-line option -fdirectives-only has been added. It enables a special preprocessing mode which improves the performance of applications like distcc and ccache.
  • Experimental support for the upcoming ISO C++ standard, C++0x
  • Fortran: The -fexternal-blas option has been added, which generates calls to BLAS routines for intrinsic matrix operations such as matmul rather than using the built-in algorithms.
  • Fortran: Support to give a backtrace (compiler flag -fbacktrace or environment variable GFORTRAN_ERROR_BACKTRACE; on glibc systems only) or a core dump (-fdump-core, GFORTRAN_ERROR_DUMPCORE) when a run-time error occurred.
  • Java: libgcj now supports all 1.5 language features which require runtime support: foreach, enum, annotations, generics, and auto-boxing.
  • x86/amd64: Tuning for Intel Core 2 processors is available via -mtune=core2 and -march=core2.
  • x86/amd64: Code generation of block move (memcpy) and block set (memset) was rewritten. GCC can now pick the best algorithm (loop, unrolled loop, instruction with rep prefix or a library call) based on the size of the block being copied and the CPU being optimized for. A new option -minline-stringops-dynamically has been added. With this option string operations of unknown size are expanded such that small blocks are copied by in-line code, while for large blocks a library call is used. This results in faster code than -minline-all-stringops when the library implementation is capable of using cache hierarchy hints.
  • x86/amd64: Support for SSSE3 built-in functions and code generation are available via -mssse3.
  • x86/amd64: Both SSE4.1 and SSE4.2 support can be enabled via -msse4.
  • x86/amd64: GCC can now utilize the ACML library for vectorizing calls to a set of C99 functions on x86_64 if -mveclibabi=acml is specified and you link to an ACML ABI compatible library.
  • MIPS: libffi and libjava now support all three GNU/Linux ABIs: o32, n32 and n64. Every GNU/Linux configuration now builds these libraries by default.
  • The configure options –with-pkgversion and –with-bugurl have been added. These allow distributors of GCC to include a distributor-specific string in manuals and –version output and to specify the URL for reporting bugs in their versions of GCC.

I’m already using GCC 4.2, but I hadn’t really looked into its changes either until now:

  • OpenMP is now supported for the C, C++ and Fortran compilers.
  • A new command-line option -Waddress has been added to warn about suspicious uses of memory addresses as, for example, using the address of a function in a conditional expression, and comparisons against the memory address of a string literal. This warning is enabled by -Wall.
  • C++: -Wextra will produce warnings for if statements with a semicolon as the only body
  • C++/libstdc++: Enabled library-wide visibility control, allowing -fvisibility to be used.
  • x86/amd64: -mtune=generic can now be used to generate code running well on common x86 chips. This includes AMD Athlon, AMD Opteron, Intel Pentium-M, Intel Pentium 4 and Intel Core 2.
  • x86/amd64: -mtune=native and -march=native will produce code optimized for the host architecture as detected using the cpuid instruction.

Who made Gentoo Linux, and when? A commit analysis

LWN just published my story of this name as premium content. I ran a number of analyses on Gentoo’s CVS commit history, kindly provided by Robin Johnson, to look at what happened to our developers and our commits over time. If you want to know the results now (rather than waiting until the content becomes free), subscribe to LWN—the best Linux and open-source software news.

HOWTO make Gentoo great

Gentoo is good. How do we make it great?

Over the past year or so, I’ve read a few books, and I want to use those ideas to build a better Gentoo. The books:

I plan to write a short series of posts discussing the lessons in these books and how to apply them to Gentoo. In this post, I’m going to summarize the concepts of “Good to Great.” I’ll discuss how we can apply them to Gentoo in a later post. The book explains what it takes to transform a good company into a great one. It’s a comparison of companies that made a transition from good to great (thus the title) with companies that remained merely good. Jim Collins and his group reduced the differences to a remarkably small set:

  • Level 5 leadership: The leaders of great companies aren’t charismatic, big-name CEOs. They’re humble, and their ambition is for the company, not for themselves.
  • First, get the right people: Before you decide what to do and where to go, get the right people in the right spots. Otherwise, you’ve got the wrong people creating the wrong vision, strategy, etc., which the right people are then forced to implement. Concrete implementations:
    • “When in doubt, don’t hire—keep looking. (Corollary: A company should limit its growth based on its ability to attract enough of the right people.)”
    • “When you know you need to make a people change, act. (Corollary: First be sure you don’t simply have someone in the wrong seat.)”
    • “Put your best people on your biggest opportunities, not your biggest problems. (Corollary: If you sell off your problems, don’t sell off your best people.)”

    The right people have the right innate abilities and character traits, not necessarily the specific knowledge and skills. For example, look for work ethic, dedication, and problem-solving ability, not ebuild-writing skills and knowledge of bash.

  • Confront the brutal facts: Don’t deny reality, or you can’t make the right decisions. To do this, create an environment where everyone can be heard, so the truth can come out. Don’t lose faith that you will win eventually, but accept the reality of now.
  • The Hedgehog Concept: Only do things that overlap in the three circles:
    • You’re passionate about it
    • You can be the best in the world at it
    • It drives your resource engine

    The resource engine is a combination of time (how well we attract contributors), money and brand (how well we create a community), drawn from an accompanying monograph for the social sectors.

  • Build a culture of discipline: Create a culture of self-disciplined people who are “fanatically consistent with the three circles.” Bureaucracy arises to compensate for incompetence and poor discipline due to having the wrong people. If you’ve got the right people, you don’t need the bureaucracy.
  • Technology accelerators: If a new technology fits into your Hedgehog Concept, become a pioneer. If not, settle for decent, or drop it altogether if you can.
  • The flywheel and the doom loop: Transforming from good to great is not quick. It’s a flywheel, slowly building momentum. From outside, all people see is the breakthrough, but from inside there’s a prior buildup. Merely good companies couldn’t build momentum—they jumped around from focus to focus, never getting anywhere with any of them. You don’t need to spend effort getting people behind your idea. Show them the results, and they’ll follow you.

Improving quality in Gentoo

I recently posted about making Gentoo a better tool. A requirement for being a good tool is being a tool that doesn’t break—thus, we need to improve our quality to a more reliable level. I’m going to mention a few ideas to start this discussion, which I hope the rest of our community will participate in.

First, we essentially have no code review. About the only time any code in Gentoo is reviewed is before and during a developer’s training, with a notable exception being the requirement to post eclasses to the gentoo-dev list. Increasing our code review ought to result in an increase in quality, in ability to justify code in words, and in a stronger community of contributors.

How do we increase code review? One idea is to require reviewer approval prior to committing, but this isn’t the best answer for Gentoo. We’ve always been a pretty open community. Developers aren’t prohibited by ACL from committing anywhere in our ebuild repository, so I don’t think they would accept additional requirements that increased the burden of contributing.

Instead, let’s create a gentoo-commits mailing list or RSS feed(s), with full diffs. We should use this tool in many different ways.

  • Each team should use it internally to review all commits to its packages.
  • Mentors should continue to follow their mentees’ commits well after they’re granted commit access (6 months minimum, and I recommend forever).
  • Mentees should also review their mentors’ commits, first to learn and later to review.
  • Every developer should have at least one reviewer and review at least one other developer. This should be formal and documented to ensure it’s happening.

These uses will require that the commit diffs be easily filterable by both committer and files affected. RSS feeds could be made available based on developer or herd, and e-mail lists could contain the information in e-mail subjects or headers.

Second, we should improve our unit testing, where the units are individual packages. This can be both automated and performed by developers and arch testers. Although a number of packages have a useful, working test suite, most lack one. For these packages, we should attempt to provide something automatable in src_test() even when a test suite is absent. Failing that, we should print out a checklist in src_test() of tests to perform before stabilizing a package. There should never be an empty src_test().

Another package-level testing approach is to create solid, automated tinderboxes. This remains unrealistic for our entire database of 10,000+ packages, but we should at least get this going for our “system” set and perhaps for some of the most common sets of packages for servers and desktops. Exactly how to set this up remains a question, since there’s a lot of tinderbox code floating around. Bonsaikitten has some almost-working code based on swegener’s work; Catalyst has some tinderboxing capability; or we could look into using Mozilla’s tinderbox.

Third, we should improve our integration testing, on the entire repository level. Our main source of testing here will be our users, because they have infinitely more combinations of build options and hardware than we can reproduce on Gentoo infrastructure. But how can we take advantage of this testing to improve our quality? By creating an additional, time-lagged set of rsync mirrors with additional QA checks, we could allow users who want to test the latest and greatest software to help those who want stable and solid software.

We already have keywords for ~arch and arch, but they’re still too mixed. A problem in ~arch ebuilds can break the entire tree for all users. They really need a stronger separation. Perhaps the separate repositories should be ~arch versus stable. But another way to do it is to add a delay to the second set of repos, anywhere from 24 hours to a week. This delay allows us time to encounter major problems in the fast-sync repos, fix them, and carry the fixes over to the slow-sync repo. But we’ll need a way to make this really easy to do. It feels like branching with periodic merges, along with cherry picks of major bugfixes, is the right way to do this. Unfortunately, CVS sucks at this. We may need to migrate to a more capable version-control system before this option becomes realistic. In addition to the user testing, we could add a tinderbox into the slow-sync repos to require that they build with the most common configurations.

To sum up, I want to increase code review, unit testing, and integration testing. These three things will strengthen Gentoo’s quality, reputation, and community.

Democracy as education in OSS projects

Reading the PressThink blog, I came across a couple quotes that apply well to open-source projects:

I kept thinking about a famous passage from Christopher Lasch, the great social critic and historian who died in 1994— before the rise of the Web. In the Revolt of the Elites, he said we learn more from argument than from information, not because opinions are weighter than facts, but because to argue for your ideas (in public) puts those ideas at risk. And that is how we learn. …

Lasch in his book:

If we insist on argument as the essence of education, we will defend democracy not as the most efficient but as the most educational form of government, one that extends the circle of debate as widely as possible and thus forces all citizens to articulate their views, to put their views at risk, and to cultivate the virtues of eloquence, clarity of thought and expression, and sound judgment… small communities are the classic locus of democracy— not because they are “self-contained,” however, but simply because they allow everyone to take part in public debates. Instead of dismissing direct democracy is irrelevant to modern conditions, we need to re-create it on a large scale.