GURU: a new model of contributing to Gentoo (WIP)

Bugzilla and e-mail submissions

Over the history of Gentoo, I think Bugzilla can clearly be classified as the primary venue of interaction between users and developers. Be it bug reports, enhancement suggestions or new package requests, all of them allowed for the users to attach patches or new ebuilds.

The most important characteristic of this venue is that the result of user's action entirely relied on an existing Gentoo developer taking care of it. The files attached by the users may help the developer but the user can not directly cause anything to happen. If user submits a new package, he can't make it happen without finding an interested developer.

Bugzilla is presented as a canonical example of this contribution model. The same model is expressed e.g. by e-mail submissions, IRC pastes, etc.

The Sunrise project (historical)

The Sunrise project was a historical Gentoo project built around user-maintained, developer-reviewed package repository. At the time when I joined Gentoo, it was the primary venue for users to work on new packages and receive training necessary to become Gentoo developers. After all, many new Gentoo developers at the time (myself included) started with Sunrise.

The Sunrise model was based on a repository with two branches: a main branch where all users worked and a reviewed branch that was meant for public consumption. All users were expected to request review on IRC for their changes, and only commit once receiving an approval from a Sunrise developer. Furthermore, Sunrise developers periodically did review late commits (again) and merge them to the reviewed branch.

Sunrise eventually become defunct due to loss of interest both on developer and user side. Eventually, it has been discontinued in June 2016. 2

Proxied maintainers

The downfall of Sunrise was paralleled by rise of proxy-maintenance. In this model, users are allowed to maintain packages straight in the Gentoo repository, with one or more developers proxying the commits for them.

While this concept has been (and still is) applied to individual user-developer pairs, forming the Proxy Maintainers project was a turning point for its growth. This project focused on gathering developers that were interested in serving users as proxies, and eventually replaced the 1:1 model with M:N where a number of developers freely proxy a number of users. 3

The historical purpose of proxy-maint project was to find new maintainers for abandoned packages in Gentoo. Today (and especially since GitHub was adopted) the majority of submissions involves new packages.

The main advantage of proxy-maint is that it allows the users to take responsibility and share their work to all Gentoo users through the official repository. However, applying those contributions requires significant effort both from users and developers, and so far the proxy-maint team is continuously overwhelmed and unable to process contributions in satisfactory time.

User repositories (overlays)

Finally, another possibility for users to publish their work is to create their own repositories, historically called ‘overlays’. There is a great multitude of kinds of repositories and their uses. We have personal, project and quasi-public overlays. We have repositories focused on a single package suite, software category, and holding all different kinds of applications. We have cross-repository dependencies, we have overrides for Gentoo packages. We have repositories of great quality, and literal ‘dumps’ of random ebuilds. Finally, we have completely unpredictable mixes of all that.

The most important advantage of third party repositories is that they are trivial to create and cheap to maintain. You can practically commit anything you like to your personal overlay, without having to pass any review or convince any Gentoo developer to accept it. With a little more effort, you can get it published on the official list, and make it possible for users to trivially start using it.

However, this implies all disadvantages of the generic case of this model. Users are subject to a great number of repositories of varying quality and size. If you need to find a particular package, you may be required to choose from a number of overlays with no clear indication which may serve your purpose better. If you add a particular overlay, you may discover it actually overrides other packages you did not want replaced. Finally, since there's no real supervision of what happens in all those repositories, it is trivial to inject malware.

Project repositories

I believe it is worthwhile to isolate a special case of third-party repositories — project overlays. Besides aforementioned Sunrise project, some examples of long-lived Gentoo project overlays are the GNOME, KDE, Science or Java overlays.

Their distinguishing quality is that they are focused on a specific yet broad topic and supervised by actual Gentoo developers. However, they frequently give commit access to various external contributors, providing them with an opportunity to work with the project members (or even become them — if you disregard the limitation of Gentoo Wiki preventing non-developer project members).

Furthermore, those project overlays frequently serve as staging ground for Gentoo repository ebuilds. Therefore, the work of users eventually makes it to the official repository.

Effort-reach comparison

The above chart attempts to describe where GURU fits graphically. The vertical axis represents the effort, i.e. how much time and work adding a package takes. The horizontal axis represents user reach, i.e. how many users will find the package added.

The middle line representing Bugzilla is taken as a reference point. In case of Bugzilla, the actual reach depends on whether any developer actually takes the patch/ebuild.

On one end of the graph, we have user repositories where the effort is relatively low and user reach depends on the popularity of particular repository. On the other end, we have commits made directly to the Gentoo repository which have the highest reach.

Proxy-maint is placed above Gentoo as reviewing and training contributors requires more effort (and of more people) than committing packages yourself. Sunrise is nearby, with its high standards still requiring significant effort and good popularity.

GURU aims to achieve similar level of popularity as Sunrise; however with lower cumulative effort. On one hand, some users will take the extra effort to review packages. On the other, it also provides for one-time package submitters.

Repository layout

GURU uses a layout similar to the one historically used by the Sunrise project. There are two branches: the development branch maintained directly by users, and the reviewed branch intended for public consumption.

All GURU users obtain access to the development branch upon requesting it. Since this opens a possibility of seriously broken or even malicious code being committed, this branch is only intended to be used by GURU contributors, and only in the scope specific to their individual development efforts (i.e. even they should not add it to repos.conf).

The reviewed branch is entirely maintained by scripts. The commits from development branch are merged into reviewed branch as soon as they are reviewed, through automated scripts.

User access and workflow

Package maintenance

Implementation requirements

The GURU project requires a git hosting and a review system. The cumulative requirements for both are:

• user access control, with accounts being associated to commits,

• git update hooks that can reject non-conformant commits,

• per-branch access control,

• possibility for minimal CI integration (i.e. commit status information visibility),

• ability to store per-user and per-commit reputation and flags,

• ability to distinguish user types and their limitations in reviews and flag control,

• good diff view.

Both elements can either be part of a single product, or be made of two or more complementary pieces. Notably, the review system will most likely require some custom coding; though it would be desirable to reuse as much of existing systems as possible.

Git hosting solutions

Custom review front-end design

GURU vs other projects

Will GURU replace proxy-maint (/ some other project)?

No, GURU is meant to be an entirely new project, aiming to fill in a specific gap. While it is possible that some users will resign from contributing via proxy-maint in favor of GURU, it should be noted that unlike proxy-maint, it does not provide a way to get your packages to the Gentoo repository.

How is GURU different from Sunrise?

There are indeed many similarities between GURU and Sunrise. However, the main difference is that Sunrise explicitly relied on developers reviewing all the commits, while GURU aims to build a self-sustainable reviewer base within the user community.

Why not revive Sunrise instead?

I believe that Sunrise died mostly because of lack of interest. While bringing it up again might get it running for some time, there is no reason to believe that it would meet the same end soon enough. Therefore, I think it is better to try something else.

GURU for users

Will GURU repository by enabled by default?

There are no such plans at the moment, and I honestly doubt it will happen in the future, at least with the currently planned contribution model. Most notably, GURU is not meant to force the high standards that are (at least theoretically) required for the Gentoo repository.

How is GURU protected against malicious users?

The primary protection is the review model. If a malicious user obtains access to the repository (which is entirely possible given no specific verification of new users), existing users should be able to notice his actions and reject them at review level.

For a malicious agent to be able to push changes to the reviewed repository itself, it would technically need to create three users first, and obtain trusted user status for each of them. While this is not impossible, it is considered costly enough to limit the attack scope. Of course, once the resulting attack is noticed, the accounts will be blocked and the attacker would have to start over.

A malicious user causing mayhem in the development repository is a more likely threat. However, this will not reach end users, so the end value of such an attack would be rather low (i.e. it would be vandalism). Furthermore, all the accounts will be confirmed manually, technically restricting the ability of quickly creating multiple accounts.

The acronym

What does GURU stand for?

It's a secret known only to the most trusted GURU users. However, the expansion most likely includes the words ‘Gentoo’, ‘User’ and ‘Repository’. Not necessarily in this order.

…but what about the second ‘U’?

Yeah, the little bugger likes to make trouble. However, it wouldn't be half as cool without it — imagine GUR, or maybe GRU (but then we would risk being despicable). Choose some U-word yourself. Suggestions include ‘Unique’, ‘Ubiqi… Ubiquitous!’, ‘Umbrella’, ‘Ubuntu’… oh, wait.

Is GURU a recursive acronym?

Sure, why not. ‘GURU Ur Repository, Umph!’

Why is there a whole section dedicated to the acronym?

Because it is the single most important quality of the project! Do you think anyone would have really look at it if it was named ‘random new user-contributed repository without a cool acronym’? Do you think I would be motivated to actually design this and deal with all the criticism if I haven't thought of that acronym in the first place and didn't want it to go to waste? I don't think so.

The logo

What's with the 9… I mean, G?

It was supposed to resemble the Gentoo ‘g’. Except after four strikes, it looked kinda non-g, so I've added the fifth strike. Then, a few minutes later I've realized that it's flipped horizontally. However, since it was a few minutes after its inception, it became classic already and I didn't change it. After all, isn't the Gentoo ‘g’ flipped actually?

What's with the V?

Look at some random ancient Latin writings, and you're going to notice ‘U’ being written as ‘V’. The logo follows that tradition. For no reason whatsoever.

Is there anything special about the R?

Nope, sorry. Incidentally it looks like R, behaves like R and nobody so far has had any reason to think otherwise. If anything changes, I'll let you know.

What are those squares in the background?

That's Braille alphabet, obviously. I do care about accessibility, and since the cost of adding Braille name was practically non-existent, why not?

What are those dots and dashes below?

Morse code, obviously. Probably doesn't have any value.

What's with the cow?

What cow?

The cow on the logo. The one below the GURU.

Someone asked for it, so I've added it.

…but who's that cow? Is that supposed to be Larry?

That's Леони́д, Larry's gender-neutral sibling who's been kidnapped by mad СССР scientists. Don't ask him about the spots.

Are you serious?

Nope.