Introducing maemo.org packages

Starting with Fremantle, packages can now be managed by developers through the maemo.org packages interface. Community testers can help in the QA process by testing applications in the queue and giving thumbs up/down for a package.

At the same time we open the extras-testing and extras repositories for Fremantle for promotion through the interface.

The interface has the following features:

• Package search
• Package listings per repository
  
• Only allow maintainers to promote a package
• Request package maintainership
• Automatic package maintainership assignment on first upload of a package
• Automatic dependency tree checking
• QA queue for testing repository
• Promotion unlock to stable (Extras) repository based on karma score
• Leaving comments when you encountered problems with a package

We index all Fremantle Extras*, SDK and Nokia-binaries repositories, so we can check for availability of every dependency. A warning will be shown on a package information page, when there are missing dependencies. Package promotion will be blocked until the issue has been resolved.

At this moment the interface is in beta testing mode, so please be aware that there might be some rough edges. Promotion will happen manually for the first few promotions, so we can make sure that everything is working. Feel free to file bugs in bugzilla, if you encounter any.

Improvements will come in the next few weeks. More integration with the autobuilder is already coming next week.

In a later post, I will discuss the new promotion process for Fremantle a bit more in depth. This is still something that needs to be refined a bit.

Improving maemo.org Downloads

As part of my tasks in the January 2009 Sprint for maemo.org, I want to gather improvement ideas for downloads.maemo.org. A list of improvements that I have come up with can be found at the wiki page. If you have ideas on how we can make Downloads work better for you or have some cool feature you want to have added, please add them to this wiki page.

At the end of this Sprint, I will pick some of the ideas to which I will commit for the next Sprint. Some items that have been added to the wiki page already are:

Category reorg

We had long discussions about what packages categories to use. Implementation and deployment is currently blocked by bug #1805. The current catalog still shows a mostly randomly-selected list of categories. It should be reorganized to use the official list of categories agreed on by the community.

OS200x vs Maemo x

In the past Nokia referred to OS releases OS2007, OS2008, etc. The use of OS200x has been deprecated in favor of Maemo x (e.g., Fremantle is Maemo 5). The catalog is currently organized by OS200x, this needs to be changed to the new versioning scheme.

Automatically create entries in Downloads when package in Extras has user/* section

At the moment not all applications which are available in Extras are listed in Downloads. A script could gather basic information for all user applications in Extras and create entries in the catalog. These entries can later be updated by users to give more details about the application.

Add Application Manager install failures feedback to downloads

There is a plan to integrate install status feedback into the Application Manager. Downloads could use statistics from installation failures to display the "quality" of a package.

Packaging Policy change proposal

After long and heated discussions, we finally managed to finish the Better Package Categories task. When I started the discussion in April, I knew it would not be an easy task. It remained on my task lists over a lot of sprints, but now we have a result.

Ryan posted about this and proposed future in his blog.

"So, what's next?"

Now we need to get these proposed changes accepted into the Packaging Policy, so we can start to apply them. I created a wiki page for the proposed changes to the policy. Please see if you would like to see the wording changed. I will push this task for the November 2008 Sprint.

The autobuilder and Extras Assistant will soon start to give warnings when your package is not using one of the official categories. These warnings will not prevent you from uploading your package, but please try to update your package to make it comply with the policy.

As we are going to change the policy, the wiki page might also be a good place to propose other changes you would like to see in the document.

New features autobuilder and Extras Assistant

Rejecting packages when the same version is uploaded for the second time.

The autobuilder has been modified to reject a package when the same version is already available in the extras-devel repository. This change will force developers to increase their package version number after each successful upload and build. In the past we've had problems in the repository where a package was uploaded twice and caused 'Size Mismatch' errors.

Package signing no longer required.

The autobuilder and Extras Assistant no longer require packages to be GPG signed. It seemed to cause a lot of grief for developers without any real benefits. This change is targetted to making uploading packages easier. Every uploader has to be authenticated to upload a package, so we can already trace back the uploader. The autobuilder signs packages which are moved into the repository.

Upload to multiple repositories at the same time.

You can now use the Extras Assistant to upload a source package to chinook and diablo at the same time. No need to do two separate uploads anymore. This feature has been requested for quite some time, let's hope it helps.

Reducing number of external repositories #1

In my talk at the Maemo summit 2008, I talked about starting an effort to reduce the number of external repositories. Talking about this helps, but actively contacting developers and repository owners helps even more. This week I started by creating a list of all repositories we need to target. When this was done, I started to contact a lot of developers to convince them to move their packages into Extras and then close their repository.

A funny thing to mention is: Nobody had any objections to this.

Most responses were apologies for not doing it sooner. When I started this, I expected to have some resistance, but so far there was none.

This is a good thing! It seems the community wants things to change too.

How can you help?

If you know somebody developing applications in their own repository, try to convince them to move their packages into Extras(-devel). Please edit the wiki page to reflect the status of each repository.

With the help of a few people, I think we can get number of repositories in the list down quickly!

Visible results

When looking at the Gronmayer listing, you can see that more and more repositories are being removed and end up as: 'Repository is offline'.

Talk: Reducing number of external repositories

At the Maemo Summit, I had a talk about the Extras repositories and the plans for the future. The slides can be found on Slideshare: The huge number of external repositories makes it increasingly hard to have a flawless experience for the user when it comes to package selection and installation. The proplems we currently face is:

• Users are unable to easily find packages
• Users experience conflicts between repositories
• Unable to do quality testing on huge amount of repositories
• maemo.nokia.com – Is not able to hand pick packages from external repositories
  

One of the many things announced by Nokia was the creation of maemo.nokia.com, where the best community made applications will be featured. I guess there will be more posts from Nokia on that topic later.

Back to the external repositories problem:
We started to retrieve all source packages from all know repositories found on Gronmayer. These packages are fed to the autobuilder to see if they would build. The status page looks like this:

Our plan to reduce the number of external repositories:

• Build all source packages available in external repositories
• Show status of builds
  
• Try to discover missing sources (a lot of packages have no source available in the repo)
  
• Help developers to fix build problems
• Convince developers to move their packages into Extras
• Convince teams inside Nokia to do the same
  

We hope to get visible results for this effort. Help with this effort would be very appreciated. Please try to convince developers to move their packages into Extras, when they haven't done so yet.

Together with Dave Neary, I did another talk. This one was about the history of the Maemo community and the changes we went through in the last few months. The slides for this talk are also available:

Maemo Summit 2008, Awesome!

The gathering of everybody Maemo has ended yesterday. 2 days filled with very interesting or even amazing meetings, talks and gossip. There was a very sharp contrast with the days before that at OSiM. I want to bet that the Maemo community was able to achieve more at the summit than all projects at OSiM together.

I've met a lot of people and it was very good to see the faces behind the email addresses or IRC nicks. We should not forget the benefit of meeting people in real life. Going to a bar or a restaurant together to have a nice dinner and have some German beers even adds more to this of course.

I'm glad that some of our rockstar hackers had a chance to meet the Nokia people involved in their favourite subjects. A lot of the explanations, about why something was decided this way, are better explained in a personal conversation.

Special props for lardman doing a talk on dsp programming in 30 minutes. The amount of information he managed to put in one talk was mind boggling.

As a maemo.org guy, I have had the opportunity to discuss a many plans and this will likely be more than enough work for me for the coming 10 years ;) Ed and Sasha(Wii tennis king!) had some valuable ideas for managing the repositories and reporting problems. I really hope we can implement those as soon as possible.

Next blog post will be about my talks and I will make the slides available there.

See you at a next meeting?

[Edit: Somehow this didn't get posted while I was at Tegel Airport yesterday]