[Mageia-dev] Status report for Mageia 1 updates, and call for help from you packagers

[Samuel Verschelde]

Hi,

I was told that QA Team's work's visibility needs to be improved, so as a team 
member I'll try to give you some sort of status report.

QA team currently mainly works on Mageia 1 updates testing and validation. We 
managed to gather various volunteers, and while we probably need still more 
volunteers, at least there's already a lot of work being done and updates are 
being pushed.


*** Updates pushed ***

Since the updates media opened, 65 updates have been pushed (in terms of 
packages, a lot more since an update can contain several packages).
- June: 9, including nvidia drivers and system-config-printer
- July: 46, including kernel, sun's java, iptables, flash-player-plugin, 
kernel, logrotate...
- August: 25 as of today, including KDE SC 4.6.5, libreoffice, flash-player-
plugin

The number of real bugfix or security updates is approximatively 3/5 of those, 
because we also pushed new packages that were present in Mandriva 2010.2 but 
absent from Mageia 1, as the policy permits (exception for Mageia 1):

There is a mailing list where updates are announced, which is what I used to 
count the updates for this report. 
See https://ml.mageia.org/wwsympa-wrapper.fcgi/arc/updates-announce


*** Pending updates ***

Right now there are 16 pending update candidates (in fact 18, but I excluded 
one that is not ready to be tested, and one which is in fact a backport 
request waiting for the backports media to open). See 
https://bugs.mageia.org/buglist.cgi?cmdtype=dorem&remaction=run&namedcmd=waiting%20for%20QA%20test&sharer_id=22

5 are recent (assigned to QA for less than one week) and being taken care of

The others are less recent (more than one week, sometimes even more than one 
month) :
- 7 are waiting for packager input, QA testing having uncovered bugs and/or 
raised questions
- 2 have been in need for testing for several weeks, here QA team must wake up 
:)
- 1 has been validated by QA one month ago, but was assigned to security team 
following updates policy for security fixes, and got not answer. We have to 
improve either the policy or the security team here (or both).
- 1 is ready to go, we just have to tag it as such and pass it to the 
sysadmins
- the last one is firefox: this is probably the longer pending update (started 
as firefox 5, now is firefox 6), and that with the higher visibility for users. 
I would like to let dmorgan and those involved in its testing comment about it 
: what caused such a long delay ? I'm not trying to blame anybody, especially 
dmorgan who did most (it not all) of the hard work: I'm genuinely curious 
about the causes of this delay so that we can explain it to our users, to 
ourselves and above all learn from it.


*** Short analysis (my opinion) ***

Most updates passed to QA went fine, quite well tested. There's still a lot to 
improve in the QA team's work, but right now we kind of manage to handle the 
update candidates. Updates are probably not as thoroughly tested as we would 
like to and often we have to trust that the bugs the upstream project claims 
to have fixed are fixed. But we always try to check that there are no 
regressions.

What can delay an update is mostly:

- bugs or other problems found during testing. An exchange starts between the 
packager and the QA team and it can become very long. The above report shows 
that currently there are more reports waiting for the packager's return than 
waiting for QA feedback.

- testers not knowing what to test, thus giving their time to other bug 
reports. We can't know every package as well as the packagers know them. When 
the packager provides good test cases, it can speed up the process 
drastically.

- lack of testers (not the majority for the current pending updates). It 
happens mostly when we need testers with specific configurations or hardware.

Here stops my own analysis, let the rest of the packagers and testers give 
their views :)


*** Need a broader view ***

This report would be more complete with information about current and past 
open bugs on Mageia 1 (and pending security issues), but this would go farther 
than the purpose of this QA team report. 

I'll let the triage team give us its report about pending bugs, and someone 
(stewb ?) tell us about pending security issues.

Best regards

Samuel Verschelde