[Mageia-dev] Mirror layout, round two

[Michael Scherer]

Le lundi 29 novembre 2010 à 18:29 +0100, Samuel Verschelde a écrit :
> Le lundi 29 novembre 2010 17:08:25, Michael Scherer a écrit :
> > 
> > So either the package is supported, and we keep, or it is not, and then
> > why should we keep it ?
> 
> Because it works, at least partially.

Having it work 'partially" is not sufficient. There is lots of things
that would work "partially", but if we want to have people to feel
confident in the distribution, we shouldn't have "partially working
packages", some with "security flaw" and hope that users will understand
the difference. Some of them will not, and shouldn't have to.

Partially working packages take time from packagers, space on mirrors,
space in hdlist size on everybody. They give bad reputation to our work.

>  Because it has users. 

If no one care to even try to update it ( not even a prospective
packager ), then it is likely not much popular. I cannot really think
that a package can have lots of users, and yet none of them being
technical enough to step and become a packager, or to convince someone
from taking it.
That would just be statistically improbable. 

> It may have flaws, 
> it may not have the latest security fixes, it may just be supported but only updated 
> once a year... That's not a reason for dropping it.

Being insecure is a reason to drop it. If no one is willing to take time
to simply dig and apply security patchs, and later fix then we should
not let people install it. 

>  That's why distiction between 
> officially supported and not officially supported is useful. There are working 
> packages, seldom updated, which don't deserve to be dropped, but which can't be 
> advertised as officially supported, and that's understandable. The world is not 
> either black or white, there are many shades of grey, and that's particularly 
> true for packages in a linux distribution.
>
> According to what you said, it looks like there will be only 2 kinds of packages :
> - in the distribution (which would be equal to "supported")
> - not in the distribution

You said that you want to have confidence in the distro capacity to
maintain rpms. So there shouldn't be something saying "here is packages
that we cannot support because there isn't enough people to do it". 

If we let people install insecure/buggy packages with just a click, they
will sooner or later. We will just increase the number of those that
complain about unmaintained packages and start to have a reputation of
not having security fixes on everything.

> In Mandriva, you can find many examples of packages in main which are not supported in reality,
>  or even maybe simply don't work. You can find also many packages in contrib which are 
> perfectly supported, in cooker as in stable releases. You gave me examples. However I 
> see very rarely security or bugfix updates for packages in contrib for stable releases 
> (or sometimes they go to backports), whereas there are many security fixes and bugfixes 
> for packages in main thanks to Mandriva's security team. There really is a difference 
> between supported packages and other, although it's far from perfect.

The difference is mainly that Mandriva has a team of 2 people full time
doing the bugfixes and security updates. We do not have them. 

So that's not because there is contribs that main got more bugfixes and
updates. That's because people are paid to do the work.

And so there is no correlation between "there is updates in main" and
"there is a split". 

> If there's no difference in term of security updates between php and a random maintained 
> game, then I won't be very confident in the distribution's quality. 

In fact, I fail to see or understand what you mean, even after trying
very hard. 

Seeing that everything is equally supported is a sign of a lack of
quality ?

In this case, you would likely have problem with gentoo, debian or
fedora security policy :/

> Let's say I want to install php on the stable Mageia 2011, will it be supported for 
> security fixes and bugfixes ? For how long ? Are security fixes applied as soon as possible ?
> [ snip ]

If you really want to discuss of support policy, then a new thread
should be started. Or this thread will be more messy. The goal is to
speak about mirror layout, as written in the subject.

> > > and the decision to merge core and extras must be taken together 
> > > with decisions on QA and support processes, 
> > 
> > Well, everything should be supported or dropped, that's all. Easy, done
> > by every other distros out there except those that place a artificial
> > separation. If there is a security bug opened and no one act on it after
> > a time, let's drop the package. If there is a severe bug and no one act,
> > the same, let's drop it.
> > 
> > If people want to resurrect a rpm, they can, there is a svn for that.
> 
> This is the most terrible thing I read so far. 
> I already tried to answer it earlier in this message, 
> however here's a last example :
> - Someone checks if postgresql is supported because if not he'll use another distribution 
> where it is
> - It is (because every package is supported, or it's dropped), but the poor user doesn't 
> know that Nanar went away doing his own fork, no one stepped in to maintain it ! 
>
> - One year later, postgresql has still security bugs opened, no one takes care => package is dropped

So you prefer :
package has security problem, we do not drop it, despite no one stepping
to take care, still offer it and users get rooted ?

> Now if there were a list of supported packages, either it would not be officially supported and 
> the user would know he could use it but maybe won't have security and bugfix updates, 
> or it is officially supported. Now take the example above :
> - Someone checks if postgresql is supported because if not he'll use another distribution where it is
> - It is !
> - However the maintainer went away doing his own fork, so he dropped maintainership. 
> - Someone in QA Team rings a bell : "this supported package isn't supported anymore, 
> but we promised we would support it for Mageia 2011 for 2 years from now ! We have 
> to do something !"
> - The package team leader, or someone else, relays the warning and finds someone 
> else to maintain the package, at least for Mageia 2011, for security and bugfix 
> updates.

Please, I would appreciate that you do not arrange facts just to support
your point, or I will seriously have to reconsider answering in the
futur.

In the first case :
package is not supported, no one step to maintain, we drop -> that's
bad.

second case :
package is not supported, someone step, we don't drop -> that's good

Why do you make the assumption that someone will step to maintain in 2nd
case and not in the first one ? 

Just saying "it should be supported because it is on some official list"
is not really something that worked that well at Mandriva for the
community. 

See for example the java issue ( no one in the community stepped to
maintain it ), kde 3 ( no one stepped to maintain it despite users
asking for it ). Or the unmaintained packages in main. 

The only reason why there was not more is because they could pay people
to do the work, something we can't and do not plan to do in the
association.

So I do not see why you think that having a list will magically make
people maintain software while there is easy to find examples of the
contrary. And why would packagers maintain anything ( and therefor do
more work ) if they can simply not care and move it to extra ?

> - (another scenario : there is a maintainer, but the package has pending issues 
> for a long time, so we have to contact the maintainer about it, and maybe find another maintainer)

another solution : "we do no promises of supporting anything".

So far, there is no BS, no packagers team and so no security team,
nothing. Basically, we have no ressources, just promises of ressources.
The experience showed us that there is a vast difference between people
who start to do some work and people who have put their name on the
wiki. We cannot guess anything for the moment.

Once we have started and done the first release of a alpha version, and
once we have a working team to package, then we can see what we can
support. For the moment, any discussion based on ressources is just
premature and likely not based on real data.

> > We _already_ do not know what is supported or not at Mandriva. People
> > push update to contribs ( I do push them for tor or puppet ), while some
> > packages in main are not updated or buggy or simply unrebuildable ( see
> > all mdk rpm still in the distro for long forgotten reasons ). See this
> > old long standing pdftk bug caused by a issue in the java stack in
> > main : https://qa.mandriva.com/show_bug.cgi?id=44372 . In main, and no
> > one did anything. 
> 
> Having packages which ought to be supported and are not in practice doesn't 
> mean that there must be no difference between officially supported and not
>  officially supported.

The same organisation of packages will likely lead to the same problem
( ie, a mess with the problem that I already highlighted ). Most
experienced packagers ( jq, boklm, tmb, among those that expressed, but
also others I have discussed with like nanar, etc ) seems to not want to
repeat again the same mistakes. 

Technically, anybody can follow security lists as they are open, see
securityfocus, or lwn to find what was updated in other distros and os.
Any packager can prepare a security update, there is nothing magic and
there is even documentation on Mandriva wiki. 

Yet almost no one did, and that's mainly because people think "I do not
need to do it, someone will do it for me" and "that's too complex". And
we need to avoid keeping this mentality, because this clearly do not
scale. This work because there is paid people for that. 

If we do say that's ok to not take care of security of a maintained
package by uploading to a unsupported media ( and later move it for
various reasons that we will not be able to avoid ), packagers will
simply tend to not take care of security because that's exactly what we
tel them. And that's exactly what happened at Mandriva.

So splitting medias based on security support is just that, sending the
wrong sign to packagers. A clear sign that not maintaining package is
ok. But we should send this kind of sign if we really value quality and
if we want to communicate clearly to our users.

> > There is also lots of duplication :
> > - apache & nginx, who was moved to main likely because oden like nginx,
> > - the various ftp servers, 
> > - sendmail & postfix, 
> > - the 4 tls/ssl implementation, 
> > - the 2 html rendering library ( webkit, gecko ) with more than 1
> > browser. 
> 
> What prevents from refining the list if it's too big ?

Just try, you will see by yourself.
But mainly : 
- technical reasons ( different API doing different things ),
- maintainers individual preferences, 
- different set of features and no clear vision of what is needed,
- forgotten requirements ( LSB, etc ),
- difficulty of digging the src.rpm requirements graph ( even if smart
or sophie could help on this regard ),

And in the case of commercial sponsor like Mandriva or Canonical:
- untold requirements ( ie client requirements that cannot be expressed,
or marketing decisions, sometimes overlap with forgotten ones )

Nothing impossible. Just nothing that no one done, and that few
attempted. I tried just once.
-- 
Michael Scherer