[Mageia-dev] Mirror layout, round two

[Michael Scherer]

Le lundi 29 novembre 2010 à 15:24 +0100, Samuel Verschelde a écrit :
> Le lundi 29 novembre 2010 14:56:20, Michael Scherer a écrit :
> > 
> > Le lundi 29 novembre 2010 à 13:14 +0100, Samuel Verschelde a écrit :
> > >
> > > It was said early that you just have to look at whether the package 
> > > has a maintainer or not to make a distinction, but this is not sufficient. 
> > > A maintainer can be very active in cauldron but not care about maintaining 
> > > for stable releases. A package can have no maintainer but be actively 
> > > supported in practice (by everybody in cauldron, by security team in 
> > > stable releases).
> > >
> > > The current main vs contrib approach in mandriva is very sensible. It has some (minor, to me) 
> > > drawbacks, but when I install packages from main I *know* there will be security updates, 
> > > bugfix updates, and a QA process that packages in contrib don't have. 
> > 
> > Since we pull lots of deps ( like perl module, python module ) to ensure
> > that everything build fine without any formal check of packages, I can
> > ensure you that packages in main do not have more QA process than
> > anything. 
> 
> May be true until the final freeze. Once the freeze happens there is a formal 
> QA process for security and bugfix updates. That's what my post revolved around.

And how having 1 merged media prevent this process ?


> > Let me remind the problems that we are having with the split, as they
> > are not so minor :
> > - complexity on build system side to take in account this
> >   - packagers time lost to understand what going on when self
> >     containment is broken. Older ones do not have the problem ( or at
> >     least, know it ), but newer one sooner or later does.
> 
> That's also what allows to block easily unwanted updates on main packages for stable releases. 

I think you misunderstood me.
I speak of the fact that in cooker, people add BuildRequires in contribs
for a package in main and get puzzled by the error message. Granted, we
need to improve the error message, but that still make us lose time.
( and the fact that no one did improve the message is imho sign that
this is not annoying enough, to warrant a patch, yet annoying enough to
lose people )

> > - complexity on the users system as they need to have twice the number
> >   of media to cope with this. This also appear in the interface of
> >   various tools, and it is imho uneeded. Most people will say that we
> >   already have too much medias.
> 
> Indeed, however it helps showing that there's a set of packages which is supported, 
> and another one which is only on behalf of the maintainer. In a community driven 
> distribution, this distinction may remains valid : some packages are officially 
> supported by the distribution, others may or may not be, depending on the maintainer 
> (or lack of maintainer).

If there is a maintainer and he has a account, then it is a official
one, and so the package is officially supported. That's all. There is no
separation of "the organisation" and "the maintainers", we are not
Mandriva. 

So either the package is supported, and we keep, or it is not, and then
why should we keep it ?


> The main complexity in current media in Mandriva comes from the updates/backports/testing/debug 
> packages which multiply the number of visible media, but this can be drastically improved on UI side.

That's exactly what i told 3 mails ago....
I have also added that we cannot do miracles on ui if we keep pushing
for more complexity. 

> 
> > - time lost managing it. moving from contribs to main, and cleaning
> >   main. Main cleaning that is seldomly done because :
> >    - no one has time to do it
> >    - people fiercly defend the inclusion of their favorite software in
> >        main 
> 
> If maintaining a package in main means a real committment to support it, 
> with proper rules and management it seems doable to have it work better 
> than it used to at Mandriva.

It was already the case at Mandriva. But it didn't work. What step do
you propose to avoid the same pitfall ( ie, packages pushed in main
without checking first with support team, packages never cleaned because
that's a hard job, endless bickering on what goes in main or contribs ) 

> > - time lost because packagers have to wait on overloaded sysadmin to do 
> >   the move
> 
> Let's give the ability to more people to do the move ?

Then they would push what they want without consideration of support.
See the previous paragraph. 

> > > If we plan to have such processes, does the merge between core and extra 
> > > make is efficient ? I guess we don't plan to have all packages (even 
> > > maintained ones) equally supported with a full QA process (doesn't seem realistic) ?
> > 
> > Again, reread the mail I sent. This is a user concern, not a mirror
> > admin concern. As such, it should not leak on mirror organisation,
> > neither on BS organisation. 
> > And the filtering proposal I made could perfectly fit, provided we find
> > a way to get the "QA note" for a package. Maybe a webapp, with 
> > 
> 
> If there's a way to really flag packages as supported, so that we can 
> query the RPM database and know if a package is supported or not, and 
> do it easily (as a user), then I have nothing more against merging 
> core and extras.

Why in the rpm db ?
A external file would perfectly do the trick. The same goes for
maintainers. The only thing needed is less talk and more code.



> In fact, mirror structure and processes (including QA and support) are 
> related, 

No. That's because you think that mirror structure is the only way we
have to act on urpmi config. And that shouldn't be, because we can't
support every possible distinction that everybody want without a
complexity explosion. See what I said in the mail. 

> and the decision to merge core and extras must be taken together 
> with decisions on QA and support processes, 

Well, everything should be supported or dropped, that's all. Easy, done
by every other distros out there except those that place a artificial
separation. If there is a security bug opened and no one act on it after
a time, let's drop the package. If there is a severe bug and no one act,
the same, let's drop it.

If people want to resurrect a rpm, they can, there is a svn for that.


And if we want to wait on the QA team for defining the mirror structure,
then we will have a loop ( qa team requires a distro to start and be
launched, a distro requires a BS the bs requires mirror structure, and
so mirror can't depend on QA or we will never start ).

> so that we don't end up with 
> a situation were no one knows what's supported or not because the media 
> separation disappeared. Without such processes I should remain against the merge.

That's already the case at Mandriva. See for example
http://lists.mandriva.com/kernel-discuss/2010-10/msg00000.php 

We _already_ do not know what is supported or not at Mandriva. People
push update to contribs ( I do push them for tor or puppet ), while some
packages in main are not updated or buggy or simply unrebuildable ( see
all mdk rpm still in the distro for long forgotten reasons ). See this
old long standing pdftk bug caused by a issue in the java stack in
main : https://qa.mandriva.com/show_bug.cgi?id=44372 . In main, and no
one did anything. 

There is also lots of duplication :
- apache & nginx, who was moved to main likely because oden like nginx,
- the various ftp servers, 
- sendmail & postfix, 
- the 4 tls/ssl implementation, 
- the 2 html rendering library ( webkit, gecko ) with more than 1
browser. 


And most people tend to simply not care to the difference between main
and contribs. If you decide to use puppet for technical reasons ( as we
did in sysadm team ), I doubt you will switch to another tool because it
is in contribs. People install web applications without using the distro
rpm also because they do not care enough about official support ( this
and maybe other reason ).

And in the end, I think most people just keep contribs and main, and
forget the difference. 

Those that care more just take a support contact at Mandriva, and with
enough money, anything can be supported. And there is not really a 1 to
1 mapping between "supported" and "main".

So the split did not help much on this regard at all. It just give a
false sense of security and more complexity on the distro side. 

> > > Packages in core get the "QA approved" stamp whereas packages in extra get none.
> > 
> > The self containment requirement would make it very hard to get such QA
> > approved stamp. 
> > 
> > We cannot have a package being tested without doing the same for its
> > requires first. And we cannot say that we support a package without
> > supporting the full requires. 
> > 
> > Look at openoffice at Mandriva. It does requires
> > tomcat5-servlet-2.4-api, hsqldb on 2010.1 . They are both non trivial to
> > test, and I am quite sure that they were not much "QA approved" besides
> > "openoffice work for basic tasks". And in fact, when eclipse was in
> > main, this was the same.
> > 
> > So in order to have openoffice to have a QA approved stamp, openjdk,
> > tomcat and hsqldb would have one. I am pretty sure they do not had
> > extensive checks..
> 
> The "QA approved" stamps, in my mind, means : "there no known blocking bug or 
> security issue, and if we find some after the release, we'll fix them rapidly".
>  This is more a guarantee of support than a guarantee of quality.

Again, I am not sure that hsqldb, being unmaintained, would be
supported. 

And there is know bugs that were not fixed rapidly in main ( see the
pdftk example I gave for the most notorious one ).

So let's be honest, let's don't start by doing promise we can't keep for
the moment.

-- 
Michael Scherer