[Mageia-dev] Mirror layout, round two

[Michael Scherer]

Le lundi 29 novembre 2010 à 02:33 +0100, Maarten Vanraes a écrit :

> > [ michael's long email ]
> >
> I agree with you partly (mostly on the basis that mirror setup should be 
> primarily for mirror admins), however:
>  - some of those big packages are pretty much core

The goal is not to place all big packages somewhere else. Just newer
packages that goes of the limit, even if they are not big. And by newer,
I mean "newer rpm", not "newer release of a existing packages".

>  - and a big core repos is having a big hdlists as well; and you should take 
> into consideration that some people have regular phone line internet.

Then, maybe a big index file is not a good idea. Using rsync + splitted
headers, like yum does, would maybe be more suitable for this situation
( nanar proposed this when we discussed of this some months ago while
eating pizza ).

And in fact, bigger hdlists come from :
- having lots of package ( 10k by rpm, estimation of Olivier )
- having lots of file ( 1k per file, same origin )
- having lots of requires/provides ( even if this is likely negligible )

A package with a 100 mb file is taking less hdlist space ( around 10k )
than a package with 100 files of 30k ( around 110k ). 

For exemple, on mandriva mirror, 
kernel-kerriged-source header : 1.3mo, rpm  32 mo.
vegastrike-data        header : 446ko, rpm 430 mo. 

And so, the explosion of kernel packages is likely a cause of contribs
hdlist size increase. On the 2010.1 stable mdv release, there is 28
kernel or kernel-sources rpm, thus using around 28 mo in the 74 mo sized
hdlist, ie 33% ).

Thus splitting games will not lead to a so big decrease of the hdlist.

 
>  - i'm not entirely sure that mirror admins would like the overflow idea:
>     - if you're a small public mirror (ie: storage size), you would not mirror 
> the overflow; however some big packages would be pretty essential. seperating 
> extra (unmaintained packages); and games would seem easier; 

Well, not mirroring is the goal of the overflow media, as explained. I
have already explained why games is not really a good idea, and why the
unmaintained idea is also not a good idea, so I will not repeat myself
( especially since you did it for me by not cleaning the mail before
responding :/ ).

> also on the 
> following up side; (ie: when problems arise); also a point is what about those 
> big packages and their dependencies (or rather other packages which depend on 
> it).

That's why I said "there is no difference between core and overflow".
This mean "core can depend on overflow or core, overflow can depend on
core or overflow". And while a user could remove overflow, I do not
think one of the goal is to prevent people from doing stupid stuff
( like removing release, or disabling updates, for urpmi related
example )

>  - i don't believe unmaintained packages is something that can be avoided

I do not think unmaintained packages is such a problem that everybody
seems to imply ( and the fact no one gave me any convincing stats or
data is not in favor of making me change my mind ). My own opinion is
that most people have seen on of their pet peeves not fixed because of
that, thus making the problem much more important in the eyes of each of
us than it is in reality. 

-- 
Michael Scherer