[Mageia-dev] Mirror layout, round two

[Maarten Vanraes]

Op zaterdag 27 november 2010 09:43:54 schreef Michael scherer:
> On Fri, Nov 26, 2010 at 10:29:14PM +0200, Thomas Backlund wrote:
> > Hi,
> > As we are getting closer to actually have something to mirror it's
> > time to get this decided.
> > 
> > And the deadline for theese discussions is December 5th, 2010 in
> > order to get a decision on the board meeting on December 6th, 2010.
> > 
> > 
> > Now this is a somewhat problematic topic but needs to be decided.
> > This has already been discussed in two threads:
> > 
> > First off we have the "basic) part:
> > "Mirror tree structure" by Olivier Thauvin
> > https://www.mageia.org/pipermail/mageia-dev/20101020/001286.html
> > 
> > And the other part (that gives some problems):
> > "Mageia repository sections, licenses, restrictions, firmware etc"
> > by Anssi Hannula.
> > https://www.mageia.org/pipermail/mageia-dev/20101012/001084.html
> > 
> > 
> > Now, in order to get somewhere, here is a suggestion that tries to
> > find a middle ground or base for discussions...
> > 
> > Now this toplevel part seems to be ok by everyone:
> > ------
> > Mageia/
> > 
> >       /distrib/
> >       
> >               /cauldron/
> >               /stable1/
> >       
> >       /iso/
> >       
> >           /cauldron/
> >           
> >                   /i586/
> >                   /srpms/
> >                   /x86_64/
> >           
> >           /stable1/
> >       
> >       /people/
> >       /software/
> > 
> > ------
> 
> > Then we come to the "problematic" part:
> This part look really too complex to me.
> 
> > ------
> > /x86_64/
> > 
> >        /media/
> >        
> >              /codecs/ (disabled by default)
> 
> so, ogg, webm, being codec, should go there or not ?
> What about patents problem about something else than codec ?
> ( freetype, image such as gif, DRM stuff )
> 
> >              /core/ (old main+contrib)
> >              
> >                   /backports/ (disabled by default)
> >                   /backports_testing/ (disabled by default)
> >                   /release/
> >                   /testing/ (disabled by default)
> >                   /updates/
> >              
> >              /extra/ (unmaintained, disabled by default)
> 
> If used by people, then why no one step to maintain anything ?
> If someone take the maintainace, does it mean that we will move the package
> ?
> 
> >              /firmware/ (disabled by default)
> 
> Why separate firmware from non_free ? What does it bring ?
> Since both of them are disabled by default, they can be simply merged.
> 
> >              /games/ (disabled by default)
> 
> That's a simplification that make no sense.
> Not all games are big, not all big packages are games ( tetex, openoffice
> ).
> 
> This only bring complexity on our side, complexity on mirror side, and
> bring few improvement to users. A rather more precise label would be to
> have /contents/ repository, as this is not the game that take space, but
> the content.

for this one, i don't really agree. I think it's purpose would be to have a 
repository that not all mirrors have to mirror (it's optional; and it'll 
probably be very big). call it whatever you will, it'll mostly contain big 
games. (imo, it could be like this: if this package would not be essential and 
more than X MB (200?; 250?) it could be in this repository, no matter if it's 
free or non_free; mirror maintainers can largely assume those to be non_free 
for mirrorring purposes) or even split those up.

this is because some mirrors will not be able to mirror core when the big 
games are in core/non_free.
 
> And a explicit policy of splitting content from big packages, with a
> explicit size or expected size for limit ( like if the package is more
> than 100 mo ). That's also a media where deltarpm would make sense, or
> someting like that. In the mean time this would only bring complexity to
> everybody else.
> 
> >              /non-free/ (disabled by default)
> >              /debug_*/ (disabled by default)
> 
> And what are the relation of requirements ?
> Ie, what can requires non_free, codecs, games, etc ?
> 
> And what about something that can goes in both media, ie a non_free
> game goes where ? A unmaintained codecs goes where ?

relations between them are important:

mirrors should:
 - always have core
 - the rest is optional; but there should be a text file somewhere which tells 
us what repositories are in here.
 - (bear in mind that i consider firmware and codecs non-existing)
 
what also needs to happen is to have mirrorlist working better:

if a mirror doesn't have some repositories, it should fetch the next one. 
also, some kind of timings could be interesting; a way of determining how long 
ago this mirror has been synced with primary mirror; ie: a way of determining 
a temporary stale mirror ==> next mirror in the list.

MD5SUM files should be forcably requested to not have a cached version; when i 
was working on urpmi-proxy, i noticed that there is a way to find out if a 
certain file has been modified.

I'm willing to spend some time on urpmi for this stuff to work well.