[Mageia-dev] Mirror tree structure
bgmilne at multilinks.com
Thu Oct 21 18:16:06 CEST 2010
On Thursday, 21 October 2010 06:37:37 Olivier Thauvin wrote:
> * J.A. Magallón (jamagallon at ono.com) wrote:
> > On Wed, 20 Oct 2010 18:34:24 +0200, Olivier Thauvin
<nanardon at nanardon.zarb.org> wrote:
> > > Hi,
> > ...
> > > Now come the question: "what is a valid mirror ?", eg, what a mirror
> > > should have as file to be valid ?
> > Some ideas (probably silly ;)).
> > Why don't you split mirrors in 2 categories:
> > - Software (RPM) repo mirrors, available for network install or urpmi.
> > (distrib+people+software).
> > - ISO mirrors. Avaliable for torrenting or ftp. MDV2010.1 release isos
> > are
> > about 15Gb. With 2 releases per year, thats 30Gb per year, 200 Gb on 6
> > years. Some places can just mirror this...you lower the 700Gb to 500,
> > not bad.
> > so mirroring can be done by version (or was this what you wanted to
> > avoid?).
> Yes, I tried to avoid it because I perfectly know mirrors will take care
> to mirror new version every 6 months (ibiblio example again, nobody had
> a look for several years, at least since 2008).
This will result in fewer mirrors (those that cannot or choose not to dedicate
~ 1TiB). I would rather consider other solutions to this problem. On my own
mirror (which has a total of 820GB available at present), I use exclude
options to rsync in my mirror scripts (but, I have to change these after each
Maybe instead we could provide exclude-list files on the mirror, so mirror
maintainers can use --exclude-from=/path/to/mageia/mirror_cfg/old_releases.cfg
Also, in order to try and expedite getting what most users need out to mirrors
first, I would prefer either:
-development tree separate from releases
-development release should be selected last by mirroring tools (e.g. follow
alphabetically all stable releases if not separate)
> > BTW
> > <nitpickin>
> > - could you kill the final 's' on all the names ?
> > distribs -> distrib
Maybe 'releases' or 'release' instead?
> > iso (you didnt named it 'isos')
> > peoples -> people
> > software (not softwares)
> Is already 'software'
> > - could you not mix case in names (SRPMS <-> x86_64), just srpms...
> This part depend on BS and how this team working on it will organise the
> distrib. I'll suggest them.
> > - could be the arch names more uniform ? in my personal scripts/setups
> > I use x86-32 and x86-64.
Is x86-32 a valid architecture for rpm etc.? While uniformity might be nice,
unfortunately vendors don't necessarily choose uniform architecure names, and
it might be better to match the repo structure to values that can be
determined directly (and not heuristcally) .
I've also never seen 'uname -m' report x86-32 or x86_32.
> > Moreover, perhaps in a not so near future some
> > adventurous soul builds Mageia on ARM or Sparc, so why not sort things
> > like
> > distrib/cauldron/srpm
> > distrib/cauldron/x86/32/iso
> > /rpm
> > /64
> > /arm/32
> > /64
I don't know if memory address space is a useful differentiator here, as
features differ substantially in different ARM cores of the same family or
architecture version. E.g., Fedora has an 'armv5tel' architecture, N900 ships
.deb's with 'armel' as the architecture. See
> > /sparc/32
> > /64
AFAIK, the valid architecture names for sparc are sparc,sparc64,sparcv9.
> This depend again on BS part.
> > The drawback is that ISOs are in the same tree so no separate mirroring
> > is possible, but I think you didn't want this anyways.
> I'll take care to this.
> > All this renaming in case it is not some kind of standard for tree layout
> > or the like...
> > </nitpickin>
> > Just my 2€cent, you will judge...
The readme file mentions that mirrors must sync every 2 hours. For tier1
mirrors, I would agree, however you may need to stagger sync intervals to
lower tiers (e.g. 4 hours at tier2 and 12 hours at tier3).
It may be worthwhile to try and estimate the actual rsync bandwidth that must
be achieved to make a specific tier.
(I am not able to sync every 2 hours, probably not 4, maybe 12, though my
mirror currently syncs daily - for Mandriva I could sync 'official' more
regularly, but not 'devel').
More information about the Mageia-dev