[Mageia-sysadm] Tier1 Mirrors

[nicolas vigier]

On Mon, 09 May 2011, Carlos Carvalho wrote:

> pterjan at gmail.com (Pascal Terjan) (pterjan at gmail.com (Pascal Terjan)) wrote on 4 May 2011 22:02:
>  >On Wed, May 4, 2011 at 21:40, Olivier Thauvin
>  ><nanardon at nanardon.zarb.org> wrote:
>  >> * nicolas vigier (boklm at mars-attacks.org) wrote:
>  >>> Hello,
>  >>>
>  >>> As we have a proposal for a new Tier1 mirror, we need to decide how many
>  >>> tier1 mirrors we want. And maybe rules for minimal bw available on those
>  >>> mirrors, and minimal refresh time.
>  >>>
>  >>> I think we can try to have one Tier1 mirror in :
>  >>> ?- europe
>  >>> ?- south america
>  >>> ?- asia
>  >>> ?- us
>  >>> ?- africa
>  >>
>  >> The goal was to limit the count of tier1 to avoid
>  >> - big bandwidth usage to our providers
>  >> - having a lot of mirrors to contact in case of change on our side.
>  >>
>  >> I don't see interest to have more primary mirror, especially in
>  >> africa or asia.
>  >
>  >Having better ones would be nice...
>  >We currently have 3 tier1 mirrors, 2 of them have only 100Mbit and the
>  >third one only syncs once a day so is useless.
>  >
>  >In asia, people in asia who want to provide a mirror have to sync from
>  >the chinese one which only have 100Mbit and is not primary
>  >
>  >In south america syncing from Europe is awful, and there is no up to
>  >date mirror in US, so I doubt we get much mirrors in America if we
>  >don't get proper primary one...
> 
> ibiblio provides only basic service, it's not adequate for a tier1,
> where commitment is necessary.
> 
> When mageia.c3sl.ufpr.br is registered we can take care of the whole
> South America. Since we have a 5Gb/s connection to Miami, I think we
> can also serve North America adequately, even if it's not optimal.
> 
> I think a tier1 mirror should indeed meet some requirements:
> - minimum bandwidth
> - mirror all of the repository, to allow other mirrors update from
>   them
> - update frequency

Good idea.

For tier1 :
- minimum BW: 1Gb/s ?
- update frequency: 15mn ?

Maybe we can also have requirement for non-tier1 :
- minimum BW: 10Mb/s ?
- update frequency: 5 hours ?

> 
> It's not only bandwidth on the master that limits the number of
> mirrors, it's also the very expensive disk scans caused by rsync. It's
> sometimes a bigger problem than bandwidth.
> 
> BTW, you say that 700GB are necessary, however the repository is only
> 73GB now. Unless Jussieu is not complete.

The repository is currently 63GB, but we only have cauldron for now. At
release 1 and for each new release the repository will be forked.
Updates and backports repositories will be added. Maybe new
architectures like ARM will be added on the mirrors. So size will
increase with time, and can reach something like 700GB after a few
years and releases. Maybe README file should be updated to explain this.

> 
> About the update frequency, you might want to provide triggering for
> the tier1s. It makes updates much more efficient. Some distros use it
> and for Debian it's a requirement for a tier1.

Interesting. The "Push Mirroring" ?
http://www.debian.org/mirror/push_mirroring