[Mageia-dev] Proposal for Mageia: implement bitorrent protocol to allow updates download

[andre999]

Motoko-chan a écrit :
>
>
>> However I feel that
>> 4) It is reasonable for downloaders to use multiple connexions per
>> mirror, as long as they don't bypass the controls put in place by the
>> mirror. Not only reasonable, but actually desirable in terms of using
>> otherwise unused download capacity that occurs from time to time, and
>> thus enhancing general download access.
>
> This method only works in the following situations:
>
> - The mirror limits the speed of each individual connection
> - You use multiple mirrors to download
> - If applying to a single file across multiple mirrors, the mirrors
> support transfer of selective portions of the requested file
>
> In the first situation, the mirror maintainer usually has a good reason
> for limiting connection speed. If their limit is very low, you should
> contact the project they are mirroring and suggest they be dropped as an
> official mirror. Going around admin-imposed limitations is bad manners
> at the very least and will either get you banned or cause further
> restrictions that will harm everyone.

Many mirror sites do restrict multiple connexions (per user), so 
evidently it is readily done.  If a mirror chooses not to make such 
restrictions, that is their choice.  It is not good form (to put it 
mildly) for some mirrors to try to dictate mirror policy for other 
mirrors, which could have different interests.
Limiting the connexion speed could be an easy way to ensure that usually 
everyone who wishes to connect can do so in busy times.  In less busy 
times, multiple connexions to such mirrors causes no problem.
Software such as aria2c monitors the download speed of connexions, and 
will avoid slower connexions, thus balancing use of available download 
ressources between mirrors.

> In the second situation, you are now taking the bandwidth of multiple
> mirrors, increasing the overall load on the mirror network. Use of four
> mirrors means you just took up resources that three other people could
> have been using. Effectively, you're being greedy to the detriment of
> the community.

So for 2 hours one uses 4x download ressources instead of 8 hours of x 
download ressources. In both cases using 8x ressources.
A simple elementary school calculation.
For large files, such as ISOs, it makes a big different to the user.
Globally, it makes no difference collectively to the mirrors.  Except 
that downloading will tend to be balanced between mirrors if software 
such as aria2c is used.  And mirrors with free download ressources are 
more likely to be more fully used.
(One could always quibble about the overhead per connexion, but that is 
relatively minor compared to the download itself.)

> The third situation presumes a certain configuration that isn't
> necessarily true. It also has the same problems as the second situation.

The third case is generally true for ftp mirrors, as well as many http 
mirrors.  Where it is not, aria2c and similar software will not use the 
mirror.

Don't forget that the major target of multiple connexions will be the 
larger/faster mirrors, thus taking load off smaller mirrors.  (Since 
applications like aria2c favour faster connexions.)

>> There is another factor that may not have been considered. Many
>> downloaders will be slowed by their internet access or other
>> non-mirror factors, which will very much increase the likelihood that
>> mirrors have unused bandwidth at various times. Increasing the utility
>> -- from both sides -- of multiple connexions.
>
> That is an argument against multi-mirror downloads. Most mirror servers
> will have good bandwidth for their region. In general, the user's
> connection will be a bottleneck. The overhead of maintaining multiple
> connections can actually _slow down_ the downloads.

So you believe that unused download bandwith should not be used ?
Sounds like that would _really_ help downloaders -- and other mirrors.

Among the potential non-mirror factors which could limit the download 
speed is the capacity of the download path.  Multiple connexions spread 
the download over multiple paths, minimising this potential bottleneck.

> As for torrents, there exists a large problem in that there are several
> ways to handle such things (torrent per file, torrent per directory,
> etc) and on every change you would need the torrent to be regenerated
> and seeds to be established. If you are doing this on something with a
> lot of file churn such as a development trunk, you run into a ton of
> management overhead for handling the torrents. Even if you start with
> web seeds, you now strain your mirrors by requesting that they be active
> in seeding a bunch of torrents.
>
> Bittorrent makes good sense in one big situation: you have a single
> large or multiple files that will be widely downloaded and distributed,
> and the downloaders are likely to leave their download tool (torrent
> application) running after download. This is especially useful when it's
> going to be a large, but temporary, demand such as ISO files at release
> time. While some mirrors might be able to handle a 200-300% increase in
> traffic, others may not. It's expensive to provision at that capacity
> when such a thing only happens twice a year for a week or maybe two. In
> such cases, torrents make very good sense as they help reduce the peak
> demand on your mirrors.

Evidently you understand the bittorent issue.
Note that it is essentially those with faster download/upload speeds 
that act as secondary mirrors that provide the benefit.  Many users 
can't offer that.
Interesting that you acknowlege the variation in demand in discussing 
bittorent.  Which implies that some capacity is not always used.  Which 
applies to the multiple connexion issue.

> - Michael

Regards :)

-- 
André