[Mageia-dev] Proposal for Mageia: implement bitorrent protocol to allow updates download

[andre999]

Olivier Thauvin a écrit :
> * andre999 (andr55 at laposte.net) wrote:
>> Michael Scherer a écrit :
>>>
>>> Le mardi 11 janvier 2011 à 20:03 +0100, Marcello Anni a écrit :
>>
>> In my case, using aria2c with 2 mirrors and the default 5 connexions is
>> at least 3 times as fast as a single connexion (to my closest mirror).
>> And a much greater improvement over other download options I've tried.
>
> This is a selfish point of view, nothing is important except the result
> on your side.

Sorry, but this can't be serious.
If a user is to download say a 4,7G DVD iso, it is 4,7G if the user uses 
one or 5 connexions, from one or 5 mirrors.
And if the 5 connexions go to 5 different mirrors, that is only 20% per 
mirror.
Globally, if the same files are downloaded, the same amount of data is 
downloaded.
What we are talking about here, from the user side, is downloading 
faster, if the capacity is _available_ on the network of mirrors.
Nothing more, nothing less.

There are many mirrors which provide multiple connexions, with a very 
large bandwidth.  If users don't use the available connexions on these 
servers, other mirrors will be more heavily impacted.
So, as I said in another post on this thread, it is win-win for both 
users and mirrors.

> But as I can see the results of such thing on distrib-coffee I can tell:
> - often the bandwidth limititation is client side (eg you, with your DSL
>    connection, and not the mirror, with 100mbits/s or more)
> - all mirrors have limit, and when the max connection will be reached
>    the mirror will stop serving you (download will be very faster !)

That is the advantage of applications like aria2c.  They search 
available connexions from the urls (mirrors) given, and choose which 
give the fastest downloads.  The number of connexions vary according to 
the conditions.  With the default of 5 connexions, aria2c tries to 
maintain 5 connexions, but in practice fluctuates (dynamically) between 
3 and 5, according to conditions.  (From my experience.)

Think of it another way.
Scenerio 1)  Suppose at this moment the mirror network has 20% free 
capacity.  We download without multiple connections or multiple mirrors, 
without using this excess capacity.  In one hour we are still 
downloading, at a time when there is no free capacity.

Scenerio 2) We download using multiple connexions and multiple mirrors, 
accelerating the downloads with the otherwise unused 20% free capacity. 
  In one hour, most of our downloads are finished, thus leaving free 
capacity.

Just who is served by scenerio 1 ?
The users that don't have access, or the mirrors that are overloaded ?

> Of course, if in the case I don't blacklist mageia and deny download via
> http/ftp to save my mirror (d-c is already under heavy load).
>
> So, no, don't donwload several times from a mirror.

The solution ?  If a mirror can't handle multiple connexions, restrict 
it in some way.  Many mirrors do.  Either all the time, or from time to 
time.  Or throttle the transfer speed.
Just do whatever works best for the mirror in question.
However let users use the capacity available.  Because it may not be 
available in one hour, or even 5 minutes.

> BTW: on distrib-coffee bandwidth is regulated by IP, not connection,
> downloading 5 times will not be faster than 1.

Essentially the same thing.  Great, if it works for the mirror.

aria2c senses the the bandwidth dynamically, and reacts accordingly. 
I'm sure that many other applications do the same thing.
Don't forget that much of the increase in download speed comes from 
using multiple mirrors. (Clearly indicated in the aria2c documentation, 
and verified in my usage.)

Why not think globally ?

another 2 cents :)

-- 
André