[Mageia-dev] Meeting tonight

[Samuel Verschelde]

Le jeudi 15 septembre 2011 11:26:41, Thierry Vignaud a écrit :
> On 14 September 2011 17:50, Samuel Verschelde <stormi at laposte.net> wrote:
> > I tend to think that fixing MageiaUpdate would
> > be a lot less work than the workaround we are trying to use which :
> > - still needs work from the sysadmins
> > - gives a lot of work to QA, that would be better used for testing the
> > updates - forces us to link a lot of packages if we really want to make
> > sure the update never fails
> 
> For now, we are in the same configuration as @ mdv:
> the updates must be self host, if they add new
> requires, these must be added to the repository to update.
> 
> This was decided a long time ago after thinking
> 
> Some people have the media networks, ie all packages
> can be fetched.
> Others will have only DVD media and thus only have a subset
> of  packages.
> Other disable certain media.
> Other activate some other media (backports).
> Some will have non-free and/or tainted enabled, some won't.
> Ie there is no guarantee that a packet with new requires
> can be updated on all machines as end users will have
> a variety of packages subset.
> 
> For almost 10 years MandrivaUpdate == "urpmi --update"
> and not "urpmi --auto-select"
> 
> I think that for the already released mga1 we should stick
> to the known good old working method, the MDV's one,
> rather than inventing a new quick & dirty thing quickly whom we'll
> find subtle bugs along the incoming month

The problem is the situation is a lot more different that what the secteam at 
mdv had to face. Would we only need to add "new" dependencies to updated 
packages, that would be no big deal and we wouldn't make all this noise, we'd 
just copy the deps and that's all. The problem is that we support migration 
from Mandriva 2010.2 to Mageia 1, and that causes various situations where 
*any* dependency of an updated package can be required to install from Core 
Release, so we have to copy the full dependency chain ! See Dave's recent mail 
about bug 2317 that explains that in more details and gives the possible 
solutions to this situation. I would be interested (really) by your opinion 
about the solution to use among those he suggests, or even another one.

As the problem is migration from Mandriva 2010.2, we could also try a 2 levels 
solution :
- copy new deps the way mdv did, this will ensure fresh mageia installations 
will never face an update problem because of a missing dep
- add to the errata that + warn users in MageiaUpdate by improving the error 
message ("you can't install, maybe it's because you have old mdv packages on 
your system, here is a way to solve your problem, do this and that..., or even 
better catch the failure, do not show it, and try again with release media 
added)

> 
> Some suggested removing the --update flag that MgaUpdate
> "gives" to urpmi.
> 
> I think most of them ignore what kind of problems that would bring,
> they haven't made any tests at all:

That's why your mail here is very valuable : we can't guess them all (that's 
what I meant with "no proof", I should have said "not enough information to 
understand with our limited knowledge").

> 
> 1)  we know there will be cases where it won't work
>      (see above those with the DVD media and not the full network
>      media, some new dependencies won't be found on the DVD, ...).
>      Multiply by those who have installed 32 DVD, 64 DVD, dual arch DVD,
> ... There are lots of different scenarios to deal with ...

I think most of us are ready to ask users to add the full media set along with 
updates media. The main difficulty is handling the transition, but this is 
doable too. For people who might disable release media, this may be a problem, 
but is there really an interesting use case for disabling release media and 
activating updates media ? And if there is one, 

> 
> 2) what's more, its not the time to do so, X months after the release

3.5 months, and the release will be supported for the coming 12.5 months, so I 
think it's worth finding a solution now.

> 
> 3) MgaUpdate will be _much_ slower (compare the startup time
>     of rpmdrake vs MgaUpdate) because all synthesis
>    have to be parsed, then we need to compute / verify a far greater
>    number of potential updates ... (and it's _not_ O(n))
>    just look at how much faster to start is mgaupdate vs rpmdrake)
>   It will also consume a lot more RAM

Here I don't understand : can't MageiaUpdate make use of the Release media 
only when it really performs the installation ? Or only if an update is 
ignore/rejected because of a missing dep ? That's extra code, sure, but there 
might be ways to do things so that extra CPU and RAM is used only when needed.

> 
> 4) that means that we must also update mgaonline to change the way
>      it computes if there any available updates (so that it
>      doesn't reject/ignore  updates with missing Requires that can
>      be resolved from */release

Indeed

> 
> 5) That means mgaapplet too will use quite a lot more time in
>      order to compute if there any updates
> 
>  6) That means mgaaplet will consume more RAM when
>    it checks the updates (more exactly the son process it
>    forks)
>    however people are already complaining about RAM usage
>    in mgapapplet
> 

Not necessarily, if you do it in 2 steps : first compute list without release 
media, and if updates have been rejected/ignored because of missing deps, try 
again with release media. You will have that extra RAM and CPU usage only when 
necessary.

> 7) rpmdrake behavior may change in unforeseen ways
>     (because a a shared algorithm will changes)

This one I can't tell, I don't know what shared algorithm you are talking 
about.

> 
> 8) Fred Lepied, Frederic Crozat & Warly have removed quite
>    a lot of code everywhere since that policy went years (nearly
>    a decade ago)
>    that means there're a lot of tools that need potential patching
>    (urpmi, rpmdrake, mgaonline, ...)
>     for eg: checking for missing media, ...
> 
> In short, it's quite a lot more work and quite a lot more risky than
> "just" not using the --update "flag" as some think.

Indeed, I suspected that, I'm just trying to find a solution, and changing 
MageiaUpdate and mgaapplet's behaviour is one of the solutions.

> They only see that it will be less work for them (the
> work previously done by mdv's secteam for years, that is check
> for new introduced requires and if yes add them to the update media)

This is unfair, as said in various places in the previous weeks, migration 
from Mandriva 2010.2 to Mageia 1 and the fact that Mageia 1 had a lot less 
packages than Mandriva is what is causing the painful situation we have now, 
we're just trying to find a way out of this nightmare.

> It can just blow up in quite a lot of places:
> - Updates that'll work for some and not for others depending
>   on which media are enabled
> - slower Mgaupdate
> - Mgaupdate consuming more RAM
> - slower Mgaapplet
> - Mgaapplet consuming more RAM
> 
> IMHO it's obviously too risky to do such a major change and I failed
> to see why we cannot work the mdv way.
> 
> For the _next_ release, we can test & try to use --media and look if:
> performance doesn't drop too much
> But we'll still have the same issues:
> - Disparate media (various DVDs flavour vs network, ...)
> - Performance: the fact that one can have quite a lot media:
>  (3 (or 6 on biarch) updates media vs 9 (or 18 on biarch) media [1]
>   even 15 or 30 (biarch) media for those who activate
>   backport ledia [2].
> - algorithm consistency: it must work for those who enable backports
>   and those who don't, for those who enables tainted/non-free but not
>   non-free/tainted , ....
> 
> This must be thought about.
> Will you?

I'm trying.

> Will you provides patches, test scenario and do the testing?

Patches, not sure. Test scenarios is doable, testing too.

> 
> my 2 cents

Thanks 

Samuel