[Mageia-dev] Updates testing

[nicolas vigier]

On Thu, 07 Jul 2011, andre999 wrote:

> nicolas vigier a écrit :
>> On Thu, 07 Jul 2011, Damien Lallement wrote:
>>
>>> Le 07/07/2011 14:18, nicolas vigier a écrit :
>>>> On Wed, 06 Jul 2011, José Jorge wrote:
>>>>
>>>>> Le mercredi 6 juillet 2011 18:12:44, nicolas vigier a écrit :
>>>>>> Hello,
>>>>>>
>>>>>> A few package updates need testing on x86_64 (they have been tested on
>>>>>> i586, thanks to Dave Hodgins) :
>>>>>>
>>>>>> https://bugs.mageia.org/show_bug.cgi?id=1944
>>>>>> https://bugs.mageia.org/show_bug.cgi?id=1485
>>>>>> https://bugs.mageia.org/show_bug.cgi?id=1892
>>>>>> https://bugs.mageia.org/show_bug.cgi?id=1939
>>>>>
>>>>> All tested, but without a testcase, some are hard to test.
>>>>
>>>> Yes, testcase need to be done for each package. Maybe we could have a
>>>> wiki page to save all test cases, and use them when the same package is
>>>> updated again ?
>>>
>>> No need to put this on the wiki as it will be useless for 90% of them as an
>>> update request must have a test case for the bug fixed, not for the whole
>>> package.
>>
>> Yes, for the test about the bug fixed. But isn't there tests to check for
>> regressions ?
>
> The general idea sounds good.
> But that would make a pretty big wiki page, if we especially if we include 
> big apps like LibreOffice & Postgresql.
> Maybe link each bigger app to a standing bugz report for that purpose ?
> It would be pretty hard to make even close to complete regression tests for 
> bigger apps as well.  (imagine 599 tests for LibreOffice ...)

I think the goal is not to do full regression tests, as we don't have
time for this. But do minimal testing, in 1 or 2 minutes, to check that
basic functionality is still working, to check that the update didn't
break everything. For LibreOffice that could be opening a test document.
Most applications are easy to test. But some require knowledge of the
software, or some configuration files, to test, so we could keep a list
of test commands or things to do and config files.