README File for the OLVER Test Suite

We've checked our test suite on the following Linux distributions and
architectures:

• Debian GNU/Linux 4.0 (etch) on IA32, IA64.
• Debian GNU/Linux 5.0.3 (lenny) on x86_64.
• Moblin release 2.0 on IA32.
• Moblin release 2.1 on IA32.
• openSuSE 11.2 on x86_64.
• RedHat Enterprise Linux 4 Update 2 on IA32 и x86_64.
• Red Hat Enterprise Linux Server release 5.4 (Tikanga) on x86_64.
• SuSE Linux Enterprise Server 10.0 on IA32, IA64, PPC32, PPC64, s390, s390x.
• SuSE Linux Enterprise Server 11.0 on IA32, x86_64.
• Ubuntu 9.04 on IA32.
• Ubuntu 9.10 on IA32.
• Fedora release 8 (Werewolf) on IA32.
• Fedora release 10 (Cambridge) on x86_64.
• Fedora release 12 (Constantine) on IA32.

All failures found on these systems have been documented and published.

For building and running the test suite additional software requirements should
be met listed in "Building the Test Suite" and "Running the Test Suite" sections.

To build OLVER one need to have the following components installed in the
system:

1. LSB 3.1 Software Development Kit (SDK). You may find it on http://www.linux-foundation.org/en/Downloads
2. GNU C/C++ Compiler 3.3.5 or later
3. Java Runtime Environment version 5 or later (Sun or IBM preferred). Note that 'java' command should be on your path.
4. autoconf
5. automake
6. libtool
7. ncurses-devel package

You should prepare Test Suite to build by running configure script. You will be asked about tester
account preferred and the architecture you want to test against.

  ./configure.sh

In order to build the test suite run script under user choosen:

  ./build_conf_tests.sh

Note: during the build you'll be prompted to enter the root password.
This is needed to set appropriate privileges for the agent in order
to test ncurses and process functionality. If you have already got root
privileges password prompt will be answered automatically.

It takes approximately 25 minutes to complete the build on Athlon XP 2700+.

After successful build you should prepare additional data for test scenarios. Run:

  ./install.sh

Additional cleaning could be performed. Run script:

  ./clean_tests.sh

To run OLVER and obtain execution reports one need to have
the following components installed in the system:

  1. LSB runtime loader: /lib/ld-lsb.so.3. If it is not found, you should install LSB 3.1 Software Development Kit (SDK). You may find it on http://www.linux-foundation.org/en/Downloads

  2. Perl language interpreter.

In order to generate reports you should additionally supply either

  3a. Java Runtime Environment version 5 or later (IBM or Sun preferred) and 'java' command need to be on your path.

(which preferred) or

  3b. any fast XML::SAX based Perl module

In order to execute the test suite run script:

  ./bin/olver_run_tests [ testplan [ timeout ] ]

  tesplan is a list of test scenarios to be run, ./bin/testplan is used by default.
  To decide which scenarios must not be included in the test run, please
  open testplan and comment out lines with scenario names.

    #str_scenario (will not be executed)
    integer_scenario (will be executed)

  timeout is a scenario max lifetime. 30 minites by default.

You could choose you own directory to store the results:

  export OLVER_REPORT_DIR=...

By default it is /var/opt/lsb/test/olver-core/<timestamp>

  ./bin/olver_report

You should point out the directory with test traces as:

  export REPORT_DIR=...

You also can choose XML::SAX based parser you want.

  export OLVER_XML_PARSER=

  /var/opt/lsb/test/olver-core/<timestamp> directory.

You can use any web browser you like to see following reports:

Test Execution Summary Reports:
  1. In OLVER format: /var/opt/lsb/test/olver-core//summary.htm
  2. In TET format: /var/opt/lsb/test/olver-core//nice_tet.log

Test Execution Report:
  /var/opt/lsb/test/olver-core//report/index.html

Requirements Coverage Report:
  /var/opt/lsb/test/olver-core//result.htm.

For detailed information about test reports see http://linuxtesting.org/downloads/olver-reports.pdf