README for the fireware test suite
==================================

Well, you got this far, which is a good thing.  

Quick Start Guide
=================

1) Dependencies: iasl, acpidump pciutils dmidecode

Please first install iasl and acpidump:

sudo apt-get install iasl acpidump pciutils dmidecode

2) To build and install (only if building from source)

./configure
make
sudo make install

To get help:

fwts --help

To run all the tests (including S3 and S4 suspend/resume hibernate/resume tests) use:

fwts

..this will dump the results into results.log


Advanced Options Guide
======================

Running Specific Tests
----------------------

Run specific tests, e.g. check syntax of DSDT:

sudo fwts syntaxcheck

You can run multiple tests too:

sudo fwts syntaxcheck fan batttery

To see all the available tests:

sudo fwts --show-tests

Changing the output
-------------------

To dump the results to another file:

sudo fwts --results-output=myresults.log

or to stderr or stdout and redirect:

sudo fwts --results-output=stderr
sudo fwts --results-output=stdout > mylog.log

The log has various columns:

0001 17/05/10 15:00:48 wakealarm       Test ACPI Wakealarm.
0002 17/05/10 15:00:48 wakealarm       ------------------------------------------------------------------------------------------
0003 17/05/10 15:00:48 wakealarm       Check existance of /sys/class/rtc/rtc0/wakealarm.
0004 17/05/10 15:00:48 wakealarm       PASSED: test 1, Check existance of /sys/class/rtc/rtc0/wakealarm.
0005 17/05/10 15:00:48 wakealarm       Trigger RTC wakealarm.
0006 17/05/10 15:00:48 wakealarm       Trigger wakealarm for 1 seconds in the future.
0007 17/05/10 15:00:48 wakealarm       PASSED: test 2, Trigger RTC wakealarm.
0008 17/05/10 15:00:48 wakealarm       Check if wakealarm is fired.
0009 17/05/10 15:00:48 wakealarm       Trigger wakealarm for 2 seconds in the future.
0010 17/05/10 15:00:51 wakealarm       PASSED: test 3, Check if wakealarm is fired.
0011 17/05/10 15:00:51 wakealarm       Multiple wakealarm firing tests.
0012 17/05/10 15:00:51 wakealarm       Trigger wakealarm for 1 seconds in the future.
0013 17/05/10 15:00:53 wakealarm       Trigger wakealarm for 2 seconds in the future.
0014 17/05/10 15:00:56 wakealarm       Trigger wakealarm for 3 seconds in the future.
0015 17/05/10 15:01:00 wakealarm       Trigger wakealarm for 4 seconds in the future.
0016 17/05/10 15:01:05 wakealarm       PASSED: test 4, Multiple wakealarm firing tests.
0017 17/05/10 15:01:05 wakealarm       ==========================================================================================
0018 17/05/10 15:01:05 wakealarm       4 passed, 0 failed, 0 warnings, 0 aborted.
0019 17/05/10 15:01:05 wakealarm       ==========================================================================================

^    ^        ^        ^
|    |        |        |
|    |        |        +--- %owner (name of test that ran)
|    |        |
|    |        +--- %time
|    |
|    +--- %date 
|
+--- %line (line number)

Also, there is an optional %field that indicates the type of message being logged:

sudo fwts wakealarm --log-format="%line %data %time %field %owner"

generates results.log containing:

0001 data 15:05:13 HED wakealarm      Test ACPI Wakealarm.
0002 data 15:05:13 SEP wakealarm      -------------------------------------------------------------------------------------------
0003 data 15:05:13 INF wakealarm      Check existance of /sys/class/rtc/rtc0/wakealarm.
0004 data 15:05:13 RES wakealarm      PASSED: test 1, Check existance of /sys/class/rtc/rtc0/wakealarm.
0005 data 15:05:13 INF wakealarm      Trigger RTC wakealarm.
0006 data 15:05:13 INF wakealarm      Trigger wakealarm for 1 seconds in the future.
0007 data 15:05:13 RES wakealarm      PASSED: test 2, Trigger RTC wakealarm.
0008 data 15:05:13 INF wakealarm      Check if wakealarm is fired.
0009 data 15:05:13 INF wakealarm      Trigger wakealarm for 2 seconds in the future.
0010 data 15:05:16 RES wakealarm      PASSED: test 3, Check if wakealarm is fired.
0011 data 15:05:16 INF wakealarm      Multiple wakealarm firing tests.
0012 data 15:05:16 INF wakealarm      Trigger wakealarm for 1 seconds in the future.
0013 data 15:05:18 INF wakealarm      Trigger wakealarm for 2 seconds in the future.
0014 data 15:05:21 INF wakealarm      Trigger wakealarm for 3 seconds in the future.
0015 data 15:05:25 INF wakealarm      Trigger wakealarm for 4 seconds in the future.
0016 data 15:05:30 RES wakealarm      PASSED: test 4, Multiple wakealarm firing tests.
0017 data 15:05:30 SEP wakealarm      ===========================================================================================
0018 data 15:05:30 SUM wakealarm      4 passed, 0 failed, 0 warnings, 0 aborted.
0019 data 15:05:30 SEP wakealarm      ===========================================================================================
                    ^
                    |
 +------------------+
 |
HED = heading
INF = information fields
SEP = separator (just pretty printing)
RES = result (PASSED or FAILED)
SUM = summary
ERR = error (something fatal happened!)
WRN = warning (not a critical failure, but may need looking at)
DBG = debug output
NLN = new line (empty field)

use fwts --log-fields to show all available fields

One can filter on these fields using --log-filter, e.g.

sudo fwts --log-filter=RES,SUM  
    .. just collect results and summary of tests

sudo fwts --log-filter=ALL,~SEP
    .. all fields except the separators

..it's quite flexible!

One can also re-arrange or modify the log format using --log-format, e.g.

sudo fwts --log-format="%line %field %time: "  mtrr

this produces:

0001 HED 15:08:38: MTRR validation.
0002 SEP 15:08:38: --------------------------------------------------------------------------------------------------------------
0003 INF 15:08:38: MTRR overview
0004 INF 15:08:38: -------------
0005 INF 15:08:38: 0x00000000 - 0x80000000   write-back
0006 INF 15:08:38: 0x80000000 - 0xc0000000   write-back
0007 INF 15:08:38: 0xbf700000 - 0xbf800000   uncached
0008 INF 15:08:38: 0xbf800000 - 0xc0000000   uncached
0009 INF 15:08:38: 0xd0000000 - 0xe0000000   write-combining
0010 INF 15:08:38:
0011 INF 15:08:38: This test validates the MTRR IOMEM setup.
0012 RES 15:08:38: FAILED [MEDIUM]: test 1, Memory range 0xd0000000 to 0xdfffffff (0000:00:02.0) has incorrect attribute
0013 RES 15:08:38: write-combining
0014 INF 15:08:39: This test validates the MTRR setup across all processors.
0015 RES 15:08:39: PASSED: test 2, All processors have the MTRR setup
0016 SEP 15:08:39: ==============================================================================================================
0017 SUM 15:08:39: 1 passed, 1 failed, 0 warnings, 0 aborted.
0018 SEP 15:08:39: ==============================================================================================================


Available fields are:

%line   - log line number
%time	- current time
%date 	- current date
%owner  - name of the test program (the owner of the the current test)
%field  - the 3 letter record identifier field (e.g. SEP, RES, etc..)

Test Progress Feedback
======================

Use the --show-progress to dump out the test progress to stderr. Output is as follows:

mtrr                : Test 1 of 2 started
mtrr                : Test 1 of 2 completed (0 passed, 1 failed, 0 aborted)
mtrr                : Test 2 of 2 started
mtrr                : Test 2 of 2 completed (1 passed, 0 failed, 0 aborted)
  ^
  test name            ^ progress info

It's a bit crude at the moment!


PASSED or FAILED?
=================

The --stdout-summary flag outputs to stdout PASSED or FAILED if test(s) passed or failed. It's essentially
for one-shot test modes where you want to run one test and see if it passed or failed without having to
look at the results log, e.g.

sudo fwts mtrr --stdout-summary

The exit code returns 1 for test failure and 0 for test success, so the
test suite can be run from a shell script and one can detect if the test worked or not, e.g.

#!/bin/bash
LOG=stdout
sudo fwts dmi_decode --results-output=$LOG
failed=$?