This is the ported Mozilla ECMAScript test suite used for tracking success, bugs
and missing features for the MJS project.

Also see the general information on http://www.mozilla.org/js/tests/library.html

1. How To Run Tests
2. What All Those Files Are Used For
3. What To Do After Fixing Bugs
4. What To Do After Finding Bugs
5. Details on the Test Runner
6. Incompatibilities with Mozilla's ECMAScript Implementations
7. License


1. How To Run Tests
===================

You can do a full run via  `mono runner.exe'  from this directory. This will run
all test files from mjs-most.tests that are not commented out. The test run will
be stopped when an unexpected critical failure is encountered. You can force
running the whole test suite by specifying the  `--full-run'  option.

You can run single tests like  `mono runner.exe ecma/Array/15.4-1.js'.


2. What All Those Files Are Used For
====================================

* rhino-n.tests:         Contains tests known to fail in Rhino (Mozilla Java
                         implementation) with reason of failure. For reference.

* spidermonkey-n.tests:  Contains tests known to fail in SpiderMonkey (Mozilla C
                         implementation) with reason of failure. For reference.

* runner.cs:             The source code for the test runner.

* runner.exe:            Test runner binary. Used to run one or more tests.

* mjs-most.fail:         Tests expected to fail. This is only used for tests
                         that cause the run-time or compiler to exit. This is
                         not used when the failure reason is shared with a lot
                         of other tests. (E.g. Invalid IL.) Tests in this file
                         will still be run, but they will not abort the test run
                         when they fail.

* mjs-most.tests:        Default test suite to run.

* mjs.preamble:          Infrastructure functions commonly used by the tests.
                         Prepended to every test before it is passed to the
                         compiler.

Directories: (Standards are available from http://www.mozilla.org/js/language/#)

* ecma:                  Tests for the first edition of ECMA-262.
* ecma_2:                Tests for changes by the second edition of ECMA-262.
* ecma_3:                Tests for changes by the third edition of ECMA-262.
* js1_1:                 First version of the Mozilla implementation.
* js1_2:                 Second version of the Mozilla implementation.
* js1_3:                 Third version of the Mozilla implementation.
* js1_4:                 Fourth version of the Mozilla implementation.
* js1_5:                 Some of these might appear in the next revision of
                         ECMA-262.

Tests for Java interaction and e4x support have been removed.

Here's an overview of different ECMAScript versions:

JavaScript 1.0:          The original version of the language. It was buggy and
                         is now essentially obsolete. Implemented by Netscape 2.

JavaScript 1.1:          Introduced a true Array object; most serious bugs
                         resolved. Implemented by Netscape 3.

JavaScript 1.2:          Introduced the switch statement, regular expressions,
                         and a number of other features. Almost compliant with
                         ECMA v1, but has some incompatibilities. Implemented by
                         Netscape 4.

JavaScript 1.3:          Fixed incompatibilities of JavaScript 1.2. Compliant
                         with ECMA v1. Implemented by Netscape 4.5.

JavaScript 1.4:          Implemented only in Netscape server products.

JavaScript 1.5:          Introduced exception handling. Compliant with ECMA v3.
                         Implemented by Mozilla and Netscape 6.

JScript 1.0:             Roughly equivalent to JavaScript 1.0. Implemented by
                         early releases of IE 3.

JScript 2.0:             Roughly equivalent to JavaScript 1.1. Implemented by
                         later releases of IE 3.

JScript 3.0:             Roughly equivalent to JavaScript 1.3. Compliant with
                         ECMA v1. Implemented by IE 4.

JScript 4.0:             Not implemented by any web browser.

JScript 5.0:             Supported exception handling. Partially compliant with
                         ECMA v3. Implemented by IE 5.

JScript 5.5:             Roughly equivalent to JavaScript 1.5. Fully compliant
                         with ECMA v3. Implemented by IE 5.5 and IE 6. (IE 6
                         actually implements JScript 5.6, but 5.6 is not
                         different from 5.5 in any way that is relevant to
                         client-side JavaScript programmers.)

ECMA v1:                 The first standard version of the language.
                         Standardized the basic features of JavaScript 1.1 and
                         added a few new features. Did not standardize the
                         switch statement or regular expression support.
                         Conformant implementations are JavaScript 1.3 and
                         JScript 3.0.

ECMA v2:                 A maintenance release of the standard that included
                         clarifications but defined no new features.

ECMA v3:                 Standardized the switch statement, regular expressions,
                         and exception handling. Conformant implementations are
                         JavaScript 1.5 and JScript 5.5. 

Excerpted from JavaScript: The Definitive Guide, Fourth Edition by David
Flanagan (ISBN: 0-596-00048-0), section 1.2.  Copyright 2001, O'Reilly &
Associates.  All rights reserved.


3. What To Do After Fixing Bugs
===============================

If this is a bug that happens for a lot of tests and that causes the compiler or
run-time to shutdown: (E.g. Invalid IL)

  1) Locate test files matching the bug that are commented out in
     mjs-most.tests. Go through these one at a time via  `mono runner.exe test`
     to confirm that you have fixed the bugs.

  2) Uncomment the tests as they succeed. (It is okay for single test cases from
     the file to fail as long as it no longer causes the run-time or compiler to
     exit.)

     If the tests still do not succeed, but now fail for different reason adjust
     the failure reasons and keep the tests commented out.

  3) Run the full test suite via  `mono runner.exe &> new-testlog.txt'  and
     compare the output to a known good state via  `diff -Nawurd old-testlog.txt
     new-testlog.txt'  to make sure that there is no regressions.

If this is a less common bug that causes the run-time or compiler to shutdown:

  1) Locate test files where the bug happens in mjs-most.fail. Go through these
     one at a time via  `mono runner.exe test'  to confirm that you have fixed
     the bug.

  2) Remove the test files from mjs-most.fail as they succeed. (It is okay for
     single test cases from the file to fail as long as it no longer causes the
     run-time or compiler to exit.)

  3) Run the full test suite via  `mono runner.exe &> new-testlog.txt'  and
     compare the output to a known good state via  `diff -Nawurd old-testlog.txt
     new-testlog.txt'  to make sure that there is no regressions.

If this is a bug that does not cause the run-time or compiler to shutdown:

  1) Run the full test suite via  `mono runner.exe &> new-testlog.txt'  and
     compare the output to a known good state via  `diff -Nawurd old-testlog.txt
     new-testlog.txt'  to make sure that there is no regressions.


4. What To Do After Finding Bugs
================================

If this is a bug that happens for a lot of tests and that causes the compiler or
run-time to shutdown: (E.g. Invalid IL)

  1) Locate the failing tests in mjs-most.tests.

  2) Comment them out and add the failure reason to a new line immediately
     before the test.

  3) Make sure that you have notified the developers. (Via bug tracker etc.)

If this is a less common bug that causes the run-time or compiler to shutdown:

  1) Add the failing test to mjs-most.fail.

  2) Make sure that you have notified the developers. (Via bug tracker etc.)

If this is a bug that does not cause the run-time or compiler to shutdown:

  1) Make sure that you have notified the developers. (Via bug tracker etc.)


5. Details on the Test Runner
=============================

The test runner runs either the test specified or the whole test suite.

It will stop running the test suite at the first test that causes the compiler
or run-time to shutdown that is not in the fail file unless it is run with the
`--full-run'  option. The test suite can be specified via $MJS_TEST_FILE. It
defaults to `mjs-most.tests'. The fail file can be specified via $MJS_FAIL_FILE.
It defaults to `mjs-most.fail'.

It will emit warnings for test files that don't exist.

It copies the test file to a temporary directory with the preamble prepended to
it. This directory can be specified via $MJS_TEST_DIR. It defaults to `/tmp' on
non-win32 platforms. The preamble can be specified via $MJS_TEST_PREAMBLE. It
defaults to `mjs.preamble'. The test file is called jstest.js.

It invokes the compiler on the test file. The compiler command can be specified
via $MJS_COMPILE_CMD. It defaults to `mono.exe' on win32 and `mjs' elsewhere.
The compiler arguments can be specified via $MJS_COMPILE_ARGS. They default to
`mjs.exe {0}' on win32 and `{0}' elsewhere. `{0}' will be replaced by the test
file name.

It invokes the run time on the the compiled test file. The run time command can
be specified via $MJS_RUN_CMD. It defaults to `mono.exe' on win32 and `mono'
elsewhere. The compiler arguments can be specified via $MJS_RUN_ARGS. They
default to `{0}'. `{0}' will be replaced by the compiled test's file name.

The test runner currently replaces eval() calls by calls to named functions that
are automatically defined. This works for simplistic cases, but doesn't replace
a real implementation of eval. It also replaces `new TestCase' with `TestCase'.


6. Incompatibilities with Mozilla's ECMAScript Implementations
==============================================================

Note that implementing proprietary features is okay as long as they do not
contradict behavior described in the ECMA-262 standard.

Netscape's JavaScript 1.2 Incompatibilities: (Note that some of these only
happen when version 1.2 is explicitely requested by the user.)

* The equality and inequality operators behave like the identity and
  non-identity operators. That is, == works like === and != works like !==.

* The default Object.toString() method displays the values of all properties
  defined by the object, returning a string formatted using object literal
  syntax.

* The Array.toString() method separates array elements with a comma and a space,
  instead of just a comma, and returns the list of elements within square
  brackets. In addition, string elements of the array are quoted, so that the
  result is a string in legal array literal syntax.

* When a single numeric argument n is passed to the Array() constructor, it
  returns an array with n as its single element, rather than an array of length
  n.

* When an array object is used in a numeric context, it evaluates to its length.
  When used in a boolean context, it evaluates to false if its length is 0 and
  otherwise evaluates to true.

* The Array.push() method returns the last value pushed rather than the new
  array length.

* When the Array.splice() method splices out a single element x, it returns x
  itself, rather than an array containing x as its only element. When splice()
  does not remove any elements from the array, it returns nothing instead of
  returning an empty array.

* When String.substring() is called with a starting position greater than its
  ending position, it returns the empty string rather than correctly swapping
  the arguments and returning the substring between them.

* The String.split() method displays special behavior inherited from Perl: if
  the specified separator character is a single space, it discards any leading
  and trailing whitespace in the string before splitting the remainder of the
  string.

Excerpted from JavaScript: The Definitive Guide, Fourth Edition by David
Flanagan (ISBN: 0-596-00048-0), section 11.6.  Copyright 2001, O'Reilly &
Associates.  All rights reserved.

If any tests are failing because of one of these incompatibilities their source
code should be changed to fit the ECMA-262 behavior.


7. License
==========

Everything is dual-licensed under the MPL and a MIT X11 style license.
