From MemberWiki

2009.03.22 Howard: I added this page.

Summary

The test suite has the following objectives, in order from highest priority (1) to lowest:

Test the reference implementation
Provide a starting point for developers of custom Container/HubClient implementations

The code of the test suite is NOT meant to be used as instructional material or examples. Separate examples are provided for this purpose.

The test suite has several parts:

MH: Managed Hub tests
UH: Unmanaged Hub tests
SEC: Security tests (for Managed Hub)
PRF: Performance profiling tests (deferred)
Utilities

The structure of the MH part mirrors that of the ManagedHub architecture. Reusable tests are provided for the Hub, HubClient and Container interfaces. These tests can be invoked by implementation-specific tests, which inject the necessary implementation objects. For example, the Hub tests accept Hub objects, which can be provided by ManagedHub and HubClient tests. HubClient tests accept HubClient objects, which can be provided by various types of HubClients.

Utilities include:

BrowerBehavior: abstracts out browser differences as an object that provides information about browser functionality rather than the name of the browser.

Namespace

The Interoperability Working Group reserves the namespace, OpenAjax.htest.*, for the official test suite.

2009.03.22 Howard: Do we need this? I added this based on my comment that the hub provides a starting point for developers of custom container/hubclient impls. If we do that, we should namespace.

MH: Managed Hub scenarios

These are the Managed Hub (vs. Unmanaged Hub) scenarios.

Specifically, these are API tests and functionality tests. They do not test responses to deliberate malicious attacks (those are in section SEC instead).

MH.BAS: Base tests

Provides testing for basic ManagedHub. Also provides reusable test cases for various base interfaces, e.g. Hub, Container, HubClient.

Reusable tests do not include creation/destruction of containers or testing of features specific to Container types. In reusable tests, container instances used in tests are provided by the caller.

MH.BAS.H: Hub interface tests

These tests perform API testing for the Hub interface.

Provides test cases that are reused by all tests that test implementations of Hub (e.g. MH.MH, MH.HC).

MH.BAS.MH: Hub interface tests

These tests perform API testing for the ManagedHub class, independent of container type.

MH.BAS.CR: Container interface tests

These tests perform API testing for the Container interface, independent of implementation. Does not include creation/destruction of containers or testing of features specific to Container types. Container instances used in tests are provided by the caller.

These tests are invoked by tests that check correctness of specific Container implementations.

MH.BAS.HC: HubClient interface tests

These tests perform API testing for the HubClient interface, independent of implementation. Does not include creation/destruction of containers or testing of features specific to Container types. Container instances used in tests are provided by the caller.

These tests are invoked by tests that check correctness of specific HubClient implementations.

MH.IC: InlineContainer/InlineHubClient tests

Provides testing for InlineContainer and InlineHubClient.

2009.03.22 Howard: Loopback tests are critical, especially in inline case, which has unusual, synchronous behavior.

MH.IC.CR. Inline Container API tests

Test Inline-specific features and params.

Also construct InlineContainer and HubClient and run MH.CR.*

MH.IC.CR.D. Container API in disconnected state

Steady-state disconnected behavior

MH.IC.CR.C. Container connected state

Steady-state connected behavior

MH.IC.CR.X. Connect and disconnect dynamics and cleanup

Behavior involving connect/disconnect, including cleanup

MH.IC.HC. Inline HubClient API tests

Test Inline-specific features and params.

Also construct InlineContainer and HubClient and run MH.HC.*

MH.IC.HC.D. HubClient API in disconnected state

Steady-state disconnected behavior

MH.IC.HC.C. HubClient connected (steady) state

Steady-state connected behavior

MH.IC.HC.X. Connect and disconnect dynamics and cleanup

Behavior involving connect/disconnect, including cleanup

MH.PC: IframePMContainer/IframePMHubClient tests

Provides testing for IframePMContainer and IframePMHubClient

Tests under MH.PC.* are analogous to those under MH.IC.*

Obviously, there are additional tests and the behavior is different (more asynchronous) and may have failure modes that are not present in the inline case.

MH.FC: IframeFIMContainer/IframeFIMHubClient tests

Provides testing for IframeFIMContainer and IframeFIMHubClient

Tests for MH.FC.* under analogous to those under MH.PC.*

Obviously, the behavior is different and may have failure modes that are not present in the inline case. The params in ctors are different as well.

MH.INT: Integration tests

Integration testing using some IframeContainers, some InlineClients and an Unmanaged Hub

UH: Unmanaged Hub scenarios

Testing the Unmanaged Hub in isolation.

Mostly this involves the v1 cases.

SEC: ManagedHub security tests

SEC. ManagedHub security tests

We need tests for PM and FIM. Some may be generic IframeContainer tests.

SEC.FP. Frame phishing attacks

SEC.MF. Message forgery attacks

SEC.MIM. Man in the middle attacks

SEC.LTO. Load timeout tests

PRF: Performance profiling

2009.03.22 Howard: We may defer these. They are not required for compliance.

PRF.THRU. Throughput tests

PRF.LAT. Latency tests

PRF.INI. Initial load time for mash-up with clients

Utilities

Browser Dependencies in Tests

I forget which framework you are using for testing. If the framework does not provide a Browser Dependent Behavior object (in an extensible way, allowing us to add the methods below), we should provide an object like this along with our test suite.

If any browser-dependent behavior that is material to the test logic is included in the tests, the behavior is abstracted in the form of a BrowserBehavior object, which has operations like:


/**
 * 
 * @returns 0 for no known limit, or a finite number for a limited number of characters
 * @type {Boolean}
 */
BrowserBehavior.getMaxUrlLength = function() {};

/**
 * @returns true if postMessage is supported, else false
 * @type {Boolean}
 */
BrowserBehavior.supportsPostMessage = function() {};

/**
 * For MIM security test. Not all browsers have same
 * API for listening at the moment, unfortunately.
 * 
 * @param ...
 * @throws ...
 */
BrowserBehavior.listenForPostMessage = function() {};
    

/*
    If the behavior of unload handlers is material to the 
    tests, we could also have something about unload 
    handler behaviors here. 
*/

Test suites should use only the BrowserBehavior object to determine browser behavior, rather than including scattered code that is specifically tuned to a particular browser or that tries to detect the behavior of a particular browser.

Different implementations could be provided, but a default implementation can dynamically determine the relevant behaviors when constructed.

A BrowserBehavior object instance should be instantiated by the test suite at the beginning of the run.

I hope that this will make it easy to test a variety of technologically evolving browsers.

Hub 2.0 Test Suite Outline