Accessibility Validation Rule Testsuite
From MemberWiki
Accessibility Home | Issue Tracker | WCAG 2.0 Rules | Rules Format 1.0 | Cache | Proposed WCAG 2.0 Techniques | Rule Review | Testsuite | Evaluation Results | Reporting Practices
Test Case Meta Data
When a rule is submitted to the task force in the format given above, the contributor is also responsible for submitting a test case for that rule. The test case is an HTML document that can demonstrate that the rule, when executed, returns a passing result on those nodes that satisfy the context of the rule and the rule logic and returns a failing result for those that satisfy the rule context but fail the rule logic. Thus, nodes that do not satisfy the rule context should be ignored.
Test cases will be managed by the support OAA Accessibility Website that includes management of Test Cases for each rule. There is a web based editor for managing the test cases and assoicated rules and pass/fail ids.
As part of the test case a JSON data structure must be included to provide information on test results. This information is encoded in JSON using a rulesCoverage object that looks like the following:
Example Test Case Meta Data
<script type="text/javascript">
var OpenAjax = OpenAjax || {} ;
OpenAjax.a11y = OpenAjax.a11y || {} ;
OpenAjax.a11y.test = OpenAjax.a11y.test || {} ;
OpenAjax.a11y.test.testId = "25";
OpenAjax.a11y.test.testTitle = "Some Form Controls Violate Rules";
OpenAjax.a11y.test.testDate = "Last updated on Dec. 12, 2010, 8:51 p.m.";
OpenAjax.a11y.test.testUrl = "http://www.oaa-accessibility.org/testsuites/test/25/";
OpenAjax.a11y.test.nextUrl = "http://www.oaa-accessibility.org/testsuites/test/26/";
OpenAjax.a11y.test.ruleTestResults = [
{
ruleId: "rule_47",
ruleLabel: "Each fieldset element should contain a legend element.",
ruleFailIds: ['id6','id8'],
rulePassIds: ['id1']
},
{
ruleId: "rule_49",
ruleLabel: "Each input element with type=text | password | checkbox | radio | file and each select and textarea element should either be referenced by the for attribute of a label element via its id attribute, or have a title attribute.",
ruleFailIds: ['id9','id10'],
rulePassIds: ['id2','id3','id4','id5','id7','id11','id12']
},
{
ruleId: "rule_50",
ruleLabel: "Input element of type=[image] must have an alt or a title attribute.",
ruleFailIds: ['id13'],
rulePassIds: []
},
{
ruleId: "rule_57",
ruleLabel: "Form controls must have unique ids.",
ruleFailIds: ['id7'],
rulePassIds: ['id2','id3','id4','id5','id9','id10','id11','id12','id13']
},
]
</script>
The ruleCoverage object should be defined in the OpenAjax.a11y namespace for clarity and interoperability of tooling and is an array of objects. Each object in this array consists of three properties:
- 'ruleId': specifies the rule for which test results are being given
- 'ruleTitle': specifies a human readable title for the rule
- 'ruleFailIds': list of IDs of node that the rule should mark as failed
- 'rulePassIds': list of IDs of node that the rule should mark as passing a rule
No test harness is maintained by this group at this time for verifying the correctness of the test case or of its validity with regard to the rule(s) that it purports to test.
