Accessibility Minutes 2010 05 05

From MemberWiki

Jump to: navigation, search

Present

  • David Bolter (Mozilla)
  • Mike Squillace (IBM - Co-Chair)
  • Rich Schwerdtfeger (IBM - Co-Chair)
  • Dylan Barrell (Deque)
  • Jon Gunderson (University of Illinois)
  • Prasanna Bale (University of Illinois)
  • Nathan Jukubiak (Parasoft)
  • Shawn Lauriat (IBM)

Minutes

[09:07am] slauriat: squillace: Made a few decisions on the test suite.

[09:07am] slauriat: squillace: Decided to keep the test rules in the repository.

[09:09am] slauriat: jongunderson: Envisioning someone going to a URL to index all of the tests.

[09:10am] slauriat: jongunderson: Want to make sure everything stays in sync as people make changes. Manually managing becomes a nightmare.

[09:10am] slauriat: jongunderson: No problem with that, and can figure out a way to export from database-stored tests to SVN.

[09:11am] slauriat: dylanb: If we took your HTML pages and incorporate the JavaScript, SVN should handle merging.

[09:13am] slauriat: davidb: Between ARIA and HTML5 and assistive techs, things will move around for a while, so we need a system that we can adapt easily as everything changes.

[09:14am] slauriat: squillace: If AT vendors do not implement things, but developers do according to to the spec…

[09:16am] slauriat: jongunderson: Best practices tied to rules and explanations in the site. The database system offers linkage between all of these things for developers to test and learn as they go.

[09:17am] slauriat: jongunderson: Also allows you to query the status of rules and rulesets, rather than manually searching through static HTML.

[09:17am] slauriat: jongunderson: And a web-based interface to edit the rules.

[09:18am] slauriat: dylanb: Alternatively, you can offer everyone access to SVN.

[09:18am] slauriat: squillace: We may run into licensing issues if we do that.

[09:19am] slauriat: jongunderson: We can export to SVN, we just need to find a way to handle that, since it dynamically assembles the pages.

[09:21am] slauriat: dylanb: We should also put a comment into all of those pages to the effect that they're not the master.

[09:21am] slauriat: jongunderson: I just want to avoid being the conduit to putting all of the best practices together.

[09:22am] slauriat: squillace: Would auto-generated code conflict with the license?

[09:22am] slauriat: jongunderson: Whatever gets checked into SVN would be static, so no.

[09:23am] slauriat: slauriat: How about going the other way, going from SVN through an automated system to generate web-based UI?

[09:24am] slauriat: squillace: Sounds great, but someone would still have to build it.

[09:26am] slauriat: squillace: Not opposed to that, so long as someone can do the work.

[09:26am] slauriat: jongunderson: How will the repository be organized? Naming conventions?

[09:27am] slauriat: squillace: We haven't talked about that yet, actually.

[09:27am] dylanb: <script language="JavaScript">

[09:27am] dylanb: /<![CDATA[

[09:27am] dylanb: var OpenAjax = OpenAjax || {} ;

[09:28am] dylanb: OpenAjax.a11y = OpenAjax.a11y || {} ;

[09:28am] dylanb: OpenAjax.a11y.ruleCoverage = [

[09:28am] dylanb: {

[09:28am] dylanb: ruleId: "imgAltAttrWithForebiddenFileExt",

[09:28am] dylanb: failedXpaths: [

[09:28am] dylanb: "//img[@id='fail-one']",

[09:28am] dylanb: "//img[@id='fail-two']",

[09:28am] dylanb: "//img[@id='fail-three']",

[09:28am] dylanb: "//img[@id='fail-four']",

[09:28am] dylanb: "//img[@id='fail-five']"

[09:28am] dylanb: ],

[09:28am] dylanb: passedXpaths: [

[09:28am] dylanb: "//img[@id='pass-one']",

[09:28am] dylanb: "//img[@id='pass-two']",

[09:28am] slauriat: squillace: Why don't we come back to this after Jon looks at exporting.

[09:28am] dylanb: "//img[@id='pass-three']"

[09:28am] dylanb: ]

[09:28am] dylanb: },

[09:28am] dylanb: {

[09:28am] dylanb: ruleId: "imgAltAttrWithForebiddenWord",

[09:28am] dylanb: failedXpaths: [

[09:28am] dylanb: "//img[@id='fail-six']",

[09:28am] dylanb: "//img[@id='fail-seven']"

[09:28am] dylanb: ],

[09:28am] dylanb: passedXpaths: [

[09:28am] dylanb: "//img[@id='pass-one']",

[09:28am] dylanb: "//img[@id='pass-two']",

[09:28am] dylanb: "//img[@id='pass-three']"

[09:28am] dylanb: ]

[09:28am] dylanb: }

[09:28am] dylanb: ] ;

[09:28am] dylanb: /]]>

[09:28am] dylanb: </script>

[09:28am] slauriat: dylanb: (just pasted in metadata he and vonut put together)

[09:29am] slauriat: nathan: Feels like it's getting complicated to me. I'd prefer something simple and based around the repository.

[09:31am] slauriat: dylanb: The main problem is making changes in Jon's system, and then exported somehow, and then incorporated into your own system, just in order to run things yourself.

[09:31am] slauriat: dylanb: People will likely end up modifying exported code in order to work, and things would fall out of sync.

[09:32am] slauriat: squillace: I assumed people would submit tests like normal, and we'd manually keep them in sync.

[09:32am] slauriat: dylanb: I'd rather we just kept things in one place, so if that's Jon's system, let's just use that system.

[09:33am] slauriat: (sounds like many people will write rules in the end, so we need something easy for us all to use)

[09:34am] slauriat: jongunderson: We'll just export to the repository, so as long as people follow the naming conventions, people can submit into the database and make it work.

[09:34am] slauriat: jongunderson: The biggest problem I see in the database system: conflicts.

[09:35am] slauriat: jongunderson: I suppose that's just a conflict, though, which they'd need to resolve anyway.

[09:35am] slauriat: jongunderson: Prasanna_Bale and I will work on that together.

[09:36am] slauriat: jongunderson: Question for dylanb: in the test case, can you have more than one rule in the metadata?

[09:36am] slauriat: dylanb: In the above example, it shows multiple rules handle via array.

[09:38am] slauriat: vonut: We keep the passed XPaths, so a user can open them and look at it. Even though automated tools would likely report the failures, it's good to keep all of that information in there.

[09:38am] slauriat: dylanb: I recommend sticking with IDs when writing these tests, to make it much easier to read and manage. You can change things around in the DOM without having to change the test.

[09:39am] slauriat: squillace: We'll post all of this in the wiki, too.

[09:40am] slauriat: jongunderson: This looks really good. I can work on this now.

[09:41am] slauriat: dylanb: Nothing in there to say which ones are dynamic and which aren't.

[09:41am] slauriat: squillace: Anything else on test cases?

[09:41am] slauriat: nathan: Which system did we decide on?

[09:42am] slauriat: jongunderson: Because we'll follow naming conventions, you can use whatever tool you want to work with these and check them into the repository.

[09:42am] slauriat: jongunderson: The database system can even read from that, as well.

[09:43am] slauriat: jongunderson: This way, we don't corner anyone into using a particular tool.

[09:43am] slauriat: nathan: So we'll use the repository to maintain the test cases?

[09:44am] slauriat: jongunderson: Yes. I'll probably build all of my test cases with the database system and continually export to the repository, but you don't have to use it.

[09:44am] slauriat: vonut: Because of the source control, you'll see any conflicts anyway.

[09:46am] slauriat: squillace: To clarify: the repository contains the definitive source.

[09:47am] slauriat: squillace: Item 3, a contuation from last meeting. What happens when a rule fails to execute?

[09:47am] slauriat: squillace: Right now, result = boolean, which doesn't reflect a not-run result.

[09:48am] slauriat: squillace: dylanb introduced the idea of undefined, while tracking the error preventing the running of the rule.

[09:48am] slauriat: squillace: Tools would then need to catch errors thrown back in this case.

[09:49am] slauriat: squillace: Since catching exceptions in test cases gets convoluted, the tool would need to try/catch the test in order to handle unexpected exceptions.

[09:50am] slauriat: vonut: We'll need a failsafe around the rules anyway. We can leave rules as they are now, and if something hits an unexpected exception, it'll get handled by the tool anyway rather than cluttering up the test cases with try/catch just in hopes of preventing leaking errors to the tool.

[09:51am] slauriat: dylanb: I completely agree with you on that. We should withdraw the other proposal and make it explicitly known that code needs to throw an exception if not run.

[09:52am] slauriat: dylanb: So tests will only return a result when it runs correctly and gets a definite pass or fail. Otherwise, they need to have an error thrown back to the tool.

[09:53am] slauriat: squillace: Would it ever return null?

[09:53am] slauriat: dylanb: No, never.

[09:54am] slauriat: squillace: We'll need to go back through the rules to see if anything needs to explicitly throw an error?

[09:54am] slauriat: dylanb: Yes, though it'll throw an exception anyway elsewhere in the code to get the same effect, but explicitly throwing an exception will make it easier to find out what happened.

[09:56am] slauriat: dylanb: Exceptions coming from the browser will end up as a manual check anyway. For example, if a browser lacks a particular feature.

[09:57am] slauriat: squillace: So for now, we'll leave the rules as they are? It sounds like we should go back through at some point to clean up.

[09:58am] slauriat: vonut: In order to distinguish between exceptions coming from the browser and exceptions coming from the code, we can extend exception with an oaa class.

[09:58am] slauriat: (10:56 jongunderson, not dylanb)

[10:00am] slauriat: nathan: I need a way to distinguish between code exceptions and browser exceptions in order to prevent browser-originating exceptions from getting shown to the user.

[10:01am] slauriat: slauriat: Would the custom Error class work?

[10:01am] slauriat: nathan: I guess. I'd rather not do it that way, but I won't oppose if everyone wants to go that route.

[10:02am] slauriat: jongunderson: Rules would need to throw in cases of rules executing in browsers that don't support a given feature.

[10:03am] slauriat: dylanb: We could set a property (oaaName?) on the Error in order to send a flag back to the tool.

[10:05am] slauriat: nathan: Related question: rules need to check for certain conditions and throw an exception if conditions not met. Why would you throw an exception if you don't plan on running the rule?

[10:05am] slauriat: squillace: Need to go, but we can continue this next meeting.

[10:05am] squillace left the chat room.

[10:07am] slauriat: rich: Does the group want to include HTML5? AT vendors won't work with it for a while, but do we want to include this?

[10:07am] slauriat: dylanb: I'd say yes.

Personal tools