Accessibility Minutes 2010 04 21
From MemberWiki
Present
- Mike Squillace (IBM - Co-Chair)
- Ann Abbott (IBM)
- Dylan Barrell (Deque)
- Nathan Jakubiak (Parasoft)
- Shawn Lauriat (IBM)
- Vio Onut (IBM)
- David Todd (IBM)
Minutes
MS: Will take turns scribing to make things fair - Nathan starting and then going in alphabetical order - no objections
MS: Browser compatibility - xpath library for IE
NJ: Suggest write own XPATH api that wraps browser-specifc xpath API
DB: Could do feature discovery inside that code to figure out which browser
MS: Is google xpath cross-browser?
NJ: probably, but not as efficient
MS: will write a wrapper api, in util package probably
DB: probably if Parasoft has been using google API, prefer to use that
NJ: currently deciding between newer google xpath API and Javascript-XPATH libraries
DB: What are rules using XPATH for?
MS: document.evaluate stuff
NJ: The other APIs don't add the document.evaluate method
DB: I wouldn't include the code for an open-source library, so we don't have to keep up with modifications to other library
DB: Just make reference that you need certain library in IE
SL: Could handle updating versions from subversion
DB: Not really helping people by including open source libraries in our code, but giving ourselves more work
MS: How handle rule that would be different in different browsers
SL: Propose to table that until we find an actual example
DB: Agreed
MS: Might matter when do more dynamic stuff
MS: Did anyone have chance to look at test suite Jon's team put together
NJ: Most have not seen, SL has a little bit
MS: Page that tests that rule running correctly, says what should expect, and links to applicable guideline
MS: options for how to go about this, IBM may contribute some of their test pages
MS: Where should test pages be? In repository, submitted to UIUC test suit?
SL: Think would belong next to rules in same repository, so easily associated
MS: That's was Nathan's suggestion too
DB: Let's start with that
DB: Jon had preference from web, Nathan for repository - suggest Jon could put stuff from repository onto web
VO: Agrees
MS: Group decision: put all test pages into repository
MS: Initial thoughts about the meta data and what should look like?
VO: One suggestion to have at least one passing and one failing example for each test
DB: Whenever find false positive, put culprit code into test case
DB: Don't build tests that pass for most part - build tests that are supposed to fail
SL: Not only have positive and negative test cases, but also comments documenting what is right and wrong
SL: Is that what we're saying?
MS: Not getting into techniques necessarily - but whether test should pass or fail should be in test page metadata
MS: Looking for something more than comments. If in meta tag of HTML, then tool could collect the info to write a batch run of the tests
MS: Don't have to resolve now, but wanted to put metadata notion out there
MS: Our test cases are more element-driven - one test page for image-related rules, with pass and fail cases
DB: that is good, we tend to have more test cases for things that fail
VO: we do same thing, test cases for each rule with multiple examples of pass and multiple examples of fail
VO: Need to be aware if rule starts failing after a change when should not
MS: Be thinking about what that metadata looks like
MS: I'm sure Jon will be ok with putting his test cases in to the repository, and once we get approval for IBM test cases we can have someone put them in correct format
MS: Might suggest to put metadata in meta tag - but not sure how much info can stuff in there
DB: We put it into JavaScript, into variables in global scope
DB: Then you can query it from JavaScript, seemed to be much cleaner way of doing it, since rules already in JavaScript
DB: We execute rules as you load the page
DB: But could do differently - put rule id, expected result into some JavaScript object
MS: Fine with me
NJ: To clarify, don't want test pages to directly pull in rules
NJ: Don't think he was saying that
DB: Correct, not saying that
VO: Shouldn't include any of the rule in the test page - but rule id, list of xpaths for failures and list of xpaths for passing cases
VO: Enough to identify what should pass/fail - also put in rule name
MS: We give all passing and failing elements ids, then could just list ids instead of xpath
<dylanb> OpenAjax.a11y.expectedRules = { deque_020101_a : 0, deque_020101_table: 18, deque_020101_form: 0, deque_020101_warn: 0} ;
VO: Problem with just ids: accessibility rule expects something to not have id, you have a problem
DB: Just pasted example of what we're doing: rule ids and expected number of failures
MS: Dylan and Vio to work out format
DB: Any comments on validation changes I suggested?
DB: I won't change rules, but will only make change to validation result
MS: I'm leery of adding try-catch to every rule.
DB: Don't want to prescribe to rule author what style to use to write rule
MS: Should return result or throw exception?
DB: Suggesting that not throw exception, but still should have try-catch around portions of rule that could throw exception
NJ: Agree that better to not throw exception back
SL: Suggest to pass back value that says what exception happened
DB: Was suggesting pass back exception object - but then what happens when non-exception error occurs?
SL: rule could package exception and return it
DB: Like suggestion that what is returned is exception - if one was not thrown, package one up
MS: Currently validation result returns boolean
MS: Dylan's suggestion is that value returns undefined if rule does not run, second parameter sends back failure information
MS: In any case is to avoid having the rule throw back exceptions
DB: Although good code should not assume that
MS: so rules engine writer does not have to handle exceptions
DB: There could be unexpected exceptions thrown, say if browser does it, but probably should minimize that
MS: Sounds like rule writer should be using try-catch if expect exceptional conditions, but we won't tell you where to put that
MS: currently you don't know if rule failed because there was a problem running rule or it was a real failure
MS: problem with Dylan's suggestion is that reporting will still report failure even if there was an error running it
VO: The other problem I see is if you leave rule writer to place try-catch, you still may get some exceptions
VO: I would almost want to have try-catch outside of the actual rule
DB: That is other approach: throw exception if error running rule, since have to catch exception anyway
MS: one of my suggestions about either of these approaches is that try-catch convolutes rule logic, spend more time doing that in the code than executing the rule
MS: Do we leave rules as they are now, and leave it up to rule engine writers to catch exceptions
DB: Suggest do 1 of 2 things: Rule should be defined to never throw exception, or we should get it to throw
DB: Think Vio's point is very valid. Anybody have another preference?
DB: As rule writer, try not to throw exceptions that are meaningless
MS: Still concerned how many times I need to check that things are right to proceed
DB: The caller does it instead of the rule writer
VO: Our experience is that rule works fine, but you find page where it throws exception
VO: Because of that better to catch outside rule, always cases you never thought of
MS: You are suggesting the engine writer catch the exception rather than the rule writer
DB: Also suggesting one other thing - if you pick up that something is wrong, also obligated to throw exception
VO: Maybe that exception would be more specialized with more info in it
VO: Would take care of rules needing to know about extra return values
DB: Agree with that
MS: To summarize - rule writer sticks with rule object, tries to anticipate likely cases to go wrong, but in general up to tool to handle exceptions
DB: This makes it clear if rule passed/failed
MS: Still have obligation to go back through existing rules
VO: Don't really have to - if there is a problem the rule will throw an exception
DB: Have to check cases where passing back pass/fail where not really a pass/fail
DB: But not urgent need to do that
MS: My only concern is that by default you don't have to check anything at all
MS: Don't think done with discussion quite yet
DB: Also Jon has written many rules
MS: Let's return to this topic in two weeks
MS: Also points 4 and 5 in two weeks
MS: Next meeting May 5
