RE: Accessibility Zoom Speak Items Under Mouse Line-by-Line Support
RE: Accessibility Zoom Speak Items Under Mouse Line-by-Line Support
- Subject: RE: Accessibility Zoom Speak Items Under Mouse Line-by-Line Support
- From: Charlie Powell <email@hidden>
- Date: Mon, 11 Apr 2016 19:20:00 +0000
- Thread-topic: Accessibility Zoom Speak Items Under Mouse Line-by-Line Support
I agree with Jonathan that letting the user configure it seems like a good approach.
Ultimately, users seem to have the expectation that they should get the same behavior for all types of controls. They don't have the notion (nor do I think they should) that certain elements get some special behavior (e.g. WebKit elements) and others don't (non-WebKit elements). So really whatever behavior WebKit has is what all elements should have.
-----Original Message-----
From: Jonathan Cohn [mailto:email@hidden]
Sent: Friday, April 1, 2016 7:41 PM
To: Josh Scotland <email@hidden>
Cc: Charlie Powell <email@hidden>; Apple Accessibility Developer List <email@hidden>
Subject: Re: Accessibility Zoom Speak Items Under Mouse Line-by-Line Support
Other systems I have worked on that speak items under the mouse have it configurable to word, line, and I believe paragraph.
Best wishes,
Jonathan
> On 1 Apr 2016, at 21:51, Josh Scotland <email@hidden> wrote:
>
> Hey Charlie,
>
> Yep, the way "Speak items under the mouse" works is different for WebKit based text engines and AppKit based text engines for the reasons you gave. The feature speaks the entire AXValue for the element under the mouse, regardless of the AXValue size.
>
> Could you file a bug for "Speak items under the mouse" to speak at better granularities for large AXValues?
>
> I'd also like to ask -- what granularity do you think would be good? Should it be line by line (could become tedious to move the mouse for every line), by sentence, or by paragraph?
>
> Thanks!
> Josh
>
>> On Mar 31, 2016, at 11:55 AM, Charlie Powell <email@hidden> wrote:
>>
>> Hi all,
>>
>> I'm looking at the functionality of the system zoom accessibility feature, specifically the piece around "Speak items under the mouse after delay" (System Preferences -> Accessibility -> Zoom -> set style to Picture-in-Picture -> More Options -> Speak items under mouse after delay). I notice that in web-based apps like Mail.app, when the mouse moves to a new line of text in the canvas, the system will read just that line of text to the user. In seemingly every other app (TextEdit, Pages, or even a sample app with just NSTextView) the system will read the entire content of the canvas instead of one line at a time.
>>
>> Looking at Mail.app under Accessibility Inspector, I see that each line of text is actually represented as an AXGroup with a single AXStaticText element underneath the AXWebArea that has focus when editing. That seems somewhat counterintuitive to me as VoiceOver doesn't seem to encounter these elements - you can't use VO+Shift+Down arrow to interact with the AXWebArea and find these individual elements. At the same time, if I enable VoiceOver and change my mouse pointer navigation preference to "Moves VoiceOver cursor" (under VoiceOver Utility -> Navigation -> Mouse pointer), VoiceOver seems to find each line individually. Again, I only see this behavior in web-based apps like Mail.app, Safari, etc. and not Pages, TextEdit, or NSTextView.
>>
>> Based on my observations, it seems that when speak items under mouse is enabled, the system will perform accessibilityHitTest: to find the relevant accessibility element, followed by asking that element for its accessibilityValue. So in the case of non-web controls, the pattern seems to be to have a single accessibility element for all the text in the canvas, hence why the system reads the entire content of that element. In the case of Mail.app, the hit testing appears to find these individual AXGroup/AXStaticText elements and reads those instead.
>>
>> Has anyone ever run into this problem before? It is expected that only things using web views get this support and other controls don't? If I'm not using a web view, does anyone know how we would need to structure our accessibility hierarchy in order to achieve this line-by-line behavior without also breaking normal VoiceOver navigation?
>>
>> Thanks,
>> Charlie
>>
>> _______________________________________________
>> Do not post admin requests to the list. They will be ignored.
>> Accessibility-dev mailing list (email@hidden)
>> Help/Unsubscribe/Update your Subscription:
>> https://na01.safelinks.protection.outlook.com/?url=https://lists.apple.com/mailman/options/accessibility-dev/jscotland%40apple.com&data=01|01|email@hidden|808adc5493d3400c62c508d35aa04831|72f988bf86f141af91ab2d7cd011db47|1&sdata=S0nKQZMbDBFxMx5KwWHiJ61cmQO/WpGgREvibTeGLq8=
>>
>> This email sent to email@hidden
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Accessibility-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
> https://na01.safelinks.protection.outlook.com/?url=https://lists.apple.com/mailman/options/accessibility-dev/jon.c.cohn%2Blists%40gmail.com&data=01|01|email@hidden|808adc5493d3400c62c508d35aa04831|72f988bf86f141af91ab2d7cd011db47|1&sdata=QnpOYNCHIPbx9zFyl7EwNaDpvygJPPjLDL9TKjbQpsI=
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Accessibility-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden