Hi all, accessibility experts and Apple's representatives on the list,
I am developing an OS X application that acts as a menu extra. Inside the app's menu I use some custom, NSView-based menu items. All things work just great, both mouse and keyboard navigation are supported and act as expected, but I am also keen to make the app as accessible (and convenient in terms of its accessibility) as I can. Indeed, I have already made it working with VO and other accessibility clients available in OS X, where I was able to do it, based on the general documentation, sample code and guidelines from Apple. Unfortunately, I still have several unresolved issues that, to my mind, would need an extra discussion and consideration, together with Apple's accessibility authorities and experts.
I have quite a few things to discuss, and I somehow feel that they do not deserve splitting them into separate topics. Thus this initial post will be somewhat lengthy (I am sorry): I will really appreciate if you can take some time to go through those details below and respond on them.
The points that I would like to raise here and discuss with you are as follows, at the moment:
1) Additions to accessibility documentation for developers creating NSView-based classes for menu items.
While NSMenuItem does support setting a custom NSView-based-class object, I believe that Apple did not provide any good guidelines or specific recommendations so far in terms of how a custom NSView-based class created for that specific case would need to implement the proper accessibility support. Certainly, such a class follows all the general guidelines in that area, thus supporting AX attributes, actions, hit and focus testing, children objects for AX (if any), and alike. But, apparently, this specific case of a custom NSView for NSMenuItem needs some more things to be done (?) Thus I hope that this post in the Accessibility mailing list will eventually collect additional useful details and recommendations, maybe.
[ A side comment: I know that custom NSView objects for NSMenuItem’s were always considered as a special, advanced thing for developers, to be used deliberately and only when the standard features of NSMenuItem or alternative means for UI and navigation are not suitable, for whatever reason. Plus the fact that menus in OS X were deeply tied to Carbon also seems to put some limitations. (Yes I see that those limitations are getting relaxed little by little as OS X is developing.) I do remember about all that as I am asking my questions here. And my decision to use custom NSView objects for some menu items in the menu was truly intentional and on purpose. ]
2) AX role for NSView inside NSMenuItem.
Assuming a custom NSView for NSMenuItem can itself group some other objects (like buttons or sliders inside it), shall that view still have its own AX role set as NSAccessibilityMenuItemRole? or can it (safely) have some other "grouping” roles too (for example, a general NSAccessibilityGroupRole)? My internal view is that it can be allowed those extra roles depending on the exact functionality of the custom menu item. But again, an official word is what I would like to have here. That also brings us to the next few questions and observations.
3) AX actions of NSView inside NSMenuItem.
Experimenting with my own custom views in menu items as well as checking how accessibility clients in OS X interact with other applications using custom views in their own menus, I see that the general accessibility attributes of those views (like position, size, description or title, children, etc.) are respected by the clients. Just as it should be. Thus, for instance, VoiceOver will correctly announce the description or title of a custom view set for menu item and draw the VO-cursor around it. That is great news.
But when it comes to the AX actions implemented for the item (like AXPress, which would be quite appropriate for many cases), those AX actions of the NSView-based-class instance do not seem to be invoked when the instance is under the VO-cursor. It is even more interesting, as when the user moves the VO cursor over such a menu item and leave the keys and mouse intact for a while, then VO always announces, "You are currently on a menu item. To choose this menu item, press Control-Option-Space." It is currently irrespective to the actual AX action names returned by the custom NSView-based implementation (even if those action names do not include "AXPress”).
And if the names of AX actions returned by the NSView-based-class instance do include "AXPress", then pressing Control-Option-Space will not invoke the accessibilityPerformAction: method on that NSView. Even that way, it could still be tolerated if pressing the keys would instead invoke the custom action assigned to the host (real) NSMenuItem enclosing that NSView-based object. But it does not happen either, based on my tests. Which, unfortunately, means that the developers cannot properly support the AX actions associated with the NSView-based menu items in their applications. And that the VO announcements to the person with disabilities, in this case, are not proper at all.
4) Highlighting menu item with custom NSView when the VO-cursor tracks the keyboard focus.
While I have noted above that the VO utility does correctly show the VO cursor for a menu item with a custom NSView, that is not completely true.
In reality, VO does it correctly only in case when the keyboard focus and VO-cursor do not follow each other — and by default they are set to track each other in the VO utility settings. When the tracking mode is enabled, the current implementation in OS X just draws the usual menu selection background (the "blue-ish" gradient-filled rectangle) for the currently selected native ("non-custom view”) menu item. (And that is all right, as the VO-cursor is tied to the keyboard focus in that mode). But when it comes to custom NSView-based menu items, then those NSView-based-class objects never receive any notification about being selected (for example, by means of the menu:willHighlightItem: method in the menu's delegate object). Nor do they get the keyboard focus notifications (like becomeFirstResponder, which, indeed, is correctly called in all other cases where the menu selection moves to an NSView-based menu item, with the keyboard input). Nor does the drawRect: method of those NSView objects get called in this case. And, apparently (based on my tests), even checking the selected children of the AX-faux object corresponding to the parent NSMenu object is also useless in this case as it only returns an empty array when the tracking mode is in effect.
Looks like I, as a developer, do not have any means to track this situation and manage re-drawing my custom NSView object properly, to be in line with how the native menu items behave in the tracking VO mode. (Or do I miss anything in existing public APIs?)
Honestly, I would consider that as a real "show-stopper" bug (or a miss, if you will, in the current design of accessibility stuff) in OS X. But I would like to first hear comments from the accessibility experts on the list. (And, again, I do understand that the situation likely has its roots in Carbon-based origins of the OS X menus.)
5) VO double-announcing that a menu item's custom view is “interactive".
When a custom NSView-based-class object inside a menu item has both children (implementing their own specific AX actions) and its own AX action(s) [for instance, AXPress], then VO first “reads" the AX title or description of that NSView object, as expected, and then it always says "interactive interactive". That is, it repeats word "interactive" twice. I consider it to be a minor defect, to be corrected going forward: To me, repeating that word twice does not help or improve the experience for a disabled person. Annoying, at best, or even confusing.
6) VO not announcing the key combination to access child objects grouped inside a custom NSView assigned to a menu item.
Again, consider an NSView-based object (set for a menu item) that groups some other faux or NSView-based objects that have their own AX actions. The user can access them with the standard Control-Option-Shift-DownArrow key combination, to start interacting with objects inside a group. I can see that that combination is announced to the user in some other cases (like when the VO cursor is on a toolbox, for example). And the combination does work properly for NSView inside menu item, I checked it. But in this specific case, no such voice description is provided by VO. And I believe that it should not be omitted in that case too. (Am I wrong?)
Okay, those are the things that I want to bring to your attention now — and to get your feedback on how they can be better approached, going forward. Can we, developers, expect any improvements on them in the near-term, in the next coming updates to OS X?
If you feel that some of the above points are worth officially opening bugs (or enhancement requests) for Apple, please, indicate that and I will be more that glad to submit such requests. Also, if you have some ready practical recommendations on how better engineer the code, to address above points (other than not using custom NSView objects in NSMenuItem’s; or replacing the whole NSMenu with a custom “menu-simulating” NSView), then I would really appreciate your feedback.
Note: All of the above observations are based on the current public version of OS X Mavericks, 10.9.2.
Thank you for your attention and for the great job you are doing to improve the accessibility in OS X and to support us, developers!
—
Vladimir
CTO, Consultant and Developer