Re: Accessibility and UIInspector to find text under cursor
Re: Accessibility and UIInspector to find text under cursor
- Subject: Re: Accessibility and UIInspector to find text under cursor
- From: Bill Cheeseman <email@hidden>
- Date: Thu, 17 Aug 2006 05:20:57 -0400
- Thread-topic: Accessibility and UIInspector to find text under cursor
on 2006-08-16 9:30 PM, Gary Yuen at email@hidden wrote:
> I've looked into this further and am not sure if Accessibility can
> give me the specific word or character under the mouse. While using
> UI Browser and TextEdit, I see that some attributes for the text area
> are range for position and string for range. Do you know if it's
> possible to pass the mouse position to get the specific character
> under the mouse?
You need to find the documentation for the parameterized attribute
accessibility functions, where all this is explained. Here's a summary:
Only one of the parameterized attributes takes a 'position' parameter, so
when you're reading the screen you have to start with that attribute: range
for position. Before you can use it, you must, of course, get the current
location of the mouse on the screen using standard Carbon or Cocoa
techniques. The location is expressed as a point value.
This attribute returns the character range of a single glyph based on its
position on the screen. The returned value is the character range of the
glyph under the mouse. In a screen reader application, use the resulting
character range returned by this attribute as a route into the other
parameterized methods which take a character index, character range, or
(indirectly from the character index) line index as input. The location
field of the character range is the character index of the first character
of the glyph, and the length field of the character range is the number of
characters to the next glyph. (It is important to respect glyph boundaries.)
You can always get the UI element under the mouse using one of the other
accessibility functions, AXUIElementCopyElementAtPosition. I assume you've
figured that out already.
So: Start by getting the current mouse position. Then use that position as
input to get the range for position of the glyph under the mouse within the
text storage contained in the UI element under the mouse, which includes
both the starting point of the glyph under the mouse expressed as an index
from the beginning of the entire text storage containing that glyph, and the
length of the glyph.
>From that point on, it's up to you to parse the text in the vicinity of that
glyph. One way -- which is what UI Browser does, for convenience -- is to
get the range of the line of text in which the glyph is located. Using the
line range, you can obtain the text of the line, then break it into words
and find the word that contains the glyph. To do all this, you just need to
keep working off of the index that you obtained from the range for position
attribute, which includes the index of the glyph under the mouse.
> If AXUIElementCopyParameterizedAttributeValue is able to do this, I'm
> not sure what format the third parameter (parameter) needs to be in.
> The 2nd one (parameterized attribute) one seems to take constant like
> kAXRangeForPositionParameterizedAttribute and
> kAXStringForRangeParameterizedAttribute.
This is all explained in the documentation. The third parameter, when you're
getting range for position, is the position of the mouse expressed as a
point value. In the accessibility API, when you're dealing with point and
range values, you have to use the accessibility AXValueCreate function,
which is well documented. Here's some Cocoa code from UI Browser that may
help you to figure this out:
- (NSValue *)AXRangeForPosition:(NSValue *)glyphPosition {
NSPoint point = [glyphPosition pointValue];
CFTypeRef parameterValue = AXValueCreate(kAXValueCGPointType, &point);
CFTypeRef value = [(NSValue *)[self
objectForAttribute:kAXRangeForPositionParameterizedAttribute
parameter:parameterValue] autorelease];
if (parameterValue) CFRelease(parameterValue);
CFRange decodedValue;
if (AXValueGetValue(value, kAXValueCFRangeType, &decodedValue)) {
return [NSValue valueWithRange:NSMakeRange(decodedValue.location,
decodedValue.length)];
}
return nil;
}
Where the method -objectForAttribute:parameter: is coded like this (leaving
out some details if an error occurs):
- (CFTypeRef)objectForAttribute:(CFStringRef)attribute
parameter:(CFTypeRef)parameter {
CFTypeRef value;
AXError err = AXUIElementCopyParameterizedAttributeValue([self
elementRef], attribute, parameter, &value);
if (err == kAXErrorSuccess) return value;
// error handling code goes here
}
> I'm most interested in text with Safari. When using the UI Browser
> with Safari, I don't see any attributes that would let me specific
> text more fine-grained than the entire area.
Safari is difficult to parse, but only because it breaks the typical HTML
page up into dozens or hundreds of groups, each of which might contain text
or something else. Once you've found a group that contains text, the
parameterized attributes work on that text the same way they work on any
other text.
You can always find the UI element under the mouse by using the basic
accessibility screen reader function, AXUIElementCopyElementAtPosition. Then
use the kAXPositionAttribute attribute of the returned element to get its
position on the screen, if needed.
I hope you find this helpful. There is documentation available, and you
really need to read it and understand it before you do any coding in the
accessibility API.
--
Bill Cheeseman - email@hidden
Quechee Software, Quechee, Vermont, USA
http://www.quecheesoftware.com
PreFab Software - http://www.prefab.com/scripting.html
The AppleScript Sourcebook - http://www.AppleScriptSourcebook.com
Vermont Recipes - http://www.stepwise.com/Articles/VermontRecipes
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Accessibility-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden