19

I need help to find out, how Dictionary app showing following popup dialog for selected text on pressing CMD+CTRL+D on any application. I want to implement the same kind of functionality for my cocoa app, where my app will run in background and showing suggestions on some hot key press for the selected text.

Dictionary app hot key suggestion dialog

I have already implemented hot key capturing, i just need to have code to get the rectangle area of selected text on screen, so i can show the dialog like dictionary app.

Thanks

2
  • how did you show the popover out of the limits of your own app? Nov 9, 2013 at 0:06
  • @AmitSri ,How did you display suggestion window on any another application? Dec 19, 2018 at 12:42

3 Answers 3

23
+50

You can use the accessibility APIs for that. Make sure that the "Enable access for assistive devices" setting is checked (in System Preferences / Universal Access).

The following code snippet will determine the bounds (in screen coordinates) of the selected text in most applications. Unfortunately, it doesn't work in Mail and Safari, because they use private accessibility attributes. It's probably possible to get it to work there as well, but it requires more work and possibly private API calls.

AXUIElementRef systemWideElement = AXUIElementCreateSystemWide();
AXUIElementRef focussedElement = NULL;
AXError error = AXUIElementCopyAttributeValue(systemWideElement, kAXFocusedUIElementAttribute, (CFTypeRef *)&focussedElement);
if (error != kAXErrorSuccess) {
    NSLog(@"Could not get focussed element");
} else {
    AXValueRef selectedRangeValue = NULL;
    AXError getSelectedRangeError = AXUIElementCopyAttributeValue(focussedElement, kAXSelectedTextRangeAttribute, (CFTypeRef *)&selectedRangeValue);
    if (getSelectedRangeError == kAXErrorSuccess) {
        CFRange selectedRange;
        AXValueGetValue(selectedRangeValue, kAXValueCFRangeType, &selectedRange);
        AXValueRef selectionBoundsValue = NULL;
        AXError getSelectionBoundsError = AXUIElementCopyParameterizedAttributeValue(focussedElement, kAXBoundsForRangeParameterizedAttribute, selectedRangeValue, (CFTypeRef *)&selectionBoundsValue);
        CFRelease(selectedRangeValue);
        if (getSelectionBoundsError == kAXErrorSuccess) {
            CGRect selectionBounds;
            AXValueGetValue(selectionBoundsValue, kAXValueCGRectType, &selectionBounds);
            NSLog(@"Selection bounds: %@", NSStringFromRect(NSRectFromCGRect(selectionBounds)));
        } else {
            NSLog(@"Could not get bounds for selected range");
        }
        if (selectionBoundsValue != NULL) CFRelease(selectionBoundsValue);
    } else {
        NSLog(@"Could not get selected range");
    }
}
if (focussedElement != NULL) CFRelease(focussedElement);
CFRelease(systemWideElement);
9
  • Hi, thanks for your code. I checked your code and seem to be working and i can get the selection bound in TextEdit, but the problem is when i try to show the popup dialog on that particular rect, then popup is showing wrong. Basically, its x position is working fine, but not y position. Please let me know, do i need to do some conversion to get the correct x and y locations relative to screen.
    – AmitSri
    Jul 11, 2011 at 4:58
  • 2
    The coordinates are vertically flipped. To get a frame for your panel, you'll have to convert the y-coordinate like this: selectionBounds.origin.y = [[NSScreen mainScreen] frame].size.height - selectionBounds.origin.y - selectionBounds.size.height (simplified a bit, assumes that you only have one screen).
    – omz
    Jul 11, 2011 at 5:52
  • Perfect. Thanks again for helping me. I am wondering if i can use the same code to get the insertion point position, just like the selection rect.
    – AmitSri
    Jul 11, 2011 at 5:59
  • 1
    Generally, it's possible, but some cases are quite difficult to handle. Without a selection, the selection range has a length of 0. You can't get the bounds of an empty range, so you could construct the range of the previous character (length 1) and get the bounds of that (the right edge of it being the insertion point). However, this won't work if the text is empty or the insertion point is at the start of a new line. You could synthesize a keypress (like space), get the bounds of the character just entered and delete it again... Not really a beautiful solution though...
    – omz
    Jul 11, 2011 at 6:30
  • 1
    @kulss Accessibility is not meant to work in sandboxed apps, so App Store will not be an option if you choose this route. Feb 22 at 9:20
7

Here you go for @omz answer in Swift

let systemWideElement = AXUIElementCreateSystemWide()
var focusedElement : AnyObject?

let error = AXUIElementCopyAttributeValue(systemWideElement, kAXFocusedUIElementAttribute as CFString, &focusedElement)
if (error != .success){
    print("Couldn't get the focused element. Probably a webkit application")
} else {
    var selectedRangeValue : AnyObject?
    let selectedRangeError = AXUIElementCopyAttributeValue(focusedElement as! AXUIElement, kAXSelectedTextRangeAttribute as CFString, &selectedRangeValue)
    if (selectedRangeError == .success){
        var selectedRange : CFRange?
        AXValueGetValue(selectedRangeValue as! AXValue, AXValueType(rawValue: kAXValueCFRangeType)!, &selectedRange)
        var selectRect = CGRect()
        var selectBounds : AnyObject?
        let selectedBoundsError = AXUIElementCopyParameterizedAttributeValue(focusedElement as! AXUIElement, kAXBoundsForRangeParameterizedAttribute as CFString, selectedRangeValue!, &selectBounds)
        if (selectedBoundsError == .success){
            AXValueGetValue(selectBounds as! AXValue, .cgRect, &selectRect)
            //do whatever you want with your selectRect
            print(selectRect)
        }
    }
}
-2

What you're looking for is a Service. With services, your app doesn't even have to be running or capture global hotkeys.

For example, the functionality of the dictionary app you described is actually a service, observable in the Services menu.

Dictionary Service Menu

Apple's Service Implementation Guide is probably the best info on services out there.

1
  • Thanks for your answer. I know about the services and already used in my app for grabbing text selection. But the requirement is to show the popup dialog like dictionary app on selected/highlighted on screen, just like dictionary app is showing in my post. Please let me know did i get the selection area relative to screen in service delegate? Thanks
    – AmitSri
    Jul 5, 2011 at 4:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.