Execute App Commands via Menus

The Problem

You know an app can do something, but you can't find the menu command. Searching through File > Edit > View > Format > Tools > Window menus wastes time. Keyboard shortcuts are impossible to remember for rarely-used commands.

The Workflow

Step 1: Ask for a Command

Describe what you want to do in natural language.

Prompt to use:

In Preview, I want to remove the background from this image.
Find the menu command for this.

Example output:

Found: Tools > Remove Background

Would you like me to run this command?

Step 2: Execute the Command

Run the command directly.

Prompt to use:

Yes, run it.

Step 3: Complex Menu Paths

For deeply nested commands, AI navigates the full path.

Prompt to use:

In Final Cut Pro, enable the comparison viewer
for color correction.

Example output:

Navigating: View > Show in Viewer > Comparison Viewer

✓ Comparison viewer enabled

Step 4: Trigger Keyboard Shortcuts

Execute shortcuts by describing the action.

Prompt to use:

Save this document and then undo my last change.

Example output:

Executing:
- ⌘S (Save)
- ⌘Z (Undo)

✓ Done

Expected Result

Instant access to any app command through natural language, without memorizing menu locations or keyboard shortcuts.

Tools Used

This workflow uses the UI Automation toolbox:

Tool	Purpose
ui_controller_select_menu	Navigate and select menu items
ui_controller_press_key	Execute keyboard shortcuts
ui_viewer_get_frontmost	Identify current app context

Related Workflows

• Batch Data Entry - Automate repetitive form filling