v1.6.0
1.6.0 — Full UI Automation
AI can now drag, scroll, manage windows, and navigate file dialogs, with visual screenshots to see what's on screen.
New Features
- New interaction tools — Drag & drop, scroll, click by position or fuzzy text match
- Window management — Close, minimize, fullscreen, move, and resize windows
- App automation — Navigate file dialogs and interact with Dock items
- Visual snapshots — Annotated screenshots with labeled UI elements for better context
Improvements
- Clearer element labels — Labels now reflect roles (B1 for buttons, T2 for text fields) for faster identification
- More reliable automation — Element references stay consistent across multi-step workflows
- Faster screenshots — Significantly faster snapshot rendering
Bug Fixes
- Mail — Fixed reading certain emails in IMAP accounts with nested folders
- Text input — Fixed append mode (
clear_before: false) replacing text instead of appending to existing content - Frontmost app — Fixed crash when querying frontmost app with no active window (e.g., all windows minimized)