keyboard.py is a core script in the Navigation & Interaction category. It provides text input and hardware button simulation for the iOS Simulator, sending keystrokes and button events primarily through idb and simctl. It is distinct from text entry via navigator.py — while navigator.py first locates an element semantically before entering text into it, keyboard.py operates at a lower level, sending raw key events and hardware button presses to the simulator's currently focused element or system-level responders.
keyboard.py handles three primary interaction types:
| Feature | Description | Implementation |
|---|---|---|
| Text Typing | Sends strings to the focused element, optionally with per-character delays. | idb ui text scripts/keyboard.py133 |
| Special Keys | Sends HID key codes for keys like return, delete, tab, and arrows. | idb ui key scripts/keyboard.py163 |
| Hardware Buttons | Simulates physical buttons like home, lock, volume, and screenshot. | idb ui button scripts/keyboard.py225 |
Sources: scripts/keyboard.py8-18 scripts/keyboard.py59-66
The script is centered around the KeyboardController class scripts/keyboard.py76 which encapsulates the logic for mapping human-readable commands to iOS HID (Human Interface Device) codes and executing them via subprocess calls to idb.
The script maintains static mappings to translate string arguments into the integers or constants required by the underlying tools.
return → 40) based on Apple's UIKeyboardHIDUsage specification. scripts/keyboard.py81-93home → HOME). scripts/keyboard.py96-104The following diagram illustrates how a user's intent (e.g., "press return") is transformed into a system-level event.
Title: Natural Language to HID Mapping
Sources: scripts/keyboard.py81-93 scripts/keyboard.py143-175
type_textThe type_text method scripts/keyboard.py110-129 supports two modes of operation:
_type_single. This is token-efficient and fast. scripts/keyboard.py129--delay is provided, it iterates through the string, calling _type_single for each character followed by a time.sleep(delay). This is used to simulate human typing or wait for UI animations. scripts/keyboard.py121-127press_key and press_key_sequencepress_key: Looks up the HID code in SPECIAL_KEYS. If not found, it attempts to treat the input as a literal integer key code. scripts/keyboard.py155-161press_key_sequence: Allows sending multiple keys in a single idb command (idb ui key-sequence), reducing the overhead of multiple subprocess calls. scripts/keyboard.py176-210press_hardware_buttonThis method uses the HARDWARE_BUTTONS dictionary to translate inputs like lock or volume-up into the specific strings required by idb ui button. scripts/keyboard.py211-234
The main() function scripts/keyboard.py237-300 handles argument parsing and initializes the KeyboardController with a UDID resolved via resolve_udid scripts/keyboard.py73
Title: CLI Command Execution Flow
Sources: scripts/keyboard.py237-300 scripts/keyboard.py133-138 scripts/keyboard.py163-169 scripts/keyboard.py225-230
| Flag | Description |
|---|---|
--type | Text to type into the focused element. |
--key | Special key name (e.g., tab, delete) or HID code. Can be repeated. |
--button | Hardware button name (e.g., home, screenshot). |
--delay | Seconds to wait between characters when typing. |
--udid | Target device identifier (resolved via common.resolve_udid). |
python scripts/keyboard.py --key return scripts/keyboard.py25python scripts/keyboard.py --key delete --key delete scripts/keyboard.py28python scripts/keyboard.py --type "Slowly..." --delay 0.2 scripts/keyboard.py37python scripts/keyboard.py --button screenshot scripts/keyboard.py57keyboard.py and navigator.py overlap in text entry, but serve different architectural layers:
navigator.py --enter-text: High-level. Performs a semantic search (Accessibility Tree), calculates coordinates, taps the element to focus it, and then enters text.keyboard.py --type: Low-level. Assumes focus is already correct. Directly injects characters into the system input buffer. scripts/keyboard.py64Use keyboard.py when focus is already established (e.g., after a previous tap) or when interacting with system-level hardware buttons that have no corresponding UI element in the app's accessibility tree.
Refresh this wiki