Getting started
Tapilot lets you ask your Android phone to open apps, tap and type. Start with one small task and check what actually happens on screen. Powered by Midscene. Everyday use needs no computer, USB cable or root.
Use an ARM64 phone running Android 11 or later and a supported vision model. The agent runs on the phone; model inference uses your configured model service.
Download and version
Get the APK from the fixed Feishu download address and allow installation from that source when Android asks. The current download is Tapilot-0.1.9-beta-arm64.apk, released on November 4, 2026. This non-debuggable beta retains the existing beta signing certificate and is not a store release. APKs with different signatures cannot update one another.
Alternative: download the same versioned APK from GitHub Releases. The release also includes SHA-256 checksums and third-party source materials.
Features depend on the installed build. Later source changes are not included in an existing APK automatically, and a capability in the source does not establish that an older download has it. Check for published updates in Settings.
Configure a model and device access
- In Settings, configure a compatible HTTPS vision-model API URL, model name and key, or choose the available ChatGPT connection option. Phone tasks require a supported vision model.
- Enable Tapilot device control in Android Accessibility settings. It supplies screenshots, gestures, system navigation and text input; Midscene locates visual targets from screenshots. Grant other permissions when needed.
- Choose automatic execution (YOLO), confirmation for critical operations, or confirmation for every operation. New installations default to automatic execution; choose the mode in the composer before sending a task.
- Start with a simple visible task and inspect the real screen and execution record. Locked or protected screens, custom text inputs and OEM background restrictions can prevent completion.
Optional background screen (experimental)
By default, phone tasks use the main screen. To try a separate screen, open Settings → Background screen (Experimental), explicitly authorize shell assistance, and follow the phone-side Wireless debugging pairing guide. No computer is needed on supported devices. Allow the floating preview, then enable Use for chat tasks. Ordinary chat tasks now use the background screen; compatibility trials are under Advanced.
Watch the floating preview to check the result. To interact manually, choose Take over and wait for execution to stop; choose Continue when ready. Closing the preview, turning the feature off or losing the helper stops the task. Reconnecting never resumes a task automatically. The feature requires Android 12 or later; app compatibility varies, and protected screens or screen-off operation are not guaranteed. Some devices require manually re-enabling Wireless debugging after a restart.
Self-hosted models over HTTP
The 0.1.9 download supports HTTP model endpoints on trusted networks.
Use an OpenAI-compatible base URL such as http://192.168.1.100:8000/v1 and a supported vision model. The phone must be able to reach the server; localhost refers to the phone itself, not your computer. The app still requires an API key. If your server does not require authentication, enter a non-empty placeholder such as local.
HTTP sends API keys, messages and screenshots without encryption. Use it only on a trusted network; prefer HTTPS otherwise. This support applies to the task model API. Optional voice polishing still requires HTTPS.
First task: calculate 12 × 8
- Make sure a Calculator app is installed and the phone is unlocked.
- Return to Tapilot and type Open Calculator and calculate 12 × 8. Send the message.
- Let Tapilot use the screen. It operates on your main display; avoid tapping or switching apps while it runs. Confirm an operation if your selected mode asks you to.
- Check that Calculator shows 96. Open the execution report to inspect the actual actions and screenshots.
- If it stops or the result is wrong, press Stop if still running, inspect the current screen and describe the problem before trying again. A reply saying “done” is not enough to verify the result.
This example uses the existing chat input. It does not require voice setup or a computer connection.
Follow a task and check its result
Add a requirement during execution, or press Stop. Retry explicitly when needed; restarting the app does not replay interrupted tasks automatically. If an outcome is uncertain, inspect the current screen before repeating an action that may already have happened.
Each phone operation ends with a separate screenshot check against the current requirements. Tasks spanning multiple pages retain each page's readings before combining them in the answer. Open the execution report to replay Midscene screenshots, model plans, actions and errors. A completion message alone is not proof when screen evidence is missing.
Voice input and conversation
Configure Volcengine or Deepgram separately in Settings → Voice input. Volcengine requires a Doubao Voice API Key, not an Ark model key. The app links to provider consoles and setup guides. Choose Check connection to save and verify the credentials.
Hold the input field to record and release to send. Move into the cancellation area and release to cancel. Optional voice polishing uses a separately configured HTTPS model; failed polishing keeps the original transcript.
On builds supporting continuous voice, use the waveform to start a conversation and configure playback for spoken replies. A foreground service maintains an active session across apps, with pause and end controls in the notification. Interruption uses a button; wake words and spoken interruption are not supported. Available controls depend on the installed build.
Operate over Wi-Fi from a computer
Use a Tapilot build with wireless control and the Android Use computer plugin. After pairing, a pinned-TLS connection carries screenshots and actions without USB or ADB.
Open pairing in the computer plugin, scan its computer QR using Tapilot, then confirm the phone. Keep the phone unlocked and let only one controller operate it. Pairing a different computer replaces the earlier authorization. Use privacy pause for private login, then resume after completing it.
Wireless and plugin support vary by build; the 0.1.9 version number alone does not establish availability.
Models can misread screens, app compatibility varies, and provider fees may apply. See privacy and permissions for screenshot, speech and report handling.