Open & local AI

BotPsy brings sub-billion parameter agentic AI to edge devices

Tether AI Research releases BotPsy, a 0.8B-parameter open-weight model optimized for on-device tool use and privacy.

A smartphone displaying a neural network on a microchip, symbolizing on-device AI.
Illustration generated for this article

Tether AI Research has introduced BotPsy, a new 0.8B-parameter language model designed specifically for on-device agentic workflows. Released in November 2026, this open-weight model aims to provide capable personal assistant features while keeping user data local and reducing reliance on cloud infrastructure.

What happened

Frontier cloud large language models offer immense capability but come with significant operational costs and privacy risks. Providers can change access terms, deprecate APIs, or modify models without notice, creating vulnerabilities for developers building dependent products. Additionally, not every task requires the full power of a massive model, leading to inefficient resource usage for simple, routine workloads.

To address these challenges, Tether AI Research developed BotPsy, a small specialized agent built on top of Qwen3.5-0.8B. The model is optimized for function calling and tool use, allowing it to perform tasks such as web search, calendar management, file handling, and device control directly on user hardware. By focusing on low to medium difficulty tasks, BotPsy delivers high efficiency and instruction-following reliability without the latency or cost of cloud-based inference.

The release includes open weights under the Apache 2.0 license, making it accessible for research and educational purposes. It is available in multiple precisions, including bf16, Q8, and Q4, and distributed as Hugging Face safetensors and GGUF files compatible with llama.cpp and the QVAC SDK.

How it works

BotPsy leverages a compact architecture to achieve state-of-the-art performance in its size class. Despite having only 0.8 billion parameters, it outperforms larger models like Qwen3.5-2B-Think on five out of six major agentic benchmarks. This efficiency is achieved through specialized training that enhances tool-use capabilities while preserving general knowledge.

The model excels at knowing when to act and when to refrain. In hallucination tests, it correctly declines to use tools when no appropriate option exists, scoring 86.1 on BFCL v4. It also handles multi-turn conversations effectively, maintaining context and following strict policies in complex scenarios like customer service interactions. This reduces the likelihood of repetitive reasoning loops, which dropped from 22.9% in the base model to just 1.4% in BotPsy.

Deployment is streamlined for edge environments. The availability of quantized GGUF formats allows developers to run the model on devices with limited memory and compute power. This local execution ensures that sensitive data, such as emails or financial records, never leaves the user's device, addressing key privacy concerns associated with cloud-based assistants.

Key details

  • BotPsy is a 0.8B-parameter model built on Qwen3.5-0.8B, optimized for on-device agentic tasks.
  • It outperforms all tested models with up to 1.2B parameters on six tool-use and agentic benchmarks.
  • The model reduces reasoning tokens by roughly 3x compared to its base model on complex tasks.
  • Safety improvements include a 21% unsafe response rate in red-teaming, lower than any comparable model.
  • Available in bf16, Q8, and Q4 precisions as safetensors and GGUF files for easy integration.
  • Licensed under Apache 2.0 for research and educational use, with clear limitations for high-stakes applications.

Why it matters

For software engineers and technical founders, BotPsy represents a shift toward decentralized, privacy-first AI applications. By enabling capable agents to run locally, developers can build products that do not depend on external API availability or incur variable cloud costs. This stability is crucial for applications requiring consistent performance and data sovereignty, such as personal finance managers or enterprise document handlers.

The efficiency gains also translate to better user experiences. Lower latency and reduced token usage mean faster response times on mobile devices and laptops. Developers can integrate intelligent features without worrying about battery drain or excessive heat generation, common issues with larger models running on edge hardware. This opens up new possibilities for always-on assistants that respect user privacy while delivering tangible utility.

What you can do

  • Download the BotPsy model weights from Hugging Face in safetensors or GGUF format.
  • Test the model using llama.cpp or the QVAC SDK to evaluate performance on your target hardware.
  • Benchmark BotPsy against your current on-device solutions using BFCL v4 or ACEBench metrics.
  • Explore integration into existing apps for tasks like calendar scheduling or local file search.
  • Review the intended use limitations to ensure appropriate application in low-to-medium difficulty domains.
  • Contribute to the open-source community by sharing feedback or fine-tuning results for specific workflows.
  • #ai
  • #edge computing
  • #open source
  • #llm
  • #privacy

Tools from the Bytechap store

$89

DocBento

Self-hosted document management that reads every scan and answers with page citations.

Live demo

Keep reading

All stories