Introducing Mobile Tethering
February 28, 2024
Today we’re releasing Mobile Tethering, a new feature that brings Backyard AI to your mobile devices.
Tethering enables you to access your local AI models from anywhere without sacrificing privacy or performance. Simply leave the Backyard AI Desktop App running in the background on your Mac or PC at home, and you’ll be able to spin up LLMs, chat with custom AI characters, and more from the convenience of your mobile browser. Everything still runs on your own hardware, so it’s 100% free to use.
Tethering is completely opt-in, not enabled by default, and can be turned off anytime. The desktop app can always be used without an internet connection.
Overview #
What is Backyard AI?
Backyard AI is a desktop app for creating custom AI chatbots (also known as characters) powered by LLMs that run locally on your Mac or PC. The installation process is dead-simple and requires no configuration; local chat “just works” out of the box. Backyard AI can run on CPU with only 8GB of RAM, and if you have a GPU, the app will automatically utilize it to speed things up.
What is Tethering?
Tethering extends all the capabilities of Backyard AI’s desktop experience to mobile, by allowing you to utilize your computer’s local resources remotely from a web browser.
That means you can access all your existing models, characters, and chats on the go, while still preserving the fundamental advantages of running Backyard AI on local hardware:
- Private - All chat data is generated and stored directly on your home computer
- Dead-simple setup - Run local AI chat with zero configuration
- 100% free to use - Unlimited generations at zero cost
You can enable Tethering with one click, and we’ve built a new mobile-friendly UI that makes all of Backyard AI's features easily accessible in the browser on small screens.
How to use Tethering
-
First, make sure you are logged in to the same Backyard AI account in your desktop app and mobile browser.
- If you don’t have the Backyard AI Desktop app, please download it.
- If you don’t have an account yet, please sign up.
-
On Desktop, click on Settings > Tethering tab > set the Tethering toggle to “Enabled”.
-
In the Mobile Browser, click on Settings > set the Server toggle to “Tethering.”
For a step-by-step walkthrough with screenshots, you can also consult our docs page on Tethering.
Building Tethering #
With Tethering, Backyard AI’s locally-running desktop infrastructure can be accessed securely from your mobile browser. This section describes how we built it and the architecture behind it.
Old Architecture
Here is what our desktop app and web infrastructure looked like before Tethering:
This should look familiar to most developers who have used tRPC. For those who haven’t, we’d recommend reading through the docs. We love tRPC at Backyard AI and are proud sponsors of the project. It enables us to develop features quickly, easily refactor code, and share logic across our desktop, web, and (soon to be released) mobile app.
A fully local tRPC pipeline (shown in the dotted blue box above) worked quite well for us when we first launched, but as soon as we wanted to give our users the ability to share and download characters online via our Character Hub, we needed to introduce some way of retrieving data from an online service and displaying it inside the desktop app.
After exploring a few different options, we introduced a Request Router to our desktop app. With some Typescript & tRPC hackery, we added the following logic:
- Have one TRPC client for both the desktop & web tRPC routes.
- If it's a Character Hub or Cloud model request, forward to the web app tRPC server.
- Otherwise, forward the request to the desktop tRPC server.
This was a massive win, as we could reuse any React component on the web or desktop without sacrificing UX or DevEx (the only unshared code is a set of typed errors that can happen when calling remote resources, such as offline, auth, 500s, etc.). We have been running this archetype in production for months and have found it easy to reason about and debug.
Implementing Tethering
To support Tethering, we came up with an elegant solution that required only one new addition to our web infrastructure: a second Request Router. Here’s a diagram of the new architecture, with the new addition highlighted in green:
The Request Router in the web app has the following responsibilities:
- If an incoming request is intended for the Character Hub, always send it to the web tRPC server.
- If the user has enabled Tethering, send all other (non-Character Hub) requests to the user’s machine to be processed by the desktop application.
Some more logic is required that we won’t cover here, such as authentication and request shape validation (Zod is helpful). These are handled once globally, rather than inside the TRPC routes.
Overall, almost all of our tethering logic lives inside of the Request Router. This is a great pattern for us, as it allows us to be able to avoid repeated branching logic across the app (e.g., is_tethering booleans in every function/tRPC route).
Notes on security
- All messages between services are encrypted in transit. Messages between the Mobile Browser, the Request Router, and the tRPC Electron server are always sent over HTTPS.
- The Request Router signs all message requests to prevent response hijacking by malicious clients.
- Chat data is only stored on your local hard drive, ensuring complete privacy. We never log or store chat data for any reason.
Unlike other solutions that offer remote access to local resources, such as port forwarding, our Tethering implementation does not require you to open any ports on your machine. This drastically reduces the surface area of possible security issues, makes HTTPS default, and enables Tethering in various network/firewall configurations.
If this interests you and you want to help solve other interesting problems at Backyard AI, we’re hiring a Senior Full-stack Engineer! See our job posting for more details.
To share your feedback, please join us on Discord, Reddit, or X (Twitter).