GPU (Graphics Card) Acceleration

Backyard Desktop automatically configures your GPU without the need for any manual configuration. Based on hardware diagnostics, the app will optimize the number of model layers offloaded to your GPU for the best possible performance.

Mac #

Backyard AI automatically utilizes Mac Metal, Apple's hardware-accelerated graphics API, to accelerate your text generations. There is no need to change any settings to enable GPU acceleration on Mac.

Windows #

Automatic Mode

Open the Settings page in the left sidebar and navigate to the "Advanced / GPU" section. Select your dedicated GPU (do not use an integrated GPU):

Keep the "Auto" setting enabled unless you are an advanced user.

Backyard AI will now use your GPU for text generation!

Manual Mode

Advanced users can set their virtual RAM (VRAM) allocation manually. Please exercise caution if you choose this option. Every system configuration uses VRAM differently. If you get within ~300MB of your GPU's available VRAM, you will start using shared memory, which is slow and inefficient.

Backyard AI's automatic GPU mode should always "just work", while manual may need regular adjustments even if it worked on previous runs.