Backyard AI | Grammars

Resources

Android App iOS App Desktop App Docs Blog Status Changelog Company

A grammar is a set of rules specifying how the model should format its output. On Backyard AI, you can add a BNF grammar to your character to ensure that it responds with a very specific structure.

Syntax #

Grammars are written in the Backus-Naur Format (BNF).

::= Assigns an expression. E.g., number ::= (0,1,2,3,4,5,6,7,8,9).
* 0 or more occurrences of the previous item. E.g., ("x")* outputs 0 or more X's.
+ 1 or more occurrences. E.g., ("x")+ outputs 1 or more X's.
? 0 or 1 occurrence. Can be replaced by [...].
[...] Anything in brackets is optional. E.g., [-] "1" outputs 1 or -1.
x | y A choice between items. ("+" | "-") outputs + or -.
(...) Groups items. In (a | b) + c, you get a or b, then c.
^... Anything but the following. [^\r\n]* outputs any character except return or enter.
"..." Outputs text directly.
"\n" Special characters in quotes.
root Every grammar must start with a root expression. Also note that expression names cannot be capitalized.

Tips #

The grammar can guide and frame the text generation, but does not control it completely. The model will still try to output the most probable next token. If you are asking it to write a token that isn't likely to occur, it will just work around your request.
If your grammar removes all of the likely next tokens from the sampling pool, your generation will slow down or stall out.
Expressions that contain layers of [...]* or [...]+ can lead to stall-outs because the LLM needs to apply multiple layers of grammar calculations, rather than just one. Ideally, you want to give any + or * statement an "exit condition", such that each one has a distinct end. For instance ["X"]+ "\n" will produce X’s until the model chooses to generate a new line, and then that statement is done.
If two consecutive grammar expressions could generate the next token, the system must track both, leading to slowdowns and possible confusion. As such, it is best practice to have an end condition for each expression to ensure they do not overlap. A simple way to do this is to end the expression with a character that cannot be generated within the expression. For instance: text =:: [^\n]+ "\n". This expression will generate text until it generates a new line, at which point it will be complete and must move to whatever is next. If you had two of these text expressions in a row and didn't include the stop condition, then the grammar must track both simultaneously and won't treat them as unique.

The quotation mark is a part of the BNF grammar syntax. As such, to output a quotation mark, you may need to write it as \" to tell the grammar that it is a character. This is not required inside a bracket statement, as they are character-based and quotations cannot function within them.