To sync prompts from a GitHub repo, place each prompt in a .json file under a folder named “prompts” in your root directory. Place each prompts test cases in a folder named “tests”. The test cases file should have the same name as the prompt file.

  • /repository-root
    • /prompts
      _ prompt-1.json
      _ prompt-2.json
    • /tests
      _ prompt-1.json
      _ prompt-2.json

We support the serliazation LangChain uses. A basic example:

{
  "input_variables": ["adjective", "content"],
  "template": "Tell me a {adjective} joke about {content}."
}

Test file:

{
  "model": "text-davinci-003",
  "params": {
    "temperature": 0.7,
    "top_p": 1
  },
  "validators": {
    "json": false,
    "links": false
  },
  "tests": [
    {
      "adjective": "dry",
      "content": "cats",
      "completion": "A joke about cats."
    },
    {
      "adjective": "funny",
      "content": "dogs",
      "completion": "A joke about dogs."
    }
  ]
}