## Run docs evals from GitHub

Use Lightsage slash commands on GitHub pull requests to run docs evals against staging documentation.

Use GitHub PR evals to test documentation changes against a staging docs URL before merging. Lightsage runs the configured custom evals for the connected docs source and stores the results in your eval history.

## Commands

| Command                       | What it does                                        |
|-------------------------------|----------------------------------------------------|
| `/lightsage run evals`      | Queues docs evals for the pull request.            |
| `/lightsage status`         | Shows the latest Lightsage eval status for the pull request. |
| `/lightsage`                | Shows available Lightsage commands.                  |
| `/lightsage help`           | Shows available Lightsage commands.                  |

Put the command on the first non-empty line of a pull request comment.

## Run evals

**1. Confirm the pull request has a staging docs URL**

The URL can appear in the PR body, a PR comment, or a deployment attached to the pull request.

**2. Comment on the pull request**

Add the run command:

```text theme={null}
/lightsage run evals
```

**3. Wait for Lightsage to reply**

Lightsage posts a comment when the evals are queued or when it needs more information.

**4. Review the eval history**

In Lightsage, open **Custom evals** and filter the history source to **PR bot**.

## Include the docs URL in the command comment

If the pull request has more than one preview URL, include the docs URL in the same comment as the command:

```text theme={null}
/lightsage run evals

Docs preview: https://example-docs-preview.vercel.app
```

Lightsage uses that URL for the eval run.

## How Lightsage finds the docs URL

Lightsage searches the pull request context for a docs URL. It checks:

* The `/lightsage run evals` comment.
* The pull request body.
* Other pull request comments.
* Deployments attached to the pull request.

If Lightsage cannot find a docs URL, it does not queue evals. It posts a comment explaining that a staging docs URL is required.

## Permissions

The comment author must have write, maintain, or admin access to the repository. This prevents drive-by comments from running evals and using credits.

The repository also needs the Lightsage GitHub App installed and connected to a Lightsage docs source.

## Check status

Use `/lightsage status` to see the latest Lightsage eval activity for the pull request:

```text theme={null}
/lightsage status
```

Lightsage replies with the latest queued or running eval status it can find for that PR.

## Troubleshooting

**Lightsage cannot find a docs URL**  
Add the staging docs URL to the `/lightsage run evals` comment, the PR body, or a PR comment. Then run the command again.

**Lightsage says the command is not authorized**  
Ask someone with write, maintain, or admin access to run the command, or update the comment author's repository permissions.

**No evals are configured**  
Configure custom evals for the connected docs source in Lightsage, then run the command again.

**The run used the wrong preview URL**  
Run the command again and include the intended URL directly in the command comment.