# API Tester — Verification Toolkit

> **Purpose:** Teach an AI agent how to test API-based skills by parsing HTTP examples from markdown, executing requests, and validating responses.
> Optional third step in the verification pipeline for `api_workflow` skills.

---

## 1. Overview

The API Tester extracts HTTP examples from a skill's markdown content, executes them against the target API, and validates that responses match expected patterns. It produces an execution trace with detailed results.

This skill is only relevant when the **Skill Classifier** identifies a skill as `api_workflow` with `api_access: true`.

---

## 2. Input

The API Tester receives:

1. **Skill content** — markdown with HTTP examples
2. **Base URL** — the API endpoint to test against
3. **Auth token** — Bearer token for authenticated requests
4. **Classification** — from the Skill Classifier (confirms this is an API skill)

---

## 3. HTTP Example Extraction

### 3a. Parsing HTTP Blocks

Look for code blocks tagged with `http`:

````markdown
```http
GET /api/skills?status=active&limit=5
Authorization: Bearer <token>
```
````

Parse these into structured requests:

```json
{
  "method": "GET",
  "path": "/api/skills",
  "query": { "status": "active", "limit": "5" },
  "headers": { "Authorization": "Bearer <token>" },
  "body": null
}
```

### 3b. Parsing Response Examples

Look for JSON response blocks immediately following HTTP blocks:

````markdown
```json
{
  "skills": [...],
  "total": 42
}
```
````

These define the expected response shape for validation.

### 3c. Common Patterns

| Pattern | Parsing Rule |
|---------|-------------|
| `GET /path` | No body expected |
| `POST /path` + JSON block | Body is the JSON block |
| `Authorization: Bearer <token>` | Replace `<token>` with actual token |
| `Content-Type: application/json` | Body should be parsed as JSON |

---

## 4. Request Execution

### 4a. Safety Rules

Before executing any request:

1. **Never execute destructive operations** (DELETE, DROP) against production endpoints
2. **Only execute GET requests** unless the test environment is confirmed as safe
3. **Replace placeholder tokens** with actual auth tokens
4. **Respect rate limits** — add delays between requests, handle 429s
5. **Do not send real credentials** in test requests

### 4b. Execution Steps

For each extracted HTTP example:

1. Build the full URL: `{base_url}{path}`
2. Set headers (replace `<token>` placeholders)
3. Set body if applicable
4. Execute the request
5. Record timing and response

### 4c. Recording Results

For each request, create trace steps:

```json
[
  {
    "type": "api_request",
    "timestamp": "2026-02-09T...",
    "method": "GET",
    "url": "https://example.com/api/skills",
    "headers": { "Authorization": "Bearer ey..." }
  },
  {
    "type": "api_response",
    "timestamp": "2026-02-09T...",
    "status": 200,
    "body_preview": "{\"skills\": [...], \"total\": 42}"
  }
]
```

---

## 5. Response Validation

### 5a. Status Code Validation

| Expected | Acceptable | Notes |
|----------|-----------|-------|
| 200 | 200 | Exact match required |
| 201 | 201 | Exact match required |
| 4xx | 400-499 | Any client error is valid if documented |
| Not specified | 200-299 | Assume success range |

### 5b. Shape Validation

Compare the response body against the expected shape:

1. **Field presence** — all documented fields should exist
2. **Field types** — types should match (string, number, array, object)
3. **Array contents** — if expected shows an array, actual should too
4. **Nested objects** — recursively validate structure

Record an assertion for each validation:

```json
{
  "type": "assertion",
  "timestamp": "2026-02-09T...",
  "description": "Response contains 'skills' array",
  "passed": true,
  "expected": "array",
  "actual": "array (5 items)"
}
```

### 5c. Error Path Testing

If the skill documents error responses:

1. Intentionally trigger the error (e.g., send invalid data)
2. Verify the error response matches the documented format
3. Verify the status code matches

---

## 6. Output Format

The API Tester produces an execution trace:

```json
{
  "version": "1.0",
  "started_at": "2026-02-09T...",
  "completed_at": "2026-02-09T...",
  "steps": [
    { "type": "info", "message": "Extracted 5 HTTP examples from skill" },
    { "type": "api_request", "method": "GET", "url": "..." },
    { "type": "api_response", "status": 200, ... },
    { "type": "assertion", "description": "Status 200", "passed": true },
    { "type": "assertion", "description": "Response has 'skills' field", "passed": true },
    ...
  ],
  "summary": "5/5 API tests passed"
}
```

---

## 7. Scoring

API test results contribute to the quality analysis scores:

| Dimension | Impact |
|-----------|--------|
| Executability | +0.1 if all tests pass, -0.1 per failure |
| Completeness | +0.05 if error paths are tested |
| Quality | +0.05 if response shapes match exactly |

---

## 8. Limitations

- Only tests HTTP-based APIs (no GraphQL, gRPC, WebSocket)
- Cannot test endpoints requiring complex authentication flows (OAuth dance)
- Cannot test endpoints that require specific server state
- Rate limit handling is best-effort — may not work with all APIs
- Does not test idempotency or concurrency

---

## 9. Integration

This tester's trace feeds into:
- The **Skill Verifier** orchestrator (as part of the execution_trace)
- The overall executability and quality scores
- The verification record's `execution_trace` field