API Tester — Verification Toolkit

Purpose: Teach an AI agent how to test API-based skills by parsing HTTP examples from markdown, executing requests, and validating responses. Optional third step in the verification pipeline for api_workflow skills.

1. Overview

The API Tester extracts HTTP examples from a skill's markdown content, executes them against the target API, and validates that responses match expected patterns. It produces an execution trace with detailed results.

This skill is only relevant when the Skill Classifier identifies a skill as api_workflow with api_access: true.

2. Input

The API Tester receives:

1. Skill content — markdown with HTTP examples
2. Base URL — the API endpoint to test against
3. Auth token — Bearer token for authenticated requests
4. Classification — from the Skill Classifier (confirms this is an API skill)

3. HTTP Example Extraction

3a. Parsing HTTP Blocks

Look for code blocks tagged with http:

```http
GET /api/skills?status=active&limit=5
Authorization: Bearer <token>
```

Parse these into structured requests:

{
  "method": "GET",
  "path": "/api/skills",
  "query": { "status": "active", "limit": "5" },
  "headers": { "Authorization": "Bearer <token>" },
  "body": null
}

3b. Parsing Response Examples

Look for JSON response blocks immediately following HTTP blocks:

```json
{
  "skills": [...],
  "total": 42
}
```

These define the expected response shape for validation.

3c. Common Patterns

4. Request Execution

4a. Safety Rules

Before executing any request:

1. Never execute destructive operations (DELETE, DROP) against production endpoints
2. Only execute GET requests unless the test environment is confirmed as safe
3. Replace placeholder tokens with actual auth tokens
4. Respect rate limits — add delays between requests, handle 429s
5. Do not send real credentials in test requests

4b. Execution Steps

For each extracted HTTP example:

1. Build the full URL: {base_url}{path}
2. Set headers (replace <token> placeholders)
3. Set body if applicable
4. Execute the request
5. Record timing and response

4c. Recording Results

For each request, create trace steps:

[
  {
    "type": "api_request",
    "timestamp": "2026-02-09T...",
    "method": "GET",
    "url": "https://example.com/api/skills",
    "headers": { "Authorization": "Bearer ey..." }
  },
  {
    "type": "api_response",
    "timestamp": "2026-02-09T...",
    "status": 200,
    "body_preview": "{\"skills\": [...], \"total\": 42}"
  }
]

5. Response Validation

5a. Status Code Validation

5b. Shape Validation

Compare the response body against the expected shape:

1. Field presence — all documented fields should exist
2. Field types — types should match (string, number, array, object)
3. Array contents — if expected shows an array, actual should too
4. Nested objects — recursively validate structure

Record an assertion for each validation:

{
  "type": "assertion",
  "timestamp": "2026-02-09T...",
  "description": "Response contains 'skills' array",
  "passed": true,
  "expected": "array",
  "actual": "array (5 items)"
}

5c. Error Path Testing

If the skill documents error responses:

1. Intentionally trigger the error (e.g., send invalid data)
2. Verify the error response matches the documented format
3. Verify the status code matches

6. Output Format

The API Tester produces an execution trace:

{
  "version": "1.0",
  "started_at": "2026-02-09T...",
  "completed_at": "2026-02-09T...",
  "steps": [
    { "type": "info", "message": "Extracted 5 HTTP examples from skill" },
    { "type": "api_request", "method": "GET", "url": "..." },
    { "type": "api_response", "status": 200, ... },
    { "type": "assertion", "description": "Status 200", "passed": true },
    { "type": "assertion", "description": "Response has 'skills' field", "passed": true },
    ...
  ],
  "summary": "5/5 API tests passed"
}

7. Scoring

API test results contribute to the quality analysis scores:

8. Limitations

• Only tests HTTP-based APIs (no GraphQL, gRPC, WebSocket)
• Cannot test endpoints requiring complex authentication flows (OAuth dance)
• Cannot test endpoints that require specific server state
• Rate limit handling is best-effort — may not work with all APIs
• Does not test idempotency or concurrency

9. Integration

This tester's trace feeds into:

• The Skill Verifier orchestrator (as part of the execution_trace)
• The overall executability and quality scores
• The verification record's execution_trace field