{"manifest":{"name":"API Tester","version":"1.0.0","description":"Parse HTTP examples from skill markdown, execute requests, validate responses, test error paths. For API-type skills.","tags":["verification","api","tester","meta","toolkit"],"standard":"agentskills.io","standard_version":"1.0","content_checksum":"977d271c5991af22107b3439b4ad21e91c6693c7d8fa4b8edb2ddbe5f71fcb19","bundle_checksum":null,"metadata":{},"files":[]},"files":{"SKILL.md":"# API Tester — Verification Toolkit\n\n> **Purpose:** Teach an AI agent how to test API-based skills by parsing HTTP examples from markdown, executing requests, and validating responses.\n> Optional third step in the verification pipeline for `api_workflow` skills.\n\n---\n\n## 1. Overview\n\nThe API Tester extracts HTTP examples from a skill's markdown content, executes them against the target API, and validates that responses match expected patterns. It produces an execution trace with detailed results.\n\nThis skill is only relevant when the **Skill Classifier** identifies a skill as `api_workflow` with `api_access: true`.\n\n---\n\n## 2. Input\n\nThe API Tester receives:\n\n1. **Skill content** — markdown with HTTP examples\n2. **Base URL** — the API endpoint to test against\n3. **Auth token** — Bearer token for authenticated requests\n4. **Classification** — from the Skill Classifier (confirms this is an API skill)\n\n---\n\n## 3. HTTP Example Extraction\n\n### 3a. Parsing HTTP Blocks\n\nLook for code blocks tagged with `http`:\n\n````markdown\n```http\nGET /api/skills?status=active&limit=5\nAuthorization: Bearer <token>\n```\n````\n\nParse these into structured requests:\n\n```json\n{\n  \"method\": \"GET\",\n  \"path\": \"/api/skills\",\n  \"query\": { \"status\": \"active\", \"limit\": \"5\" },\n  \"headers\": { \"Authorization\": \"Bearer <token>\" },\n  \"body\": null\n}\n```\n\n### 3b. Parsing Response Examples\n\nLook for JSON response blocks immediately following HTTP blocks:\n\n````markdown\n```json\n{\n  \"skills\": [...],\n  \"total\": 42\n}\n```\n````\n\nThese define the expected response shape for validation.\n\n### 3c. Common Patterns\n\n| Pattern | Parsing Rule |\n|---------|-------------|\n| `GET /path` | No body expected |\n| `POST /path` + JSON block | Body is the JSON block |\n| `Authorization: Bearer <token>` | Replace `<token>` with actual token |\n| `Content-Type: application/json` | Body should be parsed as JSON |\n\n---\n\n## 4. Request Execution\n\n### 4a. Safety Rules\n\nBefore executing any request:\n\n1. **Never execute destructive operations** (DELETE, DROP) against production endpoints\n2. **Only execute GET requests** unless the test environment is confirmed as safe\n3. **Replace placeholder tokens** with actual auth tokens\n4. **Respect rate limits** — add delays between requests, handle 429s\n5. **Do not send real credentials** in test requests\n\n### 4b. Execution Steps\n\nFor each extracted HTTP example:\n\n1. Build the full URL: `{base_url}{path}`\n2. Set headers (replace `<token>` placeholders)\n3. Set body if applicable\n4. Execute the request\n5. Record timing and response\n\n### 4c. Recording Results\n\nFor each request, create trace steps:\n\n```json\n[\n  {\n    \"type\": \"api_request\",\n    \"timestamp\": \"2026-02-09T...\",\n    \"method\": \"GET\",\n    \"url\": \"https://example.com/api/skills\",\n    \"headers\": { \"Authorization\": \"Bearer ey...\" }\n  },\n  {\n    \"type\": \"api_response\",\n    \"timestamp\": \"2026-02-09T...\",\n    \"status\": 200,\n    \"body_preview\": \"{\\\"skills\\\": [...], \\\"total\\\": 42}\"\n  }\n]\n```\n\n---\n\n## 5. Response Validation\n\n### 5a. Status Code Validation\n\n| Expected | Acceptable | Notes |\n|----------|-----------|-------|\n| 200 | 200 | Exact match required |\n| 201 | 201 | Exact match required |\n| 4xx | 400-499 | Any client error is valid if documented |\n| Not specified | 200-299 | Assume success range |\n\n### 5b. Shape Validation\n\nCompare the response body against the expected shape:\n\n1. **Field presence** — all documented fields should exist\n2. **Field types** — types should match (string, number, array, object)\n3. **Array contents** — if expected shows an array, actual should too\n4. **Nested objects** — recursively validate structure\n\nRecord an assertion for each validation:\n\n```json\n{\n  \"type\": \"assertion\",\n  \"timestamp\": \"2026-02-09T...\",\n  \"description\": \"Response contains 'skills' array\",\n  \"passed\": true,\n  \"expected\": \"array\",\n  \"actual\": \"array (5 items)\"\n}\n```\n\n### 5c. Error Path Testing\n\nIf the skill documents error responses:\n\n1. Intentionally trigger the error (e.g., send invalid data)\n2. Verify the error response matches the documented format\n3. Verify the status code matches\n\n---\n\n## 6. Output Format\n\nThe API Tester produces an execution trace:\n\n```json\n{\n  \"version\": \"1.0\",\n  \"started_at\": \"2026-02-09T...\",\n  \"completed_at\": \"2026-02-09T...\",\n  \"steps\": [\n    { \"type\": \"info\", \"message\": \"Extracted 5 HTTP examples from skill\" },\n    { \"type\": \"api_request\", \"method\": \"GET\", \"url\": \"...\" },\n    { \"type\": \"api_response\", \"status\": 200, ... },\n    { \"type\": \"assertion\", \"description\": \"Status 200\", \"passed\": true },\n    { \"type\": \"assertion\", \"description\": \"Response has 'skills' field\", \"passed\": true },\n    ...\n  ],\n  \"summary\": \"5/5 API tests passed\"\n}\n```\n\n---\n\n## 7. Scoring\n\nAPI test results contribute to the quality analysis scores:\n\n| Dimension | Impact |\n|-----------|--------|\n| Executability | +0.1 if all tests pass, -0.1 per failure |\n| Completeness | +0.05 if error paths are tested |\n| Quality | +0.05 if response shapes match exactly |\n\n---\n\n## 8. Limitations\n\n- Only tests HTTP-based APIs (no GraphQL, gRPC, WebSocket)\n- Cannot test endpoints requiring complex authentication flows (OAuth dance)\n- Cannot test endpoints that require specific server state\n- Rate limit handling is best-effort — may not work with all APIs\n- Does not test idempotency or concurrency\n\n---\n\n## 9. Integration\n\nThis tester's trace feeds into:\n- The **Skill Verifier** orchestrator (as part of the execution_trace)\n- The overall executability and quality scores\n- The verification record's `execution_trace` field"}}