AI Verification

Verify contributor proofs with AI agents

← Back to Docs

Overview

When contributors submit proof-of-work (PRs, commits, documentation) for your skills, AI verification evaluates each proof across four dimensions and assigns a contribution weight. There are two paths:

Path A: CLI Verification
Recommended

Run verification locally with your own Anthropic API key. Your key never leaves your machine. Works with Claude Code, any AI agent, or raw curl.

Path B: Server-Side (BYOK)

Store your Anthropic key in Settings and let the server verify automatically. Convenient, but your key is stored server-side.

Step 1: Get Your API Token

Since SkillSlap uses GitHub OAuth (no email/password), CLI agents need a session token. Here's how to get one:

  1. Log in to SkillSlap in your browser
  2. Visit /api/auth/token in the same browser
  3. Copy the access_token value
  4. Use it as Authorization: Bearer <token> in API calls

Response from /api/auth/token:

{
  "access_token": "eyJhbGciOiJIUzI1NiIs...",
  "expires_at": 1706745600,
  "token_type": "Bearer"
}

Tokens expire periodically. Check expires_at (Unix timestamp) and revisit the endpoint to get a fresh token when needed.

Step 2: Verify Proofs via CLI

The workflow is: fetch pending proofs evaluate with AI post results back.

Claude Code

Paste this prompt into Claude Code (or any similar agent). Replace the placeholder values with your actual data.

I need you to verify pending contribution proofs for my SkillSlap projects.

1. Fetch my pending proofs:
   curl -H "Authorization: Bearer <YOUR_TOKEN>" \
     https://<YOUR_DOMAIN>/api/proofs/pending

2. For each proof in the response, evaluate it against the skill context.
   Score each proof on 4 dimensions (0.0 to 1.0):
   - relevance: Is this contribution relevant to the skill?
   - quality: How substantive and thorough is the work?
   - evidence: How strong is the proof (URL, description)?
   - impact: What impact does this have on the skill's value?

3. Post each result back:
   curl -X POST \
     -H "Authorization: Bearer <YOUR_TOKEN>" \
     -H "Content-Type: application/json" \
     -d '{"scores":{"relevance":0.9,"quality":0.8,"evidence":0.7,"impact":0.6},"reasoning":"..."}' \
     https://<YOUR_DOMAIN>/api/proofs/<PROOF_ID>/verify-result

Any Agent (curl)

Raw curl commands you can use from any terminal or scripting environment.

1. Fetch pending proofs:

TOKEN="your_access_token_here"
BASE="https://your-domain.com"

curl -s -H "Authorization: Bearer $TOKEN" \
  "$BASE/api/proofs/pending" | jq .

2. Post verification result:

PROOF_ID="uuid-of-the-proof"

curl -s -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "scores": {
      "relevance": 0.9,
      "quality": 0.85,
      "evidence": 0.7,
      "impact": 0.8
    },
    "reasoning": "Strong PR with tests. Directly improves the skill."
  }' \
  "$BASE/api/proofs/$PROOF_ID/verify-result" | jq .

Custom Script (Node.js)

A complete Node.js script using @anthropic-ai/sdk to fetch, evaluate, and post results automatically.

import Anthropic from "@anthropic-ai/sdk";

const BASE = "https://your-domain.com";
const TOKEN = process.env.SKILLSLAP_TOKEN;
const anthropic = new Anthropic(); // uses ANTHROPIC_API_KEY env var

async function main() {
  // 1. Fetch pending proofs
  const res = await fetch(`${BASE}/api/proofs/pending`, {
    headers: { Authorization: `Bearer ${TOKEN}` },
  });
  const { proofs } = await res.json();

  if (!proofs?.length) {
    console.log("No pending proofs.");
    return;
  }

  for (const proof of proofs) {
    console.log(`Evaluating proof ${proof.id}...`);

    // 2. Ask Claude to evaluate
    const message = await anthropic.messages.create({
      model: "claude-sonnet-4-20250514",
      max_tokens: 1024,
      messages: [{
        role: "user",
        content: `Evaluate this contribution proof.

Skill: ${proof.skill?.title} - ${proof.skill?.description}
Proof type: ${proof.proof_type}
Description: ${proof.description}
Evidence URL: ${proof.evidence_url || "None"}

Score on 4 dimensions (0.0-1.0): relevance, quality, evidence, impact.
Respond with JSON only: {"scores":{...},"reasoning":"..."}`
      }],
    });

    const text = message.content[0].type === "text"
      ? message.content[0].text : "";
    const result = JSON.parse(text);

    // 3. Post result back
    const postRes = await fetch(
      `${BASE}/api/proofs/${proof.id}/verify-result`,
      {
        method: "POST",
        headers: {
          Authorization: `Bearer ${TOKEN}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify(result),
      }
    );
    const outcome = await postRes.json();
    console.log(`  -> weight: ${outcome.weight_awarded}`);
  }
}

main().catch(console.error);

Scoring Rubric

Each proof is scored across 4 dimensions. The final weight is a weighted average:

DimensionWeightDescription
Relevance25%Is this contribution relevant to the skill?
Quality35%How substantive and thorough is the work?
Evidence20%How strong is the proof? (URL to PR, detailed description)
Impact20%What impact does this have on the skill's value?

Formula:

weight = relevance * 0.25 + quality * 0.35 + evidence * 0.20 + impact * 0.20

API Reference

GET
/api/auth/token

Returns the current session's access token. Cookie auth only (visit in browser).

Response:

{ "access_token": "...", "expires_at": 1706745600, "token_type": "Bearer" }

GET
/api/proofs/pending

Fetch all pending (unverified) proofs for skills you own.

Auth: Cookie or Bearer token

Response:

{
  "proofs": [
    {
      "id": "uuid",
      "skill_id": "uuid",
      "contributor_id": "uuid",
      "proof_type": "pull_request",
      "description": "Added error handling...",
      "evidence_url": "https://github.com/...",
      "skill": { "title": "...", "description": "...", "tags": [...] },
      ...
    }
  ]
}

POST
/api/proofs/:id/verify-result

Submit AI-computed verification scores for a proof. Only the skill owner can verify.

Auth: Cookie or Bearer token

Request body:

{
  "scores": {
    "relevance": 0.9,  // 0.0 - 1.0
    "quality": 0.85,
    "evidence": 0.7,
    "impact": 0.8
  },
  "reasoning": "Explanation of the evaluation"
}

Response (200):

{ "verified": true, "weight_awarded": 0.838, "scores": {...}, "reasoning": "..." }

Error codes:

  • 400 — Invalid JSON or scores out of range
  • 401 — Not authenticated
  • 403 — Not the skill owner
  • 404 — Proof not found
  • 409 — Proof already verified

Path B: Server-Side Verification (BYOK)

If you prefer not to run verification locally, you can store your Anthropic API key on the server. The platform will automatically verify new proofs as they come in.

  1. Go to Settings
  2. Scroll to “AI Verification”
  3. Enter your Anthropic API key (sk-ant-...)
  4. New proofs will be verified automatically using your key

Your key is stored encrypted and is only used for proof verification. For maximum security, use CLI-based verification (Path A) instead.