Fleiss' kappa (N raters) API

Computes Fleiss' kappa for agreement among a fixed number of raters across many subjects from a subjects x categories count matrix, with per-category proportions. Answers 'How much do my N annotators agree beyond chance?', 'What is Fleiss kappa for this rating table?'.

Price$0.06per request

MethodPOST

Route/v1/stats/fleiss-kappa

StatusLive

MIME typeapplication/json

Rate limit120/minute

Cache0s public

psychometricsfleiss-kappainter-rateragreementreliabilitymultiple-raterskappaannotation

API URLhttps://x402.hexl.dev/v1/stats/fleiss-kappa

Integration docs

Example request

{
  "matrix": [
    [
      0,
      0,
      0,
      0,
      14
    ],
    [
      0,
      2,
      6,
      4,
      2
    ],
    [
      0,
      0,
      3,
      5,
      6
    ],
    [
      0,
      3,
      9,
      2,
      0
    ],
    [
      2,
      2,
      8,
      1,
      1
    ]
  ]
}

Example response

{
  "kappa": 0.25188,
  "meanObservedAgreement": 0.465934,
  "expectedAgreement": 0.286122,
  "nSubjects": 5,
  "nRaters": 14,
  "nCategories": 5,
  "categoryProportions": [
    0.028571,
    0.1,
    0.371429,
    0.171429,
    0.328571
  ],
  "interpretation": "fair agreement"
}

Input schema

{
  "type": "object",
  "required": [
    "matrix"
  ],
  "properties": {
    "matrix": {
      "type": "array",
      "items": {
        "type": "array",
        "items": {
          "type": "number"
        }
      },
      "description": "subjects x categories matrix of rater counts (each row sums to the rater count)",
      "examples": [
        [
          [
            0,
            0,
            0,
            0,
            14
          ],
          [
            0,
            2,
            6,
            4,
            2
          ]
        ]
      ]
    }
  }
}

Output schema

{
  "type": "object",
  "additionalProperties": true
}