{
  "id": "perfume-mcp-benchmark-2026-10",
  "title": "Perfume MCP server benchmark: 40 questions, five servers without an API key, 20 October 2026",
  "license": "CC BY 4.0",
  "license_url": "https://creativecommons.org/licenses/by/4.0/",
  "citation": "10ML (2026). Perfume MCP server benchmark: 40 questions, five servers without an API key, 20 October 2026. https://10ml.com/data/perfume-mcp-benchmark-2026-10.json. CC BY 4.0. Accessed 2026-10-20.",
  "checked": "2026-10-20",
  "method": "On Tuesday 20 October 2026, 06:24 to 06:32 UTC, five perfume MCP servers that answer without an API key were asked the same 40 fixed questions from one machine on a home connection in Southeast Asia (Node 25.9 with plain fetch, MCP Streamable HTTP, protocol 2025-06-18, one connection per server, no caching, no retries), at most one request per second to any server, 10ML included. Per server: one initialize, one tools/list, one unscored warm-up question, then the 40 questions once; no sign-up, key or credential of any kind. The questions were written before the run and not tuned to any index: 20 well-known perfumes, 10 niche or Middle-Eastern perfumes, 3 note questions, 2 similar-scent questions, 2 house questions and 3 questions in German, French and Spanish. Each server was driven the way an agent would use its own tools: one detail call where it accepts names, a search then a detail call where it needs an id, its search tool where it has no tool for the question type. A lookup is correct when the answer contains the perfume name, the house and at least 2 of the perfume's headline notes; a note question with at least 4 entries mentioning the note or 2 well-known perfumes from a fixed list; similar and house questions need 1 or 3 perfumes from fixed lists. Latency is per question, the sum of its calls, over all 40 questions; median and 95th percentile by nearest rank. Two scoring faults found after the first pass (escaped accents in JSON; note questions first scored only against a fixed perfume list) were fixed for every server and all saved responses rescored. Perfume Picks is reported twice: at one request per second, and in a supplementary run of its 19 unanswered questions, 4 seconds apart after a 60-second pause. Perfume counts are each server's own figure and were not independently counted.",
  "coverage": {
    "date": "2026-10-20",
    "questions": 40,
    "servers_scored": 5,
    "perfumes_10ml_fragrance_index": "5,406",
    "full_editorial_profiles_10ml": 481,
    "core_profiles_10ml": "4,925",
    "parfica_correct_with_translated_note_names": 34,
    "first_pass_scores": "10ML 37, ScentVerdict 32, Parfum Bible 33, Perfume Picks 18, Parfica 2",
    "not_scored": "ScentRev (claims 126,000+ scents and 22 tools), Noteboxd, Fragella API and Fraganty (claims 109,425 perfumes) need a key or sign-in"
  },
  "caveats": [
    "One run from one place; n is 40, so a one-question gap is within noise.",
    "The set leans mainstream by design; it says nothing about perfumes beyond a mid-size index, where larger indexes would pull ahead.",
    "Each server was driven by a plain script, not a language model; a model that retries or translates would lift Parfica and Perfume Picks and shrink 10ML's single-call advantage.",
    "Latency includes the trip from Southeast Asia to each server's nearest edge: one vantage point, not a global figure.",
    "Perfume counts are each server's own figure and were not independently counted.",
    "The supplementary Perfume Picks run has no comparable latency (median_ms and p95_ms are empty)."
  ],
  "sources": [
    {
      "name": "10ML Perfume Finder (10ML Fragrance Index 2.3.0)",
      "url": "https://mcp.10ml.com/mcp",
      "checked": "2026-10-20"
    },
    {
      "name": "ScentVerdict 2.0.0",
      "url": "https://scentverdict.com/mcp",
      "checked": "2026-10-20"
    },
    {
      "name": "Parfum Bible 1.0.0",
      "url": "https://parfumbible.com/mcp",
      "checked": "2026-10-20"
    },
    {
      "name": "Perfume Picks 1.0.5",
      "url": "https://mcp.perfumepicks.app/?src=registry",
      "checked": "2026-10-20"
    },
    {
      "name": "Parfica 1.0",
      "url": "https://parfica.com/mcp",
      "checked": "2026-10-20"
    }
  ],
  "columns": [
    {
      "key": "server",
      "label": "Server",
      "type": "string"
    },
    {
      "key": "version",
      "label": "Version at test",
      "type": "string"
    },
    {
      "key": "correct",
      "label": "Correct",
      "type": "integer"
    },
    {
      "key": "total",
      "label": "Questions",
      "type": "integer"
    },
    {
      "key": "known",
      "label": "Well-known (20)",
      "type": "integer"
    },
    {
      "key": "niche",
      "label": "Niche and Middle-Eastern (10)",
      "type": "integer"
    },
    {
      "key": "notes",
      "label": "Notes (3)",
      "type": "integer"
    },
    {
      "key": "similar",
      "label": "Similar (2)",
      "type": "integer"
    },
    {
      "key": "house",
      "label": "House (2)",
      "type": "integer"
    },
    {
      "key": "non_english",
      "label": "Non-English (3)",
      "type": "integer"
    },
    {
      "key": "median_ms",
      "label": "Median latency",
      "type": "number|null",
      "unit": "ms"
    },
    {
      "key": "p95_ms",
      "label": "p95 latency",
      "type": "number|null",
      "unit": "ms"
    },
    {
      "key": "calls_per_question",
      "label": "Calls per question",
      "type": "number"
    },
    {
      "key": "error_calls",
      "label": "Errors",
      "type": "integer"
    },
    {
      "key": "rate_limit_hits",
      "label": "Rate-limit hits",
      "type": "integer"
    },
    {
      "key": "tools",
      "label": "Tools",
      "type": "integer"
    },
    {
      "key": "tools_list_kb",
      "label": "tools/list size",
      "type": "number",
      "unit": "KB"
    },
    {
      "key": "output_schemas",
      "label": "Tools with output schemas",
      "type": "integer"
    },
    {
      "key": "instructions_bytes",
      "label": "Server instructions",
      "type": "integer",
      "unit": "bytes"
    },
    {
      "key": "pyramid_lookups",
      "label": "Note pyramid in lookups (of 30)",
      "type": "integer"
    },
    {
      "key": "accords_lookups",
      "label": "Accords in lookups (of 30)",
      "type": "integer"
    },
    {
      "key": "family_lookups",
      "label": "Family in lookups (of 30)",
      "type": "integer"
    },
    {
      "key": "perfumes_claimed",
      "label": "Perfumes (own figure)",
      "type": "string|null"
    }
  ],
  "rows": [
    {
      "server": "10ML Perfume Finder",
      "version": "2.3.0",
      "correct": 39,
      "total": 40,
      "known": 20,
      "niche": 10,
      "notes": 3,
      "similar": 1,
      "house": 2,
      "non_english": 3,
      "median_ms": 232,
      "p95_ms": 691,
      "calls_per_question": 1,
      "error_calls": 0,
      "rate_limit_hits": 0,
      "tools": 15,
      "tools_list_kb": 60.5,
      "output_schemas": 15,
      "instructions_bytes": 1554,
      "pyramid_lookups": 29,
      "accords_lookups": 30,
      "family_lookups": 30,
      "perfumes_claimed": "5,406"
    },
    {
      "server": "ScentVerdict",
      "version": "2.0.0",
      "correct": 36,
      "total": 40,
      "known": 20,
      "niche": 10,
      "notes": 2,
      "similar": 0,
      "house": 2,
      "non_english": 2,
      "median_ms": 698,
      "p95_ms": 899,
      "calls_per_question": 1.85,
      "error_calls": 0,
      "rate_limit_hits": 0,
      "tools": 6,
      "tools_list_kb": 39.2,
      "output_schemas": 6,
      "instructions_bytes": 1293,
      "pyramid_lookups": 0,
      "accords_lookups": 0,
      "family_lookups": 0,
      "perfumes_claimed": "31,000+"
    },
    {
      "server": "Parfum Bible",
      "version": "1.0.0",
      "correct": 33,
      "total": 40,
      "known": 20,
      "niche": 9,
      "notes": 2,
      "similar": 0,
      "house": 0,
      "non_english": 2,
      "median_ms": 3150,
      "p95_ms": 5342,
      "calls_per_question": 2.6,
      "error_calls": 0,
      "rate_limit_hits": 0,
      "tools": 4,
      "tools_list_kb": 2.2,
      "output_schemas": 0,
      "instructions_bytes": 90,
      "pyramid_lookups": 30,
      "accords_lookups": 30,
      "family_lookups": 0,
      "perfumes_claimed": null
    },
    {
      "server": "Perfume Picks, 1 request/s",
      "version": "1.0.5",
      "correct": 18,
      "total": 40,
      "known": 18,
      "niche": 0,
      "notes": 0,
      "similar": 0,
      "house": 0,
      "non_english": 0,
      "median_ms": 1237,
      "p95_ms": 2832,
      "calls_per_question": 1,
      "error_calls": 19,
      "rate_limit_hits": 19,
      "tools": 8,
      "tools_list_kb": 6.6,
      "output_schemas": 0,
      "instructions_bytes": 0,
      "pyramid_lookups": 21,
      "accords_lookups": 21,
      "family_lookups": 21,
      "perfumes_claimed": "13,000+"
    },
    {
      "server": "Perfume Picks, rate limit respected",
      "version": "1.0.5",
      "correct": 32,
      "total": 40,
      "known": 18,
      "niche": 7,
      "notes": 2,
      "similar": 1,
      "house": 2,
      "non_english": 2,
      "median_ms": null,
      "p95_ms": null,
      "calls_per_question": 1,
      "error_calls": 0,
      "rate_limit_hits": 0,
      "tools": 8,
      "tools_list_kb": 6.6,
      "output_schemas": 0,
      "instructions_bytes": 0,
      "pyramid_lookups": 30,
      "accords_lookups": 30,
      "family_lookups": 30,
      "perfumes_claimed": "13,000+"
    },
    {
      "server": "Parfica",
      "version": "1.0",
      "correct": 7,
      "total": 40,
      "known": 2,
      "niche": 1,
      "notes": 3,
      "similar": 0,
      "house": 0,
      "non_english": 1,
      "median_ms": 1099,
      "p95_ms": 2632,
      "calls_per_question": 1.88,
      "error_calls": 0,
      "rate_limit_hits": 0,
      "tools": 3,
      "tools_list_kb": 1.1,
      "output_schemas": 0,
      "instructions_bytes": 0,
      "pyramid_lookups": 30,
      "accords_lookups": 28,
      "family_lookups": 26,
      "perfumes_claimed": null
    }
  ]
}