Import unlabeled style tokens from itemdb #30

Merged
matchu merged 6 commits from armin-bot/impress:claude/itemdb-style-sync into main 2026-11-16 21:40:49 -08:00
Contributor

Builds on #28 and #29 (this branch includes their commits). Before merging, production needs an itemdb API key set as ITEMDB_API_KEY.

Why

Some of our alt styles are still unlabeled, mostly because the Styling Studio import wasn't running reliably while they were on sale. itemdb has the names and thumbnails of those styles' tokens, and sometimes their real descriptions.

What this does

  • Adds Itemdb, a small client for itemdb's v2 API, authenticated in two steps: it trades the API key in ITEMDB_API_KEY for an hour-long token at /api/auth/token, then sends that token on each request. Following itemdb's advice, it loads their official NC Pet Styles list (/api/v2/lists/official/1037/itemdata) in one request, with pagination in case the list outgrows a page. For unnamed styles that aren't on the list yet, it falls back to /api/v2/items/many by item ID, 100 at a time.

  • Adds rake itemdb:import:styles (supports DRY_RUN=1), which:

    • labels unlabeled alt styles from the token's name and thumbnail, which also creates their token items;
    • fills in real descriptions for tokens that don't have one.
  • As a sanity check, only accepts a token name that ends in the style's species (like "Blended Burlap Aisha" for an Aisha style) or belongs to a character-named series ("Essence of…", "Treasured…"), unless run with FORCE=1. Anything else is printed for a human to handle and left alone. The series name is inferred only when the name also ends in the style's color, as in the Styling Studio import.

  • Credits itemdb on every style token's item page, with their logo and a link to the item on their site, since they ask that we be clear about where we use their data.

Token item page with the itemdb credit

What it costs itemdb

One request for the list per run, using the minimal intent. The by-ID fallback only covers unnamed styles missing from the list, which should be a handful at most.

The list is maintained by hand and can include a non-token item by mistake. We only look up our own styles' IDs in it, so stray entries are ignored.

Open before merging

  • An API key in production, set as ITEMDB_API_KEY. (itemdb offered to raise our limits once we've created one.)
  • Whether to add the task to the neopets:import cron chain.

Testing

  • Service spec with stubbed HTTP: the token exchange and reuse, the list, pagination, the by-ID fallback and error cases. A view spec covers the credit.
  • Against the live API: the list endpoint returned 4,220 tokens in one page, every one with an item ID and a real description, and the client parsed it.
  • A dry run in dev, seeded with 168 of our real unnamed styles, against the live list: all 168 would be named, 135 with a series name detected.
  • Not exercised live from Ruby: the key-for-token request itself. In my sandbox the key is injected by a proxy that the app's HTTP client doesn't go through, so I made that one request with curl and handed the token to the client.

🤖 Generated with Claude Code

https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6

Builds on #28 and #29 (this branch includes their commits). Before merging, production needs an itemdb API key set as `ITEMDB_API_KEY`. ## Why Some of our alt styles are still unlabeled, mostly because the Styling Studio import wasn't running reliably while they were on sale. itemdb has the names and thumbnails of those styles' tokens, and sometimes their real descriptions. ## What this does - Adds `Itemdb`, a small client for itemdb's v2 API, authenticated in two steps: it trades the API key in `ITEMDB_API_KEY` for an hour-long token at `/api/auth/token`, then sends that token on each request. Following itemdb's advice, it loads their official NC Pet Styles list (`/api/v2/lists/official/1037/itemdata`) in one request, with pagination in case the list outgrows a page. For unnamed styles that aren't on the list yet, it falls back to `/api/v2/items/many` by item ID, 100 at a time. - Adds `rake itemdb:import:styles` (supports `DRY_RUN=1`), which: - labels unlabeled alt styles from the token's name and thumbnail, which also creates their token items; - fills in real descriptions for tokens that don't have one. - As a sanity check, only accepts a token name that ends in the style's species (like "Blended Burlap Aisha" for an Aisha style) or belongs to a character-named series ("Essence of…", "Treasured…"), unless run with `FORCE=1`. Anything else is printed for a human to handle and left alone. The series name is inferred only when the name also ends in the style's color, as in the Styling Studio import. - Credits itemdb on every style token's item page, with their logo and a link to the item on their site, since they ask that we be clear about where we use their data. ![Token item page with the itemdb credit](https://code.openneo.net/attachments/be1ca272-bfcd-4efb-9333-af15a270f8dd) ## What it costs itemdb One request for the list per run, using the `minimal` intent. The by-ID fallback only covers unnamed styles missing from the list, which should be a handful at most. The list is maintained by hand and can include a non-token item by mistake. We only look up our own styles' IDs in it, so stray entries are ignored. ## Open before merging - An API key in production, set as `ITEMDB_API_KEY`. (itemdb offered to raise our limits once we've created one.) - Whether to add the task to the `neopets:import` cron chain. ## Testing - Service spec with stubbed HTTP: the token exchange and reuse, the list, pagination, the by-ID fallback and error cases. A view spec covers the credit. - Against the live API: the list endpoint returned 4,220 tokens in one page, every one with an item ID and a real description, and the client parsed it. - A dry run in dev, seeded with 168 of our real unnamed styles, against the live list: all 168 would be named, 135 with a series name detected. - Not exercised live from Ruby: the key-for-token request itself. In my sandbox the key is injected by a proxy that the app's HTTP client doesn't go through, so I made that one request with curl and handed the token to the client. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6
Add token items for NC Pet Styles
All checks were successful
CI / rspec (pull_request) Successful in 3m25s
CI / deploy (pull_request) Has been skipped
ec77ead038
Each Pet Style has a "token" item on Neopets.com, which people want to
search for and add to their trade lists. Tokens aren't wearable, so we
never see them in modeling data. But a style's ID *is* its token's item
ID, and the Styling Studio import already tells us the token's name and
thumbnail, so we can create the item ourselves.

Now, whenever a labeled alt style is saved, we create or update an Item
with the same ID. Tokens count as fully modeled, don't fit any pet (so
they stay out of wardrobe search), and can be filtered with `is:style`.

The Styling Studio import saves every style it sees, so it will create
tokens going forward. To backfill the rest after deploying, run
`rake alt_styles:sync_token_items`.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6
Draft: Import unlabeled style tokens from itemdb
All checks were successful
CI / rspec (pull_request) Successful in 2m14s
CI / deploy (pull_request) Has been skipped
7b62f686c9
Some NC Pet Styles never got labeled, because we weren't running the
Styling Studio import while they were on sale. itemdb knows the names
and thumbnails of those styles' tokens, and sometimes their real
descriptions.

This adds a small itemdb API client, and `rake itemdb:import:styles`,
which labels unlabeled alt styles from itemdb's token data (which also
creates their token items), and fills in token descriptions.

As a safety check, we only accept a token name that ends in the style's
own color and species; anything else is flagged for a human.

Not ready to merge: this needs itemdb's blessing and an API token
(`ITEMDB_API_TOKEN`), and the response shape is from reading their
source rather than a live request.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6
Accept unusual token names in the itemdb style import
All checks were successful
CI / rspec (pull_request) Successful in 2m44s
CI / deploy (pull_request) Has been skipped
55a4f5f0bd
The latest Styling Studio import shows that many real token names don't
end in the style's color and species: "Mini Yellow Aisha" is reported
as a Blue Aisha, "Celebratory Anniversary" styles as 25th Anniversary,
and so on. Requiring that match would have rejected all of them.

Now we only require the name to end in the species (override with
FORCE=1), and infer the series name when we can, like the Styling
Studio import does. We also only ask itemdb about styles with no name
at all, rather than ones merely missing a series name.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6
Show the style itself on style tokens' item pages
All checks were successful
CI / rspec (pull_request) Successful in 3m21s
CI / deploy (pull_request) Has been skipped
b33b4ecbab
Style tokens aren't wearable, so the usual item preview (a pet wearing
the item, a species picker, zone info) doesn't make sense for them, and
would just show a bare pet.

Instead, we show the style the token unlocks, with a short explanation
and links to try it in the wardrobe, find items that fit it, and browse
related styles. We also write our own description when we don't have
the token's real one, and skip the DTI 2020 link.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6
Credit itemdb on style tokens' item pages
All checks were successful
CI / rspec (pull_request) Successful in 2m36s
CI / deploy (pull_request) Has been skipped
85a1e4619a
itemdb asks that we be clear about where we use their data, and link
back. Style token names, thumbnails and descriptions can come from
them, so each token's page now thanks them and links to the item there.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6
Load style tokens from itemdb's NC Pet Styles list
All checks were successful
CI / rspec (pull_request) Successful in 2m47s
CI / deploy (pull_request) Has been skipped
2dccc75848
itemdb recommended their v2 endpoints, which are better cached and have
slimmer responses, and pointed us at their official NC Pet Styles list:
it returns every style token in one request, so we don't need to batch
thousands of ID lookups.

The list is maintained by hand, so for unnamed styles that aren't on it
yet, we fall back to the v2 by-ID lookup, in batches small enough for
itemdb to serve from cache. The list can also include a non-token item
by mistake; we only ever look up our own styles' IDs in it, so those
are ignored.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6
armin-bot changed title from WIP: Import unlabeled style tokens from itemdb to Import unlabeled style tokens from itemdb 2026-11-16 16:33:21 -08:00
Trade our itemdb API key for a token before each request
All checks were successful
CI / rspec (pull_request) Successful in 2m48s
CI / deploy (pull_request) Has been skipped
77ce9e5721
itemdb's authentication takes two steps: the long-lived API key goes to
their token endpoint (as X-Itemdb-Key), which returns a token that lasts
an hour, and that token goes on API requests (as X-Itemdb-Token). We
were sending the key where the token belongs, which itemdb rejects.

The client now fetches a token on first use and keeps it for 50
minutes. The setting is renamed to ITEMDB_API_KEY to match what it
holds.

Checked against the live API: the NC Pet Styles list loads and parses
(4,220 tokens), and a dry run of the import names our real unnamed
styles from it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6
Accept character-named series in the itemdb style import
All checks were successful
CI / rspec (pull_request) Successful in 11m36s
CI / deploy (pull_request) Has been skipped
16a9c7da6f
Styles like "Essence of Esophagor" and "Treasured Roberta" are named
after a character, so their names never end in the species, and the
import's sanity check was holding them back for a human. These series
are an established pattern, so we now accept them, and label them with
their series name.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014PWWd1ky1PrtZvpAv7vhe6
armin-bot force-pushed claude/itemdb-style-sync from 16a9c7da6f
All checks were successful
CI / rspec (pull_request) Successful in 11m36s
CI / deploy (pull_request) Has been skipped
to e679b370bf
All checks were successful
CI / rspec (pull_request) Successful in 1m52s
CI / deploy (pull_request) Has been skipped
2026-11-16 21:38:09 -08:00
Compare
matchu merged commit 642b802587 into main 2026-11-16 21:40:49 -08:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
OpenNeo/impress!30
No description provided.