Add high-level documentation
I'm starting to learn how AI agent stuff works, and a lot of what I'm finding is that rushing them into feature development sets you up for disaster, but that having strong collaboration conversations with helpful context works wonders.
So, I'm starting by creating that context: I had a little "here's the codebase" walkthrough conversation with Claude Code, and it generated these docs as output—which came out solid from the jump, with a few tweaks from me for improved nuance.
My hope is that this can serve both as an improved starting point for human collaborators _and_ if I let future Claude instances play around in here. That's a big theme of what I've found with AI tools so far: don't try to get clever, don't expect the world, just give them the same support you'd give people—and then everybody wins 🤞
This commit is contained in:
parent
1bfacf0340
commit
d72d358135
5 changed files with 687 additions and 8 deletions
153
README.md
153
README.md
|
|
@ -2,6 +2,155 @@
|
|||
|
||||
# Dress to Impress
|
||||
|
||||
Oh! We've been revitalizing the Rails app! Fun!
|
||||
Dress to Impress (DTI) is a tool for designing Neopets outfits. Load your pet, browse items, and see how they look together—all with a mobile-friendly interface!
|
||||
|
||||
There'll be more to say about it here soon :3
|
||||
## Architecture Overview
|
||||
|
||||
DTI is a Rails application with a React-based outfit editor, backed by MySQL databases and a crowdsourced data collection system.
|
||||
|
||||
### Core Components
|
||||
|
||||
- **Rails backend** (Ruby 3.4, Rails 8.0): Serves web pages, API endpoints, and manages data
|
||||
- **MySQL databases**: Primary database (`openneo_impress`) + legacy auth database (`openneo_id`)
|
||||
- **React outfit editor**: Embedded in `app/javascript/wardrobe-2020/`, provides the main customization UI
|
||||
- **Modeling system**: Crowdsources pet/item appearance data by fetching from Neopets APIs when users load their pets
|
||||
|
||||
### The Impress 2020 Complication
|
||||
|
||||
In 2020, we started a NextJS rewrite ("Impress 2020") to modernize the frontend. We've since consolidated back into Rails, but **Impress 2020 still provides essential services**:
|
||||
|
||||
- **GraphQL API**: Some outfit appearance data still loads via GraphQL (being migrated to Rails REST APIs)
|
||||
- **Image generation**: Runs a headless browser to render outfit thumbnails and convert HTML5 assets to PNGs
|
||||
|
||||
See [docs/impress-2020-dependencies.md](./docs/impress-2020-dependencies.md) for migration status.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### Customization Data Model
|
||||
|
||||
The core data model powers outfit rendering and item compatibility. See [docs/customization-architecture.md](./docs/customization-architecture.md) for details.
|
||||
|
||||
**Quick summary**:
|
||||
- `body_id` is the key compatibility constraint (not species or color directly)
|
||||
- Items have different `swf_assets` (visual layers) for different bodies
|
||||
- Restrictions are subtractive: start with all layers, hide some based on zone restrictions
|
||||
- Data is crowdsourced through "modeling" (users loading pets to contribute appearance data)
|
||||
|
||||
### Modeling (Crowdsourced Data)
|
||||
|
||||
DTI doesn't pre-populate item/pet data. Instead:
|
||||
|
||||
1. User loads a pet (via pet name lookup)
|
||||
2. DTI fetches appearance data from Neopets APIs (legacy Flash/AMF protocol)
|
||||
3. New `SwfAsset` records and relationships are created
|
||||
4. Over time, the database learns which items fit which pet bodies
|
||||
|
||||
This "self-sustaining" approach means the site stays up-to-date as Neopets releases new content, without manual data entry.
|
||||
|
||||
## Directory Map
|
||||
|
||||
### Key Application Files
|
||||
|
||||
```
|
||||
app/
|
||||
├── controllers/
|
||||
│ ├── outfits_controller.rb # Outfit editor + CRUD
|
||||
│ ├── items_controller.rb # Item search, pages, and JSON APIs
|
||||
│ ├── pets_controller.rb # Pet loading (triggers modeling)
|
||||
│ └── closet_hangers_controller.rb # User item lists ("closets")
|
||||
│
|
||||
├── models/
|
||||
│ ├── item.rb # Items + compatibility prediction logic
|
||||
│ ├── pet_type.rb # Species+Color combinations (has body_id)
|
||||
│ ├── pet_state.rb # Visual variants (pose/gender/mood)
|
||||
│ ├── swf_asset.rb # Visual layers (biology/object)
|
||||
│ ├── outfit.rb # Saved outfits + rendering logic (visible_layers)
|
||||
│ ├── alt_style.rb # Alternative pet appearances (Nostalgic, etc.)
|
||||
│ └── pet/
|
||||
│ └── modeling_snapshot.rb # Processes Neopets API data into models
|
||||
│
|
||||
├── services/
|
||||
│ ├── neopets/
|
||||
│ │ ├── custom_pets.rb # Neopets AMF/Flash API client (pet data)
|
||||
│ │ ├── nc_mall.rb # NC Mall item scraping
|
||||
│ │ └── neopass.rb # NeoPass OAuth integration
|
||||
│ ├── neopets_media_archive.rb # Local mirror of images.neopets.com
|
||||
│ └── lebron_nc_values.rb # NC item trading values (external API)
|
||||
│
|
||||
├── javascript/
|
||||
│ ├── wardrobe-2020/ # React outfit editor (extracted from Impress 2020)
|
||||
│ │ ├── loaders/ # REST API calls (migrated from GraphQL)
|
||||
│ │ ├── WardrobePage/ # Main editor UI
|
||||
│ │ └── components/ # Shared React components
|
||||
│ └── application.js # Rails asset pipeline entrypoint
|
||||
│
|
||||
└── views/
|
||||
├── outfits/
|
||||
│ └── edit.html.haml # Outfit editor page (loads React app)
|
||||
├── items/
|
||||
│ └── show.html.haml # Item detail page
|
||||
└── closet_hangers/
|
||||
└── index.html.haml # User closet/item lists
|
||||
```
|
||||
|
||||
### Configuration & Docs
|
||||
|
||||
```
|
||||
config/
|
||||
├── routes.rb # All Rails routes
|
||||
├── database.yml # Multi-database setup (main + openneo_id)
|
||||
└── environments/
|
||||
└── *.rb # Env-specific config (incl. impress_2020_origin)
|
||||
```
|
||||
|
||||
**Documentation:**
|
||||
- [docs/customization-architecture.md](./docs/customization-architecture.md) - Deep dive into data model & rendering
|
||||
- [docs/impress-2020-dependencies.md](./docs/impress-2020-dependencies.md) - What still depends on Impress 2020 service
|
||||
|
||||
**Tests:**
|
||||
- `test/` - Test::Unit tests (privacy features)
|
||||
- `spec/` - RSpec tests (models, services, integrations)
|
||||
- Coverage is focused on key areas: modeling, prediction logic, external APIs
|
||||
- Not comprehensive, but thorough for critical behaviors
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Backend**: Ruby on Rails (Ruby 3.4, Rails 8.0)
|
||||
- **Frontend**: Mix of Rails views (Turbo/HAML) and React (for outfit editor)
|
||||
- **Database**: MySQL (two databases: `openneo_impress`, `openneo_id`)
|
||||
- **Styling**: CSS, Sass (moving toward modern Rails conventions)
|
||||
- **External Integrations**:
|
||||
- **Neopets.com**: Legacy Flash/AMF protocol for pet appearance data (modeling)
|
||||
- **Neopets NC Mall**: Web scraping for NC item availability/pricing
|
||||
- **NeoPass**: OAuth integration for Neopets account linking
|
||||
- **Neopets Media Archive**: Local filesystem mirror of `images.neopets.com` (never discards old files)
|
||||
- **Lebron's NC Values**: Third-party API for NC item trading values ([lebron-values.netlify.app](https://lebron-values.netlify.app))
|
||||
- **Impress 2020**: GraphQL for some outfit data, image generation service (being phased out)
|
||||
|
||||
## Development Notes
|
||||
|
||||
### OpenNeo ID Database
|
||||
|
||||
The `openneo_id` database is a legacy from when authentication was a separate service ("OpenNeo ID") meant to unify auth across multiple OpenNeo projects. DTI was the only project that succeeded, so the apps were merged—but the database split remains for now.
|
||||
|
||||
**Implication**: Rails is configured for multi-database mode. User auth models live in `auth_user.rb` and connect to `openneo_id`.
|
||||
|
||||
### Rails/React Hybrid
|
||||
|
||||
Most pages are traditional Rails views using Turbo for interactivity. The **outfit editor** (`/outfits/new`) is a full React app that:
|
||||
|
||||
- Loads into a `#wardrobe-2020-root` div
|
||||
- Uses React Query for data fetching
|
||||
- Calls both Rails REST APIs (in `loaders/`) and Impress 2020 GraphQL (being migrated)
|
||||
|
||||
The goal is to simplify this over time—either consolidate into Rails+Turbo, or commit fully to React. For now, we're in a hybrid state.
|
||||
|
||||
## Deployment
|
||||
|
||||
- **Main app**: VPS running Rails (Puma, MySQL)
|
||||
- **Impress 2020**: Separate VPS in same datacenter (NextJS, GraphQL, headless browser for images)
|
||||
- Both services share the same MySQL database (Impress 2020 makes SQL calls over the network)
|
||||
|
||||
---
|
||||
|
||||
**Project maintained by [@matchu](https://github.com/matchu)** • **[OpenNeo.net](https://openneo.net)**
|
||||
|
|
|
|||
|
|
@ -326,6 +326,12 @@ class Item < ApplicationRecord
|
|||
PetType.basic.released_before(released_at_estimate).
|
||||
distinct.pluck(:body_id).sort
|
||||
else
|
||||
# The core challenge: distinguish "item for Maraquan pets" from "item that
|
||||
# happens to fit the Maraquan Mynci" (which shares a body with basic Myncis).
|
||||
# We use a general rule: a color is "modelable" only if it has at least one
|
||||
# *unique* body (not shared with other colors). This filters out false
|
||||
# positives while remaining self-sustaining.
|
||||
|
||||
# First, find our compatible pet types, then pair each body ID with its
|
||||
# color. (As an optimization, we omit standard colors, other than the
|
||||
# basic colors. We also flatten the basic colors into the single color
|
||||
|
|
@ -336,6 +342,7 @@ class Item < ApplicationRecord
|
|||
Arel.sql("IF(colors.basic, 'basic', colors.id)"), :body_id)
|
||||
|
||||
# Group colors by body, to help us find bodies unique to certain colors.
|
||||
# Example: {93 => ["basic"], 112 => ["maraquan"], 47 => ["basic", "maraquan"]}
|
||||
compatible_color_ids_by_body_id = {}.tap do |h|
|
||||
compatible_pairs.each do |(color_id, body_id)|
|
||||
h[body_id] ||= []
|
||||
|
|
@ -343,17 +350,19 @@ class Item < ApplicationRecord
|
|||
end
|
||||
end
|
||||
|
||||
# Find non-basic colors with at least one unique compatible body. (This
|
||||
# means we'll ignore e.g. the Maraquan Mynci, which has the same body as
|
||||
# the Blue Mynci, as not indicating Maraquan compatibility in general.)
|
||||
# Find non-basic colors with at least one unique compatible body (size == 1).
|
||||
# This means we'll predict "all Maraquan pets" only if the item fits a
|
||||
# Maraquan pet with a unique body (like the Maraquan Acara), not if it only
|
||||
# fits the Maraquan Mynci (which shares its body with basic Myncis).
|
||||
modelable_color_ids =
|
||||
compatible_color_ids_by_body_id.
|
||||
filter { |k, v| v.size == 1 && v.first != "basic" }.
|
||||
values.map(&:first).uniq
|
||||
|
||||
# We can model on basic pets (perhaps in addition to the above) if we
|
||||
# find at least one compatible basic body that doesn't *also* fit any of
|
||||
# the modelable colors we identified above.
|
||||
# We can model on basic pets if we find a basic body that doesn't also fit
|
||||
# any modelable colors. This way, if an item fits both basic Mynci and
|
||||
# Maraquan Acara (a modelable color), we treat it as "Maraquan item" not
|
||||
# "basic item", avoiding false predictions for all basic pets.
|
||||
basic_is_modelable =
|
||||
compatible_color_ids_by_body_id.values.
|
||||
any? { |v| v.include?("basic") && (v & modelable_color_ids).empty? }
|
||||
|
|
|
|||
|
|
@ -170,6 +170,11 @@ class Outfit < ApplicationRecord
|
|||
end
|
||||
|
||||
def visible_layers
|
||||
# TODO: This method doesn't currently handle alt styles! If the outfit has
|
||||
# an alt_style, we should use its layers instead of pet_state layers, and
|
||||
# filter items to only those with body_id=0. This isn't needed yet because
|
||||
# this method is only used on item pages, which don't support alt styles.
|
||||
# See useOutfitAppearance.js for the complete logic including alt styles.
|
||||
item_appearances = item_appearances(swf_asset_includes: [:zone])
|
||||
|
||||
pet_layers = pet_state.swf_assets.includes(:zone).to_a
|
||||
|
|
|
|||
358
docs/customization-architecture.md
Normal file
358
docs/customization-architecture.md
Normal file
|
|
@ -0,0 +1,358 @@
|
|||
# Dress to Impress: Customization System Architecture
|
||||
|
||||
Dress to Impress (DTI) models Neopets's pet customization system: layered 2D images of pets wearing clothing items. This guide explains how DTI's data models represent the customization system and how they combine to render outfit images.
|
||||
|
||||
## Core Models
|
||||
|
||||
The customization system is built on these key models:
|
||||
|
||||
### Species and Color
|
||||
- **Species**: The type of Neopet (Acara, Zafara, etc.)
|
||||
- **Color**: The paint color applied to a pet (Blue, Maraquan, Halloween, etc.)
|
||||
- Colors have flags: `basic` (starter colors like Blue/Red), `standard` (follows typical body shape), `nonstandard` (unusual shapes)
|
||||
|
||||
### PetType
|
||||
**PetType = Species + Color**
|
||||
|
||||
- Represents the combination of a species and color (e.g., "Blue Acara", "Maraquan Acara")
|
||||
- **Contains the critical `body_id` field**: the physical shape/compatibility ID
|
||||
- Body ID determines what clothing items are compatible with this pet
|
||||
- Example: "Blue Acara" and "Red Acara" share the same body_id (they have the same shape), but "Maraquan Acara" has a different body_id (different shape)
|
||||
|
||||
### PetState
|
||||
**PetState = A visual variant of a PetType**
|
||||
|
||||
- Represents different presentations: gender (feminine/masculine) and mood (happy/sad/sick)
|
||||
- Standard poses: `HAPPY_FEM`, `HAPPY_MASC`, `SAD_FEM`, `SAD_MASC`, `SICK_FEM`, `SICK_MASC`
|
||||
- Special case: `UNCONVERTED` - legacy pets from before the customization system (now mostly replaced by Alt Styles)
|
||||
- Has many `swf_assets` (the actual visual layers for the pet's appearance)
|
||||
|
||||
### Item
|
||||
- Represents a wearable clothing item
|
||||
- Can have different appearances for different body IDs
|
||||
- Tracks `cached_compatible_body_ids`: which bodies this item has been seen on
|
||||
- Tracks `cached_occupied_zone_ids`: which zones this item's layers occupy
|
||||
|
||||
### Item.Appearance
|
||||
**Not a database model** - a `Struct` that represents "this item on this body"
|
||||
|
||||
- Created on-the-fly when rendering
|
||||
- Contains: the item, the body (id + species), and the relevant `swf_assets`
|
||||
- Why it exists: items can look completely different on different bodies
|
||||
|
||||
### SwfAsset
|
||||
**The actual visual layer** - a single image in the final composite
|
||||
|
||||
- Two types (via `type` field):
|
||||
- `'biology'`: Pet appearance layers (tied to PetStates)
|
||||
- `'object'`: Item appearance layers (tied to Items)
|
||||
- Has a `body_id`: either a specific body, or `0` meaning "fits all bodies"
|
||||
- Belongs to a `Zone` (which determines rendering depth/order)
|
||||
- Contains URLs for the image assets (Flash SWF legacy, HTML5 canvas/SVG modern)
|
||||
- Has `zones_restrict`: a bitfield indicating which zones this asset restricts
|
||||
|
||||
### Zone
|
||||
- Defines a layer position in the rendering stack
|
||||
- Has a `depth`: lower depths render behind, higher depths render in front
|
||||
- Has a `label`: human-readable name (e.g., "Hat", "Background")
|
||||
- Multiple zones can share the same label but have different depths (for items that "wrap around" pets)
|
||||
|
||||
### AltStyle
|
||||
**Alternative pet appearance** - a newer system for non-standard pet looks
|
||||
|
||||
- Used for "Nostalgic" (pre-customization) appearances and other special styles
|
||||
- Replaces the pet's normal layers entirely (not additive)
|
||||
- Has its own `body_id` (distinct from regular pet body IDs)
|
||||
- Most items are incompatible - only `body_id=0` items work with alt styles
|
||||
- Example: "Nostalgic Grey Zafara" is an AltStyle, not a PetState
|
||||
|
||||
---
|
||||
|
||||
## The Rendering Pipeline
|
||||
|
||||
This is the core of DTI's customization system: how we turn database records into layered outfit images.
|
||||
|
||||
### Overview
|
||||
|
||||
**Input**: An `Outfit` (a pet appearance + a list of worn items)
|
||||
**Output**: A sorted list of `SwfAsset` layers to render, bottom-to-top
|
||||
|
||||
### Step 1: Choose the Biology Layers
|
||||
|
||||
```ruby
|
||||
# If alt_style is present, use its layers; otherwise use pet_state's layers
|
||||
biology_layers = outfit.alt_style ? outfit.alt_style.swf_assets : outfit.pet_state.swf_assets
|
||||
```
|
||||
|
||||
- Alt styles completely replace pet layers (they don't layer on top)
|
||||
- Regular pets use their pet_state's layers
|
||||
- All biology layers have `type = 'biology'`
|
||||
|
||||
### Step 2: Load Item Appearances
|
||||
|
||||
```ruby
|
||||
# Get how each worn item looks on this body
|
||||
body_id = outfit.alt_style ? outfit.alt_style.body_id : outfit.pet_type.body_id
|
||||
item_appearances = Item.appearances_for(outfit.worn_items, body_id)
|
||||
item_layers = item_appearances.flat_map(&:swf_assets)
|
||||
```
|
||||
|
||||
- For each worn item, find the `swf_assets` that match this body
|
||||
- Matching logic: asset's `body_id` must equal the pet's `body_id`, OR asset's `body_id = 0` (fits all)
|
||||
- For alt styles: only `body_id=0` items will match (body-specific items are incompatible)
|
||||
- All item layers have `type = 'object'`
|
||||
|
||||
### Step 3: Apply Restriction Rules
|
||||
|
||||
This is where it gets complex. We need to hide certain layers based on zone restrictions.
|
||||
|
||||
#### Rule 3a: Items Restrict Pet Layers
|
||||
|
||||
```ruby
|
||||
# Collect all zones that items restrict
|
||||
item_restricted_zone_ids = item_appearances.flat_map(&:restricted_zone_ids)
|
||||
|
||||
# Hide pet layers in those zones
|
||||
biology_layers.reject! { |layer| item_restricted_zone_ids.include?(layer.zone_id) }
|
||||
```
|
||||
|
||||
**Example**: The "Zafara Agent Hood" restricts the "Hair Front" and "Head Transient Biology" zones.
|
||||
|
||||
**How it works**:
|
||||
- Items have a `zones_restrict` bitfield indicating which zones they restrict
|
||||
- When an item restricts zone 5, all pet layers in zone 5 are hidden
|
||||
- This allows items to "replace" parts of the pet's appearance
|
||||
|
||||
#### Rule 3b: Pets Restrict Body-Specific Item Layers
|
||||
|
||||
This rule is asymmetric and more complex!
|
||||
|
||||
> Note: This is a legacy rule, originally built for Unconverted pets.
|
||||
> Now, Unconverted pets don't exist… but pet states *do* still technically
|
||||
> support zone restrictions. We should examine whether any existing pet
|
||||
> states still use this feature, and consider simplifying it out of the
|
||||
> system if not.
|
||||
|
||||
```ruby
|
||||
# Collect all zones that the pet restricts
|
||||
pet_restricted_zone_ids = biology_layers.flat_map(&:restricted_zone_ids)
|
||||
|
||||
# Special case: Unconverted pets can't wear ANY body-specific items
|
||||
if pet_state.pose == "UNCONVERTED"
|
||||
item_layers.reject! { |layer| layer.body_specific? }
|
||||
else
|
||||
# Other pets: hide body-specific items only in restricted zones
|
||||
item_layers.reject! do |layer|
|
||||
layer.body_specific? && pet_restricted_zone_ids.include?(layer.zone_id)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Example**: Unconverted pets can wear backgrounds (body_id=0) but not species-specific clothing.
|
||||
|
||||
**Key distinction**:
|
||||
- Items restricting zones → hide **pet** layers
|
||||
- Pets restricting zones → hide **body-specific item** layers
|
||||
- Unconverted pets are special: they reject ALL body-specific items, regardless of zone
|
||||
|
||||
**Why `body_specific?` matters**:
|
||||
- Items with `body_id=0` fit everyone and are never hidden by pet restrictions
|
||||
- Items with specific body IDs are "body-specific" and can be hidden
|
||||
|
||||
#### Rule 3c: Pets Restrict Their Own Layers
|
||||
|
||||
```ruby
|
||||
# Pets can hide parts of themselves too
|
||||
biology_layers.reject! { |layer| pet_restricted_zone_ids.include?(layer.zone_id) }
|
||||
```
|
||||
|
||||
**Example**: The Wraith Uni has a horn asset, but its zone restrictions hide it.
|
||||
|
||||
**Why this exists**: Sometimes the pet data includes layers that shouldn't be visible, probably to enable the Neopets team to more easily manage certain kinds of appearance assets. This allows the pet's metadata to control which of its own layers render.
|
||||
|
||||
### Step 4: Sort by Depth and Render
|
||||
|
||||
```ruby
|
||||
all_layers = biology_layers + item_layers
|
||||
all_layers.sort_by(&:depth)
|
||||
```
|
||||
|
||||
- Pet layers first, then item layers (maintains order for same-depth assets)
|
||||
- Sort by zone depth: lower depths behind, higher depths in front
|
||||
- The sorted list is the final render order: first layer on bottom, last layer on top
|
||||
|
||||
**Important**: When a pet layer and item layer share the same zone (and thus the same depth), the item appears on top. This is achieved by putting item layers second in the concatenation and relying on sort stability.
|
||||
|
||||
---
|
||||
|
||||
## Understanding Compatibility
|
||||
|
||||
### Why body_id Is the Key
|
||||
|
||||
You might expect items to be compatible with a *species* (all Acaras) or a *color* (all Maraquan pets). But the system uses **body_id** instead.
|
||||
|
||||
**Why?**
|
||||
- Some colors share the same body shape across species (e.g., most "Blue" pets follow a standard shape)
|
||||
- Some colors have unique body shapes (e.g., Maraquan pets are aquatic and need different clothing)
|
||||
- Some species have unique shapes even in "standard" colors
|
||||
- Body ID captures the actual physical compatibility, regardless of color or species
|
||||
|
||||
**Example**:
|
||||
- Blue Acara (body_id: 93) and Red Acara (body_id: 93) → same body, share items
|
||||
- Blue Acara (body_id: 93) and Maraquan Acara (body_id: 112) → different bodies, different items
|
||||
- An item compatible with body 93 fits both Blue and Red Acaras
|
||||
|
||||
### Body ID 0: Fits All
|
||||
|
||||
- Items with `body_id=0` assets fit every pet
|
||||
- Examples: backgrounds, foregrounds, trinkets
|
||||
- These are the only items compatible with alt styles
|
||||
|
||||
### Standard vs. Nonstandard Bodies
|
||||
|
||||
- **Standard bodies**: Follow typical species shapes (usually "Blue" or other basic colors)
|
||||
- **Nonstandard bodies**: Unusual shapes (Maraquan, Baby, Mutant, etc.)
|
||||
- This distinction helps with modeling predictions (more on that below)
|
||||
|
||||
### How Compatibility Is Discovered
|
||||
|
||||
DTI doesn't know in advance which items fit which bodies. Instead, **users contribute compatibility data through "modeling"**:
|
||||
|
||||
1. A user loads their pet wearing an item
|
||||
2. DTI's backend calls Neopets APIs to fetch the pet's appearance data
|
||||
3. The response includes which asset IDs are present
|
||||
4. DTI records: "Item X has asset Y for body_id Z"
|
||||
5. Over time, the database learns which items fit which bodies
|
||||
|
||||
This crowdsourced approach is why DTI is "self-sustaining" - users passively contribute data just by using the site.
|
||||
|
||||
---
|
||||
|
||||
## Modeling and Data Sources
|
||||
|
||||
### The Modeling Process
|
||||
|
||||
**"Modeling"** is DTI's term for crowdsourcing appearance data from users:
|
||||
|
||||
1. User submits a pet name, often to use as the base of a new outfit
|
||||
2. DTI makes an API request to Neopets using the legacy Flash/AMF protocol
|
||||
3. Response contains:
|
||||
- Pet's species, color, mood, gender
|
||||
- List of biology asset IDs (the pet's appearance)
|
||||
- List of object asset IDs (items the pet is wearing)
|
||||
- Metadata for each asset (zone, manifest URLs, etc.)
|
||||
4. DTI creates/updates records:
|
||||
- `PetType` (if new species+color combo)
|
||||
- `PetState` (if new pose/mood/gender combo)
|
||||
- `SwfAsset` records for each asset
|
||||
- `ParentSwfAssetRelationship` linking assets to pets/items
|
||||
|
||||
See `app/models/pet/modeling_snapshot.rb` for the full implementation.
|
||||
|
||||
### Cached Fields
|
||||
|
||||
To avoid expensive queries, several models cache computed data:
|
||||
|
||||
- **Item**:
|
||||
- `cached_compatible_body_ids`: Which bodies we've seen this item on
|
||||
- `cached_occupied_zone_ids`: Which zones this item's assets occupy
|
||||
- `cached_predicted_fully_modeled`: Whether we think we've seen all compatible bodies
|
||||
|
||||
- **PetState**:
|
||||
- `swf_asset_ids`: Direct list of asset IDs (for avoiding duplicate states)
|
||||
|
||||
These fields are updated automatically when new modeling data arrives.
|
||||
|
||||
### Prediction Logic
|
||||
|
||||
DTI tries to predict which bodies an item *should* work on, based on which bodies we've already seen it modeled on. This helps prioritize modeling work and estimate how "complete" an item's data is.
|
||||
|
||||
#### The Maraquan Mynci Problem
|
||||
|
||||
The core challenge is avoiding **false positives**. Consider: most Maraquan pets have unique, aquatic body shapes and wear special Maraquan-themed items. But the Maraquan Mynci shares the same body_id as basic Myncis - it can wear any standard Mynci item.
|
||||
|
||||
**Naive approach**: "We saw this item on a Maraquan pet, so predict it fits all Maraquan pets"
|
||||
|
||||
**Problem**: Most items fit the Maraquan Mynci but NOT other Maraquan pets!
|
||||
|
||||
#### The Solution: Unique Body Detection
|
||||
|
||||
For each item, we check whether **this item's modeling data** shows a color in a "modelable" way: the item must fit at least one body_id that ONLY that color uses (not shared with any other color).
|
||||
|
||||
**Example 1 - Real Maraquan item**:
|
||||
- Modeled on: Maraquan Acara (body 112), Maraquan Zafara (body 98)
|
||||
- Both bodies are unique to Maraquan (no basic pets share them)
|
||||
- **Prediction**: This fits all Maraquan pets ✓
|
||||
|
||||
**Example 2 - Standard Mynci item**:
|
||||
- Modeled on: Blue Mynci (body 47), Maraquan Mynci (body 47)
|
||||
- Body 47 is shared by basic and Maraquan
|
||||
- Maraquan has NO unique body for this item
|
||||
- **Prediction**: This is a standard item for Myncis, not a Maraquan item ✓
|
||||
|
||||
**Example 3 - Maraquan item on Mynci**:
|
||||
- Modeled on: Maraquan Acara (body 112), basic Mynci (body 47)
|
||||
- Body 112 is unique to Maraquan
|
||||
- Maraquan HAS a unique body (the Acara)
|
||||
- **Prediction**: This fits all Maraquan pets (including Mynci) ✓
|
||||
|
||||
#### Basic Color Handling
|
||||
|
||||
Basic colors (Blue, Red, Green, etc.) always share the same body IDs in practice, so they're treated as a group. An item is predicted to fit all basic pets if we find at least one basic body that doesn't also fit any of the modelable colors identified above.
|
||||
|
||||
This prevents false positives: if an item fits both a unique Maraquan body AND a basic/Maraquan shared body (like the Mynci), we treat it as a Maraquan item, not a universal basic item.
|
||||
|
||||
#### Edge Cases
|
||||
|
||||
The algorithm has several early exits:
|
||||
|
||||
1. **Manual override**: If `modeling_status_hint` is set to "done", trust current data is complete
|
||||
2. **Fits all** (`body_id=0`): Item fits everyone, prediction complete
|
||||
3. **Single body**: Only one body seen - could be species-specific, stay conservative
|
||||
4. **No data**: No bodies yet - optimistically predict all basic bodies for recently-released items
|
||||
|
||||
#### Why This Works
|
||||
|
||||
This approach is **self-sustaining**: no manual color flagging needed. As users model items, the unique body pattern emerges naturally, and predictions improve automatically.
|
||||
|
||||
See `Item#predicted_body_ids` in `app/models/item.rb:306-373` for the full implementation.
|
||||
|
||||
### Flash → HTML5 Transition
|
||||
|
||||
Neopets originally used Flash (SWF files) for customization. Over time, they've migrated to HTML5 (Canvas/SVG).
|
||||
|
||||
**SwfAsset fields**:
|
||||
- `url`: Legacy SWF file URL (mostly unused now)
|
||||
- `manifest_url`: JSON manifest for HTML5 assets
|
||||
- `has_image`: Whether we generated a PNG from the SWF (legacy)
|
||||
|
||||
**Modern rendering**:
|
||||
1. Load the manifest JSON from `manifest_url`
|
||||
2. Parse it to find canvas library JS, SVG files, or PNG images
|
||||
3. Use Impress 2020 (the frontend) to render them
|
||||
|
||||
**Fallbacks**:
|
||||
- If `manifest_url` is missing or 404s, fall back to legacy PNG images
|
||||
- If those are missing too, the asset can't be rendered
|
||||
|
||||
---
|
||||
|
||||
## Summary: Key Takeaways
|
||||
|
||||
1. **body_id is the compatibility key**, not species or color
|
||||
2. **Restrictions are subtractive**: start with all layers, then hide some via restriction rules
|
||||
3. **Restrictions are asymmetric**: items hide pet layers, pets hide body-specific items
|
||||
4. **Unconverted pets are special**: they reject all body-specific items (and are no longer available on Neopets)
|
||||
5. **Alt styles replace pet layers** and only work with body_id=0 items
|
||||
6. **Data is crowdsourced** through user modeling, not pre-populated
|
||||
7. **The system evolved over time**: Flash→HTML5, UC→Alt Styles, etc.
|
||||
|
||||
---
|
||||
|
||||
## Code References
|
||||
|
||||
- **Rendering logic**: `Outfit#visible_layers` in `app/models/outfit.rb`
|
||||
- **Item appearances**: `Item.appearances_for` in `app/models/item.rb`
|
||||
- **Modeling**: `Pet::ModelingSnapshot` in `app/models/pet/modeling_snapshot.rb`
|
||||
- **Body compatibility**: `Item#compatible_body_ids`, `Item#predicted_body_ids` in `app/models/item.rb`
|
||||
- **Pet state poses**: `PetState#pose`, `PetState.with_pose` in `app/models/pet_state.rb`
|
||||
158
docs/impress-2020-dependencies.md
Normal file
158
docs/impress-2020-dependencies.md
Normal file
|
|
@ -0,0 +1,158 @@
|
|||
# Impress 2020 Dependencies
|
||||
|
||||
This document tracks how the main DTI Rails app still depends on the separate Impress 2020 service, and what would need to be migrated to fully consolidate into a single Rails app.
|
||||
|
||||
## Background
|
||||
|
||||
In 2020, we started a NextJS rewrite called "Impress 2020" to modernize the frontend. We've since decided to consolidate back into the Rails app, but migration is ongoing. The Rails app now embeds the Impress 2020 React frontend (in `app/javascript/wardrobe-2020/`) for the outfit editor, but some functionality still calls back to the Impress 2020 GraphQL API.
|
||||
|
||||
## Current State
|
||||
|
||||
### What's Migrated to Rails
|
||||
|
||||
The following have been migrated to Rails REST API endpoints (in `app/javascript/wardrobe-2020/loaders/`):
|
||||
|
||||
- **Item search** (`/items.json`) - Searching for items with filters
|
||||
- **Item appearances** (`/items/:id/appearances.json`) - Getting item layers for different bodies
|
||||
- **Outfit save/load** (`/outfits.json`, `/outfits/:id.json`) - CRUD operations on outfits
|
||||
- **Alt styles** (`/species/:id/alt-styles.json`) - Loading alternative pet appearances
|
||||
|
||||
### What Still Uses Impress 2020 GraphQL
|
||||
|
||||
The following GraphQL queries and mutations are still hitting the Impress 2020 service:
|
||||
|
||||
#### Core Wardrobe Functionality
|
||||
|
||||
- **`OutfitPetAppearance`** - Load pet appearance (biology layers) by species/color/pose
|
||||
- **`OutfitPetAppearanceById`** - Load pet appearance by ID
|
||||
- **`OutfitItemsAppearance`** - Load item appearances (object layers) for worn items
|
||||
- **`OutfitStateItems`** - Load item metadata for the outfit state
|
||||
- **`OutfitStateItemConflicts`** - Check for conflicting items in zones
|
||||
- **`PosePicker`** - Load available poses for a species/color combination
|
||||
- **`SpeciesColorPicker`** - Load all species and colors for the picker UI
|
||||
- **`SearchToolbarZones`** - Load all zones for search filtering
|
||||
- **`OutfitThumbnailIfCached`** - Check if an outfit thumbnail exists in the cache
|
||||
|
||||
#### Support/Admin Tools
|
||||
|
||||
These are staff-only features for managing modeling data:
|
||||
|
||||
- **`ItemSupportFields`** - Load item data for support drawer
|
||||
- **`ItemSupportRestrictedZones`** - Load restricted zones for an item
|
||||
- **`ItemSupportDrawerAllColors`** - Load all colors for support tools
|
||||
- **`PosePickerSupport`** - Load pet appearance data for labeling
|
||||
- **`PosePickerSupportRefetchCanonicalAppearances`** - Refresh canonical appearance data
|
||||
- **`AllItemLayersSupportModal`** - Load all layers for an item (support view)
|
||||
- **`AllItemLayersSupportModal_BulkAddProposal`** - Preview bulk layer additions
|
||||
- **`WriteItemFromLoader`** - Write item data to Apollo cache (used after REST calls)
|
||||
|
||||
#### Support/Admin Mutations
|
||||
|
||||
- **`ItemSupportDrawerSetItemExplicitlyBodySpecific`** - Mark item as body-specific
|
||||
- **`ItemSupportDrawerSetManualSpecialColor`** - Set manual special color for item
|
||||
- **`PosePickerSupportSetPetAppearancePose`** - Update pet appearance pose label
|
||||
- **`PosePickerSupportSetPetAppearanceIsGlitched`** - Mark pet appearance as glitched
|
||||
- **`ApperanceLayerSupportSetLayerBodyId`** - Update layer's body ID
|
||||
- **`AppearanceLayerSupportRemoveButton`** - Remove a layer
|
||||
- **`AllItemLayersSupportModal_BulkAddMutation`** - Bulk add layers to bodies
|
||||
|
||||
### Image Services
|
||||
|
||||
Impress 2020 also provides image generation services:
|
||||
|
||||
- **Outfit thumbnails** - Generates PNG images of outfits at various sizes (150px, 300px, 600px)
|
||||
- **Asset image rendering** - Runs a headless browser to convert HTML5 canvas movies to static images
|
||||
|
||||
These are served via:
|
||||
- Production: `https://outfits.openneo-assets.net/outfits/:id/v/:timestamp/:size.png`
|
||||
- Development: `Rails.configuration.impress_2020_origin + /api/outfitImage?id=...`
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
There are two potential paths forward:
|
||||
|
||||
### Option A: Incremental GraphQL → REST Migration
|
||||
|
||||
Continue migrating GraphQL queries to Rails REST endpoints, keeping the React wardrobe.
|
||||
|
||||
**Pros**: Lower risk, incremental progress, preserves mobile-friendly UI
|
||||
|
||||
**Cons**: Maintains complexity of React app + dual API surface
|
||||
|
||||
### Option B: Wardrobe Rewrite (Rails + Turbo)
|
||||
|
||||
Rewrite the outfit editor as a Rails view with Turbo/Stimulus, similar to the item show page.
|
||||
|
||||
**Pros**: Massive simplification—remove React, GraphQL, and complex data fetching entirely
|
||||
|
||||
**Cons**: High risk (rewrites are dangerous), significant effort, potential UI regressions
|
||||
|
||||
**Note**: Simplicity has been DTI's most valuable architectural principle long-term. The complexity of maintaining the React wardrobe + its APIs is significant. But rewrites carry inherent risk.
|
||||
|
||||
---
|
||||
|
||||
### Option A: Priority 1 - Core Data Loading
|
||||
|
||||
The most important migrations to enable turning off Impress 2020 would be:
|
||||
|
||||
1. **Pet appearances** (`OutfitPetAppearance`, `OutfitPetAppearanceById`)
|
||||
- Backend: Create `/pet-types/:species/:color/appearances.json` or similar
|
||||
- Frontend: Update `useOutfitAppearance.js` to use REST loader
|
||||
|
||||
2. **Item appearance layers** (`OutfitItemsAppearance`)
|
||||
- May already be covered by `/items/:id/appearances.json`?
|
||||
- Need to verify if this is redundant with existing loader
|
||||
|
||||
3. **Species/Color/Zone metadata** (`SpeciesColorPicker`, `SearchToolbarZones`)
|
||||
- Backend: Create endpoints for species, colors, zones listings
|
||||
- Or embed this static data in the JS bundle
|
||||
|
||||
4. **Pose availability** (`PosePicker`)
|
||||
- Backend: Add pose data to pet type endpoints
|
||||
- Frontend: Update pose picker to use REST data
|
||||
|
||||
### Priority 2: Support Tools
|
||||
|
||||
Support tools could be migrated as Rails admin pages using Turbo/Stimulus:
|
||||
|
||||
- Pet state labeling (pose picker support)
|
||||
- Item layer management
|
||||
- Manual data corrections
|
||||
|
||||
These don't need to be React-based; simpler Rails views would work fine.
|
||||
|
||||
### Priority 3: Image Services
|
||||
|
||||
This is the most complex migration:
|
||||
|
||||
- Move headless browser rendering into a Rails service or separate microservice
|
||||
- Set up image storage (S3 or similar)
|
||||
- Update outfit image URLs to point to new service
|
||||
|
||||
## Deployment Architecture
|
||||
|
||||
### Current Setup
|
||||
|
||||
- **Main Rails app**: Primary VPS server, serves web traffic and API
|
||||
- **Impress 2020**: Separate VPS in same datacenter, provides GraphQL API and image services
|
||||
- **Database**: MySQL on main Rails server, accessed by both services
|
||||
- **OpenNeo ID database**: Separate MySQL database (legacy, could be merged)
|
||||
|
||||
### After Full Migration
|
||||
|
||||
- **Single Rails app**: One VPS serving everything
|
||||
- **Image service**: Either integrated into Rails or extracted as a simple microservice
|
||||
- **Single MySQL database**: Merge OpenNeo ID schema into main database
|
||||
|
||||
## Notes
|
||||
|
||||
- The wardrobe-2020 frontend is already embedded in Rails (`app/javascript/wardrobe-2020/`)
|
||||
- Many API calls have been successfully migrated from GraphQL to REST
|
||||
- The GraphQL dependency is primarily in the core outfit rendering logic
|
||||
- Support tools are the lowest priority since they're staff-only
|
||||
|
||||
## See Also
|
||||
|
||||
- [Customization Architecture](./customization-architecture.md) - Explains the data model
|
||||
- `app/javascript/wardrobe-2020/loaders/` - Migrated REST API calls
|
||||
- `config/routes.rb` - Rails API endpoints
|
||||
Loading…
Reference in a new issue