PIM — Product Information Management

The PIM module is the product data backbone for SegOps Module 2 (Product Intelligence). It lets tenants define a custom product schema, import product catalogs via CSV or REST API, and browse/edit products in a normalized catalog UI.

Architecture#

CSV upload / REST bulk upsert
        │
        ▼
  Schema Mapping Engine (Claude Haiku)
  — infers schema from CSV headers + sample data
  — maps source columns to schema field names
        │
        ▼
  ProductSchema (per-tenant dynamic)
  Product (sku, name, data: JSONField)
        │
        └──► ClickHouse segmentation.products (raw JSON snapshot)

Key design decision: Product fields are fully dynamic. Only sku and name are fixed Postgres columns. All other attributes live in data: JSONField, driven by the tenant-owned ProductSchema. This mirrors the M6 EventSchema pattern and avoids forcing tenants into pre-defined attributes they may not need.

Models#

`ProductSchema`#

One per tenant. Defines the field vocabulary for that tenant's catalog.

Field	Type	Description
`tenant`	OneToOneField	Owner tenant
`fields`	JSONField	`[{name, type, required, description}]`
`default_currency`	CharField	e.g. `USD`, `EUR`
`default_locale`	CharField	e.g. `en_US`, `fr_FR`

Field types: string, number, boolean, array, url, price, multi_price

price — single {amount, currency} object
multi_price — array of {amount, currency, locale} for multi-locale tenants
array — comma-separated or JSON array (tags, categories, etc.)

`Product`#

Individual catalog item.

Field	Type	Description
`tenant`	ForeignKey	Owner tenant
`sku`	CharField	Business key, unique per tenant
`name`	CharField	Display name (fixed column for fast listing)
`data`	JSONField	All schema-defined attributes
`source`	CharField	`csv` or `api`
`raw_data`	JSONField	Original row before mapping
`ai_readability_score`	FloatField	AI scoring (set by M18 enrichment pipeline)

`ImportJob`#

Tracks async CSV import progress.

Field	Type	Description
`status`	CharField	`pending`, `mapping`, `importing`, `done`, `failed`
`filename`	CharField	Original file name
`row_count`	IntegerField	Total rows in CSV
`imported_count`	IntegerField	Rows upserted so far
`mapping`	JSONField	`{source_col: field_name
`csv_content`	TextField	Stored temporarily, cleared on success

CSV Import Flow#

POST /api/pim/import/csv/ — upload file
- Parses headers + first 5 rows as sample
- Calls Claude Haiku to suggest schema definition + column mapping
- Creates ImportJob in mapping status
- Returns {job_id, headers, sample_rows, suggested_schema, suggested_mapping}
User reviews schema + mapping in the UI, edits as needed
POST /api/pim/import/{id}/confirm/ — confirm and start
- Saves confirmed ProductSchema (creates or updates)
- Updates ImportJob.mapping
- Enqueues run_csv_import Celery task
Poll GET /api/pim/import/{id}/ — check status and imported_count
On done — products appear in the catalog; ClickHouse sync happens inside the task

REST API Ingestion#

POST /api/pim/products/bulk/ accepts {"products": [{sku, name, ...fields}]}. All fields beyond sku and name are stored in data. Upserts by (tenant, sku).

UI Pages#

Route	Description
`/pim`	Catalog table — dynamic columns from schema, facet filters, search
`/pim/import`	3-step import wizard — upload → map → progress
`/pim/[id]`	Product detail — schema fields rendered from `ProductSchema`, inline edit

ClickHouse Sync#

After every import or bulk upsert, _sync_products_to_ch inserts rows into segmentation.products:

sql

-- Table: segmentation.products
tenant_id String, sku String, name String, data String, updated_at DateTime
ENGINE = ReplacingMergeTree(updated_at)
ORDER BY (tenant_id, sku)

data is a raw JSON string. M13.5 product_attribute conditions query it via JSONExtractString/JSONExtractFloat.

API Reference#

See docs/api/pim.md

Integrations#

M13.5 Product Segmentation — reads segmentation.products CH table for product_attribute condition evaluation
M18 Optimization Engine — writes ai_readability_score back to Product
M19b Landing Page Generator — reads from ClickHouse products for query-to-product matching

← PreviousPlans & Billing Next →Product Segments