# Order Manager - Project Context

PyQt6 desktop app for a daily order-processing workflow: JIRA tickets -> ShipStation
labels -> Odoo fulfillment. One `Order` row per JIRA ticket; ShipStation only ever
enriches existing rows, never creates them. Two companies (Signify Health, Oak Street
Health) plus a subsidiary (RubiconMD) with its own naming quirk - see below.

This file exists because this project was built up over a very long conversation with
Claude in claude.ai, iterating and debugging interactively. It's written so a fresh
Claude Code session (or a new person) doesn't have to rediscover any of this the hard
way. Read this before making changes, especially to `shipstation_send.py`,
`ticket_validation.py`, or anything touching return labels.

## Architecture

```
main.py                              entry point
app/
  adf.py                             Atlassian Document Format -> plain text parser
  companies.py                       SKU prefix -> company resolution (COMPANY_SKU_MAP)
  config.py                          SETTINGS_SCHEMA, defaults, test-mode-aware lookups
  database.py                        SQLAlchemy engine + auto-migration (ADD COLUMN on startup)
  models.py                          Order table (single source of truth for all fields)
  queries.py                         get_open_ticket_numbers()
  return_labels.py                   is_emailed_label_order() - SH007/OK012 detection
  schedule.py                        cutoff time / past-cutoff logic
  serial_suggestions.py              device-keyword -> serial-field suggestions for Pack Ticket
  status_rules.py                    active/cancelled status list parsing
  ticket_validation.py               SKU validation rules (company mismatch, return/device mismatch)
  tracking.py                        suggest_jira_status() from tracking numbers
  external_links.py                  JIRA/Google Maps URL builders (View in JIRA, Look Up Address)
  workers.py                         QThread workers + save_orders()/load_orders_by_view()/etc.
  services/
    base.py                         NormalizedOrder TypedDict, OrderService ABC
    jira_service.py                 JIRA REST API v3 /search/jql, paginated
    shipstation_service.py          ShipStation tracking-number pull (bulk fetch)
    shipstation_send.py             ALL ShipStation write operations - see below, this is the
                                     file with the most hard-won context in the whole project
    odoo_export.py                  CSV export
  ui/
    main_window.py                  Menu bar (File/Data/Orders) + slim toolbar, 3 tabs
    settings_dialog.py               Scrollable, grouped by SETTINGS_SCHEMA category
    widgets/
      dashboard.py                  Stat cards
      orders_table.py               QAbstractTableModel - see PERFORMANCE section below
      order_detail_dialog.py        Raw payload, tracking, validation issues, JIRA/Maps links
      pack_ticket_dialog.py         Serial number entry, barcode-scanner-friendly
      return_label_dialog.py        Two-step dummy-then-return UI - see RETURN LABELS below
```

## Order model (key columns)

`id, source, external_id, ticket_number, company, skus (JSON), line_items (JSON),
shipping_info (JSON), creator, assignee, description, tracking_numbers (JSON),
shipping_method, serial_numbers (JSON), packed, packed_at, summary, status,
source_created_at, imported_at, fulfilled_at, cancelled_at, shipstation_sent_at,
dummy_outbound_label_id, raw_data`

`dummy_outbound_label_id` is the newest column - it persists step 1 of the return-label
flow (see below) so step 2 can be triggered separately, even in a later session.

All timestamps use **local time** (`dt.datetime.now()`), not UTC - this was a real,
confirmed bug early on (evening cancellations failed same-day checks under UTC).

## The three tabs

- **Active**: `status` in `ACTIVE_STATUSES` ("Created")
- **Cancelled**: cancelled status AND `cancelled_at.date() == today` - ages into Done at midnight
- **Done**: everything else

## Return labels - the hard part of this project

This is the single most iterated-on feature and the one most likely to bite you if
touched carelessly. Sequence of what was learned, in order, because the reasoning
matters for not re-breaking it:

1. **A standalone return label (`POST /v2/labels` with `is_return_label: true`, no
   linked outbound shipment) reports `status: completed` but is invisible in
   ShipStation's UI.** This is confirmed against the team's actual manual process, not
   just API docs: they always create a cheap "dummy" outbound shipment first, then
   generate the return label from it via ShipStation's own GUI.
2. **The fix**: `create_dummy_shipment()` creates a minimal outbound label (1x1x1in,
   1oz, cheapest service) first. Its `label_id` is passed as `outbound_label_id` when
   creating the real return label via `create_return_label_from_dummy()`. Both are in
   `shipstation_send.py`.
3. **This is deliberately a two-step UI flow, not one atomic action** (`return_label_dialog.py`
   + `main_window.py`'s `_on_return_label_dialog_finished`). Clicking "Create Return
   Label" the first time on a ticket only creates the dummy and stops - the button
   label changes to "Step 2" and the user is expected to verify the dummy in
   ShipStation before proceeding. This was an explicit ask: splitting the steps apart
   means a problem in one doesn't get masked by the other. `dummy_outbound_label_id` on
   the Order persists step 1's result across sessions.
4. **`external_shipment_id` populates ShipStation's "Order #" column** - confirmed via
   ShipStation's own docs. It must be **unique per account**, so the dummy uses
   `{ticket_number}-DUMMY` and the real return label uses the bare `{ticket_number}`.
5. **`warehouse_id` and `ship_from` are mutually exclusive** in a `/v2/labels` request -
   confirmed by a real ShipStation 400 error ("ship_from and warehouse_id cannot be
   provided in same request"). When a warehouse_id is configured, it's used alone
   (ShipStation resolves the address from the registered warehouse record); otherwise
   the code falls back to an explicit `ship_from` address. See
   `_create_dummy_outbound_label()`.
6. **ShipStation's `/v2/labels/{label_id}/return` endpoint exists and auto-swaps
   ship_to/ship_from, but does NOT accept a custom `shipment.packages` override** - it
   inherits the original outbound's package. That's why this project does NOT use that
   endpoint; it uses `POST /v2/labels` with `is_return_label: true` +
   `outbound_label_id` set instead (Method 1 in ShipStation's docs), which supports
   full custom packages on the return label itself. Multiple packages per return label
   already works this way (one call, packages array) - no additional work needed there.
7. **A `carrier_id` (or `store_id`/`warehouse_id`) missing the `se-` prefix produces a
   confusing "not found" error** - `_normalize_shipstation_id()` in `shipstation_send.py`
   auto-prepends `se-` to any bare numeric ID at every lookup site (store, warehouse,
   carrier), since this is an easy typo when copying IDs out of ShipStation's UI.
8. **ShipStation's own error responses appear to strip the `se-` prefix when echoing
   back `field_value`, regardless of what was actually sent.** Don't assume a bare
   number in an error message necessarily means the prefix is missing in your request -
   it may just mean the ID genuinely doesn't exist in that account (see Test Mode below,
   this is exactly what happened with carrier_id and warehouse_id there).
9. **RubiconMD (RMD-prefixed SKUs) is an Oak Street subsidiary**: classified as "Oak
   Street Health" everywhere (dashboard, filters, company mismatch checks), but the
   return label's address `name` field is overridden to "Rubicon MD"
   (`RUBICONMD_RETURN_NAME` setting). Everything else (carrier, service, address,
   phone) is identical to Oak Street's - RubiconMD does NOT have its own shipping
   account. See `_is_rubiconmd_order()` / `_return_address_for_order()`.
10. **The tracking number is never auto-copied to the clipboard** - there's an explicit
    "Copy Tracking Number" button in the success dialog instead. This was a deliberate
    reversal: an earlier version auto-copied and that was flagged as risky (staff use
    the clipboard constantly for other things; silently overwriting it loses data).

## ShipStation Test Mode

Built so the team can test the whole return-label flow without spending real money or
needing to void mistakes. Toggle: Data menu checkbox, also mirrored as a permanent
orange status-bar banner when active (`_test_mode_indicator`).

**Design**: every account-specific ShipStation setting (API key, store/warehouse ID,
return carrier/service code) has a `TEST_`-prefixed override
(`config.get_shipstation_setting()` in `config.py`). When test mode is on, **only**
the `TEST_` value is used - no fallback to production. This used to fall back to the
production value when the `TEST_` override was blank, on the theory the sandbox might
share IDs with production - confirmed against ShipStation's own docs that it never
does ("Sandbox data is isolated from production data... anything you create in the
sandbox will not be accessible in production, or vice-versa" -
docs.shipstation.com/apis/shipengine/docs/getting-started/sandbox). That fallback is
what produced three separate confusing "not found"/"invalid" errors in a row (carrier,
then warehouse, then store) before it was caught and removed - a sandbox key being
handed a production ID isn't a soft mismatch, it's a guaranteed rejection. The team's
sandbox is a separate ShipEngine-heritage test account entirely, confirmed via
`list_carriers()`/`list_warehouses()` diagnostics (Data menu - "List ShipStation
Carriers/Warehouses..."). Confirmed real test carrier: `se-6366092` (UPS) for both
companies, since there's one shared sandbox account. Because there's no fallback now,
every `TEST_*` setting a flow touches must be filled in explicitly, even ones that are
plain strings rather than account-specific IDs (e.g. `TEST_SIGNIFY_RETURN_SERVICE_CODE`/
`TEST_OAKSTREET_RETURN_SERVICE_CODE` were blank and got set to `ups_ground` to match
production - safe because a service code isn't sandbox/production-isolated data, it's
just a carrier capability string).

**Warehouses don't exist by default in a sandbox account** - `list_warehouses()`
correctly returning an empty list in test mode wasn't a bug, there was just nothing
there yet to list. Fixed with `create_warehouse()` / `create_test_warehouse_for_company()`
in `shipstation_send.py` (confirmed request shape against ShipStation's own
`POST /v2/warehouses` docs - `name` + `origin_address`, with
`address_residential_indicator` required) and a "Create Test Warehouse(s)..." Data-menu
action that reuses the existing `SIGNIFY_RETURN_*`/`OAKSTREET_RETURN_*` address settings
as the origin address (same physical address, just registered under the sandbox
account) and auto-saves the resulting `warehouse_id` into
`TEST_SHIPSTATION_SIGNIFY_WAREHOUSE_ID` / `TEST_SHIPSTATION_OAKSTREET_WAREHOUSE_ID`.

**There is no way to list store IDs, in test mode or production - confirmed a dead
end.** `list_stores()` used to call `GET /v2/stores`, which doesn't exist (plain 404,
"No route matched with those values"). Checked against ShipStation's own V2 OpenAPI
reference: there is no Stores/Marketplaces section at all - `store_id` only ever shows
up as an *input* field on label/shipment requests, never as a listable resource. Their
own help docs confirm the only ways to get a store_id are ShipStation support looking
it up, or the legacy V1 API (different auth - key+secret Basic Auth, not the single
V2 API-Key header this app uses everywhere). `list_stores()` now raises a clear error
saying so instead of a confusing 404; the Data-menu item was relabeled "About
ShipStation Store IDs..." accordingly. **The only real fix**: log into the
ShipStation account's own UI (Settings > Store Setup) - for test mode, that means the
TEST/sandbox account specifically, not production - find or create a manual store,
and copy its store_id into `TEST_SHIPSTATION_SIGNIFY_STORE_ID` /
`TEST_SHIPSTATION_OAKSTREET_STORE_ID` by hand. This was caught mid-debugging: a
"Return label creation failed" / "invalid store" error on `store_id: 367672` (Oak
Street's *production* `se-367672`, prefix stripped in the echoed error per point 8
above) - happening because `TEST_SHIPSTATION_OAKSTREET_STORE_ID` was still blank.

**Production readiness check, 2026-09-16 (read-only, no labels created, no cost)**:
before a first real production test of the return-label flow, ran `GET /v2/carriers`,
`GET /v2/warehouses`, and `GET /v2/carriers/{id}/services` directly against the
production API key and diffed the results against `.env`:
- `SIGNIFY_RETURN_CARRIER_ID` (`se-350817`) and `OAKSTREET_RETURN_CARRIER_ID`
  (`se-599657`) both exist and are the expected UPS accounts.
- `SHIPSTATION_SIGNIFY_WAREHOUSE_ID` (`se-180473`) and
  `SHIPSTATION_OAKSTREET_WAREHOUSE_ID` (`se-437417`) both exist (named "Signify" and
  "OAKM" respectively).
- `ups_ground` (both `*_RETURN_SERVICE_CODE` settings) is a valid service code on both
  carrier accounts.
- **`store_id` could NOT be checked** - no listing endpoint exists (see above). This
  is the one remaining unknown going into a real test. Per the "Known-pending" section
  below, end-to-end production verification of this flow was still unconfirmed as of
  this check - so a first real shipment is a genuine first test, not a regression
  check. If it fails, expect the same "invalid store" error shape as the test-mode one;
  the fix then is re-checking the store_id in ShipStation's own UI, not the code.

**Real limitations of ShipStation's sandbox (confirmed via their own docs, not
assumed) that constrain what test mode can actually verify:**
- Branded Labels / Branded Tracking Pages are NOT available in sandbox. This directly
  affects the emailed-return-label feature: the "Send Return Label" branded-email step
  (the actual point of the feature) cannot be tested end-to-end in test mode. Only
  label *creation* can be verified there; the branded email step still needs a real
  production test eventually.
- 20 requests/minute rate limit (much lower than production) - plausible to hit during
  heavy iterative testing, and would likely present as a confusing generic-looking error.
- Tracking events require real packages in a real carrier network - sandbox can't
  simulate them, so Pull Tracking Numbers won't return anything for test-mode shipments.
  Expected, not a bug.
- Sales Orders API (used by the emergency "Send to ShipStation" feature specifically,
  not the return-label flow) is in beta and may not work in sandbox at all.

## Performance - a real, fixed bug worth knowing about

`ticket_validation.py`'s `validate_ticket()` and `return_labels.py`'s
`is_emailed_label_order()` both read Settings from disk internally
(`config.load_settings()` / `config.get()`, which re-parse `.env` every call). Calling
either of these inside `OrdersTableModel.data()` - which Qt invokes constantly, for
every visible cell, every repaint - was a severe, real performance bug (measured:
2.7 seconds of UI freeze per refresh with 300 orders, and visibly janky scrolling).

**Fixed** by computing both once per `set_orders()` call (not per cell) and caching the
result per `order.id` (`self._issues_by_id`, `self._is_emailed_label_by_id` in
`orders_table.py`). Also hoisted the underlying Settings reads themselves out of the
per-order loop via `make_validation_context()` (fetches sku_map/keyword_map/exempt_keywords
once, passed into every `validate_ticket()` call in the batch) - measured improvement:
2749ms -> 19ms for a 300-order refresh. **If you add a new per-order computed column,
compute it once in `set_orders()` and cache it - never call anything that reads
Settings from inside `data()`.**

## Ticket validation rules (flagging only, never auto-cancels)

`ticket_validation.py` - two rules, confirmed against real business logic and real SKU
catalogs (Oak Street + Signify deliverables spreadsheets, not guessed):

1. **Company mismatch**: a ticket's SKUs resolve to more than one company (SH + OK
   present together). Simple, unambiguous.
2. **Return/device mismatch**: a "Shipping - Return Label/Box X" line item's implied
   device type doesn't match anything else on the ticket. Two valid pairings, both
   treated as satisfying the rule:
   - An actual device line item of the same type (break-fix: send the asset, return
     the same type).
   - **Another shipping item of the same type** (asset recovery: a box AND a label for
     the same device, e.g. `SH002` "Return Box iPad" + `SH011` "Return Label iPad" -
     confirmed as a valid combination, NOT a mismatch, against real examples:
     SH002/SH011, OK001/OK006, OK011/OK013).
   - Exempt return types (no device expected at all): emailed labels, DPS Device,
     scheduled pickups, padded envelopes - `RETURN_DEVICE_EXEMPT_KEYWORDS` setting.
   - Device-type keywords are shared with `serial_suggestions.py`'s
     `DEVICE_FIELD_SUGGESTIONS` (same vocabulary, different use) - includes an `IE`
     prefix note: Signify's `IE400/IE401` SKUs are **deprecated**, deliberately not
     added to `COMPANY_SKU_MAP`.

This is intentionally flag-only. Cancellation is a deliberate manual step in JIRA
(requires a reason, is audited, requires someone's JIRA account) - there is no
JIRA-write capability anywhere in this app, by design, and that's not expected to
change without a deliberate separate conversation about it.

## Other things worth knowing

- **Bitdefender ATC crash (Windows-only, resolved)**: `PackTicketDialog` and
  `ReturnLabelDialog` used to crash the whole process on close - confirmed via crash
  dump analysis to be Bitdefender Endpoint Security's Advanced Threat Control
  corrupting a stack frame inside Qt6Core.dll, not a bug in this code. Fixed by
  showing both dialogs non-modally (`.show()` instead of `.exec()` - avoids the nested
  event loop `.exec()` runs) and reusing a single persistent instance per dialog type
  via `set_order()` rather than constructing/destroying one per ticket. See the
  WORKAROUND NOTE docstrings in both dialog files before changing how they're shown.
- **Menu bar, not just a toolbar**: File / Data / Orders menus hold everything; a slim
  toolbar duplicates only the 3 highest-frequency actions (Import from JIRA, Pack
  Ticket, Create Return Label) using the *same* QAction objects, so there's nothing to
  keep in sync between the two.
- **JIRA import uses the newer `/rest/api/3/search/jql` endpoint** (the old
  `/rest/api/3/search` was removed by Atlassian) - paginated via `nextPageToken`, with
  defensive anti-loop guards (stops on empty batch, `isLast`, missing/repeated token,
  or a 200-page hard ceiling).
- **View in JIRA** defensively strips any `/rest/...` API path that might have ended up
  baked into the `JIRA_URL` setting before building the browse link - the setting is
  meant to be the bare site domain.
- **SKU company map** (`COMPANY_SKU_MAP`): `SH:Signify Health,OK:Oak Street
  Health,RMD:Oak Street Health` - RMD added after confirming it's a subsidiary, not
  its own company.

## Known-pending / not yet built

- **Immediate next step, as of the last working session**: the warehouse_id issue is
  now fixed code-side ("Create Test Warehouse(s)..." in the Data menu, see ShipStation
  Test Mode above) but hasn't been run/confirmed yet - run it, confirm the two
  `TEST_SHIPSTATION_*_WAREHOUSE_ID` settings got saved. Then a *second*, separate
  test-mode gap surfaced right after: an "invalid store" error on Oak Street's
  production store_id, because `TEST_SHIPSTATION_*_STORE_ID` is blank and there is no
  API-based fix for that one (see ShipStation Test Mode above) - both
  `TEST_SHIPSTATION_SIGNIFY_STORE_ID` and `TEST_SHIPSTATION_OAKSTREET_STORE_ID` still
  need to be filled in by hand from the sandbox account's own ShipStation UI before the
  emailed-return-label flow can be tested end-to-end in test mode.
- **EOD JIRA push**: deliberately deferred. `packed`, `serial_numbers`,
  `tracking_numbers`, `shipping_method`, `assignee` are all captured and ready for it
  whenever it's prioritized - would need the exact JIRA custom field IDs (outbound
  tracking slots 1-4, RMA number, serial fields) from the team.
- **Bulk JIRA -> ShipStation import**: still an external, separate process (their own
  CSV-conversion tool) - this app only handles the single-ticket emergency send case.
  A bigger, riskier build if ever tackled (rate limits, bulk automation behavior).
- **Return-label flow real-world verification**: the two-step dummy-then-return flow
  is fully tested at the code level (unit tests covering both success and
  void-on-failure paths), but end-to-end confirmation that it behaves correctly with
  ShipStation's actual production API, for the actual team, is still in progress as of
  the last working session - don't assume it's fully proven in production just because
  the code is correct and tests pass.
- **JIRA write access / auto-cancellation**: explicitly declined by the team so far
  (cancellation requires a JIRA account and an audited reason) - don't build this
  without a deliberate, separate conversation about it first.