Product backlog
Ideas we have agreed to explore but have not shipped yet. When work starts it moves to our dev changelog; when it ships it appears in release notes.
This sprint (2026-08) — issues, root causes, and fix plan
Research 2026-08-13/14. Alfred-prod was off ~24h then back (queue ~76). No site DB + crawl DB daily copy — one Neon production branch; protect www with cache, pool retry, and query hygiene.
Themes: (1) Reload errors = Neon pool `P2024` + `force-dynamic` weekly landings + `/report-bug` Prisma, not missing data. (2) Venue pipeline over-trusts scrape strings; Places often does not stamp out-of-860 towns so junk never hits Out of The 860 delete. (3) Hard delete + recrawl mints new slugs (Listing unavailable); bug heals FIFO behind source crawls; AXS error copy still mentions ScrapingAnt.
### P0 — Site reliability
- Weekly Trivia/Sports (`force-dynamic` explore landings) + `/report-bug`: transient Prisma retry and soft-fail; drop blanket `force-dynamic` where tagged ~15m cache already exists.
- Withdraw + recrawl (not hard delete + recrawl): keep `id`/`slug`/`externalId`; revive original row when found; hard-delete only tombstones still missing after a trusted recrawl.
- Guard `/events/null` (128 dead-slug hits); do not record `null` as a slug.
### P0 — AXS / crawl failures
- PeoplesBank: listing cards without parseable dates; stale “set SCRAPINGANT_API_KEY” copy. Prefer official arena feed; Alfred/Playwright error text; recrawl on Alfred.
### P0 — Admin
- Events search: hydrate URL `q`; Prisma-paginated search (no unbounded in-memory load); align `sourceId` + `q`.
- Venues tab: server page/cursor + slim columns; defer counts; unlinked index only when that filter is on.
- Delete venue and linked events (typed confirm) for junk; recrawl-intent paths withdraw instead.
### P1 — Venue identity / NYC / enrich
- Page-title chrome (`Home |`, Local Calendar); Places query `name, Connecticut` → address → town; Places/FS win identity.
- Enrich stamps out-of-860 truth (NYC / `, NY` / coords) so rows enter Out of The 860 delete — never skip enrich to avoid writing NYC.
- Dedup: require both towns or Places corroboration; do not merge disagreeing names on the same street unless campus.
- Closed venue (Half Door): Places `business_status` / FS closed → stamp Location, withdraw future events, block re-bind.
### P1 — Bugs / Alfred / Facebook / hygiene
- Promote bug-heal jobs to front; EVENT lane for short detail recrawls; promote already-queued source crawls.
- Cannibals: family-merge prefixed aggregator titles (`TheaterWorks Hartford: The Cannibals…`) into occurrence family; revive/merge stale CTNow.
- Social lane is used (9 completed Edge jobs Jul–Aug); verify session, no rewrite.
- Sources filter: live events with zero images.
- REMOVE stamped deploy hooks (cadence floor, town-mismatch, slug stubs, QC dismiss).
Venue Cleanup Part 2 items that overlap (Places/FS identity win, Local Calendar, merge children) are in this sprint. Gemini tiebreak / Key venue flag remain later unless already in-slice.
---
Next up — Venue Cleanup & Clarify (Part 2)
Admin light-touch + automation so junk and wrong-town Locations stop recurring. Do not add publisher/venue name lists; use structure + Places/Foursquare + Gemini only when ambiguous (`GOOGLEAI_API_KEY`).
### Implement order
1. COMPLETED parity — Admin venue cleanup treats all non-DELETED events equally (heal list/counts/relink/recrawl attribution); stop SCHEDULED-only laziness. 2. Local Calendar gap — Reject/heal `… Local Calendar` structural pattern; persist cleaned name on create; heal the 8 live junk Locations (Explorefarmington, Downtownmiddletown, etc.) via existing Heal publisher venues modal/apply (+ tests). 3. Enrich: Places/Foursquare win — Enrich with or without `websiteUrl`; always surface Places + Foursquare picks; Places/FS win identity (name/town/address/website); website optional for logo/copy. Empty state only when both miss — never “paste URL or cancel.” *(Town correction on enrich shipped via `resolveEnrichmentTown` — remaining is fuller Places/FS identity win + empty-state UX.)* 4. Gemini tiebreak — Same helper for publish bind + Enrich…; Gemini only when Places/FS ambiguous; Key freeze when `keyVenueAt` set (manual Enrich… Accept still allowed). 5. Merge alias survival — Soft-merge when matchKey/town differ (never hard-delete those losers); show merged children on keeper (“where did Hebron go?”); skip enrich on `mergedIntoId` children. *(LocationSlugAlias + stub purge deploy hook shipped — rename/merge hard-deletes no longer create `__slug_redirect__` Location rows.)* 6. Wrong-town clusters / Needs-attention — Same-name wrong-town clusters, Places miss, heal candidates → Needs-attention + Merge UX (children + Unmerge). *(One-shot SQL mismatch heal + enrich town correction shipped 2026-08-11 evening — do not rebuild as another full-catalog deploy scan.)* 7. Honest empty-delete count — COMPLETED: admin Total + “Delete all with no events (N)” use the same linked-event universe as `totalEventCountForLocation` (any region / INACTIVE), not in-region Live+Expired+Cancelled only. 8. Table / Edit / search identity — Same display identity in table and Edit; search against display label (Hawk Ridge “calendar” mismatch). 9. Key venue flag — `Location.keyVenueAt`; row + Edit toggle = freeze + content queue; filter Key · needs content (logo/hero/description/website); bind/enrich reason lines on Enrich preview + `ScrapeRun.warnings`.
### Done in this sprint (do not rebuild)
- Bingo Loco bad cross-source attach + deploy unlink (promote 2026-07-24).
- Venues table action-only badges (dropped Enriched / Auto-enriched / Scrape source / Has events listing).
- Same-name + same-street wrong-town pairs surface in Venue Dedup review (Dunkin' Park).
- Daily crawl floor + quiet-streak slowdown/speed-up + bi-weekly + Sources Frequency column / 4–6d tiers (2026-08-11 evening).
- Venue enrich town from Places/postal (`resolveEnrichmentTown`) — street-name-as-town (2026-08-11 evening).
- Targeted town-mismatch deploy heal — SQL mismatch rows only; stamp `venueTownPlacesRepairDeployAt` (2026-08-11 evening).
- Admin Venues activity default All venues (2026-08-11 evening).
- LocationSlugAlias + deploy clear of `__slug_redirect__` stubs (on `dev`; stamp `locationSlugRedirectStubsClearedAt`).
- Honest venue Total / empty-delete — Total matches delete guard (`totalEventCountForLocation`).
### Explicitly out of this slice
- Silent cross-town auto-merge without Places confidence
- Durable merge audit table (light merged-children UI is in-slice)
- Homepage Key prefer / Places-rich public `/venues` redesign (see Site / public)
- Replacing Places with Gemini for every create/enrich
- Full-catalog gazetteer sweeps in `prisma-post-db-push` (rejected — hung builds / Alfred sync)
---
Next sprint (after 2026-08 promote) — SEO / IndexNow
- IndexNow URL cool-down ↳ shipped — Event/Location stamps + quiet deploy stamp ran on prod (`indexNowCatalogStampedAt` set). Hook removed from `prisma-post-db-push.mjs`. Optional later: drop stamp columns / delete `scripts/deploy-stamp-indexnow-catalog.ts` when convenient.
- Bing traction follow-up — After cool-down is on prod: confirm sitemap index is still submitted in Webmaster Tools; spot-check URL Inspection on a few high-position keywords; do not keep hammering IndexNow hoping for impressions.
---
Deploy hygiene (operator)
After every Production promote is Ready:
1. Immediately — read the Vercel Production build log for each `deploy-*` / `prisma post-push` line, and confirm matching prod `PlatformSettings` stamps (or report still-null + remaining work). Do not treat “build green” as hook success. 2. When the next sprint starts — remove stamped hooks from `prisma-post-db-push.mjs` and clear the matching rows here. Leave unstamped hooks wired.
Current open:
1. Optional later (schema columns only): drop unused `PlatformSettings` one-shot stamps (`topPicksScoresDeployBackfillAt`, `eventInRegionDeployBackfillAt`, `scrapeCadenceRederiveDeployAt`, `scrapeCadenceDailyFloorDeployAt`, `venueTownPlacesRepairDeployAt`, `locationSlugRedirectStubsClearedAt`, `qcSweepBugReportsDismissedAt`, `venueAddressIdentityBackfillAt`, `publisherVenueHealDeployedAt`, `crawlQcRunPlansDeployedAt`, `playwrightFirstEvidenceUnlockedAt`, `venueChromeNameHealDeployAt`, `indexNowCatalogStampedAt`, `venuePublicSlugAlignDeployAt`, etc.) when convenient. 2. Belding Theater slug (manual): long name truncated at 80 chars blocked deploy stamp — operator fixing in prod admin. `scripts/deploy-align-venue-public-slugs.ts` unwired; optional manual run after rename if needed.
Do not leave “REMOVE after promote” hooks wired forever after they stamp. Do not ship one-off EventSource URL dedupe in deploy hooks — `syncEventSources` dedupes on publish. Do not reintroduce full-catalog town repair in post-db-push.
---
Account & engagement
- User accounts — Optional login to persist saved events, home town / radius, and hearts across devices. On first sign-in, merge the anonymous `the860_harted` cookie into the account. Scope auth provider (magic link vs OAuth) when there is enough traction to justify it. *(Hearts ship cookie-only today.)*
---
Site / public
- Venue detail page — Places-rich redesign ↳ partial — Event-detail layout + 860 Take / Good to know (`venueTips`) / From the source + prefiltered listing + Google rating/hours glance shipped (2026-08-12 draft). Remaining: parking/accessibility/dining highlights from snapshots, richer hero vs logo.
- Venue image gallery (curated hero + logo) ↳ partial — unified missing-photo UX + Venue Hero / Venue Logo naming shipped. Remaining: optional gallery and admin upload; AI-assisted Venue Hero candidate picking is future work (current ranking is deterministic).
- Homepage / listing Back restore ↳ partial — SSR near-you reuse + keep-alive `/events` + Back restore shipped (2026-07-23). Header Events forward nav starts at top (2026-07-24). Remaining: EventCard `prefetch={true}` site-wide; `/api/events/filters` on home mount; localStorage hydration second fetch when cookie ≠ storage.
- Homepage / detail client bundle ↳ partial — `EventDescription` server-rendered and browse/home rows slimmed. Remaining: confirm event-detail client components don’t pull the scrape graph under webpack; keep scrape/server modules out of card UI; optional lazy `EventHartControl` on listing cards.
- `/api/events` memory-pool row ceiling ↳ partial — Top Picks (including town/category/date/Near bbox/map) is DB `ORDER BY topPicksScore, startsAt`. Remaining: same discipline for `recommended` / `distance` memory pools; Top Picks still uses ≤`TOP_PICKS_CANDIDATE_CEILING` only for `q` / today / time-of-day memory passes.
- Better “cool” scoring + discover more top events ↳ partial — Persisted coolness + admin pin/edit shipped. Remaining: scoring formula tweaks / surfacing more hidden gems.
- SEO landing pages (phase 2) — More towns (Enfield, Torrington, Meriden, Simsbury, Wethersfield, Mystic/Essex) and seasonal copy; Search Console after index. *(Trivia Near Me v1 + filter UI shipped; Trivia + Live Sports weekly day-column calendars shipped. Venue sitemap + venue detail listing shipped on `dev`. IndexNow cool-down shipped on `dev` — see § Next sprint.)* Remaining after promote: Bing impressions follow-up.
- Weekly calendar for more categories — The shared day-column surface (`weekly-category-calendar.ts` + `WeeklyCategoryListing`) now powers Trivia and Live Sports. Candidates next: karaoke and open mic (recurring weeknight patterns). Needs a support-line decision per category before adding.
- Release note visitor-copy lint — Checklist in team-norms so “For visitors” passes [vision-and-brand.md](./vision-and-brand.md) before promote.
- Security doc — Admin secret model and `/api/event-image` SSRF notes (held per owner request).
---
Scrape / data
- Periodic missing-image review ↳ partial — Shipped: first missing photo → BugReport; later → soft flag; QC product defaults On via deploy hook on first prod promote. Remaining: broader offline gallery / Missing Image Heros pool (Admin / imagery).
- Adaptive ingestion — run-plan learning ↳ partial — structured feeds, locks, learned caps, Needs run plan guidance shipped. Remaining: optional richer learning from crawl patterns, per-source “learning on event recrawl,” and a persisted admin checkbox (vs derived badge).
- Facebook event sources ↳ partial — Shipped: tab outcome badges + auto-disable after repeated blocked/no-tab crawls; tiered cadence floors FB listings at weekly. Remaining: many FB listing URLs still return 0 extracted — structural listing strategy (or official API) needed before broad FB cadence; login/ToS limit browser depth.
- Per-publisher crawl session — `ScrapeTarget.extraPrompt` + run-plan locks (not global env) for market cookies (AXS Hartford zip today), logged-in Facebook/Eventbrite as the860life, etc. AXS default zip lives in `SCRAPE_PIPELINE.axsDefaultZip` until per-source prose owns it.
- Admin: unmerge / split Locations ↳ partial — Unmerge restores child + `preventAutoMerge`, relinks label-matching upcoming events, queues bounded source recrawls. Remaining: richer confirmation UI with event counts; optional `preventAutoMerge` on the winner when operators want both sides locked. *(Merge alias survival / where-did-it-go UI is in Venue Part 2.)*
- Admin: location merge audit trail — Durable log table when venue sync auto-merges (out of Venue Part 2 slice; light merged-children UI is in Part 2).
- Generalized co-production venue rules — Hartford.com co-presented shows beyond `co-production-venue.ts` unite/co-production copy parser.
- OpenAI pipeline ordering UX — Long recrawls can show publish progress then “spacing OpenAI calls”; clarify or reorder so Jobs progress matches operator mental model. *(Bulk 860 Take throttle for >12 events ships; Gemini failover + durable TPM notes ship.)*
- Alfred ingest API (optional) — If the browser worker splits off the HP machine, a small authenticated ingest endpoint for rendered HTML instead of shared DB queue drain only. ADR: [0007](./decisions/0007-playwright-oracle-vm.md); ops: [alfred-setup.md](./alfred-setup.md).
- Accessibility-tree crawl extraction — After a listing loads in Playwright on Alfred, try accessibility snapshot alongside stripped HTML and JSON-LD when rendered HTML is unusable or extract count is zero.
- Recrawl on empty-but-valid feeds ↳ partial — Trusted-empty withdraw for Facebook + trusted structured paths ships; generic HTML/AI zero-extract still skips reconcile (see [known-issues.md](./known-issues.md)). Remaining: clearer operator signal when a feed is valid but returns zero events.
- RSS / feed auto-registration — Detect new CivicLift, ICS, and Tribe feeds during crawl and surface “add as source” in admin. CivicLift/CitySpark ingestion lock for existing sources ships in `structured-feed-lock.ts`.
- E2E smoke (crawl) — One Preview crawl test against fixture HTML in CI.
- Mobile event-page screenshot — Playwright phone-width capture when listing + `og:image` fail.
- Concise crawled event titles (SEO) ↳ partial — sports strip + occurrence/date suffixes + redundant `at {venue}` strip + prefer shorter compatible titles on recrawl + concert title guidance shipped. Remaining: broader marketing-tagline shortening at extract for all publishers.
- Refresh compact occurrence titles — Re-crawl recurring series for `Sun 5/8`-style titles (Tribe occurrence URLs help HJS but not all publishers).
- Trivia image from social — Facebook/Instagram art before generated trivia cards.
- Crawl cost dashboard — Admin view rolling up per-`ScrapeTarget` cost signals from `ScrapeRun` (pages fetched, detail skips, Alfred wall time, AI token usage). Sort by spend last 7d / 30d.
- Missing-category crawl audit — Post-crawl or scheduled script listing drafts/events dropped for missing/invalid category after guidance + AI pass. Group by `ScrapeTarget` + extraction path.
- Legacy rows missing externalId — audit prod before delete ↳ partial — Events admin can filter/select/delete missing-`externalId` rows. Remaining: read-only prod count/classify script for operator sign-off before bulk delete.
- Listing-only crawl must always yield stable externalId ↳ partial — extract rejects missing id; finalize/publish block blank `externalId`. Remaining: ensure every listing-only path synthesizes a stable structural id when publisher omits one.
---
Crawl cadence & self-healing QC
Shipped (2026-07-17 promote): tiered cadence; no auto SCRAPE_RESET; Extra scrape prompt operator-edited; Jobs `EVENT_QC`; missing photo first → BugReport; live schema/ICS confirmation; QC + auto suggested recrawls On.
Shipped (2026-08-11 evening): daily auto floor; quiet-streak one-tier slowdown; quiet speed-up after zero prior while slowed (or monthly+); bi-weekly tier; Sources Frequency column; 4/5/6-day tiers.
↳ partial remaining: - Stale worker cancel thresholds — further tune Alfred hung-row cancel/re-enqueue if overlap still trips “no progress” in practice. - Periodic missing-image review gallery/backfill UX beyond `imageMissingFlaggedAt` (ties Admin / imagery).
---
Deployment / platform health
Baseline on `main`: [smoke-test.md](./smoke-test.md) (`verify:deployed:dev` / `:prod` / `verify:promote`), CI after prod→dev copy, [database-release-workflow.md](./database-release-workflow.md).
- Public deployment status page — e.g. `/status` from `/api/health` (no secrets in UI).
- Admin deployment status — e.g. `/admin/status` with schema parity, event counts, Actions links.
- `GET /api/deploy-health` — One JSON payload for monitors and status UI.
- External uptime monitoring — Document Better Stack / UptimeRobot polling www + dev health.
---
Admin / imagery
- Category on uploaded hero ↳ partial — category mapping ships on Category Missing Images (multi-select). Remaining only if HP Hero Images themselves need category tags for homepage rotation (not requested).
- Exclude from HP banner rotation — Per-image flag so a hero can serve missing-image fallbacks without entering the homepage banner carousel. *(Category Missing Images is now a separate pool — this flag may be obsolete.)*
---
Venue / admin guards
- Venue create guard ↳ partial — admin Add venue + crawl ensure share `resolveVenueLocation` (admin-guard / ensure). Remaining: surface the existing row “open instead” UX polish if operators still get confused; complements Unmerge / Venue Part 2.
