Plan scalable Recent Gamer Updates feed storage #3

New issue

Open

opened 2024-01-14 14:12:23 -06:00 by jimmyb · 6 comments

jimmyb commented

2024-01-14 14:12:23 -06:00

Owner

Copy link

The Recent Gamer Updates feed needs a scalable Laravel storage strategy so the site can keep public activity pages fast without blindly deleting historical user activity.

The original issue proposed trimming or truncating the old feed table, but later comments clarified that people do use the feed and that historical records should ideally be preserved. In the current Laravel checkout, feed entries are written to Spatie activity log table with log_name = gamer_updates, displayed through the RecentGamerUpdates presenter, and surfaced on the homepage/sidebar while a dedicated Recent Gamer Updates page is still being rebuilt. This issue tracks deciding and implementing the long-term storage, indexing, archival, and query strategy for that feed.

Scope

activity_log records used for gamer_updates.
Recent Gamer Updates homepage/sidebar card.
Future /recent-gamer-updates listing page and pagination.
Any admin/profile/feed surfaces that query activity records.
Feed query performance, indexing, archival, retention, and migration/backfill behavior.
Preservation of historical feed data unless a future product decision explicitly approves deletion.

Acceptance Criteria

Current feed producers and consumers are documented before storage changes are made.
Gamer update feed queries use bounded, indexed, newest-first reads and do not require scanning the full activity table.
The active/recent feed remains fast when the activity table contains a large historical record count.
Historical feed records are preserved by default through an archive/current-table strategy, partitioning strategy, or another documented Laravel-compatible approach.
No implementation relies on truncating the table, resetting IDs, or deleting historical data as the primary solution.
Any archival or migration process is idempotent, resumable, and safe to run in batches.
Existing gamer_updates activity entries continue to render with user, avatar, game, platform, activity type, and timestamp data.
Banned, deleted, and soft-deleted users remain excluded from public feed rendering.
Admin activity log usage remains separate from public gamer update feed behavior where appropriate.
Empty feed states render clearly.
User-facing UI supports light/dark mode and follows the existing Laravel/Tailwind site layout patterns.

Test Coverage Required

Feature/unit tests confirming Recent Gamer Updates reads are ordered newest-first and limited appropriately.
Regression tests confirming existing gamer_updates records render expected card/list data.
Tests confirming banned, deleted, and soft-deleted users are excluded from public feed output.
Tests covering pagination behavior for the dedicated feed page when that page is implemented.
Tests covering any archival/current-feed split, including no missing records, no duplicate records, and stable ordering across boundaries.
Command/job tests for archival behavior if a batch archive process is added, including dry-run or idempotency behavior where available.
Regression tests confirming admin activity log queries still work after feed storage/index changes.
Run the focused affected tests, then run vendor/bin/pint --dirty before closing the issue.

Progress Checklist

Spatie activity log table exists
gamer_updates activity entries are written from game-list status changes
RecentGamerUpdates presenter exists
Homepage/sidebar Recent Gamer Updates card exists
Sidebar link for /recent-gamer-updates exists
Activity log cleanup config exists but currently deletes by age when run
Inventory all current and planned gamer update feed producers
Inventory all public/admin consumers of activity feed data
Choose the long-term current-vs-history storage strategy
Add required indexes or migrations for bounded newest-first feed queries
Add archival or migration command/job if a split storage strategy is chosen
Ensure public feed queries preserve historical data and avoid destructive truncation
Build or coordinate with the dedicated /recent-gamer-updates page work
Add tests for feed ordering, limits, filtering, archive behavior, and regressions
Confirm the final strategy keeps Recent Gamer Updates usable at large table sizes

The Recent Gamer Updates feed needs a scalable Laravel storage strategy so the site can keep public activity pages fast without blindly deleting historical user activity. The original issue proposed trimming or truncating the old feed table, but later comments clarified that people do use the feed and that historical records should ideally be preserved. In the current Laravel checkout, feed entries are written to Spatie activity log table with `log_name = gamer_updates`, displayed through the `RecentGamerUpdates` presenter, and surfaced on the homepage/sidebar while a dedicated Recent Gamer Updates page is still being rebuilt. This issue tracks deciding and implementing the long-term storage, indexing, archival, and query strategy for that feed. ## Scope - `activity_log` records used for `gamer_updates`. - Recent Gamer Updates homepage/sidebar card. - Future `/recent-gamer-updates` listing page and pagination. - Any admin/profile/feed surfaces that query activity records. - Feed query performance, indexing, archival, retention, and migration/backfill behavior. - Preservation of historical feed data unless a future product decision explicitly approves deletion. ## Acceptance Criteria - Current feed producers and consumers are documented before storage changes are made. - Gamer update feed queries use bounded, indexed, newest-first reads and do not require scanning the full activity table. - The active/recent feed remains fast when the activity table contains a large historical record count. - Historical feed records are preserved by default through an archive/current-table strategy, partitioning strategy, or another documented Laravel-compatible approach. - No implementation relies on truncating the table, resetting IDs, or deleting historical data as the primary solution. - Any archival or migration process is idempotent, resumable, and safe to run in batches. - Existing `gamer_updates` activity entries continue to render with user, avatar, game, platform, activity type, and timestamp data. - Banned, deleted, and soft-deleted users remain excluded from public feed rendering. - Admin activity log usage remains separate from public gamer update feed behavior where appropriate. - Empty feed states render clearly. - User-facing UI supports light/dark mode and follows the existing Laravel/Tailwind site layout patterns. ## Test Coverage Required - Feature/unit tests confirming Recent Gamer Updates reads are ordered newest-first and limited appropriately. - Regression tests confirming existing `gamer_updates` records render expected card/list data. - Tests confirming banned, deleted, and soft-deleted users are excluded from public feed output. - Tests covering pagination behavior for the dedicated feed page when that page is implemented. - Tests covering any archival/current-feed split, including no missing records, no duplicate records, and stable ordering across boundaries. - Command/job tests for archival behavior if a batch archive process is added, including dry-run or idempotency behavior where available. - Regression tests confirming admin activity log queries still work after feed storage/index changes. - Run the focused affected tests, then run `vendor/bin/pint --dirty` before closing the issue. ## Progress Checklist - [x] Spatie activity log table exists - [x] `gamer_updates` activity entries are written from game-list status changes - [x] `RecentGamerUpdates` presenter exists - [x] Homepage/sidebar Recent Gamer Updates card exists - [x] Sidebar link for `/recent-gamer-updates` exists - [x] Activity log cleanup config exists but currently deletes by age when run - [ ] Inventory all current and planned gamer update feed producers - [ ] Inventory all public/admin consumers of activity feed data - [ ] Choose the long-term current-vs-history storage strategy - [ ] Add required indexes or migrations for bounded newest-first feed queries - [ ] Add archival or migration command/job if a split storage strategy is chosen - [ ] Ensure public feed queries preserve historical data and avoid destructive truncation - [ ] Build or coordinate with the dedicated `/recent-gamer-updates` page work - [ ] Add tests for feed ordering, limits, filtering, archive behavior, and regressions - [ ] Confirm the final strategy keeps Recent Gamer Updates usable at large table sizes

jimmyb added the

Rows
Columns