Implement Handling of The Feed (Database Table and Related Pages) #3

New issue

Open

opened 2024-01-14 14:12:23 -06:00 by jimmyb · 6 comments

jimmyb commented

2024-01-14 14:12:23 -06:00

Owner

There's no point in having 1 million rows in this table. We should clean it nightly and trim it down to 100,000 rows. That should be more then enough data. I can't think of any reason why we'd want to keep really old stuff.

As noted here I don't think I want to remove data, but we definitely need to figure out how to store older records. Perhaps one table can have up to 5,000 records whereas older data can be offloaded into another table or something.

~~There's no point in having 1 million rows in this table. We should clean it nightly and trim it down to 100,000 rows. That should be more then enough data. I can't think of any reason why we'd want to keep really old stuff.~~ As [noted here](https://octocat.ninja/MyVideoGameList/myvideogamelist.com/issues/4#issuecomment-1445) I don't think I want to remove data, but we definitely need to figure out how to store older records. Perhaps one table can have up to 5,000 records whereas older data can be offloaded into another table or something.

jimmyb added the

status: needs investigation

component: backend

type: improvement

labels 2024-01-14 14:12:23 -06:00

jimmyb commented

2024-01-14 14:12:30 -06:00

Author

Owner

Originally posted by josh87

Maybe you can trim it down by date?

**Originally posted by josh87** Maybe you can trim it down by date?

jimmyb commented

2024-01-14 14:12:37 -06:00

Author

Owner

@josh87 Yeah, I was thinking about how to do it. I was thinking just select 100,000 rows, truncate, then reinsert them. I have to remember which resets the auto-increment back to 1.

jimmyb commented

2024-01-14 14:12:45 -06:00

Author

Owner

Originally posted by josh87

Well truncate will drop all records from a table and reset auto_increment.

**Originally posted by josh87** Well `truncate` will drop all records from a table and reset `auto_increment`.

jimmyb commented

2024-01-14 14:12:52 -06:00

Author

Owner

I am not sure the value it would add to remove this data from the database on a periodic schedule. The only time I have noticed it going slow is when loading the Edit User page from the administrative dashboard side of things. I'd like to think this is a low priority issue at the moment.

jimmyb commented

2024-01-14 14:12:57 -06:00

Author

Owner

We need to do some research and see if this page/functionality is really even used by anyone.

jimmyb commented

2024-01-14 14:13:06 -06:00

Author

Owner

So people definitely do look at this page (I asked in our Discord server), so we'll need to figure out how to handle the feed. Ideally I don't want to remove any records but at the same time, the table is massive.