Header Storage and Compact

This is the place to help test and discuss Version 6 Beta releases.

Header Storage and Compact

Postby flat_beer » Fri Mar 29, 2024 6:52 am

When I look for music I usually do a local search. A few days ago I did some searches and it was showing what I wanted but some of it was more that 4500 and 5000 days old. At best some of my servers have out to about 4300 days. So I thought I should just purge those headers. Since I really didn't want to download the newest headers in some of the groups I thought I would manually purge the old headers. So I checked my settings and made them:
Download Age: 4300
Display Age: 4300
Storage Age: 4300

So then I right clicked on the groups Post Storage -> Purge to Display Age. Then right clicked on the groups Post Storage -> Compact Group. I went and did this to every group that I've download headers for (about 60 groups). Newsbin used about 1 core of CPU for awhile (can't remember how long... maybe 20 min). Free space on the Header Storage down and up... eventually freeing up about 10GB.

However I now have some groups that are completely empty.... but there should be stuff in them. There have been post to them in the past year. And the spool directory still has Storage.db3 and StorageData.db3 of significant size (Up to 1 GB.) Yes, I have right click on the group and done Show All Post. And I made sure all filters are off. Newsbin seems to take almost no time (less than 2 seconds) to load these groups. Like the DB is empty. I've gone to www.sqlite.org and downloaded the command line utilities. The command line utilities say the db is not corrupt.

So is there any way to recover the headers?

Hum. Just had a thought. Does Post Storage -> Purge to Display Age Purge the headers older than Display Age? Of was I thinking of it wrong and did it purge from now back to Display Age? (The opposite of what I thought it did.) If it does this then then why isn't the Storage.db3 almost 0 size? (What I did should have basically emptied the spool storage. Right?)

Is there a different way to purge the old headers without downloading new headers?

When I load a group and it shows what I've downloaded in the past. Is the status of that download stored in Storage.db3? Or DownloadMarker.db3? Or someplace else.

I'm getting tired. This probably isn't making sense. Hopefully someone can understand what I'm saying / asking and give me a meaningful response.
flat_beer
Active Participant
Active Participant
 
Posts: 56
Joined: Mon Aug 18, 2003 12:21 am

Registered Newsbin User since: 04/14/03

Re: Header Storage and Compact

Postby Quade » Fri Mar 29, 2024 9:51 am

What version are you using?

Pick a group that shows no content then "Show All Posts" on it. Does the progress bar take some time to progress but nothing shows up? Or is it a quick flip and it's done?

The question is whether the data is missing or just hidden. If you see a progress bar that takes some time, it means Newsbin is churning through some data. If it's just a quick flip and then done, it means the data isn't there anymore.

Some groups these days just have what I would consider to be spam in them anymore. A bunch of encrypted posts. Only the oldest fine contain usable results.

Usenetserver still mostly works back to 5500 days.

I'l try what you did, purge to display age on some groups and see what happens.

"Downloaded" status can be in either the Storage.db3 or DownloadMarker. It depends on when the files were downloaded.

It's a good idea to backup your data before you do this. For example when I test this, I'm going to copy the group I'm going to test in first before I test so I can restore the group.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44999
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: No registered users and 13 guests