NB won't display newly DLed headers or posts all of a sudden

[lance317]

Hello,

I'm still on 5.59 build 9826, and all has been fine prior to last night.

A few weeks ago I started DL'ing all messages for a.b.multimedia. My usenet provider can go back to August 2008, and I wanted to get some old news items for a project I'm doing at work.
and all was working fine.

Since it is such a huge group, I've been downloading posts here and there and just building it up. My goal is to just get some old stuff, and not bring ab.multimedia all the way current to 2022, as it is just too much.

Anyhow,, last night I fire up NB after work, load up a.b.multimedia, then download latest, and while it looks like posts are downloading, I notice NB is not loading new headers or posts after 12/23/08.

It looks and acts like it is - the green progress bar grows - but then it flashes and doesn't display anything new. Also, it says I have downloaded 1 GB of posts - but my hard drive capacity isn't going down?

Do I just delete the group and start over? Using "show all files" and have zero filters enabled.

Really strange.

EDIT: I will post a video capture of what I am talking about shortly.

[Quade]

What news server are you using? The only one I know that's pretty much guaranteed to have full retention are Usenetserver and their resellers.

1 - You need to look at "Storage Age". If it's too short. Say Storage Age is 1000 days then Newsbin will toss any headers older than 1000 days. You need need X days set in storage age before you even start this process.

2 - You need to look down where it says "cache". There's a number in parens "(XXX)". This represents the number of blocks of headers waiting to be imported. Until that hits zero, at least some headers won't be visible yet.

3 - When you display the group you'll probably need to use "Show all Posts".

I'd suggest upgrading to 6.90. It's essentially released. It's listed in the beta section of the forum.

[lance317]

What news server are you using? The only one I know that's pretty much guaranteed to have full retention are Usenetserver and their resellers.

1 - You need to look at "Storage Age". If it's too short. Say Storage Age is 1000 days then Newsbin will toss any headers older than 1000 days. You need need X days set in storage age before you even start this process.

2 - You need to look down where it says "cache". There's a number in parens "(XXX)". This represents the number of blocks of headers waiting to be imported. Until that hits zero, at least some headers won't be visible yet.

3 - When you display the group you'll probably need to use "Show all Posts".

I'd suggest upgrading to 6.90. It's essentially released. It's listed in the beta section of the forum.

Thanks for the quick reply.

1. storage age is set to 40000 - I have been successfully DLing content from Summer 2008 to now for many years with Newzbin.
2. not sure where the cache setting you speak of is
3. I always use "show all posts"

[lance317]

here is the video of the problem.....just fast forward through the loading of posts, etc.

you can see that NB will not load anything after 12/23/08. It downloads the headers, but it just won't display them.

show all files is set, still won't load anything after 12/23 - but it acts like it wants to (note the green progress bar activity, than an index jump - but no new posts loaded)

youtube capture of issue here: https://youtu.be/irnLdzR7Cfc

[lance317]

this is bugging the crap out of me, so I just did "download all headers" and will re-download everything and see if it stops at 12/23/08 again like it did yesterday.

Ugh.

Nothing worse than downloading 10-12 Gigabytes of headers again.

Oh well.

[Quade]

Uncheck "Hide Old" to start. I'd uncheck "Enable Filters" too.

It looks like it's loading up a bunch of data then filtering out parts of it

[lance317]

Hello. Just a follow up here...

So after my last post I started re-downloading all headers for a.b.multimedia to see if it would solve the issue.

The re-downloading of headers was going well, starting at August 15, 2008 (I am on a Highwinds provider and they do go back this far)

I have been updating the headers daily instead of just letting it sit there 24/7 as there are something like 3 billion posts in the group - as of last night I got up to September 7, 2009.

The range now for a.b.multimedia is 8/15/08 to 9/7/09

So this morning I fire up Newsbin, load the group, and start downloading headers to try and finish off 2009, and once again like my original post in this thread - it will not download or display anything after 9/7/09.

This is with no filters enabled, show all files selected, and hide old unchecked. Nothing changed on my end from last night - literally nothing.

This like 30 gigbytes of data I downloaded in the last week - I can't do this all over again...

Why is this happening now? Nothing on my PC has changed.

I have been a Newsbin user for many years at this point and have never had the client do this.

Please help, thanks.

[lance317]

I installed the same client at my office - and it is showing the same behavior as at home - it has stopped downloading posts after 9/7/09

No filters, show all files, hide old unchecked.

Retention set to 9000 days.

It will not load anything after 9/7/09

[lance317]

does Newsbin just stop displaying or downloading after a certain number of posts have been reached?

I ask becuase this is the first time in about 10 years I have tried downloading headers from such a large group like a.b.multimedia.

I know with 100% certainty that in the mid 2000s I had a.b.multimedia fully downloaded and updated with Newsbin - and it was never an issue like I have now.

[Quade]

2 - You need to look down where it says "cache". There's a number in parens "(XXX)". This represents the number of blocks of headers waiting to be imported. Until that hits zero, at least some headers won't be visible yet.

You look down at the bottom border of the Newsbin window and see a field called "Cache: X/Y (Z)"

Z is the number of header blocks waiting to import. If you keep downloading headers and the number of unprocessed header blocks just keeps increasing, it means that Newsbin will be importing headers you already have so, you won't necessarily see any changes in the post list.

As long as there are un-imported header blocks, you won't see all headers. If you repeatedly re-download headers while already having headers waiting to import, it just means you have multiple copies of the headers sitting in the import folder.

1 - Header download to the "Import" Folder
2 - Headers are then imported into the group database from "Import"
3 - When they're all processed, all the current headers will be imported and can be seen.

These days a.b.multimedia is just for obscured postings. I'm not clear when that started. I'll try what you're doing and see what I can see.

There's actually a mechanism in Newsbin to save the headers so, you can re-feed them if you want.

[Quade]

No it doesn't stop.

Ok so I added a.b.multmedia, right clicked and "Download all Headers".

I've let it run for awhile and it's showing up to 10/25/09. I'm probably going to let it load up another couple years worth before I kill the header download.

This is with 6.90 and 5000 days "Storage Age".

[lance317]

thanks for checking this out!

I don't see where this "cache" setting is in Newsbin - can you please post a screenshot? Just not seeing it

[lance317]

forgot to add that I am using v 5.59, build 9826 here at my office - I think it is the same as I am running at home...

[lance317]

These days a.b.multimedia is just for obscured postings. I'm not clear when that started. I'll try what you're doing and see what I can see.

Understood - I am mostly looking for the older material - there's still a whole treasure trove of great non-obfuscated stuff on that group from 2008 and up....I had planned on updating a.b.multimedia to about 2015, then just stop and pull posts from it as needed.

[Quade]

1 - Even if 5.59 is broken, it'll never be fixed. That's why I'm suggesting you try 6.90

2 - Open Newsbin. Look at the bottom of the window. There's a row of status information. Like disk free space and download speed. One of those blocks of status information says "Cache". That's what I want you to look at and report about. If you don't see this bar of information. Check "Status Bar" in the view menu.

[Quade]

Ok final result. 120 Gigs of header download got me to 3/27/2011.

I also found that filling the data folder can crash Newsbin.

[lance317]

thanks for the follow up.

here is a screenshot of my Newsbin - not sure which one is the cache....

Image link not allowed for unregistered users


(forum is saying I'm unregistered even though I added my key - view image here: https://imgur.com/OuJYRpH )

it will not display anything past 8/4/09...been like this for a few days.

even though Newsbin says it is downloading new headers, I noticed there are no new folders created for a.b.multimedia in the data folder - last update to a.b.multimedia data folder was 2/21/22 - even though I have selected download latest for the last 2-3 days

[lance317]

I added my registration code and the forum is still saying i'm not registered?

[Quade]

How about the cache thing I keep mentioning? I described how to turn on the status bar in case it's not on. It's pretty critical information.

[lance317]

How about the cache thing I keep mentioning? I described how to turn on the status bar in case it's not on. It's pretty critical information.

please check this screenshot of the client:

https://imgur.com/OuJYRpH

is this where the info is?

[BZee]

Looks like that information is not part of the version 5 interface.

[Quade]

I'm, pretty sure it was in 5.59. Then again 5.59 is pretty old these days.

[BZee]

How about the cache thing I keep mentioning? I described how to turn on the status bar in case it's not on. It's pretty critical information.

please check this screenshot of the client:

https://imgur.com/OuJYRpH

is this where the info is?

I don't see it there or on the status bar shown in the Newsbin.com v550 user guide. I'm using v6.82 so I can't help much.

[Quade]

Ah yes. I was thinking 6.82.

[lance317]

well I downloaded and installed the latest stable version of NB (6.82), and so far so good.

have started re-downloading a.b.multimedia headers on the new client, and am up to November 2009, seems to be going ok so far.

(new version also seems to be much faster, which is nice)

[lance317]

really blown away as to how much faster 6.82 is - I'd been using the 5.59 client going back to (what seems like) 15 years ago, and this is such an improvement!