Newsbin Choking On 1GB .rar files (50 GB set)

[Immunity]

Hi there.

I've been a loyal user for quite some time now (over a decade maybe?) and I've never really had any issue I've been unable to remedy with Newsbin until recently - maybe somebody can lend some much needed wisdom...

Lately, a lot of the posts I've been grabbing have been quite large (40 - 100 GB total) and several uploaders, for whatever reason, have started using huge .rar sizes for the set, usually 1GB. This seems to cause Newsbin to "choke" as it tries to assemble the chunks into the full .rar file. I don't have this problem with .rar's in the 200-300 MB size.

I'm on v6.80B9 Build 4978.
I've set my "Cache Size" to 1600. I've tried setting it higher, but no matter how high I up it, it will quickly deplete.
I've got my Main Download and Unrar folder on a USB 3.1 External drive.
I've got the Newsbin Data (I'm assuming this is where it downloads chunks to before they're assembled into a full .rar file?) folder on a 7200 RPM internal secondary drive.
Download rate is a steady 230 Mbps (until Cache depletes :p )
32 GB DDR4 in the system, so RAM isn't a limiting factor.

I've ran this setup for many years without issue. Sure, Unrar'ing is a bit slow as the D/L folder and Unrar destination are on the same external drive, but that doesn't really bother me. The problem I'm having now is that once that Cache depletes, download speeds plummet as my drives (or Newsbin?) struggle to catch up.

I'm betting this is a case of I/O transfer rate screwing me over and am wondering if there's anything (short of replacing hardware) that I can do on the settings side to alleviate this problem. Also, why does this only happen with large .rar's?

If anyone could provide any insight it'd be greatly appreciated.

Thanks!

[Quade]

You could dedicate more memory to the in-memory cache. Maybe make it 2400 so, it should never write chunks to disk. You're moving a gig of chunks to disk. It takes some time. Even if they're all in memory, it's still 1400 or so chunks per file.

I'd download to the internal drive too instead of out to the USB disk. You can continue unraring to the external disk, just download to the internal.

[Immunity]

Per your advice I've moved the download folder to my internal, and I've upped my cache size to 6400 (figured screw it, I've got RAM to spare, heh).

Things are better, but one issue remains however. It seems that when a RAR isn't downloaded fully (lets say a bit or two is damaged) it remains as "Downloading", un-assembled, and keeps hogging up that valuable cache space. Is there any way I can set it so that, even if it's damaged Newsbin assembles it immediately so that it gets dumped out of cache, and just let the Autopar/Unrar take care of it later? In the past I've simply used "Assemble Incompletes and Remove", but that isn't practical here due to the cache issue.

Would the "Aggressive Assembly mode" under the AutoPar options take care of that?

[Quade]

Partial file downloads automatically get pushed out of the cache as new file parts download. It does mean that when the partial files are eventually assembled, it's slower then the normal downloads.

"Aggressive Mode" is the default in B10. What Aggressive mode does is short cut the "assemble incompletes" process. It'll assemble incompletes after the first set of retries finish.