Assemble Incompletes not working - nothing happens

[Co6aka]

6.91 B5 5672... "Assemble Incompletes" does nothing; incomplete files remain on the "Downloading Files" tab and nothing is written to disk; seems as if the function is disabled. Plus, there should be enough blocks for PAR to repair the sets since only one or two files in the sets are almost complete; typically 98-99% complete. In the past, assemble incompletes worked, and I did notice that manually running PAR resulted in repaired files even though Newsbin couldn't repair.

Also, with complete files, Newsbin pauses download of PARs even though "Pause PARs" is unchecked. Checking and unchecking has no effect, and action is the same regardless of the checkbox condition.

[Co6aka]

Update... Seems that selecting "Disable PAR Repair" allows assemble incompletes to work.

FWIW, the logfile is filled with hundreds of "DEBUG FileAssembler: File Assembly Chunk Size 2 doesn't match size of size of current data buffer" messages.

[Co6aka]

Another update... After disabling automatic PAR Repair and downloading all incompletes, plus the remainder of the PARs, I was able to repair ALL of them manually with MultiPAR. Something is broken.

[Quade]

"Assemble Incompletes" is the default action these days. Meaning you don't need to select it specifically to make it assemble. Newsbin downloads "one and done" now. Meaning it downloads what it has on hand and assembles the file.

How are you downloading? If with NZB's you need to feed them directly to the download list for obscured posts to download and repair properly.

I wonder if these are the very newest files and they simply haven't propagated to your server yet? That might explain why time would impact this?

If you have NZB's to try you can send them to [email protected]. They might not be incomplete on my server though.

[Co6aka]

Downloading; three ways:
1. Open group, wait for new posts to display, select desired files, download.
2. Open group, load all posts, select desired files, download.
3. Load NZB, select desired files, download.

As I wrote before, incompletes remain in the downloading tab indefinitely, and highlighting them and selecting assemble incompletes does nothing; the incompletes remain there for days until I remove/delete then from the downloading tab. Disabling auto-PAR "enables" the assemble incompletes function.

PS- Posts are on the server; I can download them with SABNZBD.

PPS- NZBs aren't the issue, Newsbin is the issue. I can send/link a screen recording but don't understand what that would show beyond what I've already reported.

[Co6aka]

Why would Newsbin pause downloading of PARs even though "Pause PARs" is NOT checked? Maybe that's a clue? Some sort of race condition due to miltithreading maybe?

Is there a "debug logging" mode in Newsbin that might help figure this out?

[Quade]

There's a logging tab and a logging file in the data folder.

Once I get Newsbin working again, I intend to see if I can force it to fail to assemble. I'm in the middle of some big internal changes or I'd have already tried.

I've never seen B5 fail to assemble so, I'm going to generate some broken NZB's so I can simulate missing chunks.

[Co6aka]

There's something else going on too that may be related and hopefully offers a clue: When I select a lot of files and Cntl-Y to download, several seemingly random files are skipped as if their signatures matched a file that had already been downloaded... Only that feature isn't selected. If I select/download only a few files at a time this doesn't happen. Initially the green check appears next to every file whether or not it was actually downloaded. If I then close the group tab and re-open it and load posts, the files that were skipped now don't display the green check. Selecting a few of them at a time and downloading is the workaround. Guessing this is a multi-threading sync issue?

[stavros]

Hi Quade,

I've also had a few instances where downloads just hang in the download tab and assemble incompletes doesn't work.
There are no errors shown in the debug log.

The issue seems to be with the smallest .par2 - it shows as "Downloading" and remains so, even when I do a manual assemble incompletes.

When I inspect the downloaded smallest .par2 on my disk, the contents appear complete (standard start and end portion of the .par2 are present).

When I run quickpar manually (starting with the smallest .par2) it identifes that it needs a small number of blocks to fix the fileset.

I have to manually unpause a couple of the smaller .par2 files to let NB download them.

Now that all other parts needed for repair/extraction are present, when I run quickpar manually (starting with the smallest .par2) they repair properly.

Newsbin still shows the smallest ,par2 as "Downloading" and the download is still hung.

I can manually successfully extract the fileset using 7zip.

regards
Stavros

[Quade]

Only that feature isn't selected.

When Newsbin downloads. it wont assemble a file that's already on disk. Even without the signature detector, it detects exact duplicates and won't replace them. You'd have to move the matching files out of the download folder to re-download them. I suspect that's what you're seeing.

I'm still in the midst of testing the new changes. I haven't forgotten your topic.

There was a bug in one beta where if you sent a bunch of disconnected files to the download list, that were covered by par files, it was possible for the par processing to pull some of them back from the download list. I wonder if that's what you're seeing? As an experiment, you might try downloading the files first and then add the pars and see if this makes it work.

[Co6aka]

When Newsbin downloads. it wont assemble a file that's already on disk.

The files weren't already on disk, and weren't on disk until I selected and commanded the downloads (Ctrl-Y) a second time, at which time they downloaded OK.

I'm still in the midst of testing the new changes. I haven't forgotten your topic.

No prob, just updating whatever I discover in case it might help.

you might try downloading the files first and then add the pars and see if this makes it work.

I'll let you know what happens...

[Co6aka]

Downloading files before pars changes nothing. The issue happens when adding a lot of files to the DL queue at the same time. Newsbin displays the green icon as if the file were downloaded, then after reloading the group tab the green icon has reverted to a blue circle for the files that weren't downloaded. Selecting and re-downloading a few at a time works fine, then I can check (and repair if necessary) the sets with Multipar. Seems like threads are out of sync.

[Co6aka]

Another clue: When selecting and downloading just a few files at a time the issue doesn't happen.

[Quade]

How many sets of files at once seems to trigger it?

I'm assuming these are compacted sets and not individual files?

Local headers mostly or search?

As an experiment, you could pause Newsbin, then add the large block then un-pause when they're all in the list. Then see if the symptoms remain. I'm trying to figure out what's triggering it. It doesn't seem to be an "Assemble incompletes" problem at all. It sounds like a "Not getting into the download list in the first place" problem so there's nothing to assemble because the files never got there.

They're not ending up in the failed list are they?