Corrupted Files (too Fast DL?); UnRAR Error; Auto Add Back?

[ppan]

I was running original build for v6.71 and Auto UnRAR enabled.
I believe that my DL speed was too fast (100 Mbps) for my PC and I didn't have "Pause Download during UnRAR/Repair" enabled.

1. I am not sure if this could have been the reason for a lot of damaged file sets?
(I believe that most, if not all, of these sets were ungrouped)

2. I enabled "Pause Download during UnRAR/Repair" and downloaded one set again to a new directory (bypass Filters). The files downloaded again, and this time the set was unRARed properly, except:
a. I got an error: [11:00:11] ERROR UnRAR Error:RAR files Damaged, Not enough RARs, Needs Repair or Incorrect password..
Obviously this error was incorrect.
b. I got a leftover RAR files as I also reported elsewhere (Aggressive mode is disabled)

3. Is there some way that I can "automatically" add back these files to the Download queue? Now, I have to go to my hard drive, find the title of one of the bad sets, search for it in the newsgroup in Newsbin and then add it to the download queue. Then repeat for 100 or so sets....(lots of work).
I could create a text file with all the file names. Is it possible to somehow feed that list to Newsbin?

[Quade]

If speed is causing corruption then there's probably something wrong with your PC. People regularly download at 300+ Mbps without these symptoms.

I'd lean towards V scanner but it could be a hardware problem too.

You never PM'd me the name of the group to check out (or I missed it).

You should have a bunch of RARS hanging around if the unrar fails. It sounds like the unrar is working and the files are corrupted after the fact. If Newsbin is telling you there was a problem with the unrar, you should listen. Even if the output file is on disk, there's probably something wrong with it.

[ppan]

If speed is causing corruption then there's probably something wrong with your PC. People regularly download at 300+ Mbps without these symptoms.

I'd lean towards V scanner but it could be a hardware problem too.

I have no idea what caused the corrution. I saw the message about turning on the PAUSE:
"Version 6.71 has an option that tells Newsbin to continue downloading in the background while an unrar is taking place. We've had reports that files might fail downloading if your download speed is faster than your computer can handle if it is also processing an unrar. If this happens, re-downloading the files will succeed after the unrar completes. To avoid this situation, you can disable the option to unrar while downloading under Options -> Settings -> Performance. It is called "Pause Download during UnRAR/Repair". You'll want to turn this on. "

What is V scanner? Virus scanner? I have the download directory excluded.

You never PM'd me the name of the group to check out (or I missed it).

I was waiting until the next beta cycle when you will have a chance to work on the grouping.

You should have a bunch of RARS hanging around if the unrar fails. It sounds like the unrar is working and the files are corrupted after the fact. If Newsbin is telling you there was a problem with the unrar, you should listen. Even if the output file is on disk, there's probably something wrong with it.

All the RARs (except one as I mentioned above) and PARs were deleted after the unRAR, so I have no way of knowing if something was wrong at the time.
EDIT: I downloaded the same files again after turning off Auto unRAR. I then used WINRAR to unRAR the files w/o any issues. I compared the two unRARed directories (with Newsbin & error and now with WINRAR). They are identical.



I guess that there is no way to "automatically" add back these files to the Download queue (see question #3 above)? I have a program that could do this (Automate by Network Automation) but it doesn't work on 64 bit applications like Newsbin .

[Quade]

"Pause while unraring"

Is for people who have files pop into the failed list while other files are unraring. It's likely a timing problem with the retries. It doesn't involve data corruption.

When you download and unrar at the same time, the hard drive gets worked pretty hard. You have the download writing to disk and the unrar writing to disk at the same time. Most machines are just fine with it. It's possible yours isn't.

Make sure you exclude both the download and data folders. Because partial files are stored to the data folder. The fact "Pause while unrar" prevents the corruption suggest the chunks in the data folder which grow more you don't pause, are getting corrupted.

I was waiting until the next beta cycle when you will have a chance to work on the grouping.

If I know now, I can work on it between now and the next beta. In fact, I'll add it to the new download engine. Until I make changes to the code base, there's not much point do doing a beta.

I guess that there is no way to "automatically" add back these files to the Download queue (see question #3 above)? I have a program that could do this (Automate by Network Automation) but it doesn't work on 64 bit applications like Newsbin

You can always install and run the 32 bit version. They both work the same. The only time the 64 bit version is really superior is when you start using more than 2 gigs of RAM. If you do install 32, make sure you keep the chunk count below 500 (if you don't know what I mean just ignore this commend).

[ppan]

"Pause while unraring"

Is for people who have files pop into the failed list while other files are unraring. It's likely a timing problem with the retries. It doesn't involve data corruption.

When you download and unrar at the same time, the hard drive gets worked pretty hard. You have the download writing to disk and the unrar writing to disk at the same time. Most machines are just fine with it. It's possible yours isn't.

Make sure you exclude both the download and data folders. Because partial files are stored to the data folder. The fact "Pause while unrar" prevents the corruption suggest the chunks in the data folder which grow more you don't pause, are getting corrupted.

I don't know if "Pause while unrar" prevents the corruption. I don't usually see this corruption but it happened recently. It may have been due to something else.
Note that I am using different three different hard drives for data, download and unRAR. Maybe it is then not so important to use "Pause while unraring"?

[Quade]

My machine doesn't care either way. Whether I pause or not.

Since I don't know what you're downloading or even what group. I really have no guesses for your issue other than the ones I already suggested.

[stavros]

Hi,

I noticed an interaction between "Pause Download during UnRAR/Repair" and when NB is in 'scheduler mode'.

During a "Pause Download during UnRAR/Repair", if the 'scheduler' check cycle (every 30 secs or so?) kicks in, and the schedule is set to 'run' or 'speed limited' then this will reset the 'Paused' state from the "Pause Download during UnRAR/Repair" back to that set by the 'schedule'.

This is much more liklely to happen on a slower PC, when "Pause Download during UnRAR/Repair" is dealing with a large(r) file as the relative size of the time windows are more likely to overlap.

regards
Stavros.

[Quade]

Yeah that makes sense. I think the "Pause Downloads while unrar causing issues for some people" has been addressed in 6.72 as well.