Corrupt downloads when run under WINE

[Co6aka]

Bizarre issue when running under WINE... Current is NB6.90B1 on WINE6.0RC1 but issue has been going on for the past year, with earlier versions of NB and WINE of course. I've re-installed both "from scratch" but the behavior doesn't change.

Download progress displays as "normal" but certain files repeatedly end up being smaller to much smaller in size than they're supposed to be, and throughout those files will be areas ("chunks") of only 0x00-filled bytes. Logging always indicates that all chunks were downloaded successfully. Re-downloading the files usually results in different file sizes with different areas of only 0x00-filled bytes. Sometimes, after re-downloading the corrupt file(s) in the set perhaps a dozen times, with the multiples renamed, there's enough for MultiPAR to fix the file set. Doesn't matter what file type or if it's a PAR; any file type can be corrupted, but it does tend to be the larger files that are corrupt. It's always the same certain files that are corrupt.

So, earlier I installed NB on a Winderz-10 box and everything downloads perfectly. Just before posting this, I installed NB on a completely different Linux box with a fresh install of WINE and again it has the same corrupt download issue.

Again, when re-downloading the entire file set, it's always the same files in the set that are differently-corrupt, which makes me think this is an issue with NB rather than WINE.

[Quade]

I think you're going to have to log Wine and see if you see write errors. 0 byte files in the chunks folder indicate write failures. Since it's not failing in Windows, it suggests some Wine issue.

I don't run it under Wine. I guess I could try. I'd probably revert to 6.82 and see if it has the same issue.

[Co6aka]

I'll try to explain better: Not 0-byte/0-length files, but files containing "chunks" of 0x00 characters mixed in with "chunks" containing "data," and the size of the file is almost always different on each re-download, and is a fraction of the size the complete file should be. The download progress bar in NB displays as expected, no errors or retries, and the NB log shows that all of the file's parts were downloaded from the server. Maybe it's a problem with assembly of the file from the downloaded chunks?

I can supply a few corrupted files and a NZB for the source files if that might help.

6.82 and whatever previous versions for about the past year have the issue. I think all was fine in 2018, and maybe in the beginning of 2019. I haven't downloaded much over the past year so I can't be more precise.

Update: I just looked at the NB release history, and maybe the problem began with 6.80, so I'll try the 6.7X series tomorrow and let you know.

[Quade]

I can supply a few corrupted files and a NZB for the source files if that might help.

Do they work in Newsbin under windows? Sure you can send them to ts @ newsbin dot com.

The bottom line really is it works fine in Windows and not under Wine. If I'm interpreting what you're first post says correctly.

If it was corrupt both places, I'd think it was Newsbin or the file itself. The code Newsbin is running is identical in both cases. The only difference is Wine. I wonder if you're running into some weird disk space issue?

Are you using the same NBI in both Window and Linux?

[Co6aka]

Do they work in Newsbin under windows? The bottom line really is it works fine in Windows and not under Wine.

Yes, and Yes. On the Windows-10 box everything is perfect.

The code Newsbin is running is identical in both cases. The only difference is Wine. Are you using the same NBI in both Window and Linux?

Almost identical; two different computers, otherwise the Newsbin install is identical. The test NBIs are what the installer created, with one server config and group added. Problem happens with any/all NBI, my original "main" NBI and with the bare-bones testing ones I created to try to isolate the issue.

I wonder if you're running into some weird disk space issue?

4.8TB free on the Linux box, 959GB free on the Windows-10 box. The only troublesome app in WINE or Linux is this specific issue with Newsbin.

I've tried all combinations of WINE installs and original NBI, fresh-NBI, etc. I'm going to experiment with the earlier Newsbin versions today, fresh WINE+NB installs for each, and I'll update you with the results.

Pondering... The corrupt files could be similar to the resulting file(s) when you force "assemble incompletes" for incomplete files. What does NB put in place of the missing chunks; does it pad the missing part(s) of the file with 0x00 bytes?

[Co6aka]

It could be something to do with "chunks" because it looks like those 0x00-filled blocks in the corrupt download files should have been filled with the contents of files remaining in the "Chunks" folder after Newsbin reports that the download has completed. And actually, that's another thing that's been going on for a while: The "Chunks" folder continually grows with chunk-files for many months until I wonder why it's many gigabytes big...and then I manually delete the contents that date back for many months. On the Windows-10 box that's not happening, at least not during the tests I'm doing. Should the "Chunks" folder be filling up with files like that?

[Quade]

By default Newsbin purges the Chunks folder when the chunks > 2 days old. It works in Windows anyway. I agree all your problems look like chunks problems.

Doesn't really tell me why it's not working in your setup though.

[Co6aka]

Looks like the problem involves symlinked folders on another drive...

This works OK: Entire Newsbin folder plus subfolders are on another drive with a WINE drive letter assigned to it. ex- "N:\Newsbin\..."

This produces corrupted DL files as described previously, with leftover chunk files hanging around forever: "Chunks" folder is a symlink to a folder on another drive. ex- "/data/Newsbin/Chunks" is a symlink to "/temp/Newsbin/Chunks" (My "/temp" - not to be confused with "/tmp" - is a 150GB 10k-spinner for scratch purposes.)

And this also produces corrupted DL files: "Newsbin/Chunks" is a symlink to "/tmp/Newsbin/Chunks" where "/tmp" is in RAM.

It's strange how only Newsbin has difficulty with these configs. Maybe a threading/timing issue, race condition, etc? Could the chunks be getting deleted before they're completely integrated/copied into the files?

Anyway, my solution is to just have "/temp/Newsbin/..." all on the scratch drive with a WINE drive letter assigned to it, as above. At some point I'll test symlinking "/spool" back to the original SSD. Any chance of making those folders specifiable in a drive:\path format in Newsbin?

[Quade]

I just assume the disks and folders work. It the underlying hardware OR OS is broken there's not much I can do. You move the whole installation to a larger drive so you don't have to mess with the symlinks. Newsbin uses UNC pathing internally

"\\C:\folder1\folder2\"

Maybe Wine can't handle that across a symlink? That's why I suggested you log Wine.

[Quade]

I just installed the new beta under wine. It works pretty damn well. No corruption here but I'm not using symbolic links for things.