Slow performance

[samboy]

Hi,
I'm having trouble with NewBin consuming a lot of CPU cycles and the download working very slowly (many times 0/0) for long periods of time; only to speed up to my connect speed of 200Mbps for a short time. It's only happened since upgrading to 6.82/Build 5142; but also around the time with many people working from home - so I can't rule out an issue with my ISP. However, Newbin is consuming around 4% of the CPU for the 3900x system I have....... which is one core pretty much fully loaded.

I consistently see the following in the callstack:-

0, newsbinpro64.exe!pcre2_set_parens_nest_limit_8+0x6aae8
1, newsbinpro64.exe!pcre2_set_parens_nest_limit_8+0x6ae62
2, newsbinpro64.exe!pcre2_set_parens_nest_limit_8+0x56246
3, newsbinpro64.exe!pcre2_set_parens_nest_limit_8+0x560ea
4, newsbinpro64.exe!pcre2_set_parens_nest_limit_8+0x43b04
5, newsbinpro64.exe!pcre2_set_parens_nest_limit_8+0x44f20
6, newsbinpro64.exe!pcre2_set_parens_nest_limit_8+0x3e693
7, newsbinpro64.exe!pcre2_get_error_message_8+0xca95e
8, newsbinpro64.exe!pcre2_get_startchar_8+0xcedc
9, newsbinpro64.exe!pcre2_get_startchar_8+0x11a00
10, newsbinpro64.exe!pcre2_get_startchar_8+0x11915
11, newsbinpro64.exe!pcre2_get_startchar_8+0x3c3de
12, newsbinpro64.exe!pcre2_get_startchar_8+0x5ff92
13, newsbinpro64.exe!pcre2_get_mark_8+0x1378e2
14, newsbinpro64.exe!pcre2_get_mark_8+0x138feb
15, newsbinpro64.exe!pcre2_get_mark_8+0xf2d8c
16, newsbinpro64.exe!pcre2_get_mark_8+0xde2fc
17, newsbinpro64.exe!pcre2_get_startchar_8+0x27865c
18, KERNEL32.DLL!BaseThreadInitThunk+0x14
19, ntdll.dll!RtlUserThreadStart+0x21

Which is pretty consistent when it is downloading very slowly (or not at all) and taking ~4% of the CPU. I have 64GB of RAM (and have configured things for 8GB for chunks) and everything is installed on an nVME SSD; but it seems to be CPU bound not I/O bound.

I went to the "Data Folder" and cleaned out most of the "DB3" files in case something had gotten corrupted; but this has not helped.

When Newsbin is maxing my connection for the short periods it does; the CPU usage is much lower and I see several threads sharing the load. When the download is going very slowly (or not at all) I see the above core stack and it's pretty much one thread doing all the "work". Also the "Downloading Files" window is pretty much non-responsive UI wise when Newsbin is in this state.

Any suggestions for resolving this performance issue?

Thanks

[samboy]

Just a quick update........ I went back to V5.81 Build 5080 and all the performance issues have gone away

Not sure if the install cleaned anything up (in which case this may have also been a solution for the latest release as well) but for now I'm sticking with this earlier version.

[Quade]

I'd put the chunks setting back to default to start. Increasing the chunks can be counterproductive if the number is too high. Newsbin was re-designed so having a large chunk cache doesn't buy you that much anymore.

You might need to clean the password list if you download encrypted content. People have reported performance issues when the password list gets too large.

There's a guy on the IRC channel running 6.82 getting consistent 780 Mbps.

[samboy]

Thanks for the feedback Quade

The previous version is working like a charm with 250Mbps rate and no high CPU usage (it was not my ISP). I'll stick with this previous version and will try your suggestions with the next release.
I assume that the calls tacks I posted imply that my rather large chunk cache was the culprit behind the performance issue with the current release? (I don't have a password list)
One other observation is that the problem seemed to get worse the more I put into the download list.

[Quade]

Code: Select all

7, newsbinpro64.exe!pcre2_get_error_message_8+0xca95e
8, newsbinpro64.exe!pcre2_get_startchar_8+0xcedc
9, newsbinpro64.exe!pcre2_get_startchar_8+0x11a00


If you're constantly importing headers, you'll see that it spends a decent amount of time in the PCRE code. Particularly if you're downloading headers from a group that's no longer header friendly. If you look down near the cache line where it shows a number in parans "(XXX)". It's it's not zero Newsbin is importing headers.