Curious Slowdown

[Bill_M]

I seem to have developed a sudden problem in Newsbin. It appears to be operating correctly but response to any user interface inputs is very very slow. Even clicking on a tab takes up to a minute to display that window. Switching to paused improves the response but it is still very sluggish. I kind of think the problem is outside Newsbin but I'm having trouble narrowing it down. When I look at the running processes, Newsbin has two that seem to be taking all available CPU time. Even when paused.

I have been getting errors like:
ERROR Error: SSL Negociation Failed, Host: news.giganews.com Error: SSL Connection Failed: 5 Error 0 Success
(repeats 2 more times)
ERROR DG - Connection Failed: GigaNews Server Index:1
(repeats 2 more times)
On occasion and there seems to be a lot of Spam in some news groups lately. But I can't really correlate anything. I do notice that the "Cache:" line on the bottom seems to be stuck on 200/200 (19). It seems like usually the 19 is zero or briefly one or two.

So questions: Could server or network probelms be causing this? What processes would Newsbin be running when paused that would take up a lot of CPU? What can I do to get Newsbin to work off the cache? Usually when I leave it for a couple of hours the number seems to go up. I'm guessing it did an update. Could it be stuck on some set of titles? Is there some way to flush the cache and start over?

I hope this makes some sense. I'm still trying to get a handle on it. Thanks for any help.

Bill

[Quade]

Downloadmarker.db3. Try moving it out of the data folder.

Exit, move, restart. See what happens.

[Bill_M]

Well, I left Newsbin in paused and left it for a while. And the cache count finally went down to zero. And now everything seems to be back to normal. I can't duplicate the problem any more. I guess I need to mark this down as operator problems. I'm guessing my old CPU is just not up to handling a lot of posts. I've turned the speed limiter on and will watch what happens.

Thanks for the suggestion. I hope I didn't waste your time. Am I correct in assuming the error messages I got were problems with the Giganews server? I seem to have gotten a variety of errors from the lately.

Bill

[Quade]

On occasion and there seems to be a lot of Spam in some news groups lately. But I can't really correlate anything. I do notice that the "Cache:" line on the bottom seems to be stuck on 200/200 (19). It seems like usually the 19 is zero or briefly one or two.

I missed this. It's not that unusual. The server can deliver headers faster than they can be inserted into the database. They stack up and that's the number you see.

[wumpus]

I'm having some slowdown issues as well. They seem to have started when I installed a 6.62 beta version over 6.61. I did upgrade to 6.62 when it was released and things were still slow. I tried going back to 6.61, but that didn't solve it. (It's of course very possible that it was a coincidence that the slowdown started when I upgraded...)

I'm also seeing high cache numbers. 166/200 (16) if that has anything to do with this.

I was seeing normal speeds when I first started NBP, and then they would creep down to very little (1 or 2 Mbps shown in NBP). I also have an applet that shows traffic in/out on my network interface, and in that I was seeing it jump up, then drop, and jump up, etc. It seemed very chaotic, unlike it used to be when I saw steady amounts of incoming traffic.

I had also checked with my NSP and they reported no problems with their servers or network. Internet speed tests showed that, at least for HTTP, my ISP's network was running ok. I tried power cycling my router, and also switching to a wired connection from the router instead of wireless. I turn my system off when I'm not using it, so it probably doesn't have much in the way of apps with memory leaks slowly consuming memory. (It doesn't matter what other apps are running or not running either. The slowdown is the same.)

I've played with the performance settings in NBP, tried d/ling files of varying sizes, and from different newsgroups. I'm using Linux Mint 17.2 64-bit with 3.16.0-38-generic #52~14.04.1-Ubuntu SMP kernel. System RAM is 16GB and it looks like this with NBP and a few other programs running:

$ free
total used free shared buffers cached
Mem: 16432924 15872212 560712 90784 254732 10073080
-/+ buffers/cache: 5544400 10888524
Swap: 16740348 13888 16726460


I use Crossover 14.14 with NBP in its own bottle, an XP one. disks are spinning, and there is plenty of space.

The last few days things got even worse, and I can't add new items to the download queue, plus a couple of Download Headers jobs I started aren't completing, nor are downloads. The max traffic I see from NBP is 30 B/sec down. I've even tried deleting the header jobs and it doesn't help. In addition, if I shut down and restart NBP, the header jobs reappear.

I also have a Win 7 VM (32-bit) that I play with from time to time. I installed Agent 8 on this and tried it with my NSP. For NBP, I had arrived at 15 connections (SSL) in the past as seeming to do well. I tried that with Agent and saw the same kind of behavior, with chaotic actual download speeds, and overall nowhere near the speeds I should be seeing. I tried increasing the number of connections to almost 30 and didn't see any increase in speed. Then I dropped the number of connections way down to 6 and suddenly I was seeing steady download speeds of the speed I was used to in the past.

I think that narrows things down to an issue with NBP or Crossover, or an interaction between one or both of these and Linux Mint. After trying Agent, I dropped the number of connections in NBP down to 6 and then 5 and it unfortunately didn't help.

Any ideas on where I might try looking next?

[wumpus]

I just noticed that most of the sqlite dbs in my spool_v6 dir structure had 0-byte journal files and many of them also had temp files, some dated more than a year ago. Most of the db files and journals were dated several days ago, which is about when I started having trouble with the download queue becoming static.

I backed up my data directory and then deleted all of the journal files and tmp files.

NBP is much happier now. I can add items to my download queue, and downloads are processing at a decent speed. It's still slower than Agent, but it's a big improvement. I started out with the 6 connections that I had while troubleshooting, and just now bumped that up to 16. I'll try playing with the number of connections and see what gives the best speed.