newsbin.com

While working with a user with gigabit internet access, he was having problems getting much over 500 Mbps/ 62 MBp/s. He was downloading to an SSD and had the data folder on a different SSD or so we thought. Turns out they were both on the same drive. He switched the data folder to the other drive, then downloaded to a second SSD and was able to achieve a peak of:

126 MBp/s/1008Mbp/s with an average of 122ish/ 976 Mbps.

This was with Autopar enabled.

While testing, I was able to get 750ish Mbps from a local server to my data and download folder. Both are spinning disks. I was pretty impressed with that. Still, it doesn't come close to the speed this user is getting.

Quade wrote:While working with a user with gigabit internet access, he was having problems getting much over 500 Mbps/ 62 MBp/s. He was downloading to an SSD and had the data folder on a different SSD or so we thought. Turns out they were both on the same drive. He switched the data folder to the other drive, then downloaded to a second SSD and was able to achieve a peak of:

126 MBp/s/1008Mbp/s with an average of 122ish/ 976 Mbps.

This was with Autopar enabled.

While testing, I was able to get 750ish Mbps from a local server to my data and download folder. Both are spinning disks. I was pretty impressed with that. Still, it doesn't come close to the speed this user is getting.

I wonder with this extreme speed/dl amount to SSDs, what their lifetime will be after TBs of writes. If this user let us know about that, it will be a good reference point for the endurance of the SSDs including their brands of course

Best Regards

http://techreport.com/review/24841/intr ... experiment

It's a couple years old but I consider this to be the bible for SSD reliability.

All six of our subjects, including the 840 Pro and Samsung's TLC-based 840 Series, wrote hundreds of terabytes without issue. The 840 Pro has written over 1.1 petabytes to date, and it's still going strong.

The Samsung 850 Pro's come with a 10 year warranty too.

I'm still discomfortable/anxious about my SanDisk Extreme Pro even it comes with 10 years warranty

I'd look at the comparison test and check your smart variables. I tend to swap out SSD's every couple years. I just switched my work disk to a 1TB which cost less than the 512's I bought last time.

Since I've tested spinning disks up to 750 Mbps. I'm not sure an SSD is necessary unless you're really downloading faster than that.

Quade wrote:While working with a user with gigabit internet access, he was having problems getting much over 500 Mbps/ 62 MBp/s. He was downloading to an SSD and had the data folder on a different SSD or so we thought. Turns out they were both on the same drive. He switched the data folder to the other drive, then downloaded to a second SSD and was able to achieve a peak of:

126 MBp/s/1008Mbp/s with an average of 122ish/ 976 Mbps.

This was with Autopar enabled.

While testing, I was able to get 750ish Mbps from a local server to my data and download folder. Both are spinning disks. I was pretty impressed with that. Still, it doesn't come close to the speed this user is getting.

I'm using a Samsung 950Pro m.2 on an Asus x99 Deluxe II with Intel 6800K and 32GB memory. Broadband is 200MB/s. However download storage is to a 3TB 7200rpm drive. What is the bottleneck for speed download is it the 950Pro or the harddrive?

Ian

How fast is it going?

You could be bottlenecked at the server end too. What news server are you using?

Quade wrote:Since I've tested spinning disks up to 750 Mbps. I'm not sure an SSD is necessary unless you're really downloading faster than that.

I had to move the data folder to an SSD to consistently max out my 500Mbit connection. I could hear the spinning drive churning like crazy whenever the speed dropped off. My guess was that some posters use smaller individual messages, causing the HDD to get into trouble updating one of the database files with info about each one, but it could also have been some other program that decided to do something with that drive I suppose.

I have noticed NB maxes out the first core while downloading now (i7-4790K), so I'm not sure I'd be able to get more speed if I went gigabit. Several other programs seem to be CPU bound when downloading, NB is pretty much the only way to max out my bandwidth.

I have noticed NB maxes out the first core while downloading now (i7-4790K),

There's one thread that does asssembly and unrar. Repair uses multiple cores internally.

At these speeds you want to watch your logging level. Running debug logging can slow the download because the logging tab gets flooded.

Logging is set to normal. Logging tab is typically blank. I think I just need more powah, Scotty.

Good to know

I myself have seen speeds up to 460 Mbps only - downloading directly to cheap and slow external drives. But the average speed is closer to 300.
If I want to keep my downloads, I still would have to move the downloaded files to an external drive. So the question is - which is faster in the end.

I have a Gbps connection and have reached almost 900Mbps download. However, most of the time, I get 300-600Mbps. I have 7200RPM HDDs, which should be able to handle Gigabit easy. I have transferred files at 200 MBps plus, which is more than enough: 1600Mbps vs a max of 945Mbps-ish for the line. I think the bottleneck, in my case, is my Usenet provider. All of this was on v6.6. I have upgraded to v6.73 to see if there is a difference.

Also, you aren't going to do more than 945Mbps due to overhead.

New screen
Menu screen pics
How do I know what version do I have?

Think I have posted this earlier, don't remember, but FWIW...
Newsbin v6.80RC2 installed on a different drive (Samsung 850 Pro) using a 500Mbit/s subscription - fiber - with Newshosting
(25 connections, but probably any number of connections above -say- 10, 12 or so will hardly make any difference)

Nice.

1.01 Gbps/126.1 MBps

6.81B2, Intel i3 7310t. Dual 860 EVO in Raid 0. 8 gig cache on download.

Wow!

A very interesting discussion and to me it is really cool to see further developments of Newsbin in that direction for staying at the top.

However, speaking of speediness, if I may share an idea without an intention to hijack this thread: My own bottleneck is UnRAR.

I would love to see a smooth support for external applications simply because the internal UnRAR is to me a p**n in the a**.
FTTH is nice and it means to get a 65 GB ISO in a very short time but even with the best hardware, waiting ages for UnRAR to extract the archive is not cool.
Winrar can do it in a fraction of the time.

Cheers!

jamesio wrote:

1.01 Gbps/126.1 MBps

6.81B2, Intel i3 7310t. Dual 860 EVO in Raid 0. 8 gig cache on download.

Quade, now.. THAT is nice...
My 70MBps looks like just some cheap entry level subscription

@jamesio
care to post a Ookla Speednet test screenshot?

Way back in 1998 I had to dial in, pay per minute and was happy with a 4-5 kb/s ...

I still remember when Dual DSL was as fast as I could get. Aggregate speed of 3 Mbps.

[quote]I wonder with this extreme speed/dl amount to SSDs, what their lifetime will be after TBs of writes. If this user let us know about that, it will be a good reference point for the endurance of the SSDs including their brands of course
[/quote]

Well, I use an older 120GB SSD and it began slowing down noticeably after about a year and TB's of use. I swapped to another, much less used SSD and the speed came back. So I think you will notice an issue after some xxx TB of use.

BTW, the use of a separate data folder causes nothing but headaches when upgrading to new versions with filters, etc being wiped out. ALWAYS BACKUP YOUR ENTIRE CONFIG before an upgrade.

Scott

SSD's don't like to be nearly full too. The fuller they are, the slower they run.

If you don't include the headers, the data folder is pretty tiny. There's no reason you can't back up the whole thing as long as you exclude the spool_v6 folder.

Glad I came and checked the forums. Just been upgraded to 400Mbps and found my speeds were varying quite wildly when downloading, at 350Mbps it was fairly stable. Will try changing the data folder to my SSD and see if that improves it. I'm only on a 5200 rpm drive for my downloads so guess that may also be an issue.

Cheers Gents

At these speeds, non-microsoft security packages may restrict your download speed too. As an experiment, you could remove any non-MS security, reboot and try some more downloads to see if the speed stabilizes.

RayMark wrote:Good to know

I myself have seen speeds up to 460 Mbps only - downloading directly to cheap and slow external drives. But the average speed is closer to 300.
If I want to keep my downloads, I still would have to move the downloaded files to an external drive. So the question is - which is faster in the end.

If it's any help to you, my top speed (when the stars aligned) downloading directly to a WD Mybook 8TB usb3 drive is 992.9Mb/sec, so there might be some argument for upgrading your external when you next run out of space, but on the other hand, I choose to mostly keep a vpn running that slows my max d/l speed down to 300ish as the time difference in the grand scheme of things isn't worth killing all the traffic that requires a vpn running (especially as the usenet server that gives me those speeds does tend to drop to the 400mb speed frequently_) But for your other question, odds are if your setup supports higher speeds and the 460Mb limit is from the drive and not network issues, you wouldn't get any (final) speed boost from downloading them to a faster drive then moving them to the external drive. (But, what you could do to find out is set newsbin to download to a folder on your internal drive, with an extract to folder on your external drive.)

I've been a registered user of Newsbin going back many years. It's an awesome product. Until recently, I never had any issues...it did exactly what it's supposed to do without issue. The only minor issue I've had was an issue with a version that didn't like AMD for some reason (quickly resolved in the next beta!).

For my most recent issue, it was similar to the issue as described in the original post...but I was downloading the chunks on the same physical spinning disk drive as the download path directory. With larger data downloads, Newsbin couldn't keep up (I'd have to pause it for the chunks to be written to the download path). When I investigated it dawned on me that the reads and writes on the same physical disk were killing the performance. I just hadn't noticed it before because my downloads were previously never that large.

Originally my drive config was as follows:
Newsbin data directory (chunks) and download path on the same physical spinning disk drive.
UNRAR directory on a second physical drive.

I ended up saving my config and uninstalling Newsbin and reinstalling on a partition on my 1TB SSD (400GB free). Now my config looks like this:

Newsbin data directory: SSD partition
Download path directory: spinning disk drive 1
UNRAR directory: spinning disk drive 2

Dramatic improvement on speed all the way around. Newsbin has no issue with larger downloads now. I'm not worried about my SSD going belly up anytime soon (it's only a year old). I have an image of my system and my data is backed up to my NAS. When I rebuild later this year, I'm considering getting a smaller separate SSD specifically for Newsbin install and data directory.

Just for info
=

A further slight improvement

My speeds are oddly inconsistent. It might be related to how many people are using the same cable line at the same time. I'll get consistent 950's for awhile then later 450 is the best I see with the same configuration.

That is odd indeed. Not sure but I guess you are using fibre(?) There should not be any real reason for such fluctuating speeds.
Assume you have subscription for using a specific speed/bandwidth and you should get what you are paying for.

My ISP offers 800Mbit/s up and down. Practically users are getting slightly more, some 4-5%.

The Ookla speedtests shows:



The 899Mbps shown in NB, not sure how this is calculated. Maybe it is the result from using multiple connections(?)
As said, the actual subscription is 800Mbps (plus 4-5%), hence Ookla usually shows 840-850 up and down.

FWIW - Update

I was getting 950's today too. I want to think it's because I removed an entire buffer copy during download but, who knows.