Internet Search Questions

Technical support and discussion of Newsbin Version 6 series.

Internet Search Questions

Postby RangerXus » Fri Jul 22, 2016 7:29 pm

Quade or Dexter:

I'm considering trying out the Internet Search optional purchase (subscribing) and have a few questions specifically relate to Internet searches (not local):

1) Do you have a published privacy policy on the use of Internet Search?
- For example, do you store search requests? If so is there personally identifiable information logged with each search and how long is the information retained before purging?

2) How much information is returned to NewsBin for a search?
- For example, if a search has 10,000 hits do you only return a max of 2,000? And to get more of the search results using the Load Older/Newer buttons, does pressing them count as a new search against the monthly 500 or are they considered "free" and part of the original search which already counted as 1 search?
- If there is a max returned such as 2,000 out of 10,000 matches, what is considered a "hit"? For example, if a complete file consists of 1 par2, 5 pars, and 10 rars, is that file considered 1 hit returned or 16 hits returned?

3) Can you save some or all of the search results to a custom list and then use the custom list to determine what to download later?

4) Does the month run from subscription activation date for 90 days or is it calendar month starting the 1st to the 31st?
- Also, am I really restricted to 500 per 30 days or can I use the 1500 total anytime during the 3 months?

5) How does specifying a search group work?
- if I don't specify a group for a search does the search process against all groups you have available for search?
- Can I specify a specific group such as alt.binaries.whatever? And if so, do I use a.b.whatever or do I have to use the full name of the group?
- Can I create a custom group consisting of multiple groups and do a search on my custom group... will your system be able to restrict search to those groups?

6) Is your retention rolling or fixed to 2,400 days (or whatever it is for each group)?
- I.E. do you purge older posts from your search database or do you just add posts each day so that every day your search retention increases

7) Is there a way to determine my search usage? I.E is there a way to see how many searches I've used so far that period and how many are left?

8 ) Is there a list of the groups you index for searching and each groups retention?

9) Is it true you can search on subject or poster?

I know... a lot of questions, but there is not a lot of information on your Index Search and how it works.

Thanks
RangerXus
Occasional Contributor
Occasional Contributor
 
Posts: 39
Joined: Sun Mar 08, 2009 11:30 am

Registered Newsbin User since: 03/07/09

Re: Internet Search Questions

Postby Quade » Sat Jul 23, 2016 12:34 am

1 - No. We store nothing other than usage counts.

2 - 2000 hits per search, there's a button that lets you pull the next 2000 and the next. If you use a good search string there really shouldn't be hundreds of hits normally. Setting things like the age and size filters lets you focus more on what you're searching for.

3 - Yes. Or watch lists. Keep in mind that watch lists eat into your searches if you don't have unlimited, you might want to reduce the frequency to once a day.

4 - I'd have to check with dex.

5 - You create a "group of groups" as if you were grouping a bunch of related groups together using header downloads. Then if you pick this "groups of groups" when you search. It only returns results from this set of groups.

6 - I don't think we've deleted anything since we started. We have more retention than most news servers can reliably return files for.

7 - "Help/About" shows search count or unlimited if you're unlimited.

8 - I'd have to ask Dex. Only about 2000 or less groups actually have binary content. We don't do image groups at all. Pretty much any group under say 600K posts is probably just a spam trap.

9 - Yeah. Watch lists can watch for posters too.

You know you can test all this without a sub? You just can't queue files without a sub.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 45003
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Internet Search Questions

Postby zork » Fri Feb 03, 2023 6:14 pm

Quade wrote:8 - I'd have to ask Dex. Only about 2000 or less groups actually have binary content. We don't do image groups at all. Pretty much any group under say 600K posts is probably just a spam trap.


Old post but its a point I am interested in. I tried internet search with the alt.binaries.e-book.magazines group without any filters and got back only 310 results. Its not very high traffic group but it has posts very day and Newsbin shows that it has 51 million posts. Why then only 310 results?
zork
Occasional Contributor
Occasional Contributor
 
Posts: 28
Joined: Thu Jul 24, 2014 9:14 am

Registered Newsbin User since: 07/25/14

Re: Internet Search Questions

Postby Quade » Sat Feb 04, 2023 10:45 am

I did this a couple ways.

1 - I loaded the actual group so I can see directly what's posted.

2 - I right clicked the group and did a "Search in Groups"

3 - I entered "Magazine" in the main search tab.

At first I saw a bunch more results in the group then with search but, I noticed I had a "min size" filter set. Once I released the "Min Size Filter". I had the same results from search as from the group itself.

I will say the results seemed delayed. Like search was 6 hours behind the actual postings for this group. I'll try to find out why.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 45003
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Internet Search Questions

Postby zork » Sat Feb 04, 2023 1:09 pm

Seems that it was my mistake. I did not enter anything or I put just asterisk (*) there and expected to get all headers or all specified in the age filter but seems that it does not work that way. I get the latest headers added to the database and have to click on the Load older files into the list button.
zork
Occasional Contributor
Occasional Contributor
 
Posts: 28
Joined: Thu Jul 24, 2014 9:14 am

Registered Newsbin User since: 07/25/14

Re: Internet Search Questions

Postby Quade » Sat Feb 04, 2023 2:51 pm

"*" is limited to a short time span because it can return an overwhelming amount of results.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 45003
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Internet Search Questions

Postby dexter » Sat Feb 04, 2023 6:28 pm

RangerXus wrote:
4) Does the month run from subscription activation date for 90 days or is it calendar month starting the 1st to the 31st?
- Also, am I really restricted to 500 per 30 days or can I use the 1500 total anytime during the 3 months?


The subscription duration starts from the activation date and either renews or expires exactly 3 mos, 6 mos, or 1 year later depending on the subscription you choose.

For Standard subscription levels, search counts are reset at midnight after the last day of the month for everyone. So if you sign up on January 15, you have 500 searches to use until then end of the day January 31. On February 1 at 12:00:01 you will have 500 searches available until the next reset at the end of the month. Unused searches do not carry over month to month.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9514
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97

Re: Internet Search Questions

Postby dexter » Sat Feb 04, 2023 6:32 pm

RangerXus wrote:
8 ) Is there a list of the groups you index for searching and each groups retention?



No, there isn't a published list. If you download groups for your news server, filter in binaries groups, filter out picture groups, and sort on the Post Count, the first 2000 groups is pretty much the list of groups that we index. As Quade said, most everything else beyond that are low traffic, non-binary, or garbage groups.

That said, if there is a group you'd like us to index, just send a request through our support form and we'll consider it.
User avatar
dexter
Site Admin
Site Admin
 
Posts: 9514
Joined: Fri May 18, 2001 3:50 pm
Location: Northern Virginia, US

Registered Newsbin User since: 10/24/97


Return to V6 Technical Support

Who is online

Users browsing this forum: No registered users and 29 guests