External search mechanism for forums ... Log Out | Topics | Search
Moderators | Register | Edit Profile

Australian RR Forums » General Discussion » External search mechanism for forums messsages - Does one exist? « Previous Next »

Author Message
Top of pagePrevious messageNext messageBottom of page Link to this message

Brian Vogel
Grand Master
Username: guyslp

Post Number: 1811
Registered: 6-2009
Posted on Friday, 01 January, 2016 - 03:10:   Edit PostDelete PostView Post/Check IP

Hello All,

Before I even get started, please do not take this message as a complaint, but just as a sincere query as to how to operate with "the way things are."

Others have mentioned that it appears that a lot of material in older posts on the forum "that we know are there" have become unfindable using the forum's own built-in search function. I am having that same difficulty now with material I am absolutely certain is out there in these forums. Sometimes I have been lucky enough in the past to be able to do a quick scan of a specific forum that I click in to from the Topics page and recognizing the thread I'm looking for. This time, no such luck.

I have posted in the past about the wonderful, but little known and used, site: operator, available on most search engines, to allow you to conduct a search for material limited to that site. What I have been unable to do, though, is find a site designation that works with search engines for actual posts on the forum. The site options I've tried have been:

au.rrforums.net
au.rrforums.net/messages
au.rrforums.net/messages/30
au.rrforums.net/cgi-bin/forum

and no matter which, the search comes up empty. And since my search criteria paired with that are +SU +fuel +pump there is no question that posts that contain all three of these terms in a single message do exist.

There has got to be a URL under which a search engine could search actual forums content since these pages are accessible to the public and we're still using (for the most part) good, old-fashioned plain text for messages.

If anyone has any idea of what URL would be appropriate for use with a web search engine in conjunction with the site: operator would you please share it? Doing a brute force visual scan for esoterica usually results in failure, and there are many times that the built-in search is not returning results that I'm positive existed (and probably still exist).

Brian
Top of pagePrevious messageNext messageBottom of page Link to this message

David Gore
Moderator
Username: david_gore

Post Number: 1870
Registered: 4-2003
Posted on Friday, 01 January, 2016 - 08:05:   Edit PostDelete PostView Post/Check IP

Brian,

This has been a problem ever since I joined the Forum and is largely an inherent problem with the current software. It is also possible [and I am only guessing here] there may be server settings that bar web crawlers from uploading details of forum posts which prevents the external searching you mention.

We are fortunate this forum has not had the spamming and attack problems experienced by other "open access" forums and this is the result of the diligence and experience of our Administrator who works in the IT industry. As far as I am concerned, the security of the forum and lack of hacking is the most important consideration and the limitations of the current Search function are tolerable despite the annoyance of not all results being returned - I suspect this may be a function of the large number of archived posts from previous years that may not be accessible to the software search engine but I cannot be certain this is actually the case. I do know our Administrator has included accessing the existing Forum archives which contain a wealth of information as an important factor to be considered in the on-going assessment of currently available forum software.
Top of pagePrevious messageNext messageBottom of page Link to this message

Geoff Wootton
Grand Master
Username: dounraey

Post Number: 1050
Registered: 5-2012
Posted on Friday, 01 January, 2016 - 08:19:   Edit PostDelete PostView Post/Check IP

Hi Brian

I too have tried to use the site: operator in the past, with no joy. Here's a thing: If you google a known phrase contained in the forum, without the site: operator, you get a message saying the way the site's robot.txt is set up prevents searching. For instance, if you use a phrase from this site's banner page, e.g. "A site provided by Australian enthusiasts for Rolls Royce", the google search engine responds with the following message:

RROC-Australia Discussion Forums - Australian RR Forum
au.rrforums.net/cgi-bin/forum/discus.pl
A description for this result is not available because of this site's robots.txt – learn more.


More information is given here:

https://support.google.com/webmasters/answer/6062608?hl=en&rd=1

Maybe David could persuade the administrators to open the site up a little to the web crawlers used by search engines.

It would be very useful if we could search the site using external search engines.

Just to re-iterate what Brian has already said, this is in no way meant as a complaint - we are all grateful for this facility and appreciate the effort that goes in to maintaining it.

Geoff
Top of pagePrevious messageNext messageBottom of page Link to this message

Geoff Wootton
Grand Master
Username: dounraey

Post Number: 1051
Registered: 5-2012
Posted on Friday, 01 January, 2016 - 08:21:   Edit PostDelete PostView Post/Check IP

Just to say, I was writing at the same time as David and had not seen his reply.

Geoff
Top of pagePrevious messageNext messageBottom of page Link to this message

Brian Vogel
Grand Master
Username: guyslp

Post Number: 1812
Registered: 6-2009
Posted on Friday, 01 January, 2016 - 08:52:   Edit PostDelete PostView Post/Check IP

This will be my last post on the matter, because I know that drawing out the discussion any further is very likely to be misconstrued.

For myself, the inability to search adequately has, on multiple occasions, prevented me from being able to direct someone (sometimes myself) who desperately needs it to information on the forums that I absolutely, positively, know exists, and that dates from somewhere in the range of my active participation on the forum. That is detrimental.

I had not happened to try what Geoff did to reveal that the site is apparently blocked from crawling by web search engines. I would respectfully ask that this decision be revisited.

Given that the forum is moderated, guest posters must have posts approved prior to posting, and I have to imagine a "ban user" function exists that should provide adequate protection from spamming.

I have had private correspondence on several occasions with the Administrator and I remain extremely, eternally grateful for his work and for the ongoing existence of this forum.

Brian
Top of pagePrevious messageNext messageBottom of page Link to this message

Bob Reynolds
Grand Master
Username: bobreynolds

Post Number: 355
Registered: 8-2012
Posted on Friday, 01 January, 2016 - 13:06:   Edit PostDelete PostView Post/Check IP

I am totally with Brian on this matter, and it is something I myself have previously complained about.

I have posted several 'reference' articles on this site that can no longer be found. I started a whole topic on the location of all the relays on the Silver Shadow, and could only recently find it by searching for an obscure phrase that I remember was on the page. Searching for 'Relays' will not find it, even though that word appears multiple times on the page, and is, in fact, the title of the topic!

I also posted a detailed explanation and photos of how the on-screen and off-screen wiper parking function works (the only explanation I've ever seen). God knows what has happened to that.

It is disheartening to spend time and dedication in providing information here for future reference purposes; only to have it disappear off the map in a relatively short time. I have come to the conclusion that there is no point in putting any information on this site for future reference, as it will just disappear. The site as it stands is really only good for discussion purposes, not for the storage of reference material.

We have been promised for years now, that the site is going to be updated with modern forum software, but it looks like we'll all be dead before that happens. How many more years is it going to take? The ironic excuse is that we don't want to lose the information on the current site; but that's exactly what is happening! And the longer we delay, the more information that is getting lost.

Happy New Year to all!
Top of pagePrevious messageNext messageBottom of page Link to this message

David Gore
Moderator
Username: david_gore

Post Number: 1872
Registered: 4-2003
Posted on Friday, 01 January, 2016 - 17:49:   Edit PostDelete PostView Post/Check IP

Bob,

Is this the relay topic you were looking for?

http://au.rrforums.net/forum/messages/17001/12646.html

If so, I found it using the Search function using the following terms:

Header: Search by Keyword
Search for: RELAY
Keyword options: OR
Match Method: PARTS OF WORDS
Match Case: CASE INSENSITIVE
Look In: SUBJECT LINES

Header: Search Options
Search Topic: SILVER SHADOW SERIES

Then hit the "Perform Search" button on the bottom of the page.

The following link might be the one you referred to in regard to wiper parking:

http://au.rrforums.net/forum/messages/17001/15158.html

I found this using the above search method with the following alteration:

Search for: WIPER

I received 17 topics and number 17 was the above link titled: "Silver Shadow Series : Wiper Parking on Shadow II (& Intermittent Function)"

If you search using the above method with INTERMITTENT as the key word, 3 responses including the above topic will be returned.

Not the easiest search facility known but practice helps locating past topics.
Top of pagePrevious messageNext messageBottom of page Link to this message

Geoff Wootton
Grand Master
Username: dounraey

Post Number: 1052
Registered: 5-2012
Posted on Friday, 01 January, 2016 - 18:37:   Edit PostDelete PostView Post/Check IP

David

You are implying the inadequacy in the search function is due to operator error. This is not the case. Searching "subject lines" instead of the default "text of messages" will work on occasions, but it does not change the fact there are problems with the search facility. Given the current state of play, I always check the subject headings first, by using ctrl/f on my browser. If I don't get a match, then it's fingers crossed I might find what I am looking for with the search facility provided. I think we just have to accept this, but Brian and Bob's comments are valid.

Geoff
Top of pagePrevious messageNext messageBottom of page Link to this message

RR Forums Administrator
Board Administrator
Username: admin

Post Number: 92
Registered: 10-2002
Posted on Friday, 01 January, 2016 - 19:49:   Edit PostDelete PostView Post/Check IP

Members should be under no illusion that the forum's software isn't antiquated and obsolete. Fifteen years ago when it was created the software was one of the best systems available; for several years now even its maker's web site has been defunct. I'm frequently struck by this forum's longevity in the face of that.

I have sporadically tried to find suitable replacement software that would permit wholesale migration but have so far failed in that quest. Early in 2015 I did manage after much trial-and-error to migrate the 20000+ posts to different software but abandoned that effort for various reasons:
* the images in posts would not migrate correctly or at all
* the search function will not find anything in the text of migrated posts
* I could not afford the enormous amount of time it was taking to endlessly retry the bulk migration process in search of success.

Given the justifiable concern about the existing search facility I'm relieved I didn't commit to a state where searching wasn't just poor but completely futile.

When we started the forum it was run on a small server with very limited bandwidth. By far the greatest consumers of that expensive bandwidth were search engines, so I set up a robots.txt file that prevented them crawling the site. I confess I haven't even thought about that file since it was created and believe that it's probably not been needed since the forum was moved to its current host. So I've simply removed the robots.txt file and notified Google; let's see if that improves things.

As to any future upgrade/migration I don't believe it's going to happen. All I can suggest is that we stick with this forum as it is or embark on a completely fresh move to new software (or another RR forum altogether!) whilst maintaining this forum in its final state solely for reference purposes. Fifteen years is a hell of a long time to be using the same software and I'm not too proud to call "Time, gentlemen." on this if it comes to it.
Top of pagePrevious messageNext messageBottom of page Link to this message

David Gore
Moderator
Username: david_gore

Post Number: 1873
Registered: 4-2003
Posted on Friday, 01 January, 2016 - 21:21:   Edit PostDelete PostView Post/Check IP

Geoff,

I am certainly not implying the search failure is due to "operator error" as you suggest but rather is the result of the apparent limit on the number of search results that will be displayed for each search using the current software.

From my experience, the search problem appears to be 50 individual outcomes is the limit of disclosure from a single search - if the search is not confined to a single Topic, this limit is often reached before the desired topic is reached and the software closes the search before this happens.

By using the Search Topic restriction as I did in the above examples, the probability of the desired thread/post appearing is much greater. However, this requires knowledge of the topic applicable to the thread/post concerned or a possibly tedious search of each individual topic to find the relevant thread/post.

Given the Administrator's input above, I will be extremely interested in seeing whether the ubiquitous search engines can find individual topics based on common words as the search term or whether more precise wording using less-common words is needed to find the desired thread/posts.
Top of pagePrevious messageNext messageBottom of page Link to this message

Bob Reynolds
Grand Master
Username: bobreynolds

Post Number: 356
Registered: 8-2012
Posted on Saturday, 02 January, 2016 - 01:01:   Edit PostDelete PostView Post/Check IP

I would like to thank the Administrator for responding to this topic, and for doing something practical by changing the search engine access.

But, in the long term, I would vote for another forum to be set up on new software, and this one left for archive and reference purposes. Now that it has been confirmed that migration will probably never be possible; the sooner we bite the bullet and start afresh, the better IMHO. I don't see any point in further delay.

It's not just the poor search facility, it's also the lack of a decent quote facility, the weird formatting functions, and other things.

Meanwhile, perhaps a post could be placed in a prominent position explaining the best way to use the Search facility? You can't expect newcomers to be able to find stuff if we can't even find our own posts which we know are there!
Top of pagePrevious messageNext messageBottom of page Link to this message

Brian Vogel
Grand Master
Username: guyslp

Post Number: 1820
Registered: 6-2009
Posted on Thursday, 21 January, 2016 - 09:26:   Edit PostDelete PostView Post/Check IP

It appears that we are a few steps closer to Google being able to crawl the forums, but not quite there. See the below for what I'm getting now:

Google Results 2016-01-20

There was not a trace of the forums on my prior tries, the last of which was a couple of weeks ago.

If it's possible to check whether there might be another "robots.txt" or the equivalent somewhere further down the tree that's preventing indexing of the actual messages I would appreciate that effort very much.

Brian
Top of pagePrevious messageNext messageBottom of page Link to this message

RR Forums Administrator
Board Administrator
Username: admin

Post Number: 93
Registered: 10-2002
Posted on Sunday, 24 January, 2016 - 12:06:   Edit PostDelete PostView Post/Check IP

Buried deep in the forum configuration is the option to set the individual web page META tags that instruct search engines whether they may index a page.

Having rediscovered those settings I have changed all META tags from 'NOINDEX,NOFOLLOW' to 'INDEX,FOLLOW' and rebuilt all the pages. Now search engines have open slather.

It has evidently worked, as overnight the number of pages indexed by Google jumped from 32 to 1224. Over the next week or so that number should climb until the entire site has been indexed.
Top of pagePrevious messageNext messageBottom of page Link to this message

Brian Vogel
Grand Master
Username: guyslp

Post Number: 1823
Registered: 6-2009
Posted on Sunday, 24 January, 2016 - 12:09:   Edit PostDelete PostView Post/Check IP

Bless you, Mr. RR Forums Administrator!!

Brian
Top of pagePrevious messageNext messageBottom of page Link to this message

Geoff Wootton
Grand Master
Username: dounraey

Post Number: 1069
Registered: 5-2012
Posted on Sunday, 24 January, 2016 - 13:58:   Edit PostDelete PostView Post/Check IP

Many thanks.

Add Your Message Here
Post:
Bold text Italics Underline Create a hyperlink Insert a clipart image

Username: Posting Information:
This is a public posting area. Enter your username and password if you have an account. Otherwise, enter your full name as your username and leave the password blank. Your e-mail address is optional.
Please quote Chassis Numbers for all vehicles mentioned.
Password:
E-mail:
Action: