chrometweaks.org

I need to add files to my website through iPage, how?

Click Here To View All Answers...


First question I got is I need to add files to my website through iPage, how? Thanks for any comment. My other question... Ok I put the msnbot and msnbot/0.11 in spider txt but in the log files it shows the msnbot with the oscids please help..

Comments (58)

Good question... I dunno what is the answer to that question. I'll do some research in Google and get back to you if I bump into an answer. You should email the people at iPage as they probably know..

Comment #1

Hi,.

It wouldn't make any difference, because if you have both:.

Msnbot.

Msnbot/0.11.

In /catalog/includes/spiders.txt , the following code (in application_top.php), used to check:.

If (is_integer(strpos($user_agent, trim($spiders[$i])))) {.

Would result in true whether the "HTTP_USER_AGENT" was either:.

Msnbot (+http://search.msn.com/msnbot.htm).

OR.

Msnbot/0.11 (+http://search.msn.com/msnbot.htm).

I have tested this quite a bit this afternoon, and it is not an issue of values in 'spiders.txt', or anything else in osC (it seems), but something that "msnbot" is doing. For example, I forced the same user agent string that appears in my web logs like this:.

$user_agent = strtolower("msnbot/0.11 (+http://search.msn.com/msnbot.htm)");.

And the osC code resulted in:.

Spider - Yes.

Use Sessions - No.

Spider name - msnbot/0.11.

Peter..

Comment #2

The URLs were gethered by the bot before you got the spider session ID killer in place. It'll take the bot alittle while to realize that it can't get back to that URL again (with the same session id), and when it does, those URLs will drop off it's list of URLs to parse..

This will probably take a couple of weeks, maybe enven a couple of months...

Comment #3

If 'Force Cookie Usage' is set to TRUE, then SIDS are not assigned at all, spiders or no...

Comment #4

Hi,.

Yep, you can't have both, it's either one or the other..

Peter..

Comment #5

What about filtering the the yahoo bot:.

Yahoo! Slurp;.

Http://help.yahoo.com/help/us/ysearch/slurp.

Would putting.

Yahoo! Slurp.

In spiders.txt get it done ? .

Best...

Comment #6

'slurp' should already be there, and that's all you need...

Comment #7

Hi Chris,.

Thanks for that tip there, I will take a note of the cSid that 'msnbot' has used, and see if they use the same one on the next crawl..

Thanks,.

Peter..

Comment #8

Well, that's a good thought, but I think you'll find that the bot has been assigned several dozen, if not hundreds, and possibly even thousands of oscids...

Comment #9

Ok I put the msnbot and msnbot/0.11 in spider txt but in the log files it shows the msnbot with the oscids please help..

Comment #10

Hi,.

I'm having the same problem. Make sure that "FORCE COOKIE USAGE" is set to false; the Wiki said that had to be false, for the session id's to be turned off for spiders..

Peter..

Comment #11


This question was taken from a support group/message board and re-posted here so others can learn from it.