Good question... I dunno what is the answer to your question. I'll do some investigation and get back to you if I bump into an useful answer. You should email the people at iPage as they probably could give you an answer..
Where do I find my web logs on a ipower hosted server ?
Ignore my posts above, I was talking cr*p..
After checking my weblogs, all of the cookie_usage hits were from Googlebot and as a result of it trying to follow the buy_it_now link. I'm going to try the fix noted earlier on in this thread, and will report back with the results..
If you don't know where to find your weblogs, contact your host. Or try the support section of their website, it should tell you where to find them (if they are available). Though if you have ftp access, you might be able to find them on your own..
I tried this on another thread, perhaps someone here has an answer. My system does not create a session for googlebot, yet the bot keeps trying to "buy" products and then it hits the cookie usage page..
Here's some sample log entries (some are legitimate traffic).
You can see that googlebot does not have a session. Any idea how to prevent it from following the "action=buy_now&products_id=xyz" link?.
I have disallowed the SE from these pages. I only want them to index my products. Not pages that have user interaction like create_account or reviews..
This is what I have for robots.txt.
# Robots.txt for http://www.destoffenstunter.nl.
# Email: xxxx@xxxx.
#place this file in your root directory.
#Replace all / with your shop directory name.
#Remove the # to activate additional blocks.
#Block All Images.
#To block main page due to size from bots.
# Disallow: /index.php.
#To shut down iPage site completely from bots use either.
# Disallow: /.
# Disallow: /.
#If you had search engine friendly URLs on and want to clean them out of the bots.
#Block out directories bots don't need to go to.
#Block out things that are secure or login oriented.
#Optional Pages to block.
I think I managed to get rid of the robots attempting to "buy" products and then hitting the cookie_usage.php page. With the help of "stevel" I noticed that there are two kinds of "buy" in OSC: "product_info.php" uses a FORM action which is not causing any problem with robots, but "product_listing.php", the "product_reviews.php", "product_reviews_info.php", "products_new.php", "all_products.php", etc. are all using a URL action to add items to the cart..
The best way to avoid robots from flowing the URL and then hitting the cookie usage page is to turn all the links into a FORM. There is a contribution out there that someone mentioned in this thread "buy_now_link_to_button_v1.2". Install it and reduce useless page requests from robots by up to 4 times..
Hi, I have a shared SSL server, and when a robot clicks on a link that leads to a secure page, it will go to a different address, but that address is actually my iPage site still. Say the regular address is server.com and my secure page is hostxx.server.com/username.
Is it possible to block pages in my robots.txt file that use this different url? Can I say disallow hostxx.server.com/username/page.php?.
For example, I want to block login.php. I can put disallow: /login.php, but the link actually goes to hostxx.server.com/username/login.php for the Shared ssl. So it looks like the robot is not ignoring the login.php, because it changes to a.
, instead of going to.
Is there a way to block this or should it block login.php no matter what?.
I don't think you can say disallow:.
As robots text can not have actual hardlinks to ignore, or is that possible as well?..
What I don't understand, is why a spider is being directed to the cookie_usage page. It looks like this is happening in application_top, which redirects if there isn't an active session..
Surely it should not redirect if it's a spider?.
OK, now I do understand why this is happening..
If a spider comes along and it's not in the spiders.txt list, it will end up being directed to cookie_usage.php..
So, the simple solution is to look through your weblogs and identify the user agent on all requests for cookie_usage.php..
You should find that the majority, if not all, are spiders / robots..
Then simply add pattern matching for them to your spiders.txt file..
How can I stop this? Any ideas?.
All help/suggesions appreciated!.
Sorry brain fart ,set prevent spider sessions to true..
Thanks for reply - prevent spider session is set to true already. None of the pages get a session id, I also have set cookies set to true..
I think it's when the spider follows a link to buy or something like that and since it cant be able to set a cookie it comes up against the cookie usage page..
Incidentally I've just been searching for help on this on google and have come across literally 100's of people using OSC's cookie usage pages..
There must be a way around this problem?.
Sorry to bump this but does anyone have a solution? I don't really want my cookie usage page in google and sure others dont too?.
I am having exactly the same problem. Looking in my User Tracking cotrtibution I see googlebot is accessing the cookie usage page at least 100 times in 1 hour, not trying to move to other pages, and then randomally move to index other pages and return back to cookie usage. Here is an example:.
Why not just add this to the robots.txt file.
Or, be really cheeky and use it as part of your SEO campaign - I have the same problem and Google is by far the worst offendor (I currently have 6 bloddy googlebots in my store and it is killing my server, but what do you do....ban them, have no traffic, allow them, have slow site......??) but I have configured my cookie_usage.php page to be crammed with all of my keywords and target phrases. As I know the spiders will visit it regularly, it will have a high ranking in their listings for pages within my site, so why not use it......Just an idea!..
Please post your cookie_usage.php so others can see what you mean..
$Id: cookie_usage.php,v 1.1 2003/03/10 23:32:20 hpdl Exp $.
osCommerce, Open Source E-Commerce Solutions.
Copyright (c) 2003 osCommerce.
Released under the GNU General Public License.
Define('NAVBAR_TITLE', 'Cookie Usage');.
Define('HEADING_TITLE', 'Cookie Usage');.
Define('TEXT_INFORMATION', 'We have detected that your browser does not support cookies, or has set cookies to be disabled.<br><br>To continue shopping online, we encourage you to enable cookies on your browser.<br><br>For <b>Internet Explorer</b> browsers, please follow these instructions:<br><ol><li>Click on the Tools menubar, and select Internet Options</li><li>Select the Security tab, and reset the security level to Medium</li></ol>We have taken this measurement of security for your benefit, and apologize upfront if any inconveniences are caused.<br><br>Please contact the store owner if you have any questions relating to this requirement, or to continue purchasing products offline.<p><b>JEWELLERY</b><p>.
Niknakgroup.com are one of the UKs leading online jewellery and gifts retailers. The jewellery range is very varied and caters for the jewellery and gift needs of all fashion conscious people..
<a href="jewellery/silver"><b>Sterling Silver Jewellery</a>.
Jewellery, jewelry, jewellry, jewelery....no matter how you spell it, weve got loads of it!! With a collection of over 5000 silver jewellery items you should be spoilt for choice, whether it's silver jewellery for yourself or silver jewellery as a gift for someone special. The jewellery range includes a wide array of earrings, pendants, bracelets, rings and lots more in wide and varied themes. The styles vary from highly contemporary and fashionable silver jewellery to simple silver jewellery designs and so caters for all fashion conscious women and men alike..
<a href="jewellery/gold"><b>Gold Jewellery</a>.
An ever expanding gold jewellery range in both 9 and 18 carat gold. Our gold jewellery is a very high standard of design and good value for money. The niknakgroup.com gold jewellery range now boasts over 2500 jewellery items and with new jewellery additions arriving constantly, our gold jewellery range easily rivals anything else seen on the Web..
<a href="jewellery/Body+Jewellery"><b>Body Jewellery / Belly Bars</a>.
Niknakgroup.com now offers a very wide and extensive collection of body jewellery and belly bars. The body jewellery ranges now include: steel body jewellery, titanium body jewellery, 9ct gold body jewellery, 14ct gold body jewellery, 18ct gold body jewellery, ultra violet body jewellery, and sterling silver body jewellery. From navel bars to flesh plugs, the body jewellery range caters for all your body jewellery and body piercing needs. All of our body jewellery is sourced from the worlds largest manufacture of body jewellery, and as with all of our items is covered by a comprehensive guarantee and excellent product knowledge. We have over 1000 designs in countless variations and are absolutely certain that not only dow e carry one of the largest product ranges available anywhere, but that we also have extremely competetive pricing and have prices you will find extremely difficult to beat - either on the high street or on the internet..
<b>Other Designer Jewellery - Designer Jewellery.
New to the iPage website is a stunning collection of handmade contemporary designer jewellery. Specialising in sterling silver mixed with 9ct gold - it provides a truly breath-taking collection of designer jewellery. Mixing some of these styles with diamonds, semi-precious stones and even stones such as slate or granite really adds to the uniqueness of this special jewellery collection..
<b>Silver & Diamonds Jewellery - Diamond Jewellery.
Our silver and diamonds range keeps getting bigger!! The full collection includes our Diamonds with Love ranges, D for Diamonds ranges just for children, the Kit Heath White Diamonds ranges (adults and children) and a great diamond range for weddings and bridesmaids. They all include spectacular jewellery designs in a multitude of silver finishes and stone settings to complement the quality diamonds. all the ranges are supplied in excellent gift boxes to make the diamond jewellery just a little bit more special..
<b>Kit Heath Jewellery.
Niknakgroup is one of a very few places you will find the ENTIRE Kit Heath collection. The Kit Heath jewellery range is a vibrant collection of contemporary jewellery in a variety of styles and finishes, and one of the most serached for designers of jewellery on the internet worldwide. There are the diamond collections, kids jewellery and of course their contemporary jewellery ranges, all of which can be found on this website..
The mens jewellery collections have gradually grown and now boasts a range of over 300 silver jewellery items just for men. From chunky bracelets to cufflinks to the very popular dog-tag pendants, the jewellery range caters for every man’s silver jewellery needs..
<b>Childrens Jewellery - Kids Jewellery - Baby Jewellery.
Due to the over-whelming success of of childrens jewellery, we have now added lots more kids jewellery items including, earrings, studs, pendants, charm pendants, charm bracelets and lots more. There is also diamond jewellery collections for children from the D for Diamond range and by Kit Heath also..
<b>Stainless Steel & Titanium Rings - Urban Jewellery.
There has been a recent trend towards steel and titanium jewellery, and niknakgroup.com has now intoduced a stunning stainless steel jewellery collection inspired by Kit Heath Jewellery. There are trendy design steel rings, steel earrings and steel pendants. The titanium collection consists of a range of titanium rings............but what makes our range so different is that though we do sell plain titanium bands, we also sell titanium rings inset with silver, or gold, or diamonds or a combination of them all. The jewellery collection here is a must see!.
<b>Wedding Day Jewellery Range - Bridesmaids Jewellery.
Every year we have been asked for more wedding day jewellery, more bridesmaids jewellery, more jewellery gifts. So now we have introduced a whole collection dedicated just to weddings. The jewellery range includes earrings, rings, pendants, necklaces, tiaras and bracelets. All set in a variety of silver finishes, variety of semi-precious stone settings, diamond jewellery and lots more. Elegant and neutral in colouring, this range of exquisite jewellery is sure to be a best seller, and will complement every wedding colour scheme with delicate pearls and diamonds set in sterling silver and gold. The designers of our wedding jewellery collection have kept in mind that whilst you want your wedding day gems to be beautiful and stunning, they should not outshine the beauty of the bride and bridesmaids! We have also added a great collection of wedding day jewellery for the Groom and Best Man..
Niknakgroup.com has now expanded into the Pewter jewellery range, and is proud to announce our collection of over 500 high quality jewellery designs in pewter with styles from Gothic and Wicca, including pentagrams, cannabis leafs and other pagan symbolism, through to skater wear and highly contemporary well priced pewter necklaces. Our collection of pewter jewellery contains Celtic and Mystic pieces along with the jewellery designs for todays more youthful customers..
Having highlighted the Celtic designs as a firm favourite of our customers, niknakgroup.com have actively searched for quality suppliers who can add more designs to our extensive Celtic Jewellery range. The ancient artforms and symbolism of the Celts has been diligently replicated in our designers pieces, and we are absolutely certain that you will find your perfect Celtic piece right here. Available in gold, silver and pewter, our celtic designs feature diamonds, amethyst, topaz, cubic zirconia and many more gems and our celtic rings have been highlighted as one of the best collections on the internet..
<b>Rennie Mackintosh Jewellery.
This architect from the turn of the 20th Century dedicated his professional life to functional and elegant design. Whilst Charles Rennie Mackintosh himself concentrated on buildings and furniture, the stylings of his work have been faithfully translated into exceptional jewellery designs. Our Rennie Mackintosh jewellery range brings the very best styles from many British designers who have all tried with great success to put Rennie Mackintoshs stylistic designs into high quality jewellery pieces in sterling silver.</font>');.
Define('BOX_INFORMATION_HEADING', 'Cookie Privacy and Security');.
Define('BOX_INFORMATION', 'Cookies must be enabled to purchase online on this store to embrace privacy and security related issues regarding your visit to this site.<br><br>By enabling cookie support on your browser, the communication between you and this iPage site is strengthened to be certain it is you who are making transactions on your own behalf, and to prevent leakage of your privacy information.');.
See it at www.niknakgroup.com/jewellery/cookie_usage.php - the customer would still see the information they need, but the spider has lost of new links and text to chew on!..
Not sure, but I think the.
"buy now" links to forms v1.2.
Contrib would help..
Does this only apply if you have cookies enabled?..
I have had cookies turned off in my iPage site since Christmas, but they still keep getting the page; I have tried replicating what they are doing myself, and cant get the page to show up no matter what I do.......oh well, just another of those little mysteries.......
As far as I know it only happens if the sessions and cookies both don't work. And that is the case for the spiders (if you've enabled "Prevent Spider Sessions" and the spider is in your spiderlist)..
And it does not happen to you (or your customers) because you are not in the spiderlist like the spiders, so for you the sessions work as they should..
This post has been edited by.
: 08 September 2004, 07:30..
I don't have any robots.txt file, but I do have spiders.txt in my /catalog/includes.
Is that the same?..
I have "Prevent Spider Sessions" set to True, and Force Cookie Use set to False...
No Problem Mikey!.
If you need anymore help just ask!.