What If You Don't Want Your Pages To Be Crawled and Cached
?>
Download eBooks and Software
 Human Anatomy & Physiology Study Course (75% Comms
Converts At 9%! *hot Seller* Strong Niche & Few Affiliates To Compete With! Target Students, Educators, Practitioners, Trainers, Nurses, Paramedics All At One Site! Affiliates - Start Making Money With This Niche Now!
Gov-auctions.org - #1 Government & Seized Auto Auctions. Cars 95% Off!
Highest Paying Car Auction Site On Cb! #1 Trusted US Government, Police, Repos & Seized Vehicles Auction Site (incl. Real Estate). Make $33 Or $27 Real Net Profit At 75%. Genuine Product & Acclaimed Customer Service = Min Refunds. Incredible Conversions!
Surveyspaid.com
If You Promote Surveys, You Need To Give Our Site A Try- You Will Be Happy! We Use Geotargeting And Downsells. Converts Great With International Traffic. Email Followups With Your CB Id! Http://www.surveyspaid.com/affiliates.html
Articles > Site PromotionWhat If You Don't Want Your Pages To Be Crawled and Cached
by: Jerry Yu Some website owners have pages that they want to hide from general public. The pages meet the following criteria: - Only accessible by trusted users if they know page URLs.
- No links on the website that point to these pages.
- No username and password are required to gain access as long as you know page URLs.
Let's see this scenario: One day you created a page and you didn't put a link to it on your site. Then you told your family members about the page's URL. You thought nobody else would find it. You just made a mistake. Google and Yahoo would find your page if you or any family member ever visited websites with either Google toolbar PageRank enabled or Yahoo Companion Toolbar. PageRank function When you use Google toolbar with PageRank enabled, the toolbar automatically sends and records the page's URL you visited in Google's database. If a page URL is not found in Google's database, Googlebot - the robot of Google, will visit this page later to index it. Your surfing activities are tracked whether you use the toolbar to search the web or directly type a page's URL in Google search page. Google records your visits anyway. One day when you check what pages on your site have been indexed by Google, your hidden page comes up and you are worried. Furthermore, this page is cached. Even though you remove that page from your site, it can still be found and viewed from the cached version. How to check what pages have been indexed? Go to Google, type in "site:www.yoursite.com" without quotes. This query will list all the pages that have been indexed but it will only display up to 999 records as this is the limit set by Google for any queries. How to prevent your hidden pages to be indexed and cached? One simple but not sound solution is to disable PageRank function on the toolbar. To stop Google automatically track your surfing information, you can uncheck the PageRank checkbox to disable it. Steps to disable PageRank function: - Click Options button on the toolbar (you can see the word "Options" without quotes)
- In the pop-up window's Option tab, uncheck the PageRank checkbox.
See Google Toolbar Privacy Policy at http://toolbar.google.com/privacy.html for what information Google is collecting. Unfortunately, disable the PageRank function is not going to completely solve your problem because, in our example, your other family members could have PageRank enabled. A sound solution Your problem can be tackled by using meta robots html tag. The following two tags are what you need to use. Put the tag in the <head> section of your HTML documents. <meta name="robots" content="noindex,nofollow"> Search engines will read this page but will not index it and no links on this page will be traversed through to other pages. <meta name="robots" content="noarchive"> Search engines will not archive/cache the page content. How to remove an indexed and cached page If your page has already been indexed and cached, to remove from search engine databases, do this: 1. Add <meta name="robots" content="noindex,nofollow,noarchive"> to your page head tag section. Next time when Googlebot or other robots visit your page, your page will be removed from their index and cache. 2. Do what Google suggests. "If you believe your request is urgent and cannot wait until the next time Google crawls your site, use our automatic URL removal system. In order for this automated process to work, your webmaster must first insert the appropriate meta tags into the page's HTML code." (cited from Google web site Remove Content from Google's Index at http://www.google.com/remove.html) One last note. Is your page now 100% hidden? Not really. If you have outbound links on the hidden page and you click the links and navigate to other websites, your hidden page's URL will appear in other sites web traffic log as HTTP referer. You can remove outbound links from your hidden pages if that's suitable. More resources What If... Now you know how to safeguard any page on your site. What if you want to keep robots out from visiting all files in a directory? The answer is in my article Robots.txt And Search Engine Robots - http://www.WebActionGuide.com/kb/robots-txt.php About The Author
Jerry Yu is an experienced internet marketer and web developer. Visit his site http://www.WebActionGuide.com for FREE "how-to" step-by-step action guide, tips, knowledge base articles, and more. |
?>
News on Site Promotion
CreditQ.com Celebrates New Site: Begins Sweepstakes Promotion on CreditQ Social
CreditQ.com, an online financial services and resource website, recently launched a companion site called CreditQ Social. CreditQ Social operates as a combination social networking and financial information site, and is a place where companies, financial professionals, and consumers can collaborate, exchange ideas, and seek or provide financial products. As part of the roll-out, CreditQ has ... Anti-Aging Brand Announces Promotion to Reduce Age Lines on Stressed Out Facebook Investors
Anti-aging website is announcing new anti-aging products and reviews, to reduce age lines and wrinkles of the many stressed out investors of the Facebook IPO debacle.Houston, TX. (PRWEB) May 25, 2012 Anti-aging website, antiagingsupplementnews.org, is taking a new approach to the recent Facebook IPO issues that caused a frenzy among many investors, by announcing new product reviews for stressed ... Maple Grove Days offers business promotion opportunities
Maple Grove Days' "Showcase" silent auction offers one more way to join the community festival. The auction offers an inexpensive and relatively simple way for a business to support Maple Grove Days as well as promoting itself. NewHomesSection.com Anticipates Summer with Complimentary Advertising for Builders
NewHomesSection.com Anticipates Summer with Complimentary Advertising for Builders With the arrival of the summer sales season on the horizon, NewHomesSection.com is offering busy builders a completely free30-day trial of the site's extensive advertising and marketing services. About NewHomesSection.com Since 2007, NewHomesSection.com has helped shape the real estate revolution through its ... He got the promotion I wanted. How do I handle it?
Focus on what you can do to improve your performance and don’t dwell on negative thoughts about your colleague
|
