Why is my website not getting indexed?
The first thing which comes to your mind when you submit content on your website is, when will it be indexed? Sometimes it happens that you may be doing all right things, but you don’t find your website google. This is one of the most horrific nightmare of a webmaster.
So let us find out why on earth your content is not getting indexed and what could be the possible reasons?
1. Your content has not been discovered yet: Yes, if your website is quite new and it may not have been crawled yet. It may get crawled in the near future when Google attempts it next crawl of the entire web. So you have to be bit patient, because sometimes it may take a couple of weeks.
2. Robot.txt: It may be that robot.txt is blocking your website from being crawled. To check this type www.yourdomainname.com/robot.txt. It would bring up some text. Ideally, it should be something like below
The User- agent tells what a search engine is blocked and what part it is blocking it. For instance, in this robot.txt file it is blocking a page name wp-admin from getting crawled by any search engine spider or bot. So what ever i there in front of “Disallow:” is getting blocked. If you put “Disallow: /” you are actually telling search engines not to crawl your entire website. So you must be very careful when choosing what to allow and what to disallow. I could have advised not to use robot.txt at all but there are situation when you would not want some part of the website to be crawled hence it is important.
3. Meta tag is telling search engines not to index the website: Apart from the robot.txt file meta tag can be used to stop search engines from indexing your website. Check the meta description of your webpage by doing a right click on your webpage and then click on the option “view page source”. If you find anything like below this could be the reason that your web page is not getting indexed.
<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>
This meta tag tells the search engines not to index your content. If you find any such entry remove it asap.
4. .htaccess blocking your site from getting indexed: .htacess is a configuration file of apache web server. It quite important file which is loaded and executed by apache server software. It has the ability to block search engines from crawling and indexing sites. Look into the file and if you find below line this could the possible reason why your website is not getting indexed.
Header set X-Robots-Tag “noindex, nofollow”
5. Crawl error: It may be that Google has crawled your website, but there was some error while crawling and this could be the reason why your website was indexed. To find the error you need to login to the Google webmaster portal and go to “Crawl -> Crawl errors”. It gives quite a detail so as to what pages have the error and what are the reason of error. Go through this report regularly and fix them so that your site is indexed properly.
6. Very slow website: If your website takes ages to load, then this could be the reason crawlers are skipping crawling the website. Moreover, Google does not like slow website. So find out what could be the reason for slow website and fix it.
7. Too much of Downtime: If your website is down when Google bots are crawling, then it is fairly possible that your site will not be in the Google index. So chose your hosting provider wisely. Choose the one who can guarantee you good bandwidth so that your site is not slow and gives you the guarantee of maximum uptime.
8. WordPress privacy settings: If you are using WordPress and by any chance if you have turned on, your privacy setting it may still cause Google not to crawl your website. To check the status of privacy settings, go to Admin → Settings → Privacy
9. Duplicate content: If you are copying content from other pages you can’t ask Goggle to index the same content with different URL. So make sure you do not have duplicate content this will not only cause index issue, but will also get your website penalized.
10. Refurbished domain: Most of the time owners of new website get puzzled that they re doing all right things but google is not indexing the website. The cause may not be the one which you would like to hear. It is quite possible that the domain name which you have purchased may have been used earlier and it may have a website which had lots of spam links due to which Google has banned this website from being indexed. If this is the case and you want to continue with this website you will have to submit a reconsideration request to Google.
Hope this information is useful for people who have problems with indexing their sites. Apart from the reasons mentioned above, there could be several other reasons for site being not indexed. I have just tried to mention few common reasons. If you know any reason why a website would not index please share it here.
Also I would encourage you to use the forum on this website to ask question so that you get best possible answer.