4 Simple Ways to Prevent Search Engines from Indexing Your WordPress Site
If you do not want your WordPress website to be listed or indexed in search engines, you must configure the settings of your site or control panel. The good news is there are four ways you can remove or prevent search engines from indexing your website. To know what these methods are, we will discuss each below!
How to Stop Search Engines from Indexing Your Site
What Does It Mean When Search Engines Index Web Pages?
Search engine indexing works similarly as an index in a library, listing information about the available books. For web pages, search engines index them, so they can include the pages’ content in the search results page.
When a search engine visits your pages and spots updates, it updates its index as well. Although this is beneficial for websites that want to rank in search engines, other website owners might want to get rid or prevent search engines indexing their websites. The reasons vary, such as:
- Test accounts – These are websites that are created for trial and testing purposes. Owners of these websites do not want them to be seen by the public.
- Restricted websites – These are sites made for an invite-only activity. They should not be listed in the search engine results page.
- Unfinished websites – Unfinished websites located in the staging areas are not yet ready for the public because they are still at the trial and error stage.
Now that you understand what search engine indexing is, let us move on to discuss the ways in discouraging search engines from indexing WordPress sites.
Disable Search Engine Visibility on Your Website
Before you configure the settings of your WordPress, check if search engines have indexed your web pages first. From your address bar, enter your website’s URL and remove https://. Replace it with site: so, it looks like this:
site:domain.com
Prevent Search Engine Visibility
If you see some pages of your website on the search engine page, those were already indexed. If you want these pages to disappear from the results page, go to Settings from your Dashboard menu and select Reading. Scroll down the page and tick Search Engine Visibility. Click Save Changes.
This will notify search engines not to index your pages.
Configure the WordPress robots.txt File in Your Control Panel
Although you can simply configure the settings in WordPress, some search engines do not listen to that, so you can ed the robots.txt file manually in your control panel. Log in to your control panel and go to File Manager under Files.
Go to your public_html directory and look for the robots.txt file. If you cannot see that in the directory, you can create one, which is extremely easy.
In your directory, click +File at the upper left side of your screen. Enter robots.txt as the New File Name and click Create New File.
You will then see the file in your directory.
Right click on the file and select Edit.
Paste this code on the edit page:
- User-agent: *
- Disallow: /
Click Save Changes at the upper right side of the screen.
You can check robots.txt through your browser as well. Type your website’s URL on the address bar and then enter a forward slash and robots.txt after it. It looks like this:
http://domain.com/robots.txt
You will then see the code you entered when you created the file in your public directory in your control panel.
If you want to disallow indexing your blog page, for example, you can configure the robots.txt file in your control panel. To do this, right click on the file in your directory.
Edit the code and add “blog” next to “/” so it looks like this:
Click Save Changes. This will stop Search Engines Indexing the pages you blocked.
Need fast, secure, and reliable hosting for your website? Innzone Hosting provides all of this and 24/7 world-class support. Check out our plans today.
Password-Protect Your Website
Once your password-protect your pages and files, search engines cannot access them. From your control panel, go to Directory Privacy under Files.
Select public_html.
Tick Password protect this directory and then fill in the details for the name of the protected directory down to the password. For the password, you can click the Password Generator button to quickly create a strong password.
The username and password are your login details to access the directory you are naming on this page. Click the two Save buttons once done.
Configure Google Console
If Google has indexed your pages, you can remove them from its results page. To do this, set up your website’s Google search console.
Go to the search console of your website and then to Legacy tools and reports and select Removals. Tick Temporarily hide button and enter the URL of the page you want to remove from Google.
Select Clear URL from cache and temporarily remove from search from the dropdown selection of Request type. Click Submit Request once done.
Conclusion
Preventing or getting rid of search engine indexing can be done by disabling it on WordPress, configuring the robots.txt file, password-protecting your WordPress website, or configuring Google console. Using either of the methods mentioned above is quite easy.
Follow our guide to prevent or stop search engines from indexing your website now to keep your pages from the public eye!