1. Paul

    I actually thought this already happened, could this be Yoast adding the meta tag if they see the “Discourage search engines from indexing this site” option is enabled?


  2. Joshua Parker

    Without Yoast, the tag is added, and the content is ‘noindex,follow’.


  3. Ryan Hellyer

    I feel I am not the only one who thought this was already existing functionality.


  4. Peter Wilson

    Each of you are correct, there is some existing functionality to add a meta tag to private site. The tag added is `follow‘ />`

    However, the existing `robots.txt` rule prevents search engines from reading the meta tag so search engines don’t know they’re not supposed to index the site. This is like writing “to be opened by addressee only” inside an envelope.

    Removing the `robots.txt` rule in WordPress 5.3 will allow search engines to read the meta tag. The second directive of the meta tag has been changed to `nofollow` on private sites to prevent unnecessary crawling.


  5. Samuel Lavoie

    Finally! A lot of users thought the robots.txt rule was enough; it wasn’t. Small tweak but great news overall 🙂


Comments are closed.

%d bloggers like this: