DNN Forums

Ask questions about your website to get help learning DNN and help resolve issues.

Prevent Public Search Results

 17 Replies
 5 Subscribed to this topic
 40 Subscribed to this forum
Sort:
Page 1 of 212 > >>
Author
Messages
Growing Member
Posts: 31
Growing Member

I'm still on DNN 8.0.3 and we are looking to completely start anew by the end of 2024. But in the meantime we are getting various search results on commercial search engines such as Google, Bing and others that we would like to prevent from showing in the results. Is there a way to prevent pages and/or modules from being captured from these commercial search bots without breaking the 3rd party extension we use for internal searches ???

We use avt.SearchBoost for our internal searches.

Any help would be appreciated very much.

Thank You
Mike

Veteran Member
Posts: 1182
Veteran Member
MVP
MVP
You're an MVP!

You may restrict the view permissions of the page(s) and/or module(s) to specified users and/or roles (for "all" there is the "Registered Users" role). As a bot cannot login it can't index the content.

But then, your visitors have to register and log in to your portal.

Happy DNNing!
Michael

Michael Tobisch
DNN★MVP

DNN Connect
Growing Member
Posts: 31
Growing Member

Thanks Michael!

I will have to verify but I'm pretty sure as all the pages and modules are locked down to "Registered Users" since this is an Intranet site.

Veteran Member
Posts: 838
Veteran Member
3 Helpful Replier
Helpful Replier
Thanks for being such a helpful replier!
New Poster
New Poster
Congrats on posting!
My approach would be to block the crawlers in the robots.txt and allow for SearchBoost. Some examples on how to work with the parameters on: https://searchfacts.com/r...-allow-disallow-all/
Growing Member
Posts: 31
Growing Member
Thanks Tycho, I will check into this as well.

But let me ask you this. As this is a topic of discussion here in the office but who can guarantee that the search bots obey that robot.txt file. Just because I disallow them does it really prevent them from searching???
Growing Member
Posts: 31
Growing Member
I do have the robots.txt file and I am Disallowing ALL bots, but I am still seeing results on search sites.
Veteran Member
Posts: 838
Veteran Member
3 Helpful Replier
Helpful Replier
Thanks for being such a helpful replier!
New Poster
New Poster
Congrats on posting!
Well, there is no prosecution if they ignore the robots.txt. So, in theory they can search and/or display in the results. But I have not encountered any issues with this for the last say 10-15 years. If your goal is to keep a low profile with your 8.0.3 install, the robots.txt is not helping. There are other types of crawlers that help hackers identify legacy / vulnerable installs and yours is one of them. If the intranet is not connected to the internet directly (so, only via VPN or something like that), that is probably what have been keeping you safe so far. Waiting until the end of this year is really too long imho.
Growing Member
Posts: 31
Growing Member
Yeah I agree that waiting until the end of the year is a bit much, but in reality we have been working on migrating to Azure Active Directory and as far as I know the systems guys are still working on the kinks and stuff. Once they get that worked out then the company is thinking of going with Evoq if it can synch with Azure AD.
Veteran Member
Posts: 838
Veteran Member
3 Helpful Replier
Helpful Replier
Thanks for being such a helpful replier!
New Poster
New Poster
Congrats on posting!

That is old news. Evoq used to be the professional edition but about 5 years ago, the community took over the lead of the product roadmap. The platform edition is technically in better shape and there are many agencies that can provide enterprise support. If it comes to enterprise features for editors, you can suffice with a few extensions that will take care of structured content, workflows etc.

Good news: DNN Platform beats jusst about any other CMS when it comes to editor experience. Believe me I know::I have been working with over 20 CMSs and we're still up there.

Growing Member
Posts: 31
Growing Member
That sounds great! Our biggest hurdle would be to find a way to synch our users with Azure AD. Is there a vendor or extensions that could handle this?
Page 1 of 212 > >>

These Forums are for the discussion of the open source CMS DNN platform and ecosystem.

For the benefit of the community and to protect the integrity of the ecosystem, please observe the following posting guidelines:

  1. If you have (suspected) security issues, please DO NOT post them in the forums but instead follow the official DNN security policy
  2. No Advertising. This includes the promotion of commercial and non-commercial products or services which are not directly related to DNN.
  3. No vendor trolling / poaching. If someone posts about a vendor issue, allow the vendor or other customers to respond. Any post that looks like trolling / poaching will be removed.
  4. Discussion or promotion of DNN Platform product releases under a different brand name are strictly prohibited.
  5. No Flaming or Trolling.
  6. No Profanity, Racism, or Prejudice.
  7. Site Moderators have the final word on approving / removing a thread or post or comment.
  8. English language posting only, please.

Would you like to help us?

Awesome! Simply post in the forums using the link below and we'll get you started.

Get Involved