DNN Forums

Ask questions about your website to get help learning DNN and help resolve issues.

Site Asset Folders - Non-Indexable on Search Engines

 3 Replies
 3 Subscribed to this topic
 37 Subscribed to this forum
Sort:
Author
Messages
New Around Here
Posts: 1
New Around Here

Hi everyone! Is there a way to make all of the materials in an existing folder of our Site Assets to be non-indexable on Google and other search engines? We have proprietary materials that are hosted on our site that are currently indexing on Google if you know how to search for them.

It's my understanding that you can add a new folder to Site Assets and categorize it as "Secure," but I haven't found a way to mark an existing "Standard" folder type to be "Secure." We'd like to avoid having to add a new folder and migrate our materials, which would require us to manually re-link thousands of links in our web library module.

Thanks for your help!

Veteran Member
Posts: 838
Veteran Member
3 Helpful Replier
Helpful Replier
Thanks for being such a helpful replier!
New Poster
New Poster
Congrats on posting!

I think there there are (at least) 2 ways:

1. you add the folder(s) to the robots.txt

<code>Disallow: /folder-url/*</code>

<code>2. You change the setting of the folder using an SQL update statement or by using SSMS</code>

Veteran Member
Posts: 1182
Veteran Member
MVP
MVP
You're an MVP!

To change a "normal" to a "secure" folder you have to create a secure folder and then move the documents there using the Resource Manager and not the Windows Explorer on the Server directly.

This causes two issues:

  1. Links to the document like <code>/Portals/0/documents/mysecret.pdf</code> will not work anymore. In a secure folder, only the file handler (LinkClick.aspx) can access and deliver the documents. The files themselves are renamed to (originalname).(originalextension).resources. A request to <code>/Portals/0/secretdocuments/mysecret.pdf.resources</code> will be blocked by IIS. Only a Url like <code>/LinkClick.aspx?fileticket=BRV-c_Vef5M%3d&tabid=12597&portalid=0&mid=4034&forcedownload=true</code> will deliver the file to the browser in it's original name and extension.
  2. Having said that: If you have a link to the document on your public website - even in this Url - Google will be able to access the folder except you apply permissions to that folder to specific usergroups. But you should also hide the link to the document from everyone else. And - as Tycho mentioned - use robots.txt. But there are some nasty search engines that ignore it...

Happy DNNing!
Michael

 

Michael Tobisch
DNN★MVP

DNN Connect
Senior Member
Posts: 1607
Senior Member
MVP
MVP
You're an MVP!
New Poster
New Poster
Congrats on posting!

I also think it's worth noting that telling any search engine to not index any content on your website is only a request. 

It's long been known (discovered) that if a search engine like Google can see the content, it will always index anything it sees, but only serve that index when it's not specifically removed via Robots.txt and/or in the Google Search Console.  I'm not sure if anyone outside of Google knows for sure, but it's assumed (hoped) that this helps to improve the search results in general, but also from your website specifically.  

These Forums are for the discussion of the open source CMS DNN platform and ecosystem.

For the benefit of the community and to protect the integrity of the ecosystem, please observe the following posting guidelines:

  1. If you have (suspected) security issues, please DO NOT post them in the forums but instead follow the official DNN security policy
  2. No Advertising. This includes the promotion of commercial and non-commercial products or services which are not directly related to DNN.
  3. No vendor trolling / poaching. If someone posts about a vendor issue, allow the vendor or other customers to respond. Any post that looks like trolling / poaching will be removed.
  4. Discussion or promotion of DNN Platform product releases under a different brand name are strictly prohibited.
  5. No Flaming or Trolling.
  6. No Profanity, Racism, or Prejudice.
  7. Site Moderators have the final word on approving / removing a thread or post or comment.
  8. English language posting only, please.

Would you like to help us?

Awesome! Simply post in the forums using the link below and we'll get you started.

Get Involved