https://upendoventures.com/ (owner Will Strohl is also DNN MVP) took over the suite of leading AD extensions from Glanton (https://upendoventures.co...tual-property) 

Feel free to reach out to him directly on [email protected] 

Awesome!! Thanks for all the info.

Another technique is to block specific search engine crawlers with a rule in IIS.
You could adapt this (very truncated) version that I use...

<rule name="Abuse User Agents Blocking" stopProcessing="true" enabled="true">
  <match url=".*" ignoreCase="false" />
  <conditions logicalGrouping="MatchAny">
    <add input="{HTTP_USER_AGENT}" pattern="^.*(1Noonbot|1on1searchBot|3D_SEARCH|3DE_SEARCH2).*$" />
  </conditions>
  <action type="CustomResponse" statusCode="403" statusReason="Forbidden" statusDescription="Forbidden" />
</rule>

 

Whenever you need to begin fine-tuning your SEO strategy like this, you'll want to use a multi-prong approach.  

• As mentioned, update your Robots.txt to exclude the search results you don't want to show.  In my own anecdotal experience, this won't prevent search engines from indexing that content, because I've been able to see it in the Google search console.  However, those results weren't showing in the public search results.  
  • Have a section for each known bot that you wish to continue to index your website, and also add the parameters to tell them when and how frequently to index the site.  
• Update all of the pages on your site to have appropriate page index levels and also be sure to mark pages a "no index" as appropriate.  Both of these are found in each page's respective page settings.  
• Using the Google search console, remove any search results you don't want to appear in public SERPs.  
• Ideally, your website should be behind a WAF, such as CloudFlare (which starts at free).  Using your respective firewall, directly block all bots that you don't want to index your website.  
  • Using the same method, we also block any countries that aren't meant to do business with the website, and turn on the managed firewall (paid plans, only).  
• As necessary, the suggestion to use the redirectRules suggested by James can be used, but this is a last resort, in my opinion. 

Posted By Will Strohl on 1/3/2024 6:45 PM

[...]

• As necessary, the suggestion to use the redirectRules suggested by James can be used, but this is a last resort, in my opinion. 

I find this is a very charming approach. I did that very often, not for search bots, but for other suspicious activities, e.g. known login Urls for other CMSs. Eg. the Url /wp-login.php is known for WordPress login. If a hacker gets a 404 ("Not found") response, he might thinks it it not WordPress, but if he gets a 403 response ("Forbidden"), he might think it is WordPress and tries several vulnerabilities concerning WordPress - and therefore looses time (of course there are other things needed as well, e.g. supressing the "Powered-by ASP.Net" header etc.).

Happy DNNing!
Michael

 

 

 

 

Oh wow! Thanks for all the info. This is one of the top reasons I was pushing to keep DNN as our CMS. Total Community help when needed.

@Will Strohl, We are getting ready to start testing our import from our payroll partner to Azure AD and once they get that all fixed they will be hounding on me to get the next version of CMS up and running with the ability to synch and/or import from Azure AD to create user logins. Tycho had mentioned that you took over the AD extension project. Will I be able to use that extension with Azure AD? We will be using a fresh new install of DNN when we get ready to start my end of the project.

Man Will, too much high quality data in such a short post!!! You're my hero man! I already copy/paste your post in my own personal knowledge library BTW :-)

Posted By Marco Alvarado on 1/7/2024 9:42 AM

Man Will, too much high quality data in such a short post!!! You're my hero man! I already copy/paste your post in my own personal knowledge library BTW :-)

Always my pleasure!  😊