Umbraco No Node Exists / cmscontentxml out of sync

For some time now we have had a problem when modifying document type alias' in umbraco (4.x.x), because it causes a complete flush of the XML caches used in umbraco. During these flushes of the caches if unexpected errors occurs, like DB connection lost, IIS timeout, AppPool flush or similar the cmscontentxml database table could become out-of-sync. This would lead to strange errors on the frontend of the sites in the installation:

• No node exists with id 'xxxx'
• Missing menu items
• Macros throwing out errors
• etc.

The problem has become almost a show-stopper for our umbraco installation as the number of nodes has increased, we are currently having 6000+ nodes and hosting 200+ websites on a single umbraco install. 

The errors are not showing up right after a change of the doctype alias, but after roughly 10-15 minutes, due to the thread updateing of the caches, and that umbraco will automaticly flush the cmscontentxml table to disk and memory caches if any of the two caches becomes invalid. Having "random" errors happening on 6000+ nodes is not acceptable.

So far we have "fixed" the issue by calling the http://myumbraco.com/Umbraco/dialogs/republish.aspx?xml=true which will force a full republish of all pages/nodes to the cmscontentxml table, and after that flush the disk and memory cache. The normal "right-click > republish" on the root node is only flushing the cmscontentxml table to disk and memory, but is not updating the database table, which is out-of-sync.

We have found several posts about the issue, but so far no good solution:

• our.umbraco.org/../cmsContentXml-out-of-synch 
• our.umbraco.org/.../Weird-content-cache-regeneration-issue 
• forum.umbraco.org/.../No-node-exists-with-id-...
• our.umbraco.org/.../Reasons-for-the-cmsContentXml-table-to-get-wiped

So we decided to come up with a proposal for a solution, that would ensure that regardless of any connectivity issues, or AppPool recycles, the cmscontentxml table would not get out of sync (this was the main reason for errors). Also modifying a DocumentType alias should NOT be a 10-15+ minutes wait-while-saving operation on large node umbraco installs.

Our solution modifies the DocumentType save event/method, by adding each node id that is using the DocumentType into the cmstask database table bycreating a new umbraco Task. These tasks are then loaded (one-by-one) using the umbraco.presentation.publishingService, and a flush of the database, disk  and memory cache for each node is done, and only after it is completed for each node the Task is deleted.

This should ensure that even a AppPool flush will just cause the publishingService to publish the node one more time.

A umbraco work item allready exists for this, see:

• http://umbraco.codeplex.com/workitem/30610
  (Moved to: http://issues.umbraco.org/issue/U4-493)

Also our patch is uploaded to it. (download patch)

####### December 20 - 2012 Update. #######
Due to the high number of requests, both here and from other sources, about this bug in Umbraco, I have created some additional release builds of the Umbraco Source with this patch applied, just click on the version you need the patched files for to download it. (both patch, dll's and source is available)
Umbraco Version 4.7.2
Umbraco Version 4.9.1
Umbraco Version 4.10.1
Umbraco Version 4.11.1

####### July 20 - 2013 Update. #######
Friday I received a distress request to help a company with a site that was down, and I spent my vacation-day-off helping them patching the Umbraco 4.11.10 version with this fix. I reviewed the code done by Shannon at Umbraco HQ, and although his code was a HUGH improvement, it was not a fix for this issue.
So here is the patch bundle for Umbraco, the dll's are built in release mode, but with pdb files.
Umbraco Version 4.11.10

####### August 2 - 2013 Update. #######
Be very careful about the latest 4.11.10 version of Umbraco and republish xml, there is a bug which will cause the sortOrder property on the nodes to be set to "0" in the output/cache XML. I'm currently testing a new patch and will release it after some testing.
see: http://issues.umbraco.org/issue/U4-2527#comment=67-8808
A quick google search on "umbraco sort order 0" reveals that it is a old bug, and is affecting many areas of Umbraco, media, content, basicly all objects that inherit from CMSNode

Enabling correct URL's for umbraco when using hostnames on level 3+

I have in my company been running a single Umbraco installation with multiple websites for about 2½ year, each site has it's own domain, but share similar macros, and many times also functionality. But lately we have been running into issues with customers who would like to have multilingual sites under separate domain names.

Simple, right... just create a "folder" node, but each site inside and that should be it....!! But ups there is an old bug (missing feature) in Umbraco that will render out wrong URL's on both the backend and the frontend when using the Node.NiceUrl or umbraco.library.NiceUrl(NodeID) methods. The bug is that the Umbraco NiceUrl method only will filter out the nodes on level one (internally in the code called level 2, because actually the root is level 1).

Example, if your node structure looks like this:
root
+Node1 (hostname: exampleA.com)
++Node1.1
+Node2
++Node2.2 (hostname: exampleB.com)
+++Node2.3

Umbraco will construct the NiceUrl of:
Node1.1 as "/Node1.1.aspx" (correct url)
Node2.2 as "/" (correct url)
Node2.3 as "/Node2.2/Node2.3.aspx" (wrong url) should be "/Node2.3.aspx"

So to fix this/add this feature I have modified the Umbraco version 4.7.1.1 of the umbraco.presentation.library.cs
The patch is uploaded here: umbraco.codeplex.com/Download/...323577 (moved to: issues.umbraco.org/../U4-515)
The Umbraco Work Item is here: http://umbraco.codeplex.com/workitem/30662 (moved to: issues.umbraco.org/../U4-515)

Umbraco 4.7.1.1 macro parameter bug

Using the same xslt macro twice on pages causes invalid attribute values.

The bug in Umbraco 4.7.1.1 causes the parameters of following requests to load invalid or wrong parameters. The bug was discovered in the process of upgrading from version 4.5.2, and it took some time to find due to the change in loading macros in Umbraco.

How to re-produce the bug:
Load two or more macros on the same page, adding some attributes to the macro in Umbraco. If the first macro does not use all the properties (default empty values) the second macro usage will cache and set the attributes on the following requests.

This will cause rendering of invalid content....!!!

In my case it was a menu macro with an attribute defining which node to use as start node for rendering, if not set the xslt macro would use the site's rootNode/defaultNode as the start node. But this bug caused the macro to load a wrong content tree as the menu :-(

The fix is to change/update the UpdateMacroModel method code to the folowing, in the umbraco.presentation.macro.cs file:

public void UpdateMacroModel(Hashtable attributes)
        {
            foreach (MacroPropertyModel mp in Model.Properties)
            {
    if (attributes.ContainsKey(mp.Key.ToLower()))
    {
     mp.Value = attributes[mp.Key.ToLower()].ToString();
    }
    else
    {
     mp.Value = string.Empty;
    }
            }
        }

I have submitted a post here: http://our.umbraco.org/forum/developers/xslt/21851-All-XSLT-files-suddenly-giving-error-parsing-XSLT-file?p=2#comment101528

And added a diff file to umbraco's codeplex site: http://umbraco.codeplex.com/workitem/30657

Fix: Bug in .NET Roles / RoleProvider / MembershipProvider initialize

Due to a bug in the System.Web.Security.Roles class in .NET 2.0, which is caused by a singleton instance of the last exception that occurred during the Initialize of the RoleProvider, any exception's that occur during Initialize method of the RoleProvider, will be caught and the only way is to recycle the app pool in IIS.

I stumbled upon this bug due to a connectivity issue with our database, that caused a small connection lost to the database, the resulting error on the front end of our website was a database error (which was stored in the static exception inside the Roles class).

It took a little time to track this down and fix it, but the fix is to do a try catch block inside the Initialize method, preferable only around initialize DB code, and your own static object to monitor if the initialize went perfect, then in all the overridden methods do a check if the initialization went fine, and else initialize the DB again.

Update - 2012-08-13:
I sadly stumbled upon the bug in the MembershipProvider also, so to every who stumbled upon the same problem here is a example draft code that can fix the error.

public sealed class MyCustomMembershipProvider : MembershipProvider
{
 private static object _syncLock = new object();
 private static bool _isDBInitializedSuccessfully = false;

 public override void Initialize(string name, NameValueCollection config)
 {
  if (config == null)
  {
   throw new ArgumentNullException("config");
  }
  
  // .... Add your code to validate the config settings, if needed
  
  // Initialize the abstract base class.
  base.Initialize(name, config);

  // .... Add your code to initialize the class, if needed

  // Initialize the DB
  InitializeDB();
 }

 public override bool ChangePassword(string email, string oldPassword, string newPassword)
 {
  // Call InitializeDB() in all methods that require DB access in the class
  InitializeDB();
  // .... Add your code that will change the password
 }

 private static void InitializeDB()
 {
  if (_isDBInitializedSuccessfully) { return; }
  lock (_syncLock)
  {
   if (_isDBInitializedSuccessfully) { return; }
   try
   {
    // .... Add your code to check if the DB is loaded
    _isDBInitializedSuccessfully = true;
   }
   catch (Exception e)
   {
    // .... Add your code to log the Exception's
    e = null;
   }
  }
 }
}

And the code that generates the error:

namespace System.Web.Security
{
 //...
 public static class Membership
 {
  //...
  private static void Initialize()
  {
   if (s_Initialized)
   {
    if (s_InitializeException != null) // static Exception, so only a app-pool reset will fix it
     throw s_InitializeException;
    return;
   }
   if (s_InitializeException != null) throw s_InitializeException;
   //...
  }
  //...
  private static Exception s_InitializeException = null;
  //...
 }
}

Hope this helps you :-)

Fix: Visual Studio 2010 SP1 cannot open file

Today I ran into a really strange bug with my new PC (DELL Vostro, windows 7 Ultimate 64 bit, intel-i5)  and a fresh install of Visual Studio 2010, it cost me 10½ hours of searching and testing different solutions from many sources:
http://www.ninjacoding.net/archive/2010/01/17/visual-studio-hangs-when-opening-certain-files.aspx
http://social.msdn.microsoft.com/Forums/en-US/vstsdb/thread/f76ebf64-4cd3-4cac-a970-40a23e7171d3/
Some of them might help others, but they did not help me !!!

I had originally thought that it was connected only to .sql files, but quickly found out that actually all files was affected, and Visual Studio would enter a freeze/un-responsive state and sometimes throwing a random exception.

The solution that helped me was found by pure chance when checking the reset settings of visual studio, I stumbled upon a setting in: Tools>Options>Environment>General>"Use hardware graphics acceleration when available" this was checked/enabled.... and I noticed that actually when I created or opened a file i VS it was there but just not showing, unless I re-sized the application, and when typing this was also not shown until the window was re-painted. so un-checking this setting, and all problems where gone.

Hope that this can save others from spending time on a stupid bug.