Umbraco 4.7.1.1 using Safari insert image in tinymce (RTE)

Some of our new customers are using Safari, and they have stumbled upon some issues around using the Umbraco rich-text-editor (TinyMCE v3), they have problems with inserting images and macros.

We googled the problem a little, surely we couldent be the only Umbraco hosting company, which have end-clients using the Safari browser in the backend of Umbraco. But the top hits on google only revealed that there definitely was and is and long existing bug in Umbraco when it comes to the tinymce editor.

The specific problem our clients was facing was that when trying to insert an image or macro the editor just rendered out the text "mce_marker" in the spot where they where trying to insert the element.

We are currently running an Umbraco 4.7.1.1 install with a few custom patches that have yet to be implemented in the most resent versions of the Umbraco core + we are running MySQL as our db. So it's not that simple just to upgrade to the next release without testing, even though some forum entries state that the bug might be fixed in the most recent release, however I can not a resolved item in Umbraco issue tracker:
issues with mce_marker
Issue: can't paste into tinymce with Safari
Issue: can't insert youtube video in Umbraco (not sure if it applies to all browsers)
And of course the frequent dead link to the old issue tracker.

None of the issues was marked as resolved, so of to open the source and see where the underlying bug was happening. The bug was actually happening in the Safari plugin code for tinymce, but unfortunatly I'm not working on a Mac, so it was a little difficult to debug exactly why it was happening, but have a look at the file your-umbraco-site.com/umbacro_client/tinymce3/plugins/safari/editor_plugin_src.js - Line #116-117
both these lines use the "mce_marker" text as a reference to where the html that needs inserting should be placed in the tinymce html. And this is the only place in the entire Umbraco source that there exists a text called "mce_marker".

So how to fix it ?
I firstly checked if the bug was any where to be found on the TinyMCE forums and issue trackers, and I found this bug in tinymce that is a almost identical description of my clients bug. Basicly it states that the bug did exist and that actually the tinymce safari plugin had been removed more than a year before the last entry, so sometime around 2010-09-09 (unfortunately I could not find the version number of tinymce where the plugin was removed). But...

Hmmm, did Umbraco really not upgrade tinymce since 2010 (2 years ago), I looked into the tinymce implementation in umbraco and found that the file : 
your-umbraco-site.com/umbacro_client/tinymce3/tiny_mce_src.js - Line #6-10
contains the date 2011-03-10, so it seems the tinymce was updated after the Safari plugin was removed, but still the Safari plugin was included and loading in Umbraco, but properly it was no longer needed.

So I tacked down the code where the plugin is loaded in the file: \components\editorControls\tinyMCE3\TinyMCE.cs - Line #99-103
The code looks like this:

// safari compatibility
if (HttpContext.Current.Request.Browser.Browser.ToLower().Contains("safari"))
{
        _plugins += ",safari";
}


So to sum up there are two possible solutions to this bug, both are tested on a 4.7.1.1 install and seems to be working fine.

1. Remove the contents of the files (dont delete them it will cause loading errors) and remember to do a backup of the files first.
   your-umbraco-site.com/umbacro_client/tinymce3/plugins/safari/editor_plugin.js
   your-umbraco-site.com/umbacro_client/tinymce3/plugins/safari/editor_plugin_src.js
   This solution does not require any modifications to the umbraco source code and hence rebuilding of the core modules, and can be used as a "in-place" fix.
2. Uncomment line 99-103 in the file \components\editorControls\tinyMCE3\TinyMCE.cs and rebuild the umbraco.editorControls.dll, this will ensure that the plugin is not loaded.
   You can download a patch file here that can fix the changeset "umbraco_250b9089253a"

Hope it helps you :-) and as always just ping me if you need help with this

De-serialize to class with private or no empty constructor

When transferring DTO objects around in .NET sometimes you need to deserialize data to an object with no or private/internal empty constructors. What I have normally done is adding a public empty constructor to my own classes that just does nothing, but I always deemed it as an ugly hack, and have marked them all with TODO comments in the code...

Today I stumbled upon a this method in .NET FormatterServices.GetUninitializedObject that actually can instantiate a class without using any of the constructors in the class. The best part is that it works in all .NET versions, because we are targeting our DTO objects for .NET Framework 2.0 to ensure compatibility with all our clients.

I found these detailed guides about using the method:
http://www.rvenables.com/2009/08/instantiating-classes-with-internal-constructors/
http://mikeoff.blogspot.dk/2006/07/creating-object-instance-without.html

Fix : Sorting thousands of nodes in Umbraco

When drilling down into the Umbraco source code I found out that for each node in the sorting level, Umbraco uses .NET's XmlDocument.Clone() method for the entire contentxml cache file. This has a HUGH performance overhead and will cause the server to grind to a halt at some point, either when the number of nodes sorted or number of total nodes gets very high.

We have both :-(
300+ nodes needs sorting, and a total of 45000+ nodes in our umbraco install.

So I created a patch and sumbitted it to Umbraco see more here: http://umbraco.codeplex.com/workitem/30895

#### December 20 - 2012 Update ####
Since umbraco has shutdown/removed the workitems from codeplex, without any redirects or references to the new site, I have uploaded the patch here: download patch

Start writing productive code

I read this awesome post today (http://jhovgaard.net/how-i-stopped-writing-awesome-code) that really hits the nail on the head.

I have several times over the last 10 years experienced developers (my self included) falling into the trap, where new technologies is like a drug for developers "if it's new it should be used !!". I actually heard several times from our development team, that they thought we where falling behind in their skill level, because we where not implementing and using the new technologies and methodologies.

My best practice today is following the mantra "speed to market", by trying to deliver things as fast as possible and accepting bugs as a part of the development. Of course this can be frustrating at times, but whenever a bug is discovered our product is improved so it actually means that the number of bugs found equals a increased product value. And when a bug is found, that is system wide, we have built helpers to handle this. This means that we are not focusing on doing a lot of unit testing code for "what-if-scenarios", but instead evolving and improving our products instead of improving test-code.

We do however stumble upon cases where it would have been nice to be able to extend a base functionality without having to re-write 10 classes, but in real life it's not that hard to add additional functionality later to 10 classes, it just requires ten extra Ctrl-V commands.

And in my development history (15 years) whenever I tried to abstract functionality into a Interface structure, because I could foresee that down the line it could/would require changes. It actually always turned out that the changes required later was so extensive that a complete re-write would be faster and easier.

A good day for the european software industry

Functional parts of software can not be copyrighted..... a super good day for software developers in europe.

Umbraco Razor, DynamicMedia umbracoFile property empty

Some times a request for a DynamicMedia item can return an item where the property "umbracoFile" is empty/null.

dynamic mediaNode = new DynamicMedia(mediaNodeId);

var test2 = mediaNode.umbracoFile; // returning empty string

When searching for the problem on google it seems to be a problem only in version 4.7.1+ in umbraco.

As stated in theese two posts:

1. DynamicNode MediaById umbracoFile returning empty string
2. Razor Media Error on Upgrade to 4.7.1

It seems to be something with the way the media items are loaded in the dynamic node's because loading the same media node from xslt using the "umbraco.library.GetMedia" method does return the node with correct properties.

After some investigation into the issue I found out that it's actually 2 issues/bugs.

1. issue

This happens when using "<UseLegacyXmlSchema>true</UseLegacyXmlSchema>" in the umbraco 4.7.1+ (I'm using it due to our large installation, wich still uses a lot of old xslt files).

And the "umbraco.MacroEngines.ExamineBackedMedia.cs" file, it only expects the new content schema, and simply does not load any '<data alias="xx">' properties, but the code actually does load the node but not the child element values, when using the legacy schema in umbraco, it simple only accepts the schema style. 

2. issue 

The first issue only happens if the node is simply not found/missing in the Examine Indexes, this could properly happen in many ways, but in my case it was happening when cloneing the live DB to localhost for development, but it also sometines happens on live servers. (Still to investigate)

The solution(s)

To solve the first issue I modified the  "umbraco.MacroEngines.ExamineBackedMedia.cs" file so that if the item is not found in the examine index it will correctly load the node attributes and ' <data alias="xx"> ' nodes, see the workitem here (codeplex items have been removed) Instead the patch for Umbraco 4.7.1.1 is now available in my dropbox, download it here.

But this still triggers a database request, if the node is not the runtime cache (30 minutes timeout), so I still wanted to solve the second issue also, so that the Examine Indexs would be correct, to optimize our umbraco installation.

To solve the second issue I installed the Examine Dashboard package, and rebuilt the indexes manually, I had hoped that a xml-republish (http;//YOURDOMAIN/Umbraco/dialogs/republish.aspx?xml=true) would do the trick, but it apprently does not republish the media nodes and/or the Examine Indexes. I will properly investigate this later

Umbraco No Node Exists / cmscontentxml out of sync

For some time now we have had a problem when modifying document type alias' in umbraco (4.x.x), because it causes a complete flush of the XML caches used in umbraco. During these flushes of the caches if unexpected errors occurs, like DB connection lost, IIS timeout, AppPool flush or similar the cmscontentxml database table could become out-of-sync. This would lead to strange errors on the frontend of the sites in the installation:

• No node exists with id 'xxxx'
• Missing menu items
• Macros throwing out errors
• etc.

The problem has become almost a show-stopper for our umbraco installation as the number of nodes has increased, we are currently having 6000+ nodes and hosting 200+ websites on a single umbraco install. 

The errors are not showing up right after a change of the doctype alias, but after roughly 10-15 minutes, due to the thread updateing of the caches, and that umbraco will automaticly flush the cmscontentxml table to disk and memory caches if any of the two caches becomes invalid. Having "random" errors happening on 6000+ nodes is not acceptable.

So far we have "fixed" the issue by calling the http://myumbraco.com/Umbraco/dialogs/republish.aspx?xml=true which will force a full republish of all pages/nodes to the cmscontentxml table, and after that flush the disk and memory cache. The normal "right-click > republish" on the root node is only flushing the cmscontentxml table to disk and memory, but is not updating the database table, which is out-of-sync.

We have found several posts about the issue, but so far no good solution:

• our.umbraco.org/../cmsContentXml-out-of-synch 
• our.umbraco.org/.../Weird-content-cache-regeneration-issue 
• forum.umbraco.org/.../No-node-exists-with-id-...
• our.umbraco.org/.../Reasons-for-the-cmsContentXml-table-to-get-wiped

So we decided to come up with a proposal for a solution, that would ensure that regardless of any connectivity issues, or AppPool recycles, the cmscontentxml table would not get out of sync (this was the main reason for errors). Also modifying a DocumentType alias should NOT be a 10-15+ minutes wait-while-saving operation on large node umbraco installs.

Our solution modifies the DocumentType save event/method, by adding each node id that is using the DocumentType into the cmstask database table bycreating a new umbraco Task. These tasks are then loaded (one-by-one) using the umbraco.presentation.publishingService, and a flush of the database, disk  and memory cache for each node is done, and only after it is completed for each node the Task is deleted.

This should ensure that even a AppPool flush will just cause the publishingService to publish the node one more time.

A umbraco work item allready exists for this, see:

• http://umbraco.codeplex.com/workitem/30610
  (Moved to: http://issues.umbraco.org/issue/U4-493)

Also our patch is uploaded to it. (download patch)

####### December 20 - 2012 Update. #######
Due to the high number of requests, both here and from other sources, about this bug in Umbraco, I have created some additional release builds of the Umbraco Source with this patch applied, just click on the version you need the patched files for to download it. (both patch, dll's and source is available)
Umbraco Version 4.7.2
Umbraco Version 4.9.1
Umbraco Version 4.10.1
Umbraco Version 4.11.1

####### July 20 - 2013 Update. #######
Friday I received a distress request to help a company with a site that was down, and I spent my vacation-day-off helping them patching the Umbraco 4.11.10 version with this fix. I reviewed the code done by Shannon at Umbraco HQ, and although his code was a HUGH improvement, it was not a fix for this issue.
So here is the patch bundle for Umbraco, the dll's are built in release mode, but with pdb files.
Umbraco Version 4.11.10

####### August 2 - 2013 Update. #######
Be very careful about the latest 4.11.10 version of Umbraco and republish xml, there is a bug which will cause the sortOrder property on the nodes to be set to "0" in the output/cache XML. I'm currently testing a new patch and will release it after some testing.
see: http://issues.umbraco.org/issue/U4-2527#comment=67-8808
A quick google search on "umbraco sort order 0" reveals that it is a old bug, and is affecting many areas of Umbraco, media, content, basicly all objects that inherit from CMSNode