Recomposing from DirectoryCatalog

Aug 5, 2010 at 11:22 PM

Folks - The program below works fine when I add new assemblies with classes that implement IMessageSender to the Extensions directory.  However, when I try to delete an assembly, or replace an assembly with a newer version, I cannot overwrite because the file is locked.  So what's the best practice around recomposing parts from a DirectoryCatalog?  Do I need to tell my host explicitly "I want to remove all Parts associated with assembly XYZ.dll" so that it releases them and allows me overwrite the assemblies?  The Recomposition chapter of the Programming Guide states "a new extension...might become unavailable for a variety of reasons".  I'm trying to understand how to best impliment this scenario.  I thought I would be able to just overwrite the assembly - drag/drop over the old and let the new get sucked up. =)

 

using System.ComponentModel.Composition;
using System.ComponentModel.Composition.Hosting;
using Contracts;

namespace MefHost
{
    class Program
    {
        [ImportMany( AllowRecomposition = true )]
        public IMessageSender[] MessageSenders { get; set; }

        public static void Main()
        {
            var p = new Program();
            p.Run();
        }

        public void Run()
        {
            var catalog = new DirectoryCatalog( "Extensions" );
            var container = new CompositionContainer( catalog );
            container.ComposeParts( this );
            int x = 0;
            while ( true )
            {
                x++;
                catalog.Refresh();
                System.Threading.Thread.Sleep( 5000 );
                foreach ( IMessageSender sender in MessageSenders )
                {
                    sender.Send( x.ToString() );
                }

            }

        }
        

    }
}


using System.ComponentModel.Composition;
using System;
using Contracts;

namespace FirstMessageSender
{
        [PartCreationPolicy( CreationPolicy.Shared )]
        [Export( typeof( IMessageSender ) )]
        public class FirstMessageSender : IMessageSender
        {

            private readonly Guid _guid = Guid.NewGuid();

            public void Send( string message )
            {
                Console.WriteLine( "first " + _guid + " " + message );
            }
            
        }   
}


Aug 6, 2010 at 7:12 AM
Edited Aug 6, 2010 at 7:12 AM

An assembly is locked when it's loaded by the runtime, unless, it's loaded through shadow copying. You should be aware that you can not unload an assembly from an AppDomain without shutting down that app domain, thus, all loaded assemblies are locked by default.

Shadow copying is very simple in practice, all it does is that it copies the assembly somewhere else before it loads the assembly. That assembly is still locked but you'll never see it, and you will be able to drop new ones into the app without having to worry about the assemblies being locked.

However, you won't be able to use the built-in DirectoryCatalog. You'll need to use a combination of the AssemblyCatalog and AggregateCatalog. (or maybe you actually could do that, because all you really need is to copy the assemblies somewhere else temporarily before you load them, and for that you could use a DirectoryCatalog) MSDN has more information about the specifics of this, I suggest you check it out.

Aug 6, 2010 at 10:57 PM

great suggestion about shadow copy - here's what I found after implementing it:  catalog.Refresh() detects a change in the folder (you have to delete the assembly, Refresh(), then copy over the new assembly and Refresh() again - you can't just copy/paste over), however when it recomposes it uses the old loaded assembly. The new assembly is ignored completely.  This is true if the assembly filename is renamed and also true if the assembly version/fileversion are changed.  I guess that makes sense - .net cannot load two of the same assembly.  How would the assembly have to differ for it to be considered it a new assembly?  I'm not sure how AssemblyCatalog nor AggregateCatalog would help?

So I guess the bigger question is: Is it possible to refresh a plug-in without bringing down the AppDomain?  This seems like a pretty simple/straight-forward scenario.  In my case, I have a windows service where the plug-ins are continually performing important tasks.  From time-to-time I add new plug-ins.  Also, from time-to-time I need to update just a single plugin.  In both cases, its important that none of the other plug-ins are bothered.  How could this be implemented?

 

Aug 6, 2010 at 11:01 PM

to add to my last comment - its important that these plug-ins live within the same service. The easy answer is just to create separate services because then the AppDomain can be brought down.  In my case, its a feature to have them all together.  Since they implement the same contract, I can ask the service important questions like "for all plugins, execute method SomeUsefulBusinessInformation() and report that out).  Meaning, the execution of the individual services is just as important as being able to query across them to ask common questions.

Aug 7, 2010 at 1:04 AM

changing the Assembly name in IL works great - assembly is treated as new and picked up upon Refresh().  I guess that's my answer unless you think there's a better way to do this?

Aug 10, 2010 at 4:52 PM

I see the problem, but I cannot think of an easy solution. If you cant forcefully load two assemblies with the same name then I guess you'll have to go with he renaming. Although, I have a hard time understanding the practical need of a service that's pluggable but cannot be restarted.

Aug 10, 2010 at 5:07 PM

Thanks!  The service can be restarted, but not during critical operation times.  Each plugin will support a graceful Dispose, but that doesn't mean that they should be disposed just because one plugin needs to be updated.  hope that made sense.

Aug 12, 2010 at 6:05 PM

This seems to be where a lot of us find ourselves after looking at MEF for a bit.  We want a way to be able to recompose imports by replacing an older version of an assembly with a newer one.  This allows you to make runtime updates to a system without forcing a reset of IIS etc.  You guys hit on some good points about not being able to do this due to the fact that the assembly is already loaded into memory etc, which eventually leads to either changing the assembly name (so you now have multiple copies in memory) or loading the pluggin assembly in a second AppDomain.  Both of these options arn't supported by the built in Directory Catalog unfortunately.  Also, MAF doesn't seem to have the kind of support we need to do this with MEF either.

Anyhow, maybe you guys would find this thread interesting: http://mef.codeplex.com/Thread/View.aspx?ThreadId=219771.  In my final post I show a way you can import a type that was composed in a secondary AppDomain.  This also supports recomposition.  Wrapping the solution with a directory polling service, you could probably do something like what you want.  There are some limitations to the solution though.

I've started looking into writing a RecomposableDirectoryCatalog that has parts which are created in a secondary AppDomain via CreateInstanceAndUnwrap, which can be added to the CompositionContainer in the Main AppDomain.  The unfortunate bit is that several of the key MEF classes ComposablePart uses are not marked as Serializable and they do not inherit from MarshalByRefObject, which makes it incredibly hard (if not impossible) to do this.  Ideally, I think you'd want to have a class inheriting from ComposablePart (or maybe a Catalog) in the main AppDomain, which hosted the secondary AppDomain.  Then you could create the ReflectionComposablePart via an AssemblyCatalog in the second AppDomain, and provide a way to pass the ExportDefinitions and ImportDefinitions into the main AppDomain's RecomposableDirectoryCatalog.  When the parts were created, you'd call CreateInstanceAndUnwrap on the second AppDomain instead of CreateInstance in the main AppDomain.  Vuala, you've got what your looking for...

The main catch is that ExportDefinition/ImportDefinition are not Serializable and do not inherit from MarshalByRefObject so they can't cross the AppDomain boundary.

-Nick

Aug 12, 2010 at 6:13 PM
An alternative approach is shadow copying though the approach you are proposing is cleaner because the app domain can gets recycled.

Sent from my IPad.

On Aug 12, 2010, at 10:08 AM, "1quicknick" <notifications@codeplex.com> wrote:

From: 1quicknick

This seems to be where a lot of us find ourselves after looking at MEF for a bit. We want a way to be able to recompose imports by replacing an older version of an assembly with a newer one. This allows you to make runtime updates to a system without forcing a reset of IIS etc. You guys hit on some good points about not being able to do this due to the fact that the assembly is already loaded into memory etc, which eventually leads to either changing the assembly name (so you now have multiple copies in memory) or loading the pluggin assembly in a second AppDomain. Both of these options arn't supported by the built in Directory Catalog unfortunately. Also, MAF doesn't seem to have the kind of support we need to do this with MEF either.

Anyhow, maybe you guys would find this thread interesting: http://mef.codeplex.com/Thread/View.aspx?ThreadId=219771. In my final post I show a way you can import a type that was composed in a secondary AppDomain. This also supports recomposition. Wrapping the solution with a directory polling service, you could probably do something like what you want. There are some limitations to the solution though.

I've started looking into writing a RecomposableDirectoryCatalog that has parts which are created in a secondary AppDomain via CreateInstanceAndUnwrap, which can be added to the CompositionContainer in the Main AppDomain. The unfortunate bit is that several of the key MEF classes ComposablePart uses are not marked as Serializable and they do not inherit from MarshalByRefObject, which makes it incredibly hard (if not impossible) to do this. Ideally, I think you'd want to have a class inheriting from ComposablePart (or maybe a Catalog) in the main AppDomain, which hosted the secondary AppDomain. Then you could create the ReflectionComposablePart via an AssemblyCatalog in the second AppDomain, and provide a way to pass the ExportDefinitions and ImportDefinitions into the main AppDomain's RecomposableDirectoryCatalog. When the parts were created, you'd call CreateInstanceAndUnwrap on the second AppDomain instead of CreateInstance in the main AppDomain. Vuala, you've got what your looking for...

The main catch is that ExportDefinition/ImportDefinition are not Serializable and do not inherit from MarshalByRefObject so they can't cross the AppDomain boundary.

-Nick

Aug 12, 2010 at 8:22 PM
Edited Aug 12, 2010 at 8:30 PM

I feel the urge to ask why we need recomposition in the first place. I'm not saying that it's not useful, but what business case do you support with recomposition?

My take on this is that you rarely need recomposition, except maybe for the composite Silverlight app loading XAP files by demand scenario. But other than that, I have yet seen the application which couldn't just be restarted.

Edit: Oh, yeah. If it's a critical service I would design such a service that it could be told to safely restart without interrupting critical operations. Long running operation should naturally support, start, pause, resume, and stop. Also, the likelihood that I would be forced to update the service right now, shouldn't be significantly bigger than the process just crashing...

Aug 12, 2010 at 8:48 PM

Recomposition means composition is not static, as new things show up those new implementations appear. This is not only related to implementations on disk / unloading....it's about the component set changing based on changes in the system. One great place this shines is in Silverlight with the DeploymentCatalog. An app starts up with a small footprint. As time progresses and the user selects different actions, new XAPs are downloaded dyynamically. As they show up, the app recomposes and the new functionality is immediately available.

Glenn