When building projects grouped in a solution, Visual Studio figures out the project dependencies and respect the build order. However, if you’re not using Visual Studio .sln solution files when building a set of projects with MSBuild, the build order is left to the author of the MSBuild script.
I’m creating a custom Build system, in order to convert complete BizTalk solutions from source code, to Windows Installer .msi scripts. This involves not only building the projects (compiling), but also putting relevant managed assemblies to the Global Assembly Cache, deploying assemblies and third-party references to a BizTalk Application, applying bindings and exporting the resulting BizTalk application to a .msi file… I choose not to use the well-known and excellent BizTalk Deployment Framework, because, I’m primarily interested in a build system, tailored to a local development box in order to boost productivity.
In our shop, deployment is handled with a custom PowerShell-based solution that makes extensive use of the PowerShell provider for BizTalk. But for local development, there definitely was a need for improving productivity. For instance, by enabling incremental build support for BizTalk Server 2010 .btproj project files, we were able to dramatically cut the build times.
One of the things that broke when using our custom build system was to be able to perform the opposite operations – that is, undeploy – the BizTalk Application. This involves figuring out the correct chain of dependencies betweeen assemblies and remove then in the appropriate order from BizTalk.
Fortunately, MSBuild is extensible and it is not really difficult to include such features to your build scripts.
In order to alter you build system to include dependency-checking and correct build (and reverse cleanup) order, you need to apply the solution in two stages:
- First, you need to gather project dependencies
- Second, you need to figure out the correct build order
Gathering Project Dependencies
Figuring project dependencies relies on the use of ‘Project References’ in order to express project dependencies. This is what uses Visual Studio when building the projects from the solution. In order to gather the depencencies betweeen several projects, Visual Studio uses the ResolveProjectReferences builtin MSBuild task.
<Target Name="_ComputeProjectReference" Inputs="%(Project.Identity)" Outputs="%(Project.Identity)__force_output__">
<MSBuild Projects="@(Project)" Targets="ResolveProjectReferences">
<Output TaskParameter="TargetOutputs" ItemName="ResolvedProjectReferences" />
</MSBuild>
<ItemGroup>
<ProjectReferenceItemSpec Include="%(ResolvedProjectReferences.OriginalProjectReferenceItemSpec)" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="@(Project)">
<ProjectReference>@(ProjectReferenceItemSpec)</ProjectReference>
</ProjectReference> </ItemGroup>
</Target>
Hint: scanning project dependencies actually triggers a build and takes time. This can sometimes be undesirable. In order to speed things up a litte and avoid redundant builds, you can persist the project dependencies in a temporary file, with the WriteLinesToFile MSBuild task, and read the cached information on subsequent builds with the ReadLinesFromFile task.
Figuring out the Correct Build Order
When all project references have been gathered, figuring out the correct build order is trivial. In fact, this subject has been beaten to death already, and is an application of Graph Theory. More specifically, project dependencies form a Directed Acyclic Graph. The action of ordering the projects for successful build is called a Topological Sort.
The following simple custom class includes a direct translation of the corresponding Wikipedia entry on the subject:
using System;
using System.Collections.Generic;
namespace System.Collections.Generic
{
public class Graph<T>
{
#region Constructors
private IList<T> nodes_ = new List<T>();
private IDictionary<Int32, List<Int32>> edges_ = new Dictionary<Int32, List<Int32>>();
#endregion
#region Operations
public void AddEdge(T item, T dependency)
{
int index_item = AddNode(item);
int index_dependency = AddNode(dependency);
edges_[index_item].Add(index_dependency);
}
public Int32 AddNode(T item)
{
if (!nodes_.Contains(item))
nodes_.Add(item);
int index = IndexOf(item);
if (!edges_.ContainsKey(index))
edges_.Add(index, new List<Int32>());
return index;
}
public T[] Sort()
{
IList<T> Nodes = new List<T>(nodes_);
IDictionary<Int32, List<Int32>> Edges = new Dictionary<Int32, List<Int32>>(edges_);
// L ← Empty list that will contain the sorted elements
// S ← Set of all nodes with no incoming edges
List<T> L = new List<T>(Nodes.Count);
List<Int32> S = new List<int>(Nodes.Count);
foreach (Int32 index in Edges.Keys)
if (Edges[index].Count == 0)
S.Add(index);
// while S is non-empty do
// remove a node n from S
// insert n into L
// for each node m with an edge e from n to m do
// remove edge e from the graph
// if m has no other incoming edges then
// insert m into S
while (S.Count != 0)
{
T node = Nodes[S[0]];
S.RemoveAt(0);
L.Add(node);
Int32 index_from = Nodes.IndexOf(node);
foreach (KeyValuePair<Int32, List<Int32>> item in Edges)
{
if (item.Value.Contains(index_from))
{
item.Value.Remove(index_from);
if (item.Value.Count == 0)
S.Add(item.Key);
}
}
}
// if graph has edges then
// output error message (graph has at least one cycle)
// else
// output message (proposed topologically sorted order: L)
foreach (List<Int32> item in Edges.Values)
if (item.Count != 0)
throw new ApplicationException("Circular dependency detected!");
return L.ToArray();
}
public T[] Sort(bool reverse)
{
T[] array = Sort();
if (reverse)
Array.Reverse(array);
return array;
}
#endregion
#region Implementation
private Int32 IndexOf(T item)
{
return nodes_.IndexOf(item);
}
#endregion
}
}
A Custom MSBuild Task for Topological Sort
The preceding class can be used in a custom MSBuild task in order to sort the corresponding projects in the order (or reverse order) of dependencies:
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;
namespace BizTalkFactory.Automation
{
public class SortDependentProject : Task
{
private bool most_dependent_ = false;
private ITaskItem[] outputs_ = new ITaskItem[]{};
[Required]
public ITaskItem[] Project { get; set; }
[Output]
public ITaskItem[] TargetOutputs
{
get
{
return outputs_;
}
set
{
outputs_ = value;
}
}
public Boolean MostDependentFirst
{
get { return most_dependent_; }
set { most_dependent_ = value; }
}
#region Overrides
public override bool Execute()
{
if (Project.Length == 0)
outputs_ = Project;
Graph<String> graph = new Graph<string>();
foreach (ITaskItem item in Project)
{
string dependencies = (string)item.GetMetadata("ProjectReference");
string[] references = Canonicalize(Path.GetDirectoryName(item.ItemSpec), dependencies);
foreach (string reference in references)
if (String.IsNullOrEmpty(reference))
graph.AddNode(item.ItemSpec);
else
graph.AddEdge(item.ItemSpec, reference);
}
List<ITaskItem> taskItems = new List<ITaskItem>(Project.Length);
foreach (String item in graph.Sort(most_dependent_))
{
TaskItem taskItem = new TaskItem();
taskItem.ItemSpec = item;
ITaskItem sourceItem = Project.Single(element => element.ItemSpec == taskItem.ItemSpec);
sourceItem.CopyMetadataTo(taskItem);
taskItems.Add(taskItem);
}
outputs_ = taskItems.ToArray();
return true;
}
#endregion
#region Implementation
private string[] Canonicalize(string relativeRoot, string path)
{
if (String.IsNullOrEmpty(path))
return new string[]{ path };
List<String> strings = new List<String>();
foreach (string item in path.Split(new char[] { ';' }))
{
if (Path.IsPathRooted(item) && File.Exists(item))
strings.Add(item);
string resolved = Path.GetFullPath(Path.Combine(relativeRoot, item));
if (File.Exists(resolved))
strings.Add(resolved);
}
return strings.ToArray();
}
#endregion
}
}
This custom task would be used simply in an MSBuild project file like so:
<!-- Calculate projects in order of dependencies -->
<MSBuild Projects="$(MSBuildProjectFile)" Targets="_ComputeProjectReference" />
<CustomTask.Automation.SortDependentProject
Project="@(ProjectReference)"
MostDependentFirst="$(Clean)">
<Output TaskParameter="TargetOutputs" ItemName="ProjectDependencies" />
</CustomTask.Automation.SortDependentProject>
<ItemGroup>
<Project Remove="@(Project)" />
<Project Include="@(ProjectDependencies)" />
</ItemGroup>
