Creating Strongly-Typed C# Event Tracing for Windows (ETW) Assemblies with Visual Studio

The Event Tracing for Windows (ETW) Framework is a high-performance logging subsystem of Windows available to kernel-mode and user-mode developers for instrumenting mission critical and large scale applications.

Starting with Windows Vista, ETW is the foundation for instrumenting many components of Windows, including the Windows Event Log.

For this reason, ETW is primarily meant to be used from native code and the support tools and associated APIs are geared towards C++ developers. For a long time, there was no easy or straightforward way to instrument ones’s own managed application for ETW. However, things are starting to change.

The System.Diagnostics.Eventing namespace in the .Net 3.5 Framework has brought developers basic support for using ETW by wrapping core structures and APIs. With this support, it’s relatively easy to build a managed wrapper around logging and tracing. However, one still needs to declare and maintain the event structures themselves in an XML instrumentation manifest file. Surely, things will get even easier with and improved and streamlined support in the upcoming 4.5 version of the .Net framework.

If you ever wanted to add ETW support to your application, you might have stumbled upon this post where the author has made some efforts to bridge the gap between the native way and the .net way of using the tooling necessary to make use of ETW. And you might have walked away and reverted to using plain old vanilla Trace.WriteLine statements in your code… I know I nearly have !

Fear not ! In this post, I would like to walk you through a quick and easy way for using ETW in your managed application directly from Visual Studio. The solution involves using the proper tooling in an automated and integrated way so as to make it a very streamlined operation from the developer’s standpoint.

A Quick Recap

But, first, let’s recap what needs to be done for proper usage of ETW:

a) At design-time:

  • First, you need to declare and maintain the event and message structures in an Instrumentation Manifest file. This is cumbersome but, in fact, quite easy, thanks to the Manifest Generator (ecmangen.exe) authoring utility from the Windows SDK.
  • Second, you need to create a C# managed class for wrapping those event and message structures in your code. Again, this is straightforward, since the Message Compiler (mc.exe) tool includes a couple of switches to generated a C# source-code file for this purpose.
  • Third, you need to embed a binary representation of the Instrumentation Manifest, known as a Message File, as a Win32 resource in your assembly. This requires, yet again, the use of the Message Compiler (mc.exe) tool – in order to produce a Resource Script (.rc) file – followed by the Resource Compiler (rc.exe) tool in order to create a Win32 resource (.res) file. This, in turn, gets embedded in a managed assembly by specifying the resource file in the project properties and tweaking the compiler switches in order to allow the compilation of unsafe code.

In fact, these are a lot of steps, but, once you have created and authored the Instrumentation Manifest file, all of them can be automated. This is what I’m going to show you now.

Creating a Managed Wrapper Visual Studio Project

In essence, Visual Studio C# Project (.csproj) files are MSBuild-based XML files. This means that we can leverage the extensibility of MSBuild to perform the necessary steps by slightly modifying the project files. As an added bonus, such modified projects can also be built from the command-line or from a TFS build server.

Managed ETW Wrapper Project in Visual Studio

In my proposal, a developer creates a regular C# project and includes the Instrumentation Manifest XML file. Then, the developer needs to perform a one-time modification to the project, that consists in changing the build action associated with the manifest to InstrumentationManifest and add an import statement, at the end of the project file, referring to an external .targets MSBuild file in which all the required logic will be centralized.

In practice, here how it looks like, in Visual Studio:

...
  <Import= Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
  <Import= Project="$(MSBuildExtensionsPath)\Eventing\System.Diagnostics.Eventing.targets" />
</Project>

In fact, adding an import statement at the end of the file is the only manual modification required, for it will point to an external MSBuild .targets file where the custom InstrumentationManifest Build Action is registered. So, associating the Instrumentation Manifest to the appropriate Build Action can be done within Visual Studio when the project file is reloaded.

Once that’s done, the project behaves just like any regular C# project. It produces a managed assembly. It understands the Build, ReBuild and Clean actions. It even takes advantage of incremental compilation, in order to only perform the necessary actions when required. Finally, it can be referenced from other projects without any side effects. For all intents and purposes, it is a regular C# project file.

A Custom MSBuild Project for Supporting ETW from Managed Code

Here, I will show the structure and content of a custom MSBuild .targets Project that can be imported into any regular C# project file in order to include support for compiling ETW Instrumentation Manifest files into a strongly-typed C# managed wrapper:

<Project InitialTargets="_InitialTarget" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<ItemGroup>
    <AvailableItemName Include="InstrumentationManifest" />
</ItemGroup>

 <!-- Compiling ETW Instrumentation Manifest files requires Windows SDKtools -->
 <PropertyGroup>
   <WindowsSdkDir Condition="'$(WindowsSdkDir)'=='' ">C:\ProgramFiles (x86)\Microsoft SDKs\Windows\v7.0A\</WindowsSdkDir>
   <RootNamespace Condition="'$(RootNamespace)'=='' ">$(AssemblyName)</RootNamespace>
 </PropertyGroup>

 <!-- _InitialTarget -->
 <!--
 
       Normalizes the $(WindowsSdkDir) variable and retrieves the path tovarious SDK tools and dependent files.
       
       Inputs:
       =======
       (none)
       
       Outputs:
       ========
       $(Win32MessageCompiler) : path to the Message Compiler (mc.exe) SDKutility
       $(Win32ResourceCompiler) : path to the Resource Compiler (rc.exe) SDKutility
       $(Win32InstrumentationMetadata) : path to the core ETW metadatadefinitions (winmeta.xml)
       
 -->
 <Target Name="_InitialTarget">
   <PropertyGroup>
     <WindowsSdkDir Condition="!HasTrailingSlash('$(WindowsSdkDir)') ">$(WindowsSdkDir)\</WindowsSdkDir>
     <Win32MessageCompiler Condition="Exists('$(WindowsSdkDir)bin\mc.exe') ">$(WindowsSdkDir)bin\mc.exe</Win32MessageCompiler>
     <Win32ResourceCompiler Condition="Exists('$(WindowsSdkDir)bin\rc.exe') ">$(WindowsSdkDir)bin\rc.exe</Win32ResourceCompiler>
     <Win32InstrumentationMetadata Condition="Exists('$(WindowsSdkDir)include\winmeta.xml') ">$(WindowsSdkDir)include\winmeta.xml</Win32InstrumentationMetadata>
   </PropertyGroup>
 </Target>

First, some bookkeeping. Compiling ETW Intrumentation Manifests requires tools from the Windows SDK, so references to the various tools are established into a set of corresponding properties. The _InitialTarget is run first, in order to setup those properties before any other processing.

 <!-- _ComputeIntrumentationManifestResource -->
 <!--
 
       Calculates the paths (location) to the resulting .res resource file andassociated strongly-typed .cs file.

       Input ItemGroup:
       ================
       @(InstrumentationManifest) : path to the XML Instrumentation Manifest(.man) file
       
       Output ItemGroups:
       ==================
       @(InstrumentationManifestResource) : path to the resulting compiledwin32 resource (.res) file
       @(EventProviderSource) : path to the resulting strongly-typed managedwrapper class (.cs) file
 
 -->
 <Target Name="_ComputeInstrumentationManifestResource">
   <ItemGroup>
     <InstrumentationManifestResource Remove="@(InstrumentationManifestResource)" />
     <InstrumentationManifestResource Include="@(InstrumentationManifest->'$(IntermediateOutputPath)res\%(Filename).res')" />
   </ItemGroup>
   <ItemGroup>
     <EventProviderSource Remove="@(EventProviderSource)" />
     <EventProviderSource Include="@(InstrumentationManifest->'%(RootDir)%(Directory)%(Filename).cs')" />
   </ItemGroup>
 </Target>
  
 <!-- _CreateInstrumentationManifestResource -->
 <!--
 
       Compiles the resulting .res resource file.
       The win32 resource (.res) file is located in an intermediate outputpath.
       Leverages incremental compilation (i.e. only performs compilation ifrequired.)

       Inputs:
       ================
       @(InstrumentationManifest) : path to the XML Instrumentation Manifest(.man) file
       
       Outputs
       ==================
       @(InstrumentationManifestResource) : path to the resulting compiledwin32 resource (.res) file
 
 -->
 <Target Name="_CreateInstrumentationManifestResource" Inputs="@(InstrumentationManifest)" Outputs="@(InstrumentationManifest->'$(IntermediateOutputPath)res\%(Filename).res')">
   <MakeDir Condition="!Exists('$(IntermediateOutputPath)res') " Directories="$(IntermediateOutputPath)res" />
   <Exec Command="&quot;$(Win32MessageCompiler)&quot; -W &quot;$(Win32InstrumentationMetadata)&quot; &quot;@(InstrumentationManifest)&quot; -h &quot;$(IntermediateOutputPath)res&quot; -r &quot;$(IntermediateOutputPath)res&quot;" />
   <Exec Command="&quot;$(Win32ResourceCompiler)&quot; -nologo -r &quot;@(InstrumentationManifest->'$(IntermediateOutputPath)res\%(Filename).res')&quot;" />
 </Target>

 <!-- _CreateEventProviderSourceFile -->
 <!--
 
       Generates the resulting strongly-typed managed EventProvider wrapper.
       The strongly-typed C# managed wrapper class (.cs) file is located at the root of the project.
       Leverages incremental compilation (i.e. only performs generation if required.)

       Inputs:
       ================
       @(InstrumentationManifest) : pathto the XML Instrumentation Manifest (.man) file
       
       Outputs
       ==================
       @(EventProviderSource) : path to the resulting strongly-typed managed wrapper class (.cs) file
 
 -->
 <Target Name="_CreateEventProviderSourceFile" Inputs="@(InstrumentationManifest)" Outputs="@(InstrumentationManifest->'%(RootDir)%(Directory)%(Filename).cs')">
   <Exec Command="&quot;$(Win32MessageCompiler)&quot; -cs $(RootNamespace) &quot;@(InstrumentationManifest)&quot; -r &quot;$(IntermediateOutputPath)res&quot;" />
 </Target>

Then, three simple targets, each one associated with a single responsibility.

The _ComputeInstrumentationManifestResource target creates some meaningful ItemGroup created as part of the processing below. This simplifies writing subsequent targets. As you see, the resulting Win32 resource (.res) file is created under a res subdirectory of the intermediate output path and has the same base file name as the Instrumentation Manifest. Likewise, the EventProvider source file has the same base file name as the Manifest and is created in the project root.

The _CreateInstrumentationManifestResource target runs the Message Compiler (mc.exe) and Resource Compiler (rc.exe) tools in sequence, in order to create a binary representation of the manifest as a Win32 resource (.res) file.

The _CreateEventProviderSourceFile target runs the Message Compiler (mc.exe) tool in order to create a strongly-types C# managed wrapper class for an ETW EventProvider object.

Both targets take advantage of MSBuild support for incremental compilation. By comparing the timestamps of the generated artifacts relative to that of the Instrumentation Manifest, the processing only happens when necessary. Those targets are specified as dependencies of the BeforeCompile target, shown below, in order to trigger the actual custom processing.

 <!-- BeforeCompile -->
 <!--
 
       Overrides the BeforeCompile MSBuild extension point in order to generated files and alter compiler flags.
 
 -->
 <Target Name="BeforeCompile" DependsOnTargets="_ComputeInstrumentationManifestResource;_CreateInstrumentationManifestResource;_CreateEventProviderSourceFile">
   <Message Text="Createstrongly-typed ETW assembly from specified instrumentation manifest..." />
   <PropertyGroup>
     <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
     <Win32Resource>@(InstrumentationManifestResource->'%(Identity)')</Win32Resource>
   </PropertyGroup>
   <ItemGroup>
     <Compile Include="@(EventProviderSource->'%(Identity)')" />
   </ItemGroup>
   <Message Text="@(Compile)" />
 </Target>

The MSBuild BeforeCompile target is then overridden and performs the following actions:

  • Its dependencies trigger the custom generation of the C# EventProvider source file and compilation of the win32 binary resource, shown above.
  • The @(Compile) ItemGroup is modified to include the newly generated EventProvider source file.
  • The /unsafe compiler switches is turned on, in order to allow the compilation of unsafe code.
  • The /win32res compiler flag is specified with the location of the generated binary Win32 resource Message File.
 <!-- AfterClean -->
 <!--
 
       Overrides the AfterClean MSBuild extension point in order to clean generated files.
 
 -->
 <Target Name="AfterClean" DependsOnTargets="_ComputeInstrumentationManifestResource">
   <RemoveDir Condition="Exists('@(InstrumentationManifestResource->'%(RootDir)%(Directory)')') " Directories="@(InstrumentationManifestResource->'%(RootDir)%(Directory)')" />
   <Delete Condition="Exists('@(EventProviderSource)') " Files="@(EventProviderSource)"/>
 </Target>

Finally, in order to support the Clean action, the MSBuild AfterClean target is overridden to delete the generated artifacts.

Conclusion

In this post, I have focused on providing an automated and integrated way of using ETW from the perspective of a Visual Studio developer.

In order to produce ETW traces from one’s application, it is necessary to register the managed Event Provider to the system at install-time (using wevtutil.exe) and collect and analyse the resulting traces at run-time (using logman.exe, svctraceview.exe, tracerpt.exe, xperf.exe, etc.) Please, refer to the articles I linked to, as well as this excellent ETW FAQ, for more information…

About these ads
This entry was posted in ETW and tagged , , , . Bookmark the permalink.

9 Responses to Creating Strongly-Typed C# Event Tracing for Windows (ETW) Assemblies with Visual Studio

  1. Patrick Wellink says:

    Nice,
    Have you ever tried Ntrace ?
    It is a codeplex thingie that does exactly what you are doing. It’s made ETW very easy and out of the box usable….

    • Hi Patrick, no in fact I have not. I stumbled on references to it during my research but discarded it (perhaps too quickly) because I did not want to incur a dependency. Since this subject is quite new to me, I’ll investigate and reconsider.

      Thanks for the tip.

  2. Pingback: Friday Five–September 7, 2012 | UpSearchSQL

  3. Pingback: Friday Five–September 7, 2012 | UpSearchBI

  4. Pingback: Friday Five–September 7, 2012 | MSDN Blogs

  5. You may be interested to know in version 4.5 of .NET Runtime a new class called ‘System.Diagnostics.Tracing.EventSource’ has been added that can output ETW events (with a manifest). It is VERY easy to use (you don’t write the manifest it does), and produces fully manifested events. See my blog at
    http://blogs.msdn.com/b/vancem/archive/2012/07/09/logging-your-own-etw-events-in-c-system-diagnostics-tracing-eventsource.aspx and http://blogs.msdn.com/b/vancem/archive/tags/eventsource/ for more information.
    in particualr my blog has a ZIP file for a stand-alone version of the type that you can use on V3.5 or V4.0 runtimes. THus you don’t have to take the dependency on a V4.5 runtime being installed.

    Vance Morrison: Peformance architect, .NET Runtime.

    • Hi Vance,

      Thanks for your comment. The new EventSource si what I alluded to in the opening paragraph of this post, but I did not yet have the opportunity to try it out.

      I’ll grab the standalone version of the type and try this to see whether it allows me to write code that is source-compatible with the now launched .net 4.5 Framework.

      Cheers.

  6. Paul Lyon says:

    Where are you creating the InstrumentationManifest custom build action? You mention it being in the external .targets file, but I don’t see that defined anywhere.

    • Hi Paul,

      I have updated the post to include this missing piece of information. You can notice the new AvailableItemName property at the beginning of the external .targets file.

      Cheers.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s