A Happy TPE Story

I'm a big believer in BizTalk's Business Activity Monitoring (BAM) functionality, in fact I can't think of any "real" BizTalk application that wouldn't benefit from using it.

I generally use the BAM API along with the excellent Typed BAM Class Generator as a means of persisting business data and timings (aka milestones) to the BAM database. However, last week I had a simple message-only (no orchestrations) process to instrument and I thought "why not give the TPE another chance"?

The TPE, or Tracking Profile Editor, provides a graphical means of identifying the business data and milestones that are to be tracked. Previously, I had lost confidence with the TPE approach because I found it impossible to instrument what was a complex business process requiring multiple request / response ports. At the time, pressure was on for delivery and so I fell back using the BAM API. However, I didn't give up on the TPE process entirely, since it does have some advantages over the BAM API.

My new process was a simple data extract from an Oracle database. This data is transformed to a flat file format as it is sent to a remote sFTP location. The trigger for the data selection from Oracle is the receipt of a particular file via the File adpater.

The first step when "BAMing" a process is to create a BAM activity model using Excel. After doing this, I then imported my BAM activity definition into the TPE - as shown below:

TPE UI

One thing that needs to be understood then using the TPE for a process of any complexity is the use of continuations. In my case I need a continuation for:

  • the trigger file --> oracle select request
  • oracle select request --> select result
  • select result --> sFTP send

These continuations are illustrated by the 'expanded' nodes in the previous screen grabs.
The 1st of the "Trigger2Oracle" continuations (without the key icon) is mapped to the File receive port for the trigger file. The 2nd "Trigger2Oracle" continuation is mapped to the send portion of the Oracle request response port. It's the next piece I think I've missed previously. The 1st of the "Select2Result" continuations (without the key icon) is also mapped to the send portion of the Oracle request response port, with the 2nd continuation mapped to the response portion of the same port.

In order to give a full picture, I've pasted the XML of the completed tracking profile below:

<?xml version="1.0" encoding="utf-16"?>
<TrackingProfile xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" VersionGuid="00000000-0000-0000-0000-000000000000" Name="Silverpop">
  <Dimension Name="ActivityID" DataType="TraceID" />
  <Dimension Name="FileContentType" DataType="NVARCHAR">
    <DataLevel Name="ReceivePortName" SourceTypeSelected="Messaging Context Property" SchemaName="BTS.bts_system_properties,Microsoft.BizTalk.GlobalPropertySchemas, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://schemas.microsoft.com/BizTalk/2003/system-properties']/*[local-name()='ReceivePortName' and namespace-uri()='http://schemas.microsoft.com/BizTalk/2003/system-properties']" XPath="/*[local-name()='ReceivePortName' and namespace-uri()='http://schemas.microsoft.com/BizTalk/2003/system-properties']">
      <Port Name="RcvSilverpopCustServey" Direction="Receive" />
    </DataLevel>
  </Dimension>
  <Dimension Name="TimeRequestedFromOracle" DataType="DATETIME">
    <DataLevel Name="PortStartTime" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']">
      <Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Send" />
    </DataLevel>
  </Dimension>
  <Dimension Name="TimeReceivedFromOracle" DataType="DATETIME">
    <DataLevel Name="PortEndTime" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortEndTime' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortEndTime' and namespace-uri()='']">
      <Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Receive" />
    </DataLevel>
  </Dimension>
  <Dimension Name="TimeSentToSilverpop" DataType="DATETIME">
    <DataLevel Name="PortStartTime" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']">
      <Port Name="SilverpopSendCustSurvey_sFTP" Direction="Send" />
    </DataLevel>
  </Dimension>
  <Dimension Name="SentFilename" DataType="NVARCHAR">
    <DataLevel Name="MessageID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageID' and namespace-uri()='']">
      <Port Name="SilverpopSendCustSurvey_sFTP" Direction="Send" />
    </DataLevel>
  </Dimension>
  <Dimension Name="TimeOfException" DataType="DATETIME">
    <DataLevel Name="PortStartTime" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='PortStartTime' and namespace-uri()='']">
      <Port Name="ExceptionHandling.SubmitFault" Direction="Receive" />
    </DataLevel>
  </Dimension>
  <Dimension Name="InterchangeId" DataType="NVARCHAR">
    <DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
      <Port Name="RcvSilverpopCustServey" Direction="Receive" />
    </DataLevel>
  </Dimension>
  <Dimension Name="Trigger2Oracle" DataType="Continuation">
    <DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
      <Port Name="RcvSilverpopCustServey" Direction="Receive" />
    </DataLevel>
  </Dimension>
  <Dimension Name="Trigger2Oracle" DataType="ContinuationID">
    <DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
      <Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Send" />
    </DataLevel>
  </Dimension>
  <Dimension Name="Select2Result" DataType="Continuation">
    <DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
      <Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Send" />
    </DataLevel>
  </Dimension>
  <Dimension Name="Select2Result" DataType="ContinuationID">
    <DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
      <Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Receive" />
    </DataLevel>
  </Dimension>
  <Dimension Name="Result2sFTP" DataType="Continuation">
    <DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
      <Port Name="SilverpopSendGetCustSurvey_ORACLE" Direction="Receive" />
    </DataLevel>
  </Dimension>
  <Dimension Name="Result2sFTP" DataType="ContinuationID">
    <DataLevel Name="InterchangeID" SourceTypeSelected="Messaging Property" SchemaName="Messaging Property Schema" SomXPath="/*[local-name()='&lt;Schema&gt;' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']" XPath="/*[local-name()='MessageProperties' and namespace-uri()='http://OrderProcess.Schema1']/*[local-name()='InterchangeID' and namespace-uri()='']">
      <Port Name="SilverpopSendCustSurvey_sFTP" Direction="Send" />
    </DataLevel>
  </Dimension>
</TrackingProfile>

Once deployed, this tracking profile worked perfectly! It resulted in a single "Completed" activity record (containing milestones for receipt of trigger, request & response from Oracle and send via sFTP) for each trigger file processed. My previous efforts had resulted in multiple records, some of which never made their way from the "Active" to "Completed" BAM tables. If you run into a similar issue, it may be worth checking you haven't missed a continuation.