Tellago Announces SQL Server 2008 R2 BI Quick Adoption Programs

Tellago Announces SQL Server 2008 R2 BI Quick Adoption Programs

Read about it here: http://tinyurl.com/22uftd5

Advertisements

Attaching an ESB 2.0 Itinerary To WCF Message Header

Attaching an ESB 2.0 Itinerary To WCF Message Header

Yesterday I was building a unit test to test an Http Adapter provider ( http://esbextlibrary.codeplex.com ) that we needed for communication with a Http based endpoint; and before you go down the road of why use HTTP Adapter vs the WCF Adapter and all its endless possibilities, the HTTP Legacy adapter didn’t have issues with badly formed messages like the WCF Adapter did.

When building the unit test, I wanted to use the ESB WCF Generic on ramps, and the WCF Adapter to send a message into the ESB. For those that don’t know this, there is a sample test client that comes with the ESB Samples. This test client contains code to embed an Itinerary and send this itinerary using a SOAP header. The sample also contains code to send to an ASMX web service, and a WCF service. In my opinion the samples are fairly cumbersome and awkward in regards to sending an itinerary into the ESB. Also, keep in mind that this test client was built for ESB 1.0, and ASMX focused doesn’t really take WCF and it’s power into account.

Thus the reasoning for this post…. I wanted to show that there are other ways accomplish this same goal, however using WCF, and the WCF InboundHeaders promoted property.

Let’s say I have an itinerary that looks as simple as this:

This itinerary uses a custom Http Adapter provider through the Resolve_Http_Endpoint resolver configuration to send in the message using the BizTalk HTTP Adapter.

To embed this itinerary into a WCF message, I’ll need to export this itinerary as XML and get the Itinerary content as such:

<Itinerary xmlns:xsi=\”http://www.w3.org/2001/XMLSchema-instance\” xmlns:xsd=\”http://www.w3.org/2001/XMLSchema\” uuid=\”\” beginTime=\”\” completeTime=\”\” state=\”Pending\” isRequestResponse=\”false\” servicecount=\”1\” name=\”HttpLegacyItinerary\” version=\”1.0\” xmlns=\”http://schemas.microsoft.biztalk.practices.esb.com/itinerary\”><BizTalkSegment interchangeId=\”\” epmRRCorrelationToken=\”\” receiveInstanceId=\”\” messageId=\”\” xmlns=\”\” /><ServiceInstance name=\”DTSTwoWaySyncResponse\” type=\”Messaging\” state=\”Pending\” position=\”0\” isRequestResponse=\”true\” xmlns=\”\” /><Services xmlns=\”\”><Service uuid=\”dde8337957054bb79fbdb8be148d5c02\” beginTime=\”\” completeTime=\”\” name=\”DTSTwoWaySyncResponse\” type=\”Messaging\” state=\”Pending\” isRequestResponse=\”true\” position=\”0\” serviceInstanceId=\”\” /></Services><ResolverGroups xmlns=\”\”><Resolvers serviceId=\”DTSTwoWaySyncResponse0\”>&lt;![CDATA[STATIC:\\transportType=HTTP;transportLocation=http_legacy://localhost/MyApp/default.aspx;action=;endpointConfig=RequestTimeout=30&amp;ResponseStatusCode=200;jaxRpcResponse=false;messageExchangePattern=;targetNamespace=;transformType=;]]&gt;</Resolvers></ResolverGroups></Itinerary>

Now that I have the itinerary content, I can use one of the Generic WCF Onramps, or create my own, which is actually what I did. To create your own WCF Generic on ramp:

  1. Create a BizTalk Receive Location inside the ESB BizTalk Application using the BizTalk Admin Console.

  2. Configure the Receive Location to use a WCF-Custom adapter. I choose to use the netTCP Binding with all the default settings. The Uri for this adapter was set to: net.tcp://localhost:8888/ReceiveMsg/service.svc

  3. Configure the Receive Location to use the ItineraryReceivePassthrough pipeline with all the defaults.

  4. Create a Dynamic Send port that matches the filter expressions you created inside the Itinerary. Mine were set to

    1. … ServiceName=DTSTwoWaySyncResponse

    2. … ServiceType=Messaging

    3. … ServiceState=Pending

    4. … IsRequestResponse=True

After I configured all the BizTalk receive and send ports, I needed the client code in my unit test project to invoke and send the test message to the WCF Generic on ramp, along with the attached itinerary as a Soap header (WCF Message Header).

To Configure a client with the message header I created a unit test with a method marked up with the [TestMethod] attribute and added the following code:

NetTcpBinding b = new NetTcpBinding ();

EndpointAddress epa = new EndpointAddress ( “net.tcp://localhost:8888/ReceiveMsg/service.svc” );

IBizTalkReceiveLocation proxy = ChannelFactory < IBizTalkReceiveLocation >.CreateChannel(b, epa);

Message msg = Message .CreateMessage( MessageVersion .Default, “SendToBizTalk” ,

XmlTextReader .Create(

new StringReader (

“<?xml version=\”1.0\” ?><TestMessage>Post To Http Adapter</TestMessage>” )));

Write_ItineraryTo_WCF_InboundHeaders( ref msg);

proxy.SendToBizTalk(msg);

The Method of interest here is the “ Write_ItineraryTo_WCF_InboundHeaders(ref msg);” line of code.

This method is outlined here:

private void Write_ItineraryTo_WCF_InboundHeaders( ref Message wcfMessage)

{

// code taken from msdn: http://technet.microsoft.com/en-us/library/bb246105%28BTS.20%29.aspx

//const string PropertiesToPromoteKey = “http://schemas.microsoft.com/BizTalk/2006/01/Adapters/WCF-properties/Promote&#8221;;

const string PropertiesToWriteKey = http://schemas.microsoft.com/BizTalk/2006/01/Adapters/WCF-properties/WriteToContext&#8221; ;

//XmlQualifiedName propItinerary = new XmlQualifiedName(“Itinerary”, “http://schemas.microsoft.biztalk.practices.esb.com/itinerary&#8221;);

XmlQualifiedName writeItinerary = new XmlQualifiedName ( “Itinerary” , http://schemas.microsoft.biztalk.practices.esb.com/itinerary&#8221; );

//Create a List of KeyValuePairs that indicate properties to be promoted to BizTalk message context.

//A Property Schema must be deployed and string values have a limit of 256 characters

//List<KeyValuePair<XmlQualifiedName, object>> promoteProps = new List<KeyValuePair<XmlQualifiedName, object>>();

//promoteProps.Add(new KeyValuePair<XmlQualifiedName, object>(propItinerary , HttpLegacyItinerary ));

//wcfMessage.Properties[PropertiesToPromoteKey] = promoteProps;

//Create a List of KeyValuePairs that indicate properties to be written to BizTalk message context

KeyValuePair < XmlQualifiedName , object > pair = new KeyValuePair < XmlQualifiedName , object >(writeItinerary,

HttpLegacyItinerary);

List < KeyValuePair < XmlQualifiedName , object >> writeProps = new List < KeyValuePair < XmlQualifiedName , object >>();

writeProps.Add(pair);

XmlSerializer ser = new XmlSerializer ( typeof ( Itinerary ), http://schemas.microsoft.biztalk.practices.esb.com/itinerary&#8221; );

//XmlSerializerNamespaces ns = new XmlSerializerNamespaces();

//ns.Add(“itin”, “http://schemas.microsoft.biztalk.practices.esb.com/itinerary&#8221;);

Itinerary itin =

( Itinerary )ser.Deserialize( XmlDictionaryReader .Create( new StringReader (HttpLegacyItinerary)));

MessageHeader header = MessageHeader .CreateHeader( “Esb” , “” , itin, new myItinerary (itin, “i” , http://schemas.microsoft.biztalk.practices.esb.com/itinerary&#8221; ));

// code for testing the Itinerary pipeline component…

//string str2 = header.ToString(); // .Replace(“\r\n”, “”);

//string pattern = “<[a-zA-Z0-9]:Itinerary.*?</[a-zA-Z0-9]:Itinerary>”;

//string str = Regex.Match(str2, pattern).Value;

wcfMessage.Headers.Add(header);

wcfMessage.Properties.Add(PropertiesToWriteKey, writeProps);

//wcfMessage.Properties[PropertiesToWriteKey] = writeProps;

}

This method takes the itinerary content as a string, and deserializes it into an actual Itinerary class from the ESB Itinerary OM v1.0 assembly. This class then used to create a WCF Message header using the MessageHeader.Create static method call.

There’s one major note here to pay attention to…

The ESB Itinerary pipeline Component that is used with the ItineraryReceivePassthrough pipeline retrieves the Itinerary from a SOAP 1.x message header which is written (not distinguished/nor promoted) into the BizTalk Message context by either the SOAP adapter, or the WCF Adapter. When using the SOAP legacy adapter in BizTalk, SOAP headers are automatically written to the message context, again I must stress, that they are written, not promoted, nor distinguished. This means that the values are not available to orchestrations, nor routing capabilities by default. To get these values promoted, then you have to deploy a property schema with the correct SOAP target namespace. For WCF adapters, however, to get the headers promoted or written you must follow this msdn article:

http://technet.microsoft.com/en-us/library/bb246105%28BTS.20%29.aspx

The component uses a regular expression in order to find the itinerary inside the Soap Headers content. The Regular expression is: string pattern = “<[a-zA-Z0-9]:Itinerary.*?</[a-zA-Z0-9]:Itinerary>”;

What this regular expression tells us is that we must take care to add a prefix qualifier with exactly 1 character to the Itinerary Message Header, otherwise, your itinerary will not be found. Also make sure that you don’t use the “ a” prefix qualifier, as this is already take by WCF and SOAP specifics. Thus any other value will do fine.

Well in order to do this I had to use the overridden MessageHeader.Create() static method, which supports the use of a custom XmlObjectSerializer. This will allow me to control the serialization of the ESB Itinerary so that I may achieve this functionality. This is outlined in the following code:

publicclassmyItinerary:XmlObjectSerializer

{

private Itinerary itin = null ;

private XmlSerializerNamespaces ns = null ;

private XmlSerializer ser = null ;

public myItinerary( Itinerary itin, string prefix, string nameSpace)

{

this .itin = itin;

ns = new XmlSerializerNamespaces ();

ns.Add(prefix, nameSpace);

ser = new XmlSerializer ( typeof ( Itinerary ), http://schemas.microsoft.biztalk.practices.esb.com/itinerary&#8221; );

}

public override bool IsStartObject( XmlDictionaryReader reader)

{

return true ;

}

public override object ReadObject( XmlDictionaryReader reader, bool verifyObjectName)

{

return ser.Deserialize(reader);

}

public override void WriteEndObject( XmlDictionaryWriter writer)

{

}

public override void WriteObjectContent( XmlDictionaryWriter writer, object graph)

{

ser.Serialize(writer , this .itin, this .ns);

}

public override void WriteStartObject( XmlDictionaryWriter writer, object graph)

{

}

}

This custom XmlObjectSerializer class just uses the XmlSerializer behind the scenes, and a XmlSerializerNamespaces instance to add the prefix and namespace setting when serializing the Itinerary ESB OM v1.0 class back into the message stream with a single character prefix qualifier.

After all this was done, I was able to send the test message into the WCF Channel, with the attached WCF Message Header containing the ESB 2.0 strict Itinerary.

Case Closed!!!

BizTalk Dynamic Ports vs. Static Ports – Performance

BizTalk Dynamic Ports vs. Static Ports – Performance

(from the trenches…)

During our implementation of a Microsoft ESB 2.0 solution, we attempted to follow the original design of ESB 2.0 in that we use dynamic adapters/ports for the outgoing transport mechanism when sending data to various endpoints.

Overall the design was elegant in my humble opinion. It successfully sent data in various formats to various endpoints using transport protocols such as MSMQ, HTTP, TCP. It communicated with SQL databases, Oracle Databases, and even custom endpoints with unique TCP based protocols which communicated to MITEL PBX and Voice mail systems. The design basically utilized the Business Rules Engine to dynamic determine which transformation/map to use, and which endpoint to communicate with be it 1-way communication, 2-way communication or some awkward 2-way async-correlated response.

In the initial development and Q&A phases, all appeared to work well and fine. Of course not withstanding the normal headaches of an integration solution with vary sparse requirements, such as the famous “Oh that system doesn’t send that type of message, and this system doesn’t need to process the other one…”. For the most part, the system worked as designed.

Lo and behold we move into a quasi-stress test/performance test environment where the normal load, and stress load is placed on the solution…. what do we get???

We Learn that dynamic adapters / ports are not what they’re hyped up to be!!!

Especially when using them with the WCF Custom Adapter, or WCF LOB Adapter framework, and this includes the WCF-SQL, WCF-Oracle. The issues we ran into were “TimeOut” issues. Be they SQL timeouts, MSMQ Timeouts, Transaction Timeouts, or just plain transmission failures communicating with the endpoint in question. These “Timeout” issues were noticed especially on the WCF adapters, and then noticed on some of the non WCF adapters such as MSMQ, and SQL (non WCF). Now before I get into the specifics, let me explain more about our environment, and solution…

Our environment was not best, but it surely was not worst of environments either. We had 1 server running inside a VM Ware VM with an allocated 3.5 gb of RAM. The Physical Server was an HP DSX server physically with at least 4 2ghz processors, not sure of exactly how much physical RAM, however enough to power roughly 50 different vm images with at least 2gb ram each…. However only 1 processor was allocated to the VM at the time. The MessageBox ran on another VM with the same allocated RAM and a different processor was allocated to it, however running on the same physical server. Disk space for both systems was well over 40gb available on both systems. Both SQL and BizTalk were running within the same AD Domain, thus no crazy configurations in that department. We used the default out of the box configuration for BizTalk and SQL, as this is the start of the Optimization phase of our solution… The Endpoints we were sending to, did not have the latest software necessarily, nor service packs, sometimes we communicated with SQL2000 servers, SQL 2k5, Windows 2003 MSMQ transactional queues, IIS 6.0 Asmx web services, and Raw TCP sockets.

The Solution was a simple Messaging only solution, no orchestrations. It starts by messages are retrieved or received in from one of the endpoints, such as a MSMQ Queue or WCF Web Service. The ESB determines the path of the message through a business rule. The ESB then determines the endpoint to route to through a business rule (BRE). The ESB then determines the transformation to convert the message to,for sending to the selected endpoint.

In development and the initial Q/A, all endpoints passed with small challenges here and there. However, when placing the normal load, timeouts started occurring upon sending to SQL and MSMQ outgoing endpoints. We first noticed the issue when sending a load of about 5000 xml messages of an average size of about 3kb into the BizTalk system. What we saw was that the ESB processed the message fine, transformed it, and sent it to the internal dynamic send port queue. It was then the job of the dynamic send port to take the message and actually send it to the corresponding endpoint. After the first 1100 or so messages, things appeared to work fine, however at a certain point, “Timeouts”.

Thus we took a look at the inner workings of the adapter in question and noticed especially for WCF Custom when sending to a transactional based endpoint, there is a major performance overhead as the adapter must determine how to build the channel stack when sending messages. That’s when we ran into a little advice from the MS WCF LOB team: http://blogs.msdn.com/mdoctor/archive/2009/12/18/performance-tip-when-using-wcf-custom-with-dynamic-send-ports-and-custom-bindings-on-biztalk-server-2009.aspx?CommentPosted=true#commentmessage

We applied this setting, as shown by one of my developers here: http://geekswithblogs.net/BizTalkUnleashed/archive/2010/03/31/configure-enabletransaction-and-isolationlevel-property-in-business-rules-for-dynamic.aspx

However, no success. We did see a performance gain, instead of timing out at 1100 or so messages, we got it up to about 1700 or so. We started to wonder if it was really the dynamic adapter.

Static Ports

We decided to run the exact same messaging scenario, however this time with Static ports. The first test was to just create a static port with the same filter conditions as the dynamic port, use the same send pipeline configuration and the exactly same Endpoint Resoluted adapter values etc. As we ran the solution, SUCCESS!!!. No timeouts, No errors, No issues, No warnings, and what appeared to be 5/6 messages being sent through per second, dynamic dramatically increased by 2 – 3 times the amount, in other words 15-20 messages per second. I’d say pretty good, without any throttling settings, all out of the box behavior. For those of you saying what about the baseline… we continually asked for a baseline for the hardware, which we were never provided, nor allowed to do ourselves thus we really don’t know if this is GOOD metrics according to the hardware or not, however, it’s acceptable for out client’s requirements- which is Good enough- ya know!!!

By the way, we also saw timeouts with MSMQ o.o.b (out of the box), WCF-SQL o.o.b and SQL o.o.b adapters when using them in dyanmic ports.

So MS what gives? An explanation would be nice here, besides the WCF create binding elements weak excuse. I welcome any advice, recommendation, or explanations.