Using SO-Aware with BizTalk 2010 and ESB 2.1 part 2 of 2

As a quick start, for those that don’t know, SO-Aware is a service metadata repository, very similar to UDDI, except without the complexity, all based on a REST-ful architecture. If you do know, you can skip to using SO-Aware with BizTalk 2010 and ESB 2.1 section. It supports registry of SOAP, REST, and OData based services, written using Microsoft’s WCF technology stack, as well and Java based web services. However, that’s not all. SO-Aware has five major capability buckets: Centralized Configuration, Service Testing, Dependency Modeling, Activity Monitoring and last but not least Service Cataloging.

Centralized Configuration allows you take full advantage of Microsoft’s WCF based services, by containing a central repository database for all WCF Configurations. This feature allows you to allows you to store and retrieve all information pertaining to WCF configurations. You can retrieve endpoints, bindings, behaviors, Url’s, security information, just to name a few. You can also dynamically change your bindings and configurations so that all you existing services can point to this central location for your configuration.

 
 

Service Testing allows you to test registered services. One derived idea about Service Cataloging and Centralized Configuration, is that if SO-Aware knows about your service and communication protocol, then testing becomes simple. It’s simple because if you registered any security, binding, message type format information about the service, then all SO-Aware needs to do is query this information and build the communication stack and messages to send to the service for testing. What better tool to test than the one that understands how to communicate with the Service.

 
 

Dependency Modeling allows you to build a diagram of service versions, and the dependencies of the service version. Thus if you needed to see which services depended each other, you have a view into this.

 
 

Activity Monitoring allows you see tracked events and aggregations about registered services, such which operations were invoked, and how many services were sent message over a period of time, and many other dimensions and measurements.

 
 

Service Cataloging allows you to store and retrieve custom metadata about Services, Service Versions, and environment details. Using these features, allow you to query the catalog for information about services, and service versions dynamically.

 
 

Which leads us into the next discussion, using SO-Aware with BizTalk 2010 and ESB 2.1. The first part of this post covered how SO-Aware can use its Service Cataloging, Centralized Configuration, Dependency Modeling, and Dynamic Resolution capabilities to enhance BizTalk server 2010. This post covers how we can use SO-Aware with BizTalk’s ESB 2.1toolkit.

What is BizTalk ESB 2.1?

The BizTalk ESB Toolkit is a collection of tools and libraries that extend BizTalk Server capabilities of supporting a loosely coupled and dynamic messaging architecture. It functions as middleware that provides tools for rapid mediation between services and their consumers. Enabling maximum flexibility at run time, the BizTalk ESB Toolkit simplifies loosely coupled composition of service endpoints and management of service interactions. The ESB Toolkit contains many components that allows BizTalk Server solutions to be agile in its execution. For example, BizTalk Server contains Adapters which are connectors into different systems and protocols. It also contains Maps, which are Xml stylesheet transformation documents that have been precompiled to transform documents from one format into another. There are Pipelines which are components that allow you to translate and convert one file into another such as a Adobe PDF document into a comma seperated text file. There are also Schemas which support validation of different messages, both xml and non xml. Lastly there are Orchestrations which are graphically modeled business processes that support long running transactions, compensation models, error handling, correlation, and custom logic flow.

Adapters in BizTalk are one of the key components for communicating with BizTalk Server. There are sending adapters and receiving adapters. There’s an adapter for practically every protocol that exists such as ftp, http, smtp, tcp, ftps (new in 2010), Msmq and others. There are application type of adapters that adhere to the rules and policies of Line of Business applications, such as Sharepoint, SAP, Oracle E-Business Suite, and other CRM applications. There are even adapters for databases such as SQL, Oracle, and DB2. When developing solutions with BizTalk Adapters are usually configured statically and there’s little room for agility. Thus BizTalk supports the notion of Dynamic adapters for sending data. These adapters are configured at runtime, which will yield great agile solutions, at the sacrifice of  High performance. , naturally. Using Dynamic adapters requires knowledge of which properties must be set for a particular adapter, and custom code inside of Orchestrations and/or custom pipeline components. This is where some aspects of the ESB 2.1 toolkit come in. The ESB 2.1 toolkit contains Adapter Providers which can dynamically configure adapters for use in a agile solution. All the Adapter provider needs is the runtime configuration values.

The ESB toolkit utilizes a component called Resolvers which feed the Adapter provider the runtime configuration values so that the Adapter Provider can dynamically configure the Dynamic Adapter at runtime. Resolvers can not only discern which Adapter runtime configuration to use, but also which Map, Pipeline, Schema, and Orchestration. Resolvers are simply a .Net component which build up a dictionary collection of entries which then tell other ESB components what values to use for the various BizTalk Components mentioned earlier.

Using Resolvers dramatically changes the way architects and developers can build, design and implement BizTalk Server 2010 solutions. As a quick example, most designs of BizTalk prior to the ESB toolkit hard coded, and statically configured maps. A Map contains a Source schema associated with an xslt transformation, and a destination schema. If either the source, destination or transformation stylesheet changes, it can potentially render the map invalid and cause all kinds of errors. Using a more agile design allows architects to use different versions of maps at runtime, and even dynamically pre-compile a auto generated map all at runtime.

The ESB Toolkit not only contains Resolvers, but also Routing Services, Transformation Services, Itinerary Services, Exception Management services and a  portal. With  exception or the Error portal, Resolvers play an integral role, communicating with the Routing Services, Itinerary Services and Transformation Services. All this to say Resolvers are the key compoonent of the ESB toolkit. The ESB Toolkit comes with many Resolvers: Static, Business Rules, XPath, UDDI, WSMEX, and LDAP, all of which allow for the dynamic resolution of Maps, and adapter providers.

Which leads us to SO-Aware…. SO-Aware provides an ESB Resolver to allow you to dynamically resolve Adapter providers, Maps, schemas, pipelines, orchestrations, WCF Service: Bindings, Versions, Behaviors and other configurations. Inside the SDK, you can find samples on how to use the SO-Aware resolver which show how to resolve Adapter providers, WCF Service: Bindings, Versions, and Behaviors. The SO-Aware resolver and database can be easily extended to resolve Maps, schemas, pipelines and orchestrations.

 
 

Using the SO-Aware ESB Resolver

 
 

After installing the SO-Aware SDK,  you can find the Solution and samples inside the C:\Program Files (x86)\Tellago Studios\SO-Aware\SDK\Samples\BizTalkESB\vs.net2k10\. You can install the SO-Aware resolver by running the setup on a BizTalk 2010 ESB 2.1 system. The Install can be found here: C:\Program Files (x86)\Tellago Studios\SO-Aware\SDK\Samples\BizTalkESB\vs.net2k10\Tellago.SOA.ESB.Extensions.Solution\Tellago.SOA.ESB.Extensions.Setup\Debug\setup.exe .

After installing the SO-Aware  ESB Resolver, you can use the Resolver inside of Business Rules, Itineraries, Orchestrations, or pipeline components, basically anywhere resolvers can be used. Below is an example of how to use the SO-Aware resolver inside an Itinerary.

Steps.

1. Create a new BizTalk 2010 Itinerary inside a C# Library project

2.

3. Add An On-Ramp, Off-Ramp, and 2 Itinerary Service steps into the Itinerary

4. Right click on the Resolver on one of the Itinerary Service Steps and add a new Resolver:


5. In the properties windows select the Tellago SO-Aware ESB Resolver


6. Name the resolver SoAware, and set the properties:


The SO-Aware Resolver contains these properties:

Endpoint Name: –This is the Name of the endpoint registered inside the SO-Aware Repository

Environment: This is the configuration category such as “Production” “Q&A” “Testing” that the service version is registered under.

Operation Name- This is the name of the operation of the Service version that is registered inside SO-Aware

Service Name: This is the name of the Service You would like to retrieve its configuration from.

SO-Aware URL – this is the URL location of where the SO-Aware Service Repository  Service is installed.

Version – This is the version number of the Service registered inside SO-Aware.

7. Configure the remaining Itinerary steps, choosing Receive Port/Location, Dynamic Send Port for the Off Ramp, and adding the Itinerary connectors, completing the Itinerary design.


8. This itinerary is a simple Itinerary that resolves the People Finder  version 1.0 GetAllRegisteredPeople Operation.

Note: The BizTalk 2009 Version of this Resolver is a more generic example, whereas the BizTalk 2010 version contains more functionality.

Advertisements

Using SO-Aware with BizTalk 2010 and ESB 2.1 part 1 of 2

As a quick start, for those that don’t know, SO-Aware is a service metadata repository, very similar to UDDI, except without the complexity, all based on a REST-ful architecture. If you do know, you can skip to using SO-Aware with BizTalk 2010 and ESB 2.1 section. It supports registry of SOAP, REST, and OData based services, written using Microsoft’s WCF technology stack, as well and Java based web services. However, that’s not all. SO-Aware has five major capability buckets: Centralized Configuration, Service Testing, Dependency Modeling, Activity Monitoring and last but not least Service Cataloging.

Centralized Configuration allows you take full advantage of Microsoft’s WCF based services, by containing a central repository database for all WCF Configurations. This feature allows you to allows you to store and retrieve all information pertaining to WCF configurations. You can retrieve endpoints, bindings, behaviors, Url’s, security information, just to name a few. You can also dynamically change your bindings and configurations so that all you existing services can point to this central location for your configuration.

 

Service Testing allows you to test registered services. One derived idea about Service Cataloging and Centralized Configuration, is that if SO-Aware knows about your service and communication protocol, then testing becomes simple. It’s simple because if you registered any security, binding, message type format information about the service, then all SO-Aware needs to do is query this information and build the communication stack and messages to send to the service for testing. What better tool to test than the one that understands how to communicate with the Service.

 

Dependency Modeling allows you to build a diagram of service versions, and the dependencies of the service version. Thus if you needed to see which services depended each other, you have a view into this.

 

Activity Monitoring allows you see tracked events and aggregations about registered services, such which operations were invoked, and how many services were sent message over a period of time, and many other dimensions and measurements.

 

Service Cataloging allows you to store and retrieve custom metadata about Services, Service Versions, and environment details. Using these features, allow you to query the catalog for information about services, and service versions dynamically.

 

Which leads us into the next discussion, using SO-Aware with BizTalk 2010 and ESB 2.1. First, for those that don’t know BizTalk and ESB 2.1, here’s a quick blurb from Microsoft’s web site on each technology:

  What is BizTalk?

BizTalk Server is Microsoft’s Integration and connectivity server solution. A mature product on its seventh release, BizTalk Server 2010 provides a solution that allows organizations to more easily connect disparate systems. Including over 25 multi-platform adapters and a robust messaging infrastructure, BizTalk Server provides connectivity between core systems both inside and outside your organization. In addition to integration functionality, BizTalk also provides strong durable messaging, a rules engine, EDI connectivity, Business Activity Monitoring (BAM), RFID capabilities and IBM Host/Mainframe connectivity.

What is BizTalk ESB 2.1?

The BizTalk ESB Toolkit is a collection of tools and libraries that extend BizTalk Server capabilities of supporting a loosely coupled and dynamic messaging architecture. It functions as middleware that provides tools for rapid mediation between services and their consumers. Enabling maximum flexibility at run time, the BizTalk ESB Toolkit simplifies loosely coupled composition of service endpoints and management of service interactions.

Using SO-Aware with BizTalk 2010

One of the main features of BizTalk Server is its adapter component design. BizTalk uses the Adapters to communicate to various systems, such as Line of Business systems, Databases, Mainframe Applications, and most notably Web Services. To be more specific on web services, BizTalk contains WCF adapters that facilitate communication to different types of web services: REST/ODATA and SOAP.  The more involved BizTalk becomes in integration solutions, the more WCF adapter, and Web services are used to compliment or run the entire solution.

Maintaining such as solution can easily become a maintenance nightmare. When a particular web service changes, or is versioned there are no tools that help keep track of this. Usually trying to remember all the configuration values are impossible, and the solution becomes brittle to the slightest change. SO-Aware easily alleviates this by Service Cataloging, Centralized Configuration, Dependency Modeling, and Dynamic Resolution.

With Service Cataloging, SO-Aware you can register the WCF Adapter configurations for Receive Locations (BizTalk as a Service Provider), and Web Service configurations that the WCF Adapters communicate with through Send Ports (BizTalk as a Service Consumer/client). Below is an example of what may be registered.

BizTalk WCF Adapters Cataloged

Centralized Configuration allows a BizTalk Architect to store configurations about WCF Adapter bindings (individual components that make up a complete communication channel for the WCF Adapter). Using the Centralized confguration of SO-Aware, you can store binding information, behavior information, extension information, security details, transport protocol information, line of business information and etc. We include binding and behavior templates that even provide a dynamic user interface  for configuring the commonly changed settings. This feature facilitates the use of BizTalk’s WCF LOB adapters such as SAP, Oracle and Oracle E-Business Suite, and SQL. Below is an example of registering the Oracle E-Business Suite LOB Adapter binding and  binding template.

image

With Dependency Modeling, you can quickly view which services depend on each other. A reason why this is important in BizTalk is simple. Say you have a business process flow that requires order information to be uploaded to two different web services in one atomic transaction, such as an Order Header Service and a Order Detail Service. This business process can be modeled using BizTalk’s Orchestration design. Below is an image depicting the business process flow.

image

When these services change later, who will remember that these services are dependent upon each other, and that this orchestration depends on both of these services? SO-Aware can easily depict this dependency:

image

Dynamic Resolution

Another option we have with BizTalk Server is using Dynamic Adapters. Dynamic adapters allow for runtime resolution of adapters such as WCF Adapters. This resolution can be performed in two BizTalk components: Orchestrations and pipeline components. Using SO-Aware, dynamic runtime resolution is very easy to do. SO-Aware yields a REST-ful interface to every aspect of its repository. This means that any component which supports invoking .Net code, such as BizTalk’s orchestrations and pipeline components do, will support querying SO-Aware for adapter configuration. Included in the SDK is an example of how to use a BizTalk Orchestration with Dynamic ports and adapters to implement a runtime resolution solution. Below is an example of code inside the BizTalk Orchestration of invoking dynamic runtime resolution.

image

In the above example, the ProcessOrderInformation Orchestration is dynamically resolving the OrderDetailPort  which is configured at runtime to send data to the OrderDetailService registered inside SO-Aware. At runtime, when BizTalk Receives an Order, it will split the order into a Header message and a Order Details message and send both messages to their respective services in one atomic transaction. The differenece being that the OrderDetailService’s binding, configuration, address, and transport protocol will be dynamically resolved using SO-Aware api calls. The Order Detail Service is registered inside the SO-Aware Repository and when queried will return the WCF service configuration details. Below are some depictions of the Order Detail Service SO-Aware registration.

image

 

image

Remember this example was using an Orchestration, however the same applies to BizTalk Pipeline components as well. In the next posting, I will discuss how SO-Aware can be utilized with ESB 2.1 and BizTalk 2010 together.

ESB 2.1 First Look Usage Notes

Here are some notes on my initial try of Microsoft BizTalk ESB 2.1, and BizTalk Server 2010 beta on an x64 image.

To Install the Itinerary Designer, first download VS_VMSDK.exe
http://www.microsoft.com/downloads/details.aspx?FamilyID=0def949d-2933-49c3-ac50-e884e0ff08a7&displaylang=en

then goto -> Start-> All Programs -> Microsoft BizTalk Server 2010 ESB Toolkit -> Install Itinerary Designer extensions. This installs the DSL Package for the Itinereary Designer.

When I first installed it the Itinerary designer did not show up, so I googled and found this:
http://social.msdn.microsoft.com/Forums/en/biztalkesb/thread/d65d2c8c-5fb0-4ae7-87d0-e5e732a6caae

Basically copy all files and folders inside this:
C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\Tools\Itinerary Designer\Lib

to this:
C:\Users\<login name>\AppData\Local\Microsoft\VisualStudio\10.0\Extensions\Microsoft\BizTalk ESB Toolkit Itinerary Designer\2.1.0.0\Lib

After this I restarted VS.NET 2010, and the new Itinerary Designer showed up inside VS.NET….. YAY!!!

However I wasn’t out of the woods yet. When I tried creating a new Itinerary by adding on-ramps and off-ramps, and itinerary services, I noticed a few new features:

1. THere’s a Require Encryption Certificate : true | false setting that allows you to control whether the itinerary needs to be encrypted or not. This is kewl because the last version required you to go into the Registry and turn it off, or go into the project the xml of the file along with other files and change the export policies.(You can google the two methods…) As I noticed this, I also noticed that my Model Exporter was empty.

I said this is not good, I can’t choose Database nor Xml as the Export model. So I venturued further and saw that I couldnt’ choose On-ramps extensions, off-ramp extensions, the Messaging Extender or anything.

So I went further and addded a resolver, at least I could do that…. And noticed that I had my custom resolvers but none of the standard resolvers loaded.

WTF!!!

So I decided to look for some more Dll’s to copy into that 2.1.0.0\Lib folder since apparently this is where the new 2010 Extensions are loading the DSL extensions. So I copied the all the DLL’s files from the Itinerary Designer folder: Microsoft.Practices.Modelling.COmmon.dll, Microsoft.Practices.Modelling.ExtensionProvider.dll, Microsoft.Practices.Modelling.Services.dll, Microsoft.Practices.Services.Itinerary.Dsl.dll, Microsoft.Practices.Services.Itinerary.DslPackage.dll into the 2.1.0.0\Lib folder.

Now, before I can absolutely say that this worked, I also re-paired the ESB Toolkit 2.1 twice by running the installer and choosing the repair mode. And I installed the Itinerary Designer extensions again by cling the link from the program files menu. And as a last step, I copied all the \Lib files from the ESB\Tools\Itinerary Designer folder into the 2.1.0.0\Lib folder again. However it wasn’t until after I copied the 5 dll’s mentioned above did the regulart extender services and off-ramps start working…

YAY!!!

Hernan de Lahitte’s has a quick posting on some of the new features:
http://weblogs.asp.net/hernandl/archive/2010/05/27/biztalk-esb-toolkit-2-1-beta.aspx

I was also in the mood to update some of the Custom Resolvers and Messaging (Itinerary Services) I’ve built in the past and below are my notes on this thus far:

For Custom Resolver:
Had to Re-Reference ESB Assemblies for 2.1 version
Had to Download VS_VMSDK.exe for 2010 (Modeling SDK 2010)
http://www.microsoft.com/downloads/details.aspx?FamilyID=0def949d-2933-49c3-ac50-e884e0ff08a7&displaylang=en

Be careful to check all assembly and assembly versions, because if you have VS.NET 2k8 installed with the VS.NET 2k8 SDK 1.1 then you have version 9.0 along with 10.0.

My projects referenced the xx.Modelling.Common dll for 9.0, but I needed to re-reference it to the 10.0 assemblies, along with the xx.Modelling.ExtensionProvider.

Verify the xxx.Itinerary.DSL and xx.xx.DSLPackage is version 10.0 as well.

That’s it for now, it’s a rough read, but my attempts are a brute force right now, considering I’m so busy.