Using SO-Aware with BizTalk 2010 and ESB 2.1 part 2 of 2

As a quick start, for those that don’t know, SO-Aware is a service metadata repository, very similar to UDDI, except without the complexity, all based on a REST-ful architecture. If you do know, you can skip to using SO-Aware with BizTalk 2010 and ESB 2.1 section. It supports registry of SOAP, REST, and OData based services, written using Microsoft’s WCF technology stack, as well and Java based web services. However, that’s not all. SO-Aware has five major capability buckets: Centralized Configuration, Service Testing, Dependency Modeling, Activity Monitoring and last but not least Service Cataloging.

Centralized Configuration allows you take full advantage of Microsoft’s WCF based services, by containing a central repository database for all WCF Configurations. This feature allows you to allows you to store and retrieve all information pertaining to WCF configurations. You can retrieve endpoints, bindings, behaviors, Url’s, security information, just to name a few. You can also dynamically change your bindings and configurations so that all you existing services can point to this central location for your configuration.

 
 

Service Testing allows you to test registered services. One derived idea about Service Cataloging and Centralized Configuration, is that if SO-Aware knows about your service and communication protocol, then testing becomes simple. It’s simple because if you registered any security, binding, message type format information about the service, then all SO-Aware needs to do is query this information and build the communication stack and messages to send to the service for testing. What better tool to test than the one that understands how to communicate with the Service.

 
 

Dependency Modeling allows you to build a diagram of service versions, and the dependencies of the service version. Thus if you needed to see which services depended each other, you have a view into this.

 
 

Activity Monitoring allows you see tracked events and aggregations about registered services, such which operations were invoked, and how many services were sent message over a period of time, and many other dimensions and measurements.

 
 

Service Cataloging allows you to store and retrieve custom metadata about Services, Service Versions, and environment details. Using these features, allow you to query the catalog for information about services, and service versions dynamically.

 
 

Which leads us into the next discussion, using SO-Aware with BizTalk 2010 and ESB 2.1. The first part of this post covered how SO-Aware can use its Service Cataloging, Centralized Configuration, Dependency Modeling, and Dynamic Resolution capabilities to enhance BizTalk server 2010. This post covers how we can use SO-Aware with BizTalk’s ESB 2.1toolkit.

What is BizTalk ESB 2.1?

The BizTalk ESB Toolkit is a collection of tools and libraries that extend BizTalk Server capabilities of supporting a loosely coupled and dynamic messaging architecture. It functions as middleware that provides tools for rapid mediation between services and their consumers. Enabling maximum flexibility at run time, the BizTalk ESB Toolkit simplifies loosely coupled composition of service endpoints and management of service interactions. The ESB Toolkit contains many components that allows BizTalk Server solutions to be agile in its execution. For example, BizTalk Server contains Adapters which are connectors into different systems and protocols. It also contains Maps, which are Xml stylesheet transformation documents that have been precompiled to transform documents from one format into another. There are Pipelines which are components that allow you to translate and convert one file into another such as a Adobe PDF document into a comma seperated text file. There are also Schemas which support validation of different messages, both xml and non xml. Lastly there are Orchestrations which are graphically modeled business processes that support long running transactions, compensation models, error handling, correlation, and custom logic flow.

Adapters in BizTalk are one of the key components for communicating with BizTalk Server. There are sending adapters and receiving adapters. There’s an adapter for practically every protocol that exists such as ftp, http, smtp, tcp, ftps (new in 2010), Msmq and others. There are application type of adapters that adhere to the rules and policies of Line of Business applications, such as Sharepoint, SAP, Oracle E-Business Suite, and other CRM applications. There are even adapters for databases such as SQL, Oracle, and DB2. When developing solutions with BizTalk Adapters are usually configured statically and there’s little room for agility. Thus BizTalk supports the notion of Dynamic adapters for sending data. These adapters are configured at runtime, which will yield great agile solutions, at the sacrifice of  High performance. , naturally. Using Dynamic adapters requires knowledge of which properties must be set for a particular adapter, and custom code inside of Orchestrations and/or custom pipeline components. This is where some aspects of the ESB 2.1 toolkit come in. The ESB 2.1 toolkit contains Adapter Providers which can dynamically configure adapters for use in a agile solution. All the Adapter provider needs is the runtime configuration values.

The ESB toolkit utilizes a component called Resolvers which feed the Adapter provider the runtime configuration values so that the Adapter Provider can dynamically configure the Dynamic Adapter at runtime. Resolvers can not only discern which Adapter runtime configuration to use, but also which Map, Pipeline, Schema, and Orchestration. Resolvers are simply a .Net component which build up a dictionary collection of entries which then tell other ESB components what values to use for the various BizTalk Components mentioned earlier.

Using Resolvers dramatically changes the way architects and developers can build, design and implement BizTalk Server 2010 solutions. As a quick example, most designs of BizTalk prior to the ESB toolkit hard coded, and statically configured maps. A Map contains a Source schema associated with an xslt transformation, and a destination schema. If either the source, destination or transformation stylesheet changes, it can potentially render the map invalid and cause all kinds of errors. Using a more agile design allows architects to use different versions of maps at runtime, and even dynamically pre-compile a auto generated map all at runtime.

The ESB Toolkit not only contains Resolvers, but also Routing Services, Transformation Services, Itinerary Services, Exception Management services and a  portal. With  exception or the Error portal, Resolvers play an integral role, communicating with the Routing Services, Itinerary Services and Transformation Services. All this to say Resolvers are the key compoonent of the ESB toolkit. The ESB Toolkit comes with many Resolvers: Static, Business Rules, XPath, UDDI, WSMEX, and LDAP, all of which allow for the dynamic resolution of Maps, and adapter providers.

Which leads us to SO-Aware…. SO-Aware provides an ESB Resolver to allow you to dynamically resolve Adapter providers, Maps, schemas, pipelines, orchestrations, WCF Service: Bindings, Versions, Behaviors and other configurations. Inside the SDK, you can find samples on how to use the SO-Aware resolver which show how to resolve Adapter providers, WCF Service: Bindings, Versions, and Behaviors. The SO-Aware resolver and database can be easily extended to resolve Maps, schemas, pipelines and orchestrations.

 
 

Using the SO-Aware ESB Resolver

 
 

After installing the SO-Aware SDK,  you can find the Solution and samples inside the C:\Program Files (x86)\Tellago Studios\SO-Aware\SDK\Samples\BizTalkESB\vs.net2k10\. You can install the SO-Aware resolver by running the setup on a BizTalk 2010 ESB 2.1 system. The Install can be found here: C:\Program Files (x86)\Tellago Studios\SO-Aware\SDK\Samples\BizTalkESB\vs.net2k10\Tellago.SOA.ESB.Extensions.Solution\Tellago.SOA.ESB.Extensions.Setup\Debug\setup.exe .

After installing the SO-Aware  ESB Resolver, you can use the Resolver inside of Business Rules, Itineraries, Orchestrations, or pipeline components, basically anywhere resolvers can be used. Below is an example of how to use the SO-Aware resolver inside an Itinerary.

Steps.

1. Create a new BizTalk 2010 Itinerary inside a C# Library project

2.

3. Add An On-Ramp, Off-Ramp, and 2 Itinerary Service steps into the Itinerary

4. Right click on the Resolver on one of the Itinerary Service Steps and add a new Resolver:


5. In the properties windows select the Tellago SO-Aware ESB Resolver


6. Name the resolver SoAware, and set the properties:


The SO-Aware Resolver contains these properties:

Endpoint Name: –This is the Name of the endpoint registered inside the SO-Aware Repository

Environment: This is the configuration category such as “Production” “Q&A” “Testing” that the service version is registered under.

Operation Name- This is the name of the operation of the Service version that is registered inside SO-Aware

Service Name: This is the name of the Service You would like to retrieve its configuration from.

SO-Aware URL – this is the URL location of where the SO-Aware Service Repository  Service is installed.

Version – This is the version number of the Service registered inside SO-Aware.

7. Configure the remaining Itinerary steps, choosing Receive Port/Location, Dynamic Send Port for the Off Ramp, and adding the Itinerary connectors, completing the Itinerary design.


8. This itinerary is a simple Itinerary that resolves the People Finder  version 1.0 GetAllRegisteredPeople Operation.

Note: The BizTalk 2009 Version of this Resolver is a more generic example, whereas the BizTalk 2010 version contains more functionality.

Rough Notes for setting up ESB 2.1 Portal on a System with Sharepoint 2010 on port 80

I just wanted to log my notes for everyone to use when trying to setup ESB 2.1 portal on a System that has Sharepoint 2010 running on port 80. I’ll first start with my computer system environment. So that you can compare yours.

My Environment: (Win2k8R2Ent – VMWare 7.0 64bit image)

  • Windows Server 2008 R2 (64bit)
  • VS.NET 2010
  • BizTalk 2010 Beta (64bit)
  • ESB 2.1 (Installed but not configured)
  • SQL 2k8 R2 (64bit)
  • Sharepoint Portal 2010 (64bit)
  • MS Office 2010 (64bit)
  • BizTalk Adapter pack 2.1 (64bit & 32bit)
  • WCF LOB SDK (64bit)

Again I’d just like to stress this scenario is where Sharepoint Portal is installed on port 80, and your IIS default web site is running on another port. Mine is running on port 8090. If you’re not doing this you probably won’t have as much trouble, nor have to run through as many steps.

  1. I remember from last time when install ESB Portal, you have a couple of options, using the source code, solution, and running the setup. Or using the Setup Script. I decided to use the Powershell script.
  2. So I First went to the Management Portal folder/Install/Scripts
  3. I had to fix Management_Install.ps1 powershell file.
  4. Script was looking in "Program files" folder for Visual Studio .Net 2010, instead of "Program Files (x86)" program files folder.
  5. On line 12 of script file change to:
  6.    1: $env:VS="${env:ProgramFiles} (x86)\Microsoft Visual Studio 10.0\Common7\IDE"

  7. Next had to create an SNK file in Keys folder – just followed the Readme.txt file inside the "Keys" folder of the ESBSource extraction. I extracted my ESBSource to my documents folder.
  8. Next I ran the script as an administrator… (Important)!!!
  9. Next had to change identity of App Pool to have secure access to the unzipped folder where ESB Portal will reside…
  10. I used the Administrator account for the app pool.
  11. Next I had to modify web.config file of Management Portal site, to use port 8090 on BAM, Exception, Itin, BizTalk Operation services. Because I had sharepoint installed on port 80
  12. So what I did was change this in web.config file using VS.net text editor
  13.    1: <applicationSettings>

       2:         <Microsoft.Practices.ESB.Portal.Properties.Settings>

       3:             <setting name="Microsoft_Practices_ESB_Portal_BizTalkOperationsService_Operations" serializeAs="String">

       4:                 <value>http://localhost:8090/ESB.BizTalkOperationsService/Operations.asmx</value>

       5:             </setting>

       6:             <setting name="Microsoft_Practices_ESB_Portal_ProcessItinerary_Process" serializeAs="String">

       7:                 <value>http://localhost:8090/ESB.ItineraryServices/ProcessItinerary.asmx</value>

       8:             </setting>

       9:             <setting name="Microsoft_Practices_ESB_Portal_UDDIService_UDDIService" serializeAs="String">

      10:                 <value>http://localhost:8090/ESB.UDDI.Service/UDDIService.asmx</value>

      11:             </setting>

      12:         </Microsoft.Practices.ESB.Portal.Properties.Settings>

      13:     </applicationSettings>

  14. Then in WCF Client Endpoints section:
  15.    1: <client>

       2:             <endpoint address="http://localhost:8090/ESB.ItineraryServices.WCF/ProcessItinerary.svc" binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_ITwoWayAsyncVoid" contract="ProcessRequest" name="WSHttpBinding_ITwoWayAsyncVoid"/>

       3:             <endpoint bindingConfiguration="webBam" address="http://localhost:8090/ESB.BAM.Service/BAMService.svc" behaviorConfiguration="bamservice" binding="webHttpBinding" contract="ESB.BAM.Service.Implementation.IBAMService" name="BAMService"/>

       4:             <endpoint bindingConfiguration="webBam" address="http://localhost:8090/ESB.Exceptions.Service/ExceptionService.svc" behaviorConfiguration="bamservice" binding="webHttpBinding" contract="ESB.Exceptions.Service.Implementation.IExceptionService" name="ExceptionService"/>

       5:         </client>

  16. At this point, I Forgot that my ESB wasn’t configured, and none of my ESB Web Services was configured so you can use the ESBConfiguration wizard, or manually configure web services like I did below.
  17. Manually Configuring ESB Web Services without using the Config wizard.
  18. I created web applications in IIS for each service basically configuring with the settings below.
  19. For certain web sites I had to configure differently, which are outlined below, everything else used these setttings here:
  20. Used the default ESB Portal Network App Pool which was created from running the Portal install script
  21. I changed the default Network settings account to use admin account for this app pool -user id/pwd
  22. Used Anon Auth -Disabled; ASP.Net Impersonation – Enabled; Windows Auth – Enabled
  23. .Net 2.0 app pool, Integrated
  24. For ESB.ExceptionHandlingServices.WCF, and all BizTalk Receive locations based Services –
  25. I created a web app in IIS
  26. Used ESB Portal Network App Pool – admin user id/pwd
  27. Used Anon Auth-Enabled, Wind Auth -Enabled, ASP.NET Imper -Enabled
  28. .NET 2.0 app pool, integrated
  29. Started BizTalk Receive Location
  30. For ESB.Resolver Service
  31. Used the Network App Pool – admin user id
  32. Used Anon Auth -Disabled; ASP.Net Impersonation – Enabled; Windows Auth – Enabled
  33. .Net 2.0 app pool, Integrated
  34. For ESB.ResolverServices.WCF, ESB.TransformService, ESB.TranformService.WCF, ESB.ExceptionService and ESB.Portal
  35. Created a New App pool – managed Pipeline: Classic – .Net 2.0 – admin user id
  36. Used Anon Auth -Disabled; ASP.Net Impersonation – Enabled; Windows Auth – Enabled
  37. * – just for ESB.TranformService.WCF, I had to turn off Custom Errors = off and then back on for this to work using VS.NET text editor to edit web config
  38. Now because my ports are 8090, I had to open up the portal Web References folder and change all the url’s to the correct port number, in this case it was only the ASMX BizTalkOperations Service, all the others used the application settings from the web config I changed in like step 2.
  39. I noticed a slight issue, problem was that the web site was referencing this ESB.Portal.DLL which contained all the web service references.
  40. So I had to recompile this dll.
  41. If you look inside the web ESB.Portal site folder you’ll notice a vs.net 2k8 csproj file. Open this project with vs.net 2010, heck I just added it to the Web Site project convert it to 2010 format, and make the changes in this project. Make sure you keep the web site using the .Net framework 2.0.
  42. Even though I changed mine to be targeted to .Net framework 3.5.
  43. Rebuild here.
  44. For example the BizTalkOperations service – navigate to Reference.cs – look for code this.Url = "" and modify it to reflect the correct port 8090.
  45. In the Reference.cs file I used the web.config application settings property for the Operations service
  46. code is:
  47. I had to change UDDI Service as well:
  48. Update the Service References by right-clicking on Service Reference of each web service look at the properties window for the default url and point to the new port number, refresh references and rebuild.
  49. Lastly I had to rebuild the web site.
  50. Restart IIS and my esb.portal came up…
  51. image

ESB 2.1 First Look Usage Notes

Here are some notes on my initial try of Microsoft BizTalk ESB 2.1, and BizTalk Server 2010 beta on an x64 image.

To Install the Itinerary Designer, first download VS_VMSDK.exe
http://www.microsoft.com/downloads/details.aspx?FamilyID=0def949d-2933-49c3-ac50-e884e0ff08a7&displaylang=en

then goto -> Start-> All Programs -> Microsoft BizTalk Server 2010 ESB Toolkit -> Install Itinerary Designer extensions. This installs the DSL Package for the Itinereary Designer.

When I first installed it the Itinerary designer did not show up, so I googled and found this:
http://social.msdn.microsoft.com/Forums/en/biztalkesb/thread/d65d2c8c-5fb0-4ae7-87d0-e5e732a6caae

Basically copy all files and folders inside this:
C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\Tools\Itinerary Designer\Lib

to this:
C:\Users\<login name>\AppData\Local\Microsoft\VisualStudio\10.0\Extensions\Microsoft\BizTalk ESB Toolkit Itinerary Designer\2.1.0.0\Lib

After this I restarted VS.NET 2010, and the new Itinerary Designer showed up inside VS.NET….. YAY!!!

However I wasn’t out of the woods yet. When I tried creating a new Itinerary by adding on-ramps and off-ramps, and itinerary services, I noticed a few new features:

1. THere’s a Require Encryption Certificate : true | false setting that allows you to control whether the itinerary needs to be encrypted or not. This is kewl because the last version required you to go into the Registry and turn it off, or go into the project the xml of the file along with other files and change the export policies.(You can google the two methods…) As I noticed this, I also noticed that my Model Exporter was empty.

I said this is not good, I can’t choose Database nor Xml as the Export model. So I venturued further and saw that I couldnt’ choose On-ramps extensions, off-ramp extensions, the Messaging Extender or anything.

So I went further and addded a resolver, at least I could do that…. And noticed that I had my custom resolvers but none of the standard resolvers loaded.

WTF!!!

So I decided to look for some more Dll’s to copy into that 2.1.0.0\Lib folder since apparently this is where the new 2010 Extensions are loading the DSL extensions. So I copied the all the DLL’s files from the Itinerary Designer folder: Microsoft.Practices.Modelling.COmmon.dll, Microsoft.Practices.Modelling.ExtensionProvider.dll, Microsoft.Practices.Modelling.Services.dll, Microsoft.Practices.Services.Itinerary.Dsl.dll, Microsoft.Practices.Services.Itinerary.DslPackage.dll into the 2.1.0.0\Lib folder.

Now, before I can absolutely say that this worked, I also re-paired the ESB Toolkit 2.1 twice by running the installer and choosing the repair mode. And I installed the Itinerary Designer extensions again by cling the link from the program files menu. And as a last step, I copied all the \Lib files from the ESB\Tools\Itinerary Designer folder into the 2.1.0.0\Lib folder again. However it wasn’t until after I copied the 5 dll’s mentioned above did the regulart extender services and off-ramps start working…

YAY!!!

Hernan de Lahitte’s has a quick posting on some of the new features:
http://weblogs.asp.net/hernandl/archive/2010/05/27/biztalk-esb-toolkit-2-1-beta.aspx

I was also in the mood to update some of the Custom Resolvers and Messaging (Itinerary Services) I’ve built in the past and below are my notes on this thus far:

For Custom Resolver:
Had to Re-Reference ESB Assemblies for 2.1 version
Had to Download VS_VMSDK.exe for 2010 (Modeling SDK 2010)
http://www.microsoft.com/downloads/details.aspx?FamilyID=0def949d-2933-49c3-ac50-e884e0ff08a7&displaylang=en

Be careful to check all assembly and assembly versions, because if you have VS.NET 2k8 installed with the VS.NET 2k8 SDK 1.1 then you have version 9.0 along with 10.0.

My projects referenced the xx.Modelling.Common dll for 9.0, but I needed to re-reference it to the 10.0 assemblies, along with the xx.Modelling.ExtensionProvider.

Verify the xxx.Itinerary.DSL and xx.xx.DSLPackage is version 10.0 as well.

That’s it for now, it’s a rough read, but my attempts are a brute force right now, considering I’m so busy.