Using SO-Aware with BizTalk 2010 and ESB 2.1 part 1 of 2

As a quick start, for those that don’t know, SO-Aware is a service metadata repository, very similar to UDDI, except without the complexity, all based on a REST-ful architecture. If you do know, you can skip to using SO-Aware with BizTalk 2010 and ESB 2.1 section. It supports registry of SOAP, REST, and OData based services, written using Microsoft’s WCF technology stack, as well and Java based web services. However, that’s not all. SO-Aware has five major capability buckets: Centralized Configuration, Service Testing, Dependency Modeling, Activity Monitoring and last but not least Service Cataloging.

Centralized Configuration allows you take full advantage of Microsoft’s WCF based services, by containing a central repository database for all WCF Configurations. This feature allows you to allows you to store and retrieve all information pertaining to WCF configurations. You can retrieve endpoints, bindings, behaviors, Url’s, security information, just to name a few. You can also dynamically change your bindings and configurations so that all you existing services can point to this central location for your configuration.

 

Service Testing allows you to test registered services. One derived idea about Service Cataloging and Centralized Configuration, is that if SO-Aware knows about your service and communication protocol, then testing becomes simple. It’s simple because if you registered any security, binding, message type format information about the service, then all SO-Aware needs to do is query this information and build the communication stack and messages to send to the service for testing. What better tool to test than the one that understands how to communicate with the Service.

 

Dependency Modeling allows you to build a diagram of service versions, and the dependencies of the service version. Thus if you needed to see which services depended each other, you have a view into this.

 

Activity Monitoring allows you see tracked events and aggregations about registered services, such which operations were invoked, and how many services were sent message over a period of time, and many other dimensions and measurements.

 

Service Cataloging allows you to store and retrieve custom metadata about Services, Service Versions, and environment details. Using these features, allow you to query the catalog for information about services, and service versions dynamically.

 

Which leads us into the next discussion, using SO-Aware with BizTalk 2010 and ESB 2.1. First, for those that don’t know BizTalk and ESB 2.1, here’s a quick blurb from Microsoft’s web site on each technology:

  What is BizTalk?

BizTalk Server is Microsoft’s Integration and connectivity server solution. A mature product on its seventh release, BizTalk Server 2010 provides a solution that allows organizations to more easily connect disparate systems. Including over 25 multi-platform adapters and a robust messaging infrastructure, BizTalk Server provides connectivity between core systems both inside and outside your organization. In addition to integration functionality, BizTalk also provides strong durable messaging, a rules engine, EDI connectivity, Business Activity Monitoring (BAM), RFID capabilities and IBM Host/Mainframe connectivity.

What is BizTalk ESB 2.1?

The BizTalk ESB Toolkit is a collection of tools and libraries that extend BizTalk Server capabilities of supporting a loosely coupled and dynamic messaging architecture. It functions as middleware that provides tools for rapid mediation between services and their consumers. Enabling maximum flexibility at run time, the BizTalk ESB Toolkit simplifies loosely coupled composition of service endpoints and management of service interactions.

Using SO-Aware with BizTalk 2010

One of the main features of BizTalk Server is its adapter component design. BizTalk uses the Adapters to communicate to various systems, such as Line of Business systems, Databases, Mainframe Applications, and most notably Web Services. To be more specific on web services, BizTalk contains WCF adapters that facilitate communication to different types of web services: REST/ODATA and SOAP.  The more involved BizTalk becomes in integration solutions, the more WCF adapter, and Web services are used to compliment or run the entire solution.

Maintaining such as solution can easily become a maintenance nightmare. When a particular web service changes, or is versioned there are no tools that help keep track of this. Usually trying to remember all the configuration values are impossible, and the solution becomes brittle to the slightest change. SO-Aware easily alleviates this by Service Cataloging, Centralized Configuration, Dependency Modeling, and Dynamic Resolution.

With Service Cataloging, SO-Aware you can register the WCF Adapter configurations for Receive Locations (BizTalk as a Service Provider), and Web Service configurations that the WCF Adapters communicate with through Send Ports (BizTalk as a Service Consumer/client). Below is an example of what may be registered.

BizTalk WCF Adapters Cataloged

Centralized Configuration allows a BizTalk Architect to store configurations about WCF Adapter bindings (individual components that make up a complete communication channel for the WCF Adapter). Using the Centralized confguration of SO-Aware, you can store binding information, behavior information, extension information, security details, transport protocol information, line of business information and etc. We include binding and behavior templates that even provide a dynamic user interface  for configuring the commonly changed settings. This feature facilitates the use of BizTalk’s WCF LOB adapters such as SAP, Oracle and Oracle E-Business Suite, and SQL. Below is an example of registering the Oracle E-Business Suite LOB Adapter binding and  binding template.

image

With Dependency Modeling, you can quickly view which services depend on each other. A reason why this is important in BizTalk is simple. Say you have a business process flow that requires order information to be uploaded to two different web services in one atomic transaction, such as an Order Header Service and a Order Detail Service. This business process can be modeled using BizTalk’s Orchestration design. Below is an image depicting the business process flow.

image

When these services change later, who will remember that these services are dependent upon each other, and that this orchestration depends on both of these services? SO-Aware can easily depict this dependency:

image

Dynamic Resolution

Another option we have with BizTalk Server is using Dynamic Adapters. Dynamic adapters allow for runtime resolution of adapters such as WCF Adapters. This resolution can be performed in two BizTalk components: Orchestrations and pipeline components. Using SO-Aware, dynamic runtime resolution is very easy to do. SO-Aware yields a REST-ful interface to every aspect of its repository. This means that any component which supports invoking .Net code, such as BizTalk’s orchestrations and pipeline components do, will support querying SO-Aware for adapter configuration. Included in the SDK is an example of how to use a BizTalk Orchestration with Dynamic ports and adapters to implement a runtime resolution solution. Below is an example of code inside the BizTalk Orchestration of invoking dynamic runtime resolution.

image

In the above example, the ProcessOrderInformation Orchestration is dynamically resolving the OrderDetailPort  which is configured at runtime to send data to the OrderDetailService registered inside SO-Aware. At runtime, when BizTalk Receives an Order, it will split the order into a Header message and a Order Details message and send both messages to their respective services in one atomic transaction. The differenece being that the OrderDetailService’s binding, configuration, address, and transport protocol will be dynamically resolved using SO-Aware api calls. The Order Detail Service is registered inside the SO-Aware Repository and when queried will return the WCF service configuration details. Below are some depictions of the Order Detail Service SO-Aware registration.

image

 

image

Remember this example was using an Orchestration, however the same applies to BizTalk Pipeline components as well. In the next posting, I will discuss how SO-Aware can be utilized with ESB 2.1 and BizTalk 2010 together.

Advertisements

Rough Notes for setting up ESB 2.1 Portal on a System with Sharepoint 2010 on port 80

I just wanted to log my notes for everyone to use when trying to setup ESB 2.1 portal on a System that has Sharepoint 2010 running on port 80. I’ll first start with my computer system environment. So that you can compare yours.

My Environment: (Win2k8R2Ent – VMWare 7.0 64bit image)

  • Windows Server 2008 R2 (64bit)
  • VS.NET 2010
  • BizTalk 2010 Beta (64bit)
  • ESB 2.1 (Installed but not configured)
  • SQL 2k8 R2 (64bit)
  • Sharepoint Portal 2010 (64bit)
  • MS Office 2010 (64bit)
  • BizTalk Adapter pack 2.1 (64bit & 32bit)
  • WCF LOB SDK (64bit)

Again I’d just like to stress this scenario is where Sharepoint Portal is installed on port 80, and your IIS default web site is running on another port. Mine is running on port 8090. If you’re not doing this you probably won’t have as much trouble, nor have to run through as many steps.

  1. I remember from last time when install ESB Portal, you have a couple of options, using the source code, solution, and running the setup. Or using the Setup Script. I decided to use the Powershell script.
  2. So I First went to the Management Portal folder/Install/Scripts
  3. I had to fix Management_Install.ps1 powershell file.
  4. Script was looking in "Program files" folder for Visual Studio .Net 2010, instead of "Program Files (x86)" program files folder.
  5. On line 12 of script file change to:
  6.    1: $env:VS="${env:ProgramFiles} (x86)\Microsoft Visual Studio 10.0\Common7\IDE"

  7. Next had to create an SNK file in Keys folder – just followed the Readme.txt file inside the "Keys" folder of the ESBSource extraction. I extracted my ESBSource to my documents folder.
  8. Next I ran the script as an administrator… (Important)!!!
  9. Next had to change identity of App Pool to have secure access to the unzipped folder where ESB Portal will reside…
  10. I used the Administrator account for the app pool.
  11. Next I had to modify web.config file of Management Portal site, to use port 8090 on BAM, Exception, Itin, BizTalk Operation services. Because I had sharepoint installed on port 80
  12. So what I did was change this in web.config file using VS.net text editor
  13.    1: <applicationSettings>

       2:         <Microsoft.Practices.ESB.Portal.Properties.Settings>

       3:             <setting name="Microsoft_Practices_ESB_Portal_BizTalkOperationsService_Operations" serializeAs="String">

       4:                 <value>http://localhost:8090/ESB.BizTalkOperationsService/Operations.asmx</value>

       5:             </setting>

       6:             <setting name="Microsoft_Practices_ESB_Portal_ProcessItinerary_Process" serializeAs="String">

       7:                 <value>http://localhost:8090/ESB.ItineraryServices/ProcessItinerary.asmx</value>

       8:             </setting>

       9:             <setting name="Microsoft_Practices_ESB_Portal_UDDIService_UDDIService" serializeAs="String">

      10:                 <value>http://localhost:8090/ESB.UDDI.Service/UDDIService.asmx</value>

      11:             </setting>

      12:         </Microsoft.Practices.ESB.Portal.Properties.Settings>

      13:     </applicationSettings>

  14. Then in WCF Client Endpoints section:
  15.    1: <client>

       2:             <endpoint address="http://localhost:8090/ESB.ItineraryServices.WCF/ProcessItinerary.svc" binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_ITwoWayAsyncVoid" contract="ProcessRequest" name="WSHttpBinding_ITwoWayAsyncVoid"/>

       3:             <endpoint bindingConfiguration="webBam" address="http://localhost:8090/ESB.BAM.Service/BAMService.svc" behaviorConfiguration="bamservice" binding="webHttpBinding" contract="ESB.BAM.Service.Implementation.IBAMService" name="BAMService"/>

       4:             <endpoint bindingConfiguration="webBam" address="http://localhost:8090/ESB.Exceptions.Service/ExceptionService.svc" behaviorConfiguration="bamservice" binding="webHttpBinding" contract="ESB.Exceptions.Service.Implementation.IExceptionService" name="ExceptionService"/>

       5:         </client>

  16. At this point, I Forgot that my ESB wasn’t configured, and none of my ESB Web Services was configured so you can use the ESBConfiguration wizard, or manually configure web services like I did below.
  17. Manually Configuring ESB Web Services without using the Config wizard.
  18. I created web applications in IIS for each service basically configuring with the settings below.
  19. For certain web sites I had to configure differently, which are outlined below, everything else used these setttings here:
  20. Used the default ESB Portal Network App Pool which was created from running the Portal install script
  21. I changed the default Network settings account to use admin account for this app pool -user id/pwd
  22. Used Anon Auth -Disabled; ASP.Net Impersonation – Enabled; Windows Auth – Enabled
  23. .Net 2.0 app pool, Integrated
  24. For ESB.ExceptionHandlingServices.WCF, and all BizTalk Receive locations based Services –
  25. I created a web app in IIS
  26. Used ESB Portal Network App Pool – admin user id/pwd
  27. Used Anon Auth-Enabled, Wind Auth -Enabled, ASP.NET Imper -Enabled
  28. .NET 2.0 app pool, integrated
  29. Started BizTalk Receive Location
  30. For ESB.Resolver Service
  31. Used the Network App Pool – admin user id
  32. Used Anon Auth -Disabled; ASP.Net Impersonation – Enabled; Windows Auth – Enabled
  33. .Net 2.0 app pool, Integrated
  34. For ESB.ResolverServices.WCF, ESB.TransformService, ESB.TranformService.WCF, ESB.ExceptionService and ESB.Portal
  35. Created a New App pool – managed Pipeline: Classic – .Net 2.0 – admin user id
  36. Used Anon Auth -Disabled; ASP.Net Impersonation – Enabled; Windows Auth – Enabled
  37. * – just for ESB.TranformService.WCF, I had to turn off Custom Errors = off and then back on for this to work using VS.NET text editor to edit web config
  38. Now because my ports are 8090, I had to open up the portal Web References folder and change all the url’s to the correct port number, in this case it was only the ASMX BizTalkOperations Service, all the others used the application settings from the web config I changed in like step 2.
  39. I noticed a slight issue, problem was that the web site was referencing this ESB.Portal.DLL which contained all the web service references.
  40. So I had to recompile this dll.
  41. If you look inside the web ESB.Portal site folder you’ll notice a vs.net 2k8 csproj file. Open this project with vs.net 2010, heck I just added it to the Web Site project convert it to 2010 format, and make the changes in this project. Make sure you keep the web site using the .Net framework 2.0.
  42. Even though I changed mine to be targeted to .Net framework 3.5.
  43. Rebuild here.
  44. For example the BizTalkOperations service – navigate to Reference.cs – look for code this.Url = "" and modify it to reflect the correct port 8090.
  45. In the Reference.cs file I used the web.config application settings property for the Operations service
  46. code is:
  47. I had to change UDDI Service as well:
  48. Update the Service References by right-clicking on Service Reference of each web service look at the properties window for the default url and point to the new port number, refresh references and rebuild.
  49. Lastly I had to rebuild the web site.
  50. Restart IIS and my esb.portal came up…
  51. image

ESB 2.1 First Look Usage Notes

Here are some notes on my initial try of Microsoft BizTalk ESB 2.1, and BizTalk Server 2010 beta on an x64 image.

To Install the Itinerary Designer, first download VS_VMSDK.exe
http://www.microsoft.com/downloads/details.aspx?FamilyID=0def949d-2933-49c3-ac50-e884e0ff08a7&displaylang=en

then goto -> Start-> All Programs -> Microsoft BizTalk Server 2010 ESB Toolkit -> Install Itinerary Designer extensions. This installs the DSL Package for the Itinereary Designer.

When I first installed it the Itinerary designer did not show up, so I googled and found this:
http://social.msdn.microsoft.com/Forums/en/biztalkesb/thread/d65d2c8c-5fb0-4ae7-87d0-e5e732a6caae

Basically copy all files and folders inside this:
C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\Tools\Itinerary Designer\Lib

to this:
C:\Users\<login name>\AppData\Local\Microsoft\VisualStudio\10.0\Extensions\Microsoft\BizTalk ESB Toolkit Itinerary Designer\2.1.0.0\Lib

After this I restarted VS.NET 2010, and the new Itinerary Designer showed up inside VS.NET….. YAY!!!

However I wasn’t out of the woods yet. When I tried creating a new Itinerary by adding on-ramps and off-ramps, and itinerary services, I noticed a few new features:

1. THere’s a Require Encryption Certificate : true | false setting that allows you to control whether the itinerary needs to be encrypted or not. This is kewl because the last version required you to go into the Registry and turn it off, or go into the project the xml of the file along with other files and change the export policies.(You can google the two methods…) As I noticed this, I also noticed that my Model Exporter was empty.

I said this is not good, I can’t choose Database nor Xml as the Export model. So I venturued further and saw that I couldnt’ choose On-ramps extensions, off-ramp extensions, the Messaging Extender or anything.

So I went further and addded a resolver, at least I could do that…. And noticed that I had my custom resolvers but none of the standard resolvers loaded.

WTF!!!

So I decided to look for some more Dll’s to copy into that 2.1.0.0\Lib folder since apparently this is where the new 2010 Extensions are loading the DSL extensions. So I copied the all the DLL’s files from the Itinerary Designer folder: Microsoft.Practices.Modelling.COmmon.dll, Microsoft.Practices.Modelling.ExtensionProvider.dll, Microsoft.Practices.Modelling.Services.dll, Microsoft.Practices.Services.Itinerary.Dsl.dll, Microsoft.Practices.Services.Itinerary.DslPackage.dll into the 2.1.0.0\Lib folder.

Now, before I can absolutely say that this worked, I also re-paired the ESB Toolkit 2.1 twice by running the installer and choosing the repair mode. And I installed the Itinerary Designer extensions again by cling the link from the program files menu. And as a last step, I copied all the \Lib files from the ESB\Tools\Itinerary Designer folder into the 2.1.0.0\Lib folder again. However it wasn’t until after I copied the 5 dll’s mentioned above did the regulart extender services and off-ramps start working…

YAY!!!

Hernan de Lahitte’s has a quick posting on some of the new features:
http://weblogs.asp.net/hernandl/archive/2010/05/27/biztalk-esb-toolkit-2-1-beta.aspx

I was also in the mood to update some of the Custom Resolvers and Messaging (Itinerary Services) I’ve built in the past and below are my notes on this thus far:

For Custom Resolver:
Had to Re-Reference ESB Assemblies for 2.1 version
Had to Download VS_VMSDK.exe for 2010 (Modeling SDK 2010)
http://www.microsoft.com/downloads/details.aspx?FamilyID=0def949d-2933-49c3-ac50-e884e0ff08a7&displaylang=en

Be careful to check all assembly and assembly versions, because if you have VS.NET 2k8 installed with the VS.NET 2k8 SDK 1.1 then you have version 9.0 along with 10.0.

My projects referenced the xx.Modelling.Common dll for 9.0, but I needed to re-reference it to the 10.0 assemblies, along with the xx.Modelling.ExtensionProvider.

Verify the xxx.Itinerary.DSL and xx.xx.DSLPackage is version 10.0 as well.

That’s it for now, it’s a rough read, but my attempts are a brute force right now, considering I’m so busy.