Web Service Discovery in SO-Aware (Part 2)

In my last post we discussed what was Web Service Discovery, and how it’s two design patterns provide us with different implementations, such as UDDI, SO-Aware, and WCF 4.0 Discovery. In this post we will discuss WCF 4.0 Discovery, and how SO-Aware handles the two modes of WCF Discovery.

How Discovery works today

WS-Discovery provides a protocol to discover services that are coming into and leaving from a network. As a service joins the network, it informs its peers of its arrival by broadcasting a Hello message; likewise, when services drop off the network they multicast a Bye message. WS-Discovery doesn’t rely on a single node to host information about all available services as UDDI and SO-Aware. Which SO-Aware alleviates this for WCF registered services by downloading a cached copy of the configuration to the WCF Service, such that the WCF Service Host retrieves it’s setting from the cached copy if SO-Aware is down. Rather, for WS-Discover each node forwards information about available services in two ways, an ad hoc fashion or a managed mode. This reduces the amount of network infrastructure needed to discover services and facilitates bootstrapping. My Business partner, Jesus Rodriguez has an excellent blog posting on WS-Discover with WCF 4.0. When using WS-Discovery with WCF 4.0, we can use the SO-Aware Service Repository as both the ad hoc and managed mode implementation.

How SO-Aware Handles WCF 4.0 Discovery ad hoc mode

Using WCF 4.0 Discovery requires running the SO-Aware Web Site and ServiceRepository service with .Net 4.0. You can tell ASP.NET to dymanically recompile the web site and data service to use the .Net 4.0 framework. Follow the steps outlined in this post (http://tellagostudios.com/how-setup-so-aware-run-net-40) to do this.

Next is to turn on WCF Add Hoc Discovery in the Service Host. The Current SO-Aware Service Repository host uses the default WCF DataService Service host. By Default this host does not know anything about the discovery mechanism. Thus I have created a new Service Host Factory which injects details about the discovery mechanism. This host is named: Tellago.ServiceModel.Governance.Data.DataServiceHostFactory, which derives from the default WCF DataServiceHost, and adds the Discovery behavior, Annoucement and Discovery Endpoints to the DataServiceHost, making it support the WCF 4.0 Discovery mechanism. Open the ServiceRepository.svc markup with a text editor, and change it’s Factory to point to the new DataServiceHostFactory as such:

Factory = "Tellago.ServiceModel.Governance.Data.DataServiceHostFactory, Tellago.ServiceModel.Governance.Data"

Look for the Tellago.ServiceModel.Governance.Data project inside the SDK folder (\Samples\.Net40\WCF Discover) and compile the project. Copy the resulting dll into the bin directory of the SO-Aware web site, run the site, and now you have Ad hoc discovery mechanisms. The ability to add a service version for (OData, REST, or Soap) and remove a service version entry generates "Online" and "Offline" announcement broadcasts to all listening announcement endpoint clients. (Note: this example only works when the client announcement endpoints are on the same subnet as the SO-Aware ServcieRepository service) It also uses the managed mode approach by creating a Discoverable Host and proxy which clients can use to listen for these announcements. I stress, this is sample code only as it shows how to implement such a thing with SO-Aware.

The way the custom ServiceHost Factory works is by injecting a WCF Message Inspector into the DataServiceHost. The Message Inspector parses the DataService message contents. You know, a funny thing about WCF Data Services is that, the message content is sent and received in a binary, base64 encoded, multi-part message format. Basically it’s a batch message of Xml, post/merge/put/delete messages all delimited by "\r\n\r\n—changeset_{guid}". This was not fun to parse let me tell you. In anycase the inspector looks within the Batch message for the Xml contents, Post method verb in the message properties, and a term="" the entity type in question. If the Entity type is "ServiceVersion" and the Post contains an term="restdescription" or "odatadescription" or "soapdescription" then I know the insert or update message is a Service Version. Upon "Offline" broadcasts, the message inspector looks for a Http Delete verb method in the message properties of the batch and looks for a "ServiceVersion" term as well as the "restdescription", "odatadescription" and "soapdescription" entity names (loweredcased…) to determine if it should broadcast "Offline" messages to announcement endpoints.

To create a client application that uses the Ad Hoc discovery service is rather simple. First you create a DiscoveryClient instance, and add a UdpDiscoveryEndpoint instance to its constructor. The UdpDiscoveryEndpoint already contains a hardcoded Uri which points so some standard urn:docs-oasis-open-org:ws-dd:ns:discovery:2009:01
address version, believe it or not, this is a valid Uri address. Once the discovery client is created, create event handlers to listen for when the Discovery Client has completed it Find operation. Start or invoke the Find operation by using the FindAsync() method of the Discovery Client and wait for the responses, you’re done.

Here’s a code example:

   1: class Program

   2: {

   3:     static DiscoveryClient client;

   4:     static AnnouncementService svc = new AnnouncementService();

   5:     static ServiceHost announcementHost = null;

   6:  

   7:     static void Main(string[] args)

   8:     {

   9:         client = new DiscoveryClient(new UdpDiscoveryEndpoint());

  10:         client.FindProgressChanged += new EventHandler<FindProgressChangedEventArgs>(client_FindProgressChanged);

  11:  

  12:         client.FindCompleted += new EventHandler<FindCompletedEventArgs>(client_FindCompleted);

  13:         client.FindAsync(new FindCriteria(typeof(ServiceRepositoryDataService)));

  14:        

  15:  

  16:         announcementHost = new ServiceHost(svc);

  17:         announcementHost.AddServiceEndpoint(new UdpAnnouncementEndpoint());

  18:         svc.OnlineAnnouncementReceived += new EventHandler<AnnouncementEventArgs>(svc_OnlineAnnouncementReceived); svc.OfflineAnnouncementReceived += new EventHandler<AnnouncementEventArgs>(svc_OfflineAnnouncementReceived);

  19:        

  20:         announcementHost.BeginOpen((result) =>

  21:         {

  22:             announcementHost.EndOpen(result);

  23:         }, null);

  24:  

  25:         Console.WriteLine("Searching...\n");

  26:         Console.ReadLine();

  27:  

  28:     }

  29:  

  30:     static void svc_OfflineAnnouncementReceived(object sender, AnnouncementEventArgs e)

  31:     {

  32:         Console.WriteLine("Service went offline: {0} ", e.EndpointDiscoveryMetadata.Address );

  33:     }

  34:  

  35:     static void svc_OnlineAnnouncementReceived(object sender, AnnouncementEventArgs e)

  36:     {

  37:         Console.WriteLine("Service came online: {0} ", e.EndpointDiscoveryMetadata.Address );

  38:     }

  39:  

  40:     static void client_FindCompleted(object sender, FindCompletedEventArgs e)

  41:     {

  42:         if (e.Error == null)

  43:         {

  44:             if (e.Result.Endpoints.Count > 0)

  45:             {

  46:                 Console.WriteLine("Find Completed Ran\n{0}", e.Result.Endpoints[0].Address);                 

  47:             }

  48:             if (client.InnerChannel.State == System.ServiceModel.CommunicationState.Opened)

  49:             {

  50:                 client.Close();

  51:             }

  52:             Console.ReadLine();

  53:         }

  54:         else

  55:         {

  56:  

  57:             Console.WriteLine(e.Error.ToString());

  58:             Console.ReadLine();

  59:         }

  60:     }

  61:  

  62:     static void client_FindProgressChanged(object sender, FindProgressChangedEventArgs e)

  63:     {

  64:         Console.WriteLine("Percentage Complete:\n{0}%\n", e.ProgressPercentage.ToString());

  65:     }

  66: }

  67:  



How SO-Aware Handles WCF 4.0 Discovery in managed mode

One of the disadvantages to using Ad Hoc discovery is that all the clients must reside on the same subnet due to the UDP multicasting protocol. Managed mode Discovery, provides a way to discover services on other protocols besides the UDP multicast protocol. It works by way of a custom "Discoverable Proxy". The discoverable proxy can query for online services through discovery endpoints, or through any custom means. The concept is simple, a client/service interested in discovery first calls the discoverable proxy and asks it for registered services. The discoverable proxy then returns a list of "Online" services as announcements or some other transport mechanism. This is where SO-Aware can really shine. The SO-Aware Discoverable proxy uses the SO-Aware central repository to discover services being "Online" and "Offline".

The steps to use this discoverable proxy are simple, you can continue to use the announcement client endpoints, as long as they reside on the same subnet, and you can create a client that queries directly into the proxy using the address of the discoverable proxy. In this release, the address is hard coded to net.tcp://localhost:7777/discoverableproxy endpoint address. The next release, v2 of the SO-Aware implementation will allow you to take full advantage of the configuration and repository to control which binding and what uri listening address.

The way the SO-Aware discoverable proxy works is by creating a class that derives from the DiscoveryProxy base abstract class. This class allows you to implement four asynchronous paired (Beginxx and Endxx) methods, OnBeginOnlineAnnouncement, OnBeginResolve, OnBeginFind, OnBeginOfflineAnnoucement

Here’s a sample OnBeginFind method example:

   1:  

   2:      protected override IAsyncResult OnBeginFind(FindRequestContext findRequestContext, AsyncCallback callback, object state)

   3:      {            List<EndpointDiscoveryMetadata > services = new List<EndpointDiscoveryMetadata>();

   4:          var ctxt = new ResourceRepositoryContext(this.ServiceRepositoryUri);

   5:          ctxt.Credentials = System.Net.CredentialCache.DefaultCredentials;

   6:  

   7:          var Soaps = from sp in ctxt.SoapDescriptions.Expand("ServiceVersion")

   8:                      select sp;

   9:          foreach (var soapDescription in Soaps)

  10:          {                

  11:              foreach (var ep in soapDescription.Endpoints )

  12:              {

  13:                  EndpointDiscoveryMetadata discovery = new EndpointDiscoveryMetadata();    

  14:                  discovery.Address = new EndpointAddress(ep.Address );

  15:                  discovery.Extensions.Add(XElement.Parse(String.Format("<ServiceVersionName>{0}</ServiceVersionName>", soapDescription.ServiceVersion.Name)));

  16:                  services.Add(discovery );                    

  17:              }

  18:          }

  19:          var odatas = from od in ctxt.ODataDescriptions.Expand("ServiceVersion")

  20:                       select od;

  21:          foreach (var oDataDescription in odatas)

  22:          {

  23:              EndpointDiscoveryMetadata discovery = new EndpointDiscoveryMetadata();

  24:              discovery.Address = new EndpointAddress(oDataDescription.MetadataURI );

  25:              discovery.Extensions.Add(XElement.Parse(String.Format("<ServiceVersionName>{0}</ServiceVersionName>", oDataDescription.ServiceVersion.Name)));

  26:              services.Add(discovery);

  27:          }

  28:          var rests = from rs in ctxt.RestDescriptions.Expand("ServiceVersion")

  29:                      select rs;

  30:          foreach (var restDescription in rests)

  31:          {

  32:              EndpointDiscoveryMetadata discovery = new EndpointDiscoveryMetadata();

  33:              discovery.Address = new EndpointAddress(restDescription.BaseURI );

  34:              discovery.Extensions.Add(XElement.Parse(String.Format("<ServiceVersionName>{0}</ServiceVersionName>", restDescription.ServiceVersion.Name)));

  35:              services.Add(discovery);

  36:          }

  37:          var query = from service in services

  38:                      where findRequestContext.Criteria.IsMatch(service)

  39:                      select service;

  40:  

  41:          var queryCache = from service in cache

  42:                           where findRequestContext.Criteria.IsMatch(service)

  43:                           select service;

  44:  

  45:          foreach (var metadata in query)

  46:          {

  47:              findRequestContext.AddMatchingEndpoint(metadata);

  48:          }

  49:  

  50:          foreach (var endpointDiscoveryMetadata in queryCache)

  51:          {

  52:              findRequestContext.AddMatchingEndpoint(endpointDiscoveryMetadata );

  53:          }

  54:          return new OnFindAsyncResult(callback, state);          

  55:  

  56:      }



Briefly let’s talk about each of the 4 asynchronous methods, if you want details, MSDN is the best resource for that. The names are self explanatory. BeginFind allows you to lookup services when you find them you just call the findRequestContext.AddMatchingEndpoint(). This method announces to the clients a service has be found immediately. The BeginOnlineAnnouncement and BeginOfflineAnnouncement methods are there for you to cache them for clients calling into your proxy for this information. And lastly BeginResolve allows you to query your cache or in our case the SO-Aware ServiceRepository for service versions to return an Endpoint Address and Service Version Metadata.

To create a client application that uses this managed mode design is also simple. First you create a DiscoveryEndpoint instance and use the same binding and endpoint address of the discoverable proxy you are communicating with. In the case of SO-Aware, it’s binding is a NetTcpBinding with all the default settings, with the endpoint address set to: net.tcp://localhost:7777/discoverableproxy. After you create the DiscoveryClient, you can add the events are mentioned earlier (FindProgressChanged, FindCompleted) to listen for the "Online" and "Offline" messages. To start the process you invoke the FindAsync() method accordingly

Here’s a code example:

   1: Console.WriteLine("Now using Managed proxy on net.tcp://localhost:7777/discoverableproxy");

   2:  

   3: DiscoveryEndpoint ept = new DiscoveryEndpoint(new NetTcpBinding(), new EndpointAddress("net.tcp://localhost:7777/discoverableproxy"));

   4:  

   5: client = new DiscoveryClient(ept);

   6:  

   7: client.FindProgressChanged += new EventHandler<FindProgressChangedEventArgs>(client_FindProgressChanged);

   8:  

   9: client.FindCompleted += new EventHandler<FindCompletedEventArgs>(client_FindCompleted);

  10:  

  11: client.FindAsync(new FindCriteria(typeof(ServiceRepositoryDataService)));

  12:  

  13: Console.ReadLine();

  14:  



Putting it all together: Summary

Once you take advantage of the WCF 4.0 ws-discovery mechanisms you can have clients listen for registered services, such that when service versions are registered with SO-Aware, a client can immediately be notified the service metadata is now in SO-Aware. In the upcoming version, the Service "Online" and "Offline" announcement features will be more exact and technically correct. We will do this by actually checking if the registered Service is truly online. SO-Aware can do this through it’s pinging mechanism, which is included in the current version of SO-Aware. It’s the "Supports Is Alive" option in SO-Aware. We’ll talk more about this option in another posting.

Figure 1 SO-Aware Pinging Mechanism – Supports Is Alive

So, to make a long story short, there are a ton of possibilities here. SO-Aware can integrate into WMI, and SCOM to provide for a realtime monitoring of services going "Online" and "Offline". SQL Reporting mechanisms can be built to monitor who adds service versions, who takes them offline, and other things. Sharepoint can be used and cataloged itself using it’s REST based services. Mobile devices can have clients created that go through SO-Aware to retrieve its configuration and all be notified when new services are available to take advantage of them.

You need to Become SO-AWARE!!!

Take a look at some of the screen shots:

Figure 2 Client Using Broadcast Ad Hoc Discovery

Figure 3 Client Using Managed Mode Discover through SO-Aware

Figure 4 Adding and Removing Service Versions in SO-Aware