May
23

Consuming OData on Windows Phone 7

odataOData is a great concept, it provides the strongly-typed data formats of SOAP-based services, with the low payload highly flexible REST-based services. It also brings its own twists such as URL based operators to allow for SQL-like querying, sorting, and filtering. This lightweight data protocol has seen great adoption through both Microsoft, and non-Microsoft based products on both the client, and server and also throughout popular sites such as Netflix, eBay, and StackOverflow.

The construct and design of the OData protocol lend it to also be a great solution for mobile application development and Microsoft have even created client SDKs and code generator tools for all of the major mobile platforms, including their very own Windows Phone 7. The use of OData on Windows Phone is however very much a confusing story at this points in time. The Windows Phone API, as you more than likely know is built upon the .NET Compact Framework running a cut down version of Silverlight. Due to this it does not have fully fledging support for the .NET framework and thus has caused some issues with the OData API’s being ported across. If you fired up Visual Studio and created a new ASP.NET application to access OData you can simply use the built-in Add Service Reference code generator (or via the command line DataSvcUtil.exe) and you’ll automatically be presented with strongly typed entities (in the same fashion as a SOAP-based service) and a fluent LINQ-style data context you can use for data entry. Unfortunately within Windows Phone 7 development we don’t have these luxuries (yet) which has meant guides to using the OData client are misleading and sometimes even wrong (this is because early CTPs and beta’s of the WP7 client SDK where different to the released version). So I’d like to take some time out to create a basic demo and show you how easy(ish) it is to get up and running with OData on Windows Phone 7.\

Getting Set-up with MVVM Light

So open up Visual Studio and create a new Windows Phone 7 Silverlight project (if you don’t have this template you can download it from here).

SNAGHTML2aa5232_634416408501332021

Once you have done that we need to create our framework skeleton for our phone application. As this is a Silverlight project we are going to use Model View ViewModel (MVVM), and currently I prefer the MVVMLight framework. So I’m going to add this in using Nuget

SNAGHTML2acde99_634416408538460497

Once MVVMLight has installed itself, you’ll notice it would have created a ViewModel folder, and added two classes MainViewModel.cs and ViewModelLocator.cs. We’ll need to wire these up to our application, so first open up App.xaml and add the following code:

<Application.Resources>   <!--Global View Model Locator-->   <vm:ViewModelLocator x:Key="Locator" d:IsDataSource="True" /></Application.Resources>

Then open up the code behind (AppCode.xaml.cs) and add the following:

private void Application_Launching(object sender, LaunchingEventArgs e){    DispatcherHelper.Initialize();}

 

You will also need to add the namespace for the ViewModel (vm) :

xmlns:vm="clr-namespace:ODataApplication.ViewModel" 

Next you need to wire up the ViewModel to to the View, so open up MainPage.xaml and add the following to the phone:phoneApplication node:

DataContext="{Binding Main, Source={StaticResource Locator}}"

 

Getting OData

So now we have our MVVM framework wired up, we need to somehow get our data plugged in. First create a directory within your projet called Service, this is where we will keep the OData classes.  Nexy we need to download the OData Windows Phone 7 client SDK. You’ll find this on CodePlex here. There are a few different flavours of the download, but the one you want is ODataClient_BinariesAndCodeGenToolForWinPhone.zip which contains both the necessary assemblies and also a customised version of the Code Generator DataSvcUtil.exe for use with the WP7 Silverlight runtimes .

Once downloaded, add both System.Data.Services.Client.dll and System.Data.Services.Design.dll as references to your project (remember to unblock the assemblies first).

Next open up a command prompt, and set a path to the location of where your downloaded copy of DataSvcUtil.exe is located. Now change the path to your location of your Visual Studio project, and the Service directory you just created. From the command prompt run DataSvcUtil.exe, which takes the following arguments:

/in:<file>               The file to read the conceptual model from

/out:<file>              The file to write the generated object layer to

/language:CSharp         Generate code using the C# language

/language:VB             Generate code using the VB language

/Version:1.0             Accept CSDL documents tagged with m:DataServiceVersion=1.0 or lower

/Version:2.0             Accept CSDL documents tagged with m:DataServiceVersion=2.0 or lower

/DataServiceCollection   Generate collections derived from DataServiceCollection

/uri:<URL>               The URI to read the conceptual model from

/help                    Display the usage message (short form: /?)

/nologo                  Suppress copyright message

We are going to use the following combination (NB: for this example we are using the NetFlix OData Service):

datasvcutil /uri:http://odata.netflix.com/Catalog /out:Model.cs /version:2.0 /DataServiceCollection

You will see the application execute, and it should complete with no errors or warnings.

image_634416408657802027

 

Now open up Visual Studio, and click the show hidden files icon, find the newly created file (Model.cs) and include it in your solution.

Using OData Models

The file that was just created actually contains multiple objects, including a data context for accessing data and also model representations of the OData entities. As we used the /DataServiceCollection prefix all of our models have been created to support the INotifyPropertyChanged interface so we can actually exposes them all the way through to our View. So what we’ll do next is set-up the data binding between the Model –> ViewModel –> View.  Open up your ViewModel (MainViewModel.cs) and create the following property:

private ObservableCollection<Genre> _genres;

public ObservableCollection<Genre> Genres{    get { return _genres; }    set    {        _genres = value;        RaisePropertyChanged("Genres");

    }}


Next (for simplicity) we’ll wire-up this property to the MainPage.xaml file using a ListBox that will repeat for each genre that is retrieved from the Netflix service and list the name out in a text block. Here is the XAML we need to implement:

<!--ContentPanel - place additional content here--><Grid x:Name="ContentPanel" Grid.Row="1" Margin="12,0,12,0">    <ListBox ItemsSource="{Binding Genres, Mode=OneWay}">        <ListBox.ItemTemplate>            <DataTemplate>                <TextBlock Text="{Binding Name, Mode=OneWay}" />            </DataTemplate>        </ListBox.ItemTemplate>    </ListBox></Grid>

Connecting OData EndPoint to a ViewModel

The last thing we need to do in our project is to populate our ViewModel with data. To do this we are going to create a new class within our Service directory, called NetflixServiceAgent.cs. For simplicity we’ll create a single method called GetGenres(Action<ObservableCollection<Genre>> sucess) which takes a single argument of an async Action to pass back data to our ViewModel. Populate the method with the following code:

public void GetGenres(Action<ObservableCollection<Genre>> success){    var ctx = new NetflixCatalog.Model.NetflixCatalog(new Uri("http://odata.netflix.com/Catalog/"));    var collection = new DataServiceCollection<Genre>(ctx);

    collection.LoadCompleted += (s, e) =>                                        {                                            if (e.Error != null)                                            {                                                throw e.Error;                                            }                                            else                                            {                                                if (collection.Continuation != null)                                                {                                                    collection.LoadNextPartialSetAsync();                                                }                                                else                                                {                                                    success(collection);                                                }                                            }                                };    collection.LoadAsync(new Uri(@"Genres/", UriKind.Relative));}    

What we are doing here is creating a new data context, which acts as an aggregate handling all of the interaction with the OData Service. Next we create a new DataServiceCollection (of the type of the Entity model we wish to return) and pass in the context to it’s constructor. DataSericeCollection is actually derived from ObservableCollection (the Collection type we use in Silverlight databinding) but it also contains methods to synchronous and asynch data retrieval as well as continuation where by the OData service is handling pagination. As with everything in Silverlight we are using the asynchronous data retrieval mechanism, and we pass in an anonymous method to the event handler which ensures an error has not been returned and then returns the collection. The actual LoadAsync method accepts an argument of type URI. This URI should be relative to the one we’ve already passed into the data context, and is representative of the both the entity and query we want to return. This is the primative form of the fluent-LINQ API we have in the full .NET framework, so for instance if we wanted to apply filters to the OData query we would change the URI to reflect this, such as “Genres/$skip=1&$top5”.

Once we have created our service agent, our final task to get us up and running is to call it from within the ViewModel. So open up MainViewModel.cs and within the constructor add the following:

var repo = new NetFlixServiceAgent();repo.GetGenres((success) => DispatcherHelper.CheckBeginInvokeOnUI(() =>                                                     { Genres = success; }));

And that’s it. By running the application we’ll see the not-so overly fancy list of genres being populated from the OData Service.

SNAGHTML306ef2e_634416408811776001

 

Over the next few weeks I’ll upload some more posts around OData clients and WCF Data Services to try and expand on this example so more.

May
08

WCF WEB API – Building Restful APIs

wcf_logo_634404094849306723When .NET 3 launched, Microsoft developers where introduced to a drastically new way of dealing with application communication. Windows Communication Foundation (WCF) had taken the web service concept we knew from ASP.NET and radicalised it drastically to cater for the most common networking stacks and communication protocols. We where no longer dealing with simply SOAP/XML services, our world was now opened up to TCP Steams, bi-directional networking traffic, and integration into existing networking components such as MSMQ. Yet the revolution that was .NET and its evolution in .NET 3.5 never supported the API concepts we gradually saw materialising within the Web 2.0 applications that where making their presence in not only our lives, our phones, but even our Hello World demos we would soon see in tutorials and presentations around the world.

RESTful Services

Made infamous by applications such as Twitter, the concept of Representational State Transfer (REST) is not necessarily new. First introduced in 2000 by Roy Fielding, one of the original authors of the HTTP specification, REST is an architecture based on client-server communication whereby clients initiate a request to a server, with the server processing requests and returning appropriate resource. Usually associated with HTTP, it is not necessary limited to the protocol however many of the RESTful API’s we see today do base themselves natively on the HTTP architecture. The primary reason for this is the HTTP protocol has a rich vocabulary of verbs used to define resources and associated service methods thus helping to confirm to one of the architectures core principals of Uniform Interfaces. The remaining RESTful principals of client – server , stateless, cacheable, and layered systems (see table 1 for definitions) are also tightly coupled to HTTP but can easily be adopted in other protocols such as  Remote Procedure Call (RPC).

Table 1 – RESTful Architecture Principals

  • Client – Server-Clients should be separated from a server by a uniformed interface. This separation of concerns means that clients are not concerned with data storage, which remains internal to each server so that the portability of each client code is improved. Server are not concerned with the user interface or user state, so that servers can be simpler and more scalable.
  • Stateless – The client-server communication is further constrained by no client context being stored on the server between requests. Each request from any client contains all of the information necessary to service the request, and any session state is held on the client. The server can be stateful; however this is constraint requires that server-side state by addressable by a URI.
  • Cacheable – Responses must implicitly or explicitly define themselves as cacheable or not prevent clients reusing stale or inappropriate data in response to further requests.
  • Layered system – A client cannot ordinarily tell whether it is connected directly to the end server, or to an intermediary along the way. Intermediary servers may improve systems scalability by enabling load balancing and by providing shared caches. They may also enforce security policies.
  • Uniformed interface – The uniform interface between client and server simplifies and decouples the architecture which enables each part to evolve independently.
  • When the popularity of RESTful services become apparent within the industry, it was unfortunate that .NET developers struggled somewhat with the architecture. WCF was never originally designed around with REST support in mind, and at the time the alternative of ASP.NET was not necessarily the best framework either, although the introduction of ASP.NET MVC did provide a stop-gap solution for many people. In late 2008 the WCF team released the WCF REST toolkit which extended the System.ServiceModel assembly that provided the underlying WCF engine and allowed service endpoints to become REST enabled. The solution was adequate for people familiar with the WCF architecture, but at the same time was complicated to get up and running and difficult to extend. It was for this, and other reasons that it was not widely adopted.

    With the introduction of .NET 4 the latest version of the WCF framework for the first time natively supported REST services.  WCF 4.0 simplified the deployment of REST services using the .NET framework, removing the new from .SVC files, service contract interfaces, and all of the configuration required within the web.config creating services now followed similar structure to that found in ASP.NET MVC routes,easily done with apply attribute to a class, and its methods and registering routes in the Global.asax. Simplicity was good, but unfortunately it also brought with it restrictions with extensibility and flexibility.

    WCF Web API

    In late October 2010, Glenn Block and the WCF Team provided us with the first insight in to the future of REST services within WCF through a CodePlex project currently known as WCF Web API. With the last version released within the past month at MIX 11 the new framework builds upon the simplicity of REST support within .NET 4 but provides us the the advanced support for customisation and personalisation of our REST APIs.  The primary difference between WCF WEB API and the REST Support is within its layered architecture (fig 1), (which in itself is a natural progression of the REST principals). Unlike the native support within .NET 4, the Web API channels all incoming request from its HTTP endpoint (or listener as it is called within this implementation) via a HTTP Channel Stack. At this low level within the service, extensions  to the framework can be injected (via configuration) to handle transport-level concerns such as security, logging or error handling. The HTTP Channel Stack is also responsible for the serialisation/deserialization of the incoming raw messages into the frameworks proprietary objects HttpRequestMessage. 

    Once the object is successfully passed through the channel stack it is then the responsibility of the dispatcher to ensure that the message is routed to the correct method on the service implemented by the developer. Within Web API the server interface can be defined by more than just the URI. The dispatcher natively supports all of the HTTP Verbs, it also has an extensibility point at this level for defining custom media types, called a HTTP Processor. The dispatcher is also responsible for de-serialising the body of the request message into .NET types.  With the combination of the URI, Verb, and Content Type the dispatcher will then pass the HttpRequestMessage  to the HTTP Service.

    The Http Service is the where the developer actually builds out the logic of the service. This resource is similar to a service contract in terms of traditional WCF, and even carry the same class attributes. As with WCF 4 native REST support, and other WCF implementations to expose methods to the service you decorate them with attributes.

    Figure 1 – Web API Architecture (source CodePlex) 

    image_thumb_3_634404094876763251 

    The final part of the Web API architecture is a client library for .NET closely tied to the HttpRequestMessage and HttpResponseMessage objects found within the Web API framework. Although traditional HTTP libraries such as WebRequest will work the WCF team have built this client to conform to a tighter representation of the HTTP specification to ensure it works with all HTTP REST calls, as well as provide a set of extension methods that handles the deserialisation of the HTTP Body to .NET types and classes.

    The extensible architecture are not the only bonuses provided by the Web API framework, as of the latest release their is now container support for IOC libraries which in turn provides better support for separation of concerns and testability. The configuration model now also uses a fluent API, which provides a cleaner, easier method of handling extensibility. Web API also supports IQueryable services in a similar fashion to WCF Data Services thus allowing data queries to be passed into the service using the OData URI protocol.

    Getting Started with Web API

    So I’ve talked a lot about simplicity, so how easy is it to get a REST service up and running? Well lets give it a go. First up Visual Studio, and open up a new Empty Web Application project. From there I am going to use Nuget (package name WebApi.All) to add the assemblies in to my project, but if you prefer just download the latest from the CodePlex site

     

    SNAGHTML5154e0a_634404094909211875

    Once you’ve added the assemblies the first thing we want to do is create some data classes so we have something for our service to return. So lets start by creating some domain objects around a store catalog:

    public class Category {     public string Description { get; set; }
    
         public string Name { get; set; }
    
         public List<Product> Products { get; set; }
    
     }
    
    public class Product   {       public int Id { get; set; }
    
           public string Name { get; set; }
    
           public string Description { get; set; }
    
           public double Price { get; set; }
    
       }

     

    Once we’ve done that we want to create a stub data context, and basic repository to allow us to have some data access flowing:

    public class CatalogContext   {       public IEnumerable<Category> GetData()       {           return new[]                      {                          new Category                              {                                  Description = "A selection of technology titles",                                  Name = "Technology",                                  Products = new List<Product>()                                                 {                                                     new Product {Id = 1, Description = @"Mixing provocative insights and cliched criticisms, Postman defines the U.S. as a society in which technology is deified to a near-totalitarian degree.", Name = " Technopoly: The Surrender of Culture to Technology", Price = 12.99},                                                     new Product {Id = 2 ,Description = @"Applying the lessons of history to modern-day dilemmas, Nye defies much common wisdom about the power of technology in society. With irony and wit, he exhorts us not to succumb to defeatist notions of technological determinism but to take charge of our own human destinies", Name = "Technology Matters: Questions to Live With ", Price = 10.76},                                                     new Product {Id = 3, Description = @"This historical account achieves its basic aim of demonstrating that, with the exception of quite recent history, technology has always influenced science, not the other way round.", Name = "Science and Technology in World History: An Introduction", Price = 17.82}                                                 }                              },                              new Category                              {                                  Description = "A selection of titles to read with the light on",                                  Name = "Horror",                                  Products = new List<Product>()                                                 {                                                     new Product {Id = 4, Description = @"Dracula begins with the journal of Jonathan Harker, a young solicitor on the way to Transylvania to give information to the mysterious Count Dracula about his new estate in London. Dracula takes the young man prisoner, and Jonathan sees many strange and evil things in the castle before escaping and fleeing into the night. He later decides that he must have been mad.", Name = "Dracula", Price = 15.95},                                                     new Product {Id = 5, Description = @"The story of Victor Frankenstein's monstrous creation and the havoc it caused has enthralled generations of readers and inspired countless writers of horror and suspense. Includes the author's own 1831 introduction.", Name = "Frankenstein", Price = 2.99},                                                     new Product {Id = 6,Description = @"This University of Nebraska Press edition is a small, exquisitely produced paperback. The book design, based on the original first edition of 1886, includes wide margins, decorative capitals on the title page and first page of each chapter, and a clean, readable font that is 19th-century in style. Joyce Carol Oates contributes a foreword in which she calls Dr. Jekyll and Mr. Hyde a mythopoetic figure like Frankenstein, Dracula, and Alice in Wonderland, and compares Stevenson's creation to doubled selves in the works of Plato, Poe, Wilde, and Dickens.", Name = "The Strange Case of Dr. Jekyll and Mr. Hyde ", Price = 1.59}                                                 }                              }                      };       }   }

     
     
    public class CatalogRespository    {        private readonly CatalogContext _context;
    
            public CatalogRespository()        {            _context = new CatalogContext();        }
    
            public IEnumerable<Category> GetCategories()        {            return _context.GetData();        }
    
            public Category GetCategoriesByName(string name)        {            return _context.GetData().First(x => x.Name == name);        }
    
            public IEnumerable<Product> GetProductsByCategory(string category)        {            return GetCategoriesByName(category).Products;        }
    
            public Product GetProductById(int id, string category)        {            return GetProductsByCategory(category).First(x => x.Id == id);        }    }

     
    So far, so good. As of yet we haven’t actually done anything relevant to Web API, so lets dive into the WCF component and create a service contract. Create another new class, and decorate it with the [ServiceContract] attribute.

     
    [ServiceContract]public class CatalogService{     ...}

     

    This is what you would see in a standard WCF service. The next thing we want to do is add some methods. For each method we want exposed to the REST service we also need to apply an attribute. For Web API this attribute is [WebInvoke()], and it accepts a few different properties. In the interests of simplicity the only two properties we are interested in is UriTemplate and Method.  Starting with the first, UriTemplate is a familiar property seen with ASP.NET MVC and is used for defining a URI to that particular resource. The URI template is a string that can consist of a route, and parameters, an example of this is “/Product/{id}” where Product is the route which is used to seperate resources, and {id} is mapped to the parameter the method accepts. You can also nest multiple parameters for a method in a UriTemplate too such as “{Category}/{ProductId}”. The next property we are going to use is Method, this is simply the HTTP Verb we want associated with our resource, and in this example we simply want to use “GET”.

    So inside our Service Contract lets construct some methods, and add the necessary attributes (you’ll also notice in my hold I am retrieving the data using the repository we created earlier):

    [ServiceContract]   public class CatalogService   {       private readonly CatalogRespository _repository; 
    
           public CatalogService()       {           _repository = new CatalogRespository();       }
    
           [WebInvoke(UriTemplate = "", Method = "GET")]       public IQueryable<Category> GetCategories()       {           return _repository.GetCategories().AsQueryable();       }
    
           [WebInvoke(UriTemplate = "{category}", Method = "GET")]       public Category GetCategoryByName(string category)       {           return _repository.GetCategoriesByName(category);       }
    
           [WebInvoke(UriTemplate = "/{category}/{id}", Method = "GET")]       public Product GetProductByCategoryAndId(string category, int id)       {           return _repository.GetProductById(id,category);       }   }

     

    So that is our service now finished. The last thing to do is to wire-up our routes. To do this add a new Global Application file (Global.asax) to the project. Once added we want to edit the Application_Start event and add in a Route Table object. This is exactly the same class as used by ASP.NET MVC (System.Web.Routing) , but Web API provides some extensions with the Microsoft.ApplicationServer.Http.Activation assembly. What this class essentially is going to do is intercept requests to the URI and instead of passing them to ASP.NET resources it is going to hand off to the HTTP Channel Stack as discussed earlier. This is done via the MapServiceRoute<T>() method,  so lets add this in to our file:

    public class Global : System.Web.HttpApplication   {       protected void Application_Start(object sender, EventArgs e)       {           RouteTable.Routes.MapServiceRoute<CatalogService>("catalog");       }
    
       }

     

    The MapRoute<T>() method accepts a type which should relate to our HTTP Service which we have defined by our service contract. The constructor parameter it accepts is the base resource URI which in our instance is “catalog”. With that we are done, so if you now fire up the application and open up the URI in the browser you should see the following:

    http://{yourURL}/catalog

    image_634404094966152970

     http://{yourURL}/catalog/Technology

    image_634404095010301819

    http://{yourURL}/catalog/Technology/1

    image_634404095041502419

    As you may have noticed on our base resource (/catalog) our method implemented the IQueryable interface, so this means we can actually implement the OData URI protocols to also sort out catalog, so try loading this URI in the browser to see the impact:

    http://{yourURL}Catalog?$top=1&$orderby=Name

    image_634404095089863349

    If you want to create a full CRUD solution, simply swap our the verbs you declared on the METHOD property of the WebInvoke attribute, and have you method accept your domain objects, or HttpRequest<T> where T is your domain object (depending on whether you need full access to the incoming request) Web API will handle the serialisation for you.

    So that’s all it takes to now get up and running with WCF and REST. It’s along way from where we’ve come from; it’s simple, it extensible, and you can be up and running in minutes. So I urge you now to go forth and experiment with the API and provide your feedback to the WCF team, so they can improve the process along the way – and hopefully make WCF 5 the frame work we want it to be.

    I’ll be posting a few more advanced posts on Web API in the coming weeks, so please stay tuned to my blog for more details. If you would like the source code for this sample project you can find it here.

    Apr
    25

    Using HTML5 Labs WCF WebSockets Prototype

    In December 2010 the Interoperability Strategy  Team at Microsoft made a pioneering attempt into the world of Web Sockets. For a while the team had been playing around with the emerging HTML5 specifications, and prototyping them in a sandbox environment they have affectionately entitled http://html5labs.interoperabilitybridges.com. As you more than likely know with the latest release of IE 9 many of the developed HTML5 standard had progressed out of the labs and into the RTM build of Microsoft’s latest web browser. Due to the delays faced in the ratification process to the TCP Web Socket protocol definition it was clear that Web Sockets would not make it into this release. The prototype does however, and is continually being developed with each new draft published by the IEFT.

    Based on the most recent WebSockets 06 specification, the prototype is actually a new WCF endpoint which is built on top of the existing WCF support for TCP bindings. When you download the prototype you will receive an installer which deploys the required assemblies and a whole bunch of demo solutions to help highlight the features of HTML 5 WebSockets.

    Getting Started

    So to get started with actually build a WebSocket service, first fire up Visual Studio. As this is only a prototype currently there is not support for hosting the WS service within IIS, so we are going to create a new Console Application and host the service ourselves.

    Once loaded up we need to add a reference the prototype assemblies, which are installed in C:\Program Files (x86)\Microsoft SDKs\WCF WebSockets\11.03.10\bin :

    SNAGHTML747b2a_634393597786781589

    As shown above, out of the 5 assemblies within the downloaded package, only 2 are relevant to the the actual server side implementation of WebSockets, the others are for WPF and Silverlight demos. As we are using WCF we also need to adda reference to the System.ServiceModel assembly from the GAC too.

    Once you have added all of the assemblies to the solution, you Solution Explorer within Visual Studio should now look like this

    image_634393597813613589

     

    Creating the Service

    Before we can host our service we need to create a concrete implementation of the service that will actually execute our code. To make this easier the WCF team have create an abstract WebSocketsService class which houses all of the relevant methods  allowed on the service, as defined within the IETF 06 specification. If you open up WebSocketsService in the Visual Studio Object Browser you will see the makeup of the class:

    image_634393597839197589

    To product a simple service that is capable of retrieving and sending a message from JavaScript we are going to focus on only two methods found within this class, OnMessage(string) and SendMessage(string). These method stubs are relatively self explanatory and are featured within the service to send and receive a UTF-8 encoded string to the client. As the client we intend to target is going to be the browser, and JavaScript currently doesn’t support byte the string methods are the only ones we can use.

    So to construct the service, create a new class which derives from Microsoft.ServiceModel.WebSockets.WebSocketsService

    public class ReverseEchoService : WebSocketsService   {
    
       }

    Next override the OnMessage(string) method, and to provide some logic we are simply going to take the incoming string, reverse it, and return it back to the client via the SendMessage(string) method.

     
    public class ReverseEchoService : WebSocketsService    {        public override void OnMessage(string value)        {            value.Reverse();            SendMessage(value);        }    }
     
    Hosting the Service
    Now our service is complete we are ready to host it. Much the same as other WCF services, you can embed the WebSocket prototype inside of an executable and expose the TCP endpoint however the limitations of the prototype do not allow us to configure the endpoint via an external config file so instead we are going create all of the configuration in code inside of our console app.
     
    Open up the Program.cs and inside the Main(string[] args) static method place the following code
     
    static void Main(string[] args)       {           var host = new WebSocketsHost<ReverseEchoService>(new Uri("ws://localhost:4502/reverseecho"));           host.AddWebSocketsEndpoint();           host.Open();           Console.ReadKey();           host.Close();       }
     
    Like with all self-hosted WCF endpoints we are simply creating a host container, which is generated from the prototype class Microsoft.ServiceModel.WebSockets.WebSocketsHost<T> and in doing so we are passing it the service type we have just created.  As one of the constructor arguments we can optionally establish a URI. As we are self-hosting, this will be required so we simply create a new URI object with passing it a string. Note we are using the new WS:// protocol which has been established as part of the IETF 06 draft spec . Typically we would be using port 80 as it is supported as part of the spec, but to avoid conflicts with other local applications we use a random port number for our URI.
     
    Once we have created our host, we then need to add an endpoint that is capable of receiving the incoming message.  In the case of the WebSocketsHost class, a method has been added to take care of this for us, so we simply call AddWebSocketsEndPoint() . When this has been done we can open us the service, and allow traffic to be received. As this is a console application, it would automatically close once all of the logic has been parsed, so to keep the service active we place a Console.ReadKey() method in to our code. One the console detects a key strike it will then close the application and service down.
     
    Building a Client

    So once our console is up and running we need to write a client that we can use to contact the service. Currently Internet Explorer does not support the WebSocket standard, however FireFox , iOS and Google Chrome both do on their current release (just be sure to check it is version 0.6 of the draft they have implemented). The actual client is Javascript working with an API built in to the browser engine and is not part of the prototype. We can build the Javascript based on the following implementation:

     

    var output;
    
    function init() {    output = document.getElementById("output");
    
        //Create a new Web Socket API    websocket = new WebSocket("ws://localhost:4502/reverseecho");
    
        //Handle the OnOpen event        websocket.onopen = function (evt) {        output("CONNECTED");        send("WebSocket rocks");    };
    
        //Handle the OnClose event    websocket.onclose = function (evt) {        output("DISCONNECTED");    };
    
        //Handle the OnMessage event    websocket.onmessage = function (evt) {        output('<span style="color: blue;">RESPONSE: ' + evt.data + '</span>');        websocket.close();    };
    
        //Handle the onError Event    websocket.onerror = function (evt) {        output('<span style="color: red;">ERROR:</span> ' + evt.data);    };}
    
    //Send a message back to the servicefunction send(message) {    output("SENT: " + message);    websocket.send(message);}
    
    //Output the message to the screenfunction output(message) {    var pre = document.createElement("p");    pre.style.wordWrap = "break-word";    pre.innerHTML = message; output.appendChild(pre);}
    
    window.addEventListener("load", init, false); 

     

    The import sections here to note a first the creation of the WebSocket connection, to do so we simply create a new WebSocket object from the browser API (which is part of the W3C HTML 5 WebSocket specification), and pass it the URL of our service. Next we attach method to catch the events that the API create. Significantly we want to receive our message and output it. To do so we simply use the websocket.onmessage event and write the content back to the DOM. If we want to send a message to our server (so in our case we receive a response) we use websocket.send. Bot the method/event receive send and receive arguments that are simply a UTF-8 string so we can freely transmit JSON,XML,SVG  or any other text based data type across the wire.  There are a few other events we may want to consider if we where building a production implementation, and so it is worth reading up in the spec the details of the API before building anything in a live environment.

    Graceful Degradation

    Although the WCF Services are a prototype, we are starting to see HTML5 WebSocket service enter into production via other providers. As not all browsers support the WebSocket specification it is worth considering how you can open up the functionality to a wider user-base. One option would be to run to end-points, with an alternative to WebSockets being a class HTTP connection using AJAX functionality. This would however require two clients, and two services to be built. Fortunately the team who developed the WCF prototype also considered the graceful degradation of WebSockets across browsers, such as Internet Explorer  and have provided a solution. Included within the installer for the WCF WebSockets prototype is a Silverlight .XAP file, and some Javascript snippets, the purpose of which is if the browser does not natively support WebSockets the script will instead load the Silverlight plugin to act as a gateway and broker the exchange of data.

    Some Things to Remember

    Currently the WCF WebSocket prototype is functional, and meets the IETF 06 specification – however it has its limitations. As a web server solution, we would ideally want IIS support, and the ability to configure via .config files. The specification also calls for WebSocket connections to be made via a HTTP Upgrade request, which is currently not built out of the box within the prototype and would be an easier solution to construct if both services are within the realms of the web server. It is also extremely important to note that HTML5 Web Sockets is still an evolving specification and can change regularly, so anything you build with one version of the prototype may be prone to breaking changes within the next. WebSockets however are an extremley easy to use, light weight method of send real-time data to both rich clients and the browser and I encourage you to explore this toolkit and get used to the possibilities we will start seeing materialise over the coming months,

    Apr
    25

    HTML 5 Web Sockets: The Real Time Web

    html5_logo_128_634393295476156017The latest buzzword on the Internet has got to be HTML5. Unlike previous W3C standards, HTML5 is not simply mark-up guidelines but more importantly an expansion and standardization of rich media across Internet Browsers. Having heavy investment from companies such as Mozilla, Microsoft, and Google, and even though it is more than 10 years away from final ratification HTML5 is extremely dominant in today’s browsers wars providing competition through plug-in free experiences whilst still offering the full rich internet (RIA) experience we have come to know and enjoy on the Internet today.

    The HTML5 specification covers the rendering of media such as video and audio, semantic mark up, as well as browser specific APIs such as geo-location and off-line capabilities through browser storage. In conjunction with the IETF the W3C have also considered how the next generation of Internet sites should feed data to these rich browser experience, without the overhead that exists in today’s browsers through the complex nature of AJAX API’s or long running HTTP requests. This has been delivered via the W3C Web Sockets API draft in conjunction with the IETF Hybi working group WebSocket Protocol.

    The Past, the Present – HTTP Requests.

    To truly understand the benefits Web Sockets play in the future of Internet Browsing we should take a moment to look at where we are, and where we have come from when we build our webpages today. Enter HTTP.

    Prior to the Web 2.0 revolution, getting data from our web servers was pretty simple. On loading a web page, we would make a HTTP request from the browser, to the server, and the server would acknowledge the request and return with a response. Typically this response would be HTML to be parsed by the browser which may include links to other resource which are required to help render the page so the process starts once again to retrieve images, style sheets, etc.

    Progressively the number of devices and application we had connected to the Internet grew, and the demands and requirements we had for accessing resources from the server increased and diversified yet we are still tied to using the 20 year old HTTP specification to access our servers. To cater for change we altered our implementation, leveraging off newly formed Javascript API’s that was able to make smaller HTTP requests in an asynchronous manner, interrupt the response and manipulates and update the browser DOM to give the appearance of a real time, and responsive application. The asynchronous JavaScript (AJAX) is one of the most common implementations of the HTTP Polling.

    image_634393295480776281

    HTTP Polling is exactly the same HTTP request/response pattern that are used by the browser for retrieving the HTML but fired off in a short-interval frequency. This solution is optimal for retrieving small packets of data that change quite regularly, such as a stock-ticker as the requesting client can then handle synchronization and collation of the changes. Yet there are very few scenarios where real-time data is this predictable, and thus bombarding the server was regular requests can use unnecessary resources and cause performance issues within the application.

    If we know the retrieval of data is going to be completely ad-hoc, such as the receipt of e-mail to an SMTP server, it would be better practice to establish a HTTP request and allow the server to hold on to it, until either an event occurs (such as an e-mail arrives) or the request times out. Although long-running polling does not offer better performance on the server (and could possibly detriment performance due to an excess number of resources being held on to) it does reduce network traffic and load on the client as we are reducing the number of requests we are firing off and simply handling the receipt of events.

     

    image_634393295481826341

    The last implementation we could consider for improving network load, and client resource efficiency would be to remove the need for the client to make continual requests. By establishing a stream of information from the server the client would only need to make a single HTTP request, to which the server would retain and intermittently pass updates to the client. HTTP Streaming  or Comet implementations offer the most efficient way of “pushing” data to the client with reduced network latency, network traffic, and client load – yet HTTP Streaming is still HTTP and still have to negotiate proxy servers and firewalls which can buffer messages  add unnecessary latency to streams due to this the majority of Comet  servers do fall back to long-running  polling to prevent this.

     

    image_634393295482686390

    The Problems with HTTP

    When working with services the primary goal is to remove network latency, reduce the network payload. Unfortunately though  when Berners-Lee and his team designed the original implementation of HTTP it was never engineered to be serve real-time data, or offer full-duplex connectivity. The biggest overhead introduced by HTTP is the unnecessary size of the payload being sent back and forth. As part of the HTTP specification a header is attached to each request to, and response from the server and is a way of defining the operating parameters of the transaction. When this negotiation is being handled by the browser headers are critical in maintaining the key elements of HTTP such as session state, compression, cookies, etc. However when we simply wish to request a trivial piece of data back from the server we are still obliged by the HTTP protocol to have the header information sent via the network each time.

    To see the impact of this yourself you can open up a HTTP Proxy, such as Fiddler and examine the traffic that your site/services emit. You are more than likely going to see something like this:

    HTTP Request:

    GET /PollingStock/PollingStock HTTP/1.1 Host: localhost:8080 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://www.example.com/PollingStock/ Cookie: showInheritedConstant=false; showInheritedProtectedConstant=false; showInheritedProperty=false; showInheritedProtectedProperty=false; showInheritedMethod=false; showInheritedProtectedMethod=false; showInheritedEvent=false; showInheritedStyle=false; showInheritedEffect=false

    HTTP Response:

    HTTP/1.1 200 OKCache-Control: privateContent-Type: text/plain; charset=utf-8Server: Microsoft-IIS/7.5X-Powered-By: UrlRewriter.NET 2.0.0X-AspNet-Version: 4.0.30319X-Powered-By: ASP.NETDate: Mon, 25 Apr 2011 00:07:02 GMTContent-Length: 19471

    To put this in context what you see above is 1204 bytes of data being transferred across the network to retrieve a 20 bytes JSON or XML request containing stock information which equates to approximately 6000% extra network traffic per request. When you multiply this across multiple requests, and multiple concurrent sessions the numbers soon start to add up. 

    HTTP also suffers from an increase in latency due to its request/response nature. If you remember the above real-time data architecture implemented with HTTP, all but the last required a request to be sent prior to a response being received so if a message from your client takes 10ms to arrive to the server, it is going to take at least 10ms for the server to respond to the client thus causing a delay of double the amount of network latency involved.

    The Future – HTML 5 Web Sockets

    To help solve both the issues of the complicated client-server API, and the limitations of the HTTP protocol the W3C and IETF are developing a new-real time data method commonly known as Web Sockets. Defined within the communications section of the HTML 5 specification Web Sockets are a true full-duplex communication channel, capable of sending both UTF-8 string and binary frames in any direction at the same time, and operating from a single socket across the web.

    As Web Sockets are part of HTML5 the application of the client interface will become native to all modern browsers, removing the complexities of the client/server architectures we currently have with HTTP. The W3C have already documented the JavaScript API for Web Sockets and it is now going through the ratification process. To establish a Web Socket connection, the browser or client simply makes a request to the server for an upgrade from HTTP to a Web Socket protocol. NB: As this is a HTTP request, cookie-based authentication for the resource can still be applied.

    HTTP Upgrade Request

    
    GET /text HTTP/1.1Upgrade: WebSocketConnection: UpgradeHost: localhost
    
    
    

    HTTP Upgrade Response

    HTTP/1.1 101 WebSocket Protocol HandshakeUpgrade: WebSocketConnection: Upgrade

    Once the connection has been established the socket is moved to a Web Socket TCP connection, with its own protocol of ws:// (or wss:// for secured sockets). One of the crucial features of the WS protocol is that its runs via port 80 (or 443 for WSS) making it proxy server and firewall friendly and will work with current network configurations.   The WS TCP connection has an overhead of only 2 bytes per frame, some 60,000% less than the headers involved in the HTTP connection. Once the socket is open it is also capable of sending both UTF-8 text based data, such as JSON, XML, HTML etc. or binary between client and server (although the Javascript currently doesn’t support raw binary data, so currently ignores the message).

    var webSocket = new WebSocket("ws://localhost"); 
    
    //Events for receiving datamyWebSocket.onopen = function(evt) { alert("Connection open ..."); }; myWebSocket.onmessage = function(evt) { alert( "Received Message: " + evt.data); };myWebSocket.onclose = function(evt) { alert("Connection closed."); }; 
    
    //Events for sending datamyWebSocket.send("Hello Web Sockets!");myWebSocket.close();

    The actual client/browser API is purely event based too. As seen with the JavaScript code snippet, there is no requirement for the browser to issue requests for updating of data which removes the latency we suffered from our previous HTTP requests. Also as we are not continually polling, the server resources used within the service are now reduced.

    The value of the Web Socket protocol in terms of efficiency has promoted interest from large industry players with Google openly remarking:

     “Reducing kilobytes of data to 2 bytes…and reducing latency from 150ms to 50ms is far more than marginal. In fact, these two factors alone are enough to make Web Sockets seriously interesting to Google.”

    The simplicity of the API does however does lead to some extra planning when working on implementing the primarily around keep-alive, timeouts, and message queuing/retries. In the ever-growing world of partially connected devices the need to deal with these scenarios are ever likely however the W3C API does not cater for any of these scenarios so extra planning, and use of the OnClose event need to be carefully considered upon implementation.

    The Future – is still the future

    The simplicity of the Web Socket specification, along with its enormous benefits in real-time data service design has meant a lot of people have been keen to jump on and develop alongside the emerging specification. Usually this innovation is applauded, however in the case of Web Sockets caution is actually being urged. As two organisation are jointly-developing this standard there has been some unfortunate delays in the ratification and standardisation process. Most notably the IETF has recently revised the Web Socket upgrade request approach based on advise from an independent security review of the protocol. It was for this reason Mozilla since disabled its support for HTML5 web sockets within the recent release of Firefox 4.

    It would also be highly advisable when developing with web-sockets to consider graceful degradation for browsers that do not natively support the protocol. This could be done with alternative server side implementation, using some of the HTTP methodologies seen above, or with plugins such as Flash and Silverlight to act as a gateway client between the browser and the server.

    The actual release of Web Sockets will be intertwined the official release date of the HTML5 standard – which at the time of writing is not for another 10 years. Yet like all standards, the best time for implementation will be when all of the major browsers start to implement – which will certainly be sooner than the official release. As the list will be ever changing of who support which HTML5 standard, the best source of information would be: Comparison of layout engines HTML 5 .

    Summary

    The HTML5 standards over the next decade will help revolutionise the Internet once again. We are moving from an Internet designed 20+ years ago with the purpose of single request/response of resources to a rich internet application with a constant need to access real time data, quickly, efficiently,  from any device. The HTTP protocol have got us so far, however it has now been recognised by the W3C and IETF that to efficiently get data to the new breed of rich clients a simple and more efficient way of transporting and accessing data was required. HTML 5 Web Sockets offers us just this, with a low overhead network protocol using existing HTTP ports in conjunction with native browser event driven API’s accessing real-time streams of data will be standard practice within the next-generation of Internet Applications.

    Oct
    29

    Augmented Reality–coming to a mirror near you soon

    Earlier this year I spent a lot of time talking to some friends and colleagues in the industry around where I see augmented reality heading within the retail industry.

    From my Microsoft background I had recently been playing with technology such as Microsoft Surface, and the rich multi-touch controls within Windows 7 and WPF 4. Around the same time Microsoft was also starting to promote its Kinect, XBox’s camera, motion sensor and augmented reality offering.

    When you spend a lot of time playing with such technology, you start to wonder what it would be like to piece them together and how that would fit in to a retail scenario, which would provide both fun and business value

    The best solution I came up with at the time was an augmented reality mirror, which couples the multi-touch capabilities of Surface and Windows 7 and couples it with the augmented reality you see within the Kinect prototype videos. I thought the idea had merits, so started to shop around to see what was on the market already. The first thing I came across was an offering by Lit Studios:

    All though this is impressive, it more forms the basis of a multi-touch screen with a reflective background. It would have merits in a retail environment with the ability to display information, and the user may interact with the mirror but the abilities I felt where still limiting. The abilities now of augmented reality should now allow you to interact in an immersive and social experience with this mirror, and not simply augment extra information and user interaction. If this mirror was simply a big touch screen monitor connected to a PC, why couldn’t we super impose clothing for cross-sell and up-sell purposes. The experience would very much be the screen could recognise the garment you are trying on, and could suggest accessories and super impose them into the image. The social experience also call for consumers to be able to engage outside of simply the mirror, so why couldn’t the PC also capture your image, post it to Facebook and provide real time and immediate feedback?

    Shoppers can digitally "try on" clothes at Macy's flagship store in New York.This was 6 months ago, when I first started looking into this project. My aim was to convince someone to fund the prototype, so I could show case the experience at conferences – but unfortunately I failed . So it was much to my surprise when earlier this week CNN announced (http://edition.cnn.com/2010/TECH/innovation/10/14/macys.virtual.mirror/index.html)  Macy’s the New York department store launching a similar idea. The Macy’s mirror provides a full augmented reality experience for trying on clothes, and posting the images back and forth to Facebook. The experience they have achieve is amazing, and certainly adds value and direction into where I believe the retail industry is heading.

    I do ask myself however how far can we go with this experience? If I have access to augmented reality, and real-time photos of my customers, why not link loyalty programs to social networking sites? If I stand in front of the mirror the camera, and PC should use facial recognition to link me to my Facebook profile. The social links should also be able to connect me to real time trends between my network and provide cross-sell information on not just from a stores perspective but from my peers and their purchases.

    This may all seem overly complex, and in some ways scary – but the technology to achieve these types of retail experiences are common place in todays IT Market. My personal believe if we will see a greater shift in the merging of our digital and physical lives, and this will play out very shortly in retail stores globally over the next few years. The difficulty now is no longer technical limitations, but a paradigm shift for retailers who are still very stuck in the physical world.

    This is just my two cents, or a much larger industry debate.. but I’m sure more fuel can be added to the fire – in the mean time I’ll carrying on plotting more ideas…

    Older posts «