Showing posts with label Silverlight. Show all posts
Showing posts with label Silverlight. Show all posts

Sunday, May 6, 2012

Running RIA Services on the top of Raven DB

In this post I will describe how to integrate RIA Services with document database called Raven DB. If you are not familiar with those frameworks watch this video and this one (my 2 presentations in polish). RIA Services integrates very easily with relational database by providing base implementations of domain services out of the box. Using the same pattern I have created the base implementation for services working with Raven DB. We will see later in this post what exactly the domain service is and how we can use it to build n-tier business application, but now let’s look at the Raven DB API. Let’s say we have the following data model:

public partial class Person 
{
public string Id { get; set; }
public string Name { get; set; }

public string Biography { get; set; }
public string Photos { get; set; }
public double AverageRating { get; set; }
}

public partial class Movie
{
public string Id { get; set; }

public string Name { get; set; }
public string Synopsis { get; set; }
public double AverageRating { get; set; }
public int? ReleaseYear { get; set; }

public string ImageUrl { get; set; }

public List<string> Genres { get; set; }
public List<string> Languages { get; set; }

public List<PersonReference> Cast { get; set; }
public List<PersonReference> Directors { get; set; }

public List<Award> Awards { get; set; }

public int CommentsCount { get; set; }
public string CommentsId { get; set; }
}

public class PersonReference
{
public string PersonId { get; set; }
public string PersonName { get; set; }
}

public class Award
{
public string Type { get; set; }
public string Category { get; set; }
public int? Year { get; set; }
public bool Won { get; set; }

public string PersonId { get; set; }
}

With this we can do some base CRUD operation:

using (var store = new DocumentStore() { Url = "http://localhost:8080", }.Initialize())
{
using (var session = store.OpenSession())
{
var davidDuchowny = new Person() { Name = "David Duchovny" };
var demiMoore = new Person() { Name = "Demi Moore" };
session.Store(davidDuchowny); // Id property is set here
session.Store(demiMoore); // Id property is set here

var theJoneses = new Movie()
{
Name = "The Joneses",
Synopsis = "A seemingly perfect family moves into a suburban neighborhood, but when it comes to the " +
"truth as to why they're living there, they don't exactly come clean with their neighbors.",
ReleaseYear = 2009,
AverageRating = 6.5,
Genres = new List<string>() { "Comedy", "Drama" },
Cast = new List<PersonReference>()
{
new PersonReference() { PersonId = davidDuchowny.Id, PersonName = davidDuchowny.Name},
new PersonReference() { PersonId = demiMoore.Id, PersonName = demiMoore.Name},
}
};
session.Store(theJoneses); // Id property is set here

session.SaveChanges();
}

using (var session = store.OpenSession())
{
var movie = (from m in session.Query<Movie>()
where m.Name == "The Joneses"
select m).FirstOrDefault();
var people = session.Load<Person>(movie.Cast[0].PersonId, movie.Cast[1].PersonId);

session.Delete(movie);
session.Delete(people[0]);
session.Delete(people[1]);

session.SaveChanges();
}
}

Raven DB stores data as a JSON documents:

image

Once we know how the Raven DB works we can look at the RIA Services. RIA Services gives us tools (Visual Studio extensions) and libraries that allow us to build n-tier business solution with the Silverlight or JavaScript application as a client and .Net Web Service as a server. We start building RIA Services solution from the domain service which is just a class deriving directly from DomainService or any other derived class depending on the type of a data store.

image

If the relation database is a data store then we can use LinqToSqlDomainService, LinqToEntitesDomainService or DbDomainService (LINQ to Entities Code First approach). TableDomainService class allows us to store data inside Windows Azure Table. If we want to store data inside any other kind of data source we have to derive directly from the DomainService class. My base service called RavenDomainService derives from DomainService and it encapsulates the logic of initializing and saving the data inside Raven DB. Let’s look at the sample usage of this class:

[EnableClientAccess]
public class NetflixDomainService : RavenDomainService
{
protected override IDocumentSession CreateSession()
{
var session = base.CreateSession();
session.Advanced.UseOptimisticConcurrency = true;
return session;
}

public IQueryable<Person> GetPeople()
{
return Session.Query<Person>();
}
public void InsertPerson(Person entity)
{
Session.Store(entity);
}
public void UpdatePerson(Person entity)
{
var original = ChangeSet.GetOriginal(entity);
Session.Store(entity, original.Etag.Value); // optimistic concurrency
}
public void DeletePerson(Person entity)
{
var original = ChangeSet.GetOriginal(entity);
Session.Store(entity, original != null ? original.Etag.Value : entity.Etag.Value); // optimistic concurrency
Session.Delete(entity);
}

public IQueryable<Movie> GetMovies()
{
return Session.Query<Movie>();
}
public void InsertMovie(Movie entity)
{
Session.Store(entity);
}
public void UpdateMovie(Movie entity)
{
var original = ChangeSet.GetOriginal(entity);
Session.Store(entity, original.Etag.Value); // optimistic concurrency
}
public void DeleteMovie(Movie entity)
{
var original = ChangeSet.GetOriginal(entity);
Session.Store(entity, original != null ? original.Etag.Value : entity.Etag.Value); // optimistic concurrency
Session.Delete(entity);
}
}

After adding “WCF RIA Services link” from the Silverlight project to projects containing domain service implemented above and rebuilding the whole solution, Visual Studio will generate appropriate code on the client side and we are ready to write code very similar to that presented earlier:

async public void BlogPost()
{
var context = new NetflixDomainContext();

var davidDuchowny = new Person() { Name = "David Duchovny" };
var demiMoore = new Person() { Name = "Demi Moore" };
context.Persons.Add(davidDuchowny);
context.Persons.Add(demiMoore);

await context.SubmitChanges().AsTask(); // Id property is set here

var theJoneses = new Movie()
{
Name = "The Joneses",
Synopsis = "A seemingly perfect family moves into a suburban neighborhood, but when it comes to the " +
"truth as to why they're living there, they don't exactly come clean with their neighbors.",
ReleaseYear = 2009,
AverageRating = 6.5,
Genres = new List<string>() { "Comedy", "Drama" },
Cast = new List<PersonReference>()
{
new PersonReference() { PersonId = davidDuchowny.Id, PersonName = davidDuchowny.Name},
new PersonReference() { PersonId = demiMoore.Id, PersonName = demiMoore.Name},
}
};
context.Movies.Add(theJoneses);

await context.SubmitChanges().AsTask(); // Id property is set here


var context2 = new NetflixDomainContext();

var movie = (await context2.Load
(
from m in context2.GetMoviesQuery()
where m.Name == "The Joneses"
select m
)).FirstOrDefault();

var people = (await context2.Load
(
from p in context2.GetPeopleQuery()
where p.Id == movie.Cast[0].PersonId || p.Id == movie.Cast[1].PersonId
select p
)).ToArray();

context2.Movies.Remove(movie);
context2.Persons.Remove(people[0]);
context2.Persons.Remove(people[1]);
await context2.SubmitChanges().AsTask();
}

RIA Services gives us a very nice implementation of Unit Of Work and Identity Map patterns so the code using DAL frameworks like LINQ to SQL, LINQ to Entities, NHibernate or Raven DB is very similar to that written on the client side. There is one thing worth mentioning at this point, RIA Services needs to know which property identifies the entity object. I didn’t show you how to do it yet but it will be presented in a moment, I am writing about it here because without this little hint, the code above wouldn’t even compile.

The last thing I would like to explain is how the optimistic concurrency pattern has been implemented in case of Raven DB. Raven DB has metadata mechanism which means that each JSON document besides the actual data has additional information like document type, last modification time, etag and so on. Etag is a single value of type Guid and its goal is very similar to Timestamp or Row Version column data type in SQL Server. Raven DB server changes the value of Etag internally every time the document is changing. With this Raven DB client API allows us to enable optimistic concurrency checking  on the session object level via UseOptimisticConcurrency property. There is one problem when it comes to integration with RIA Services optimistic concurrency mechanism. Etag is part of the metadata instead of document itself and it is handled by the session object internally. RIA Services doesn’t have any notion of metadata information, all data is stored inside the entity objects so we need to add additional property storing etag value which will be used on the RIA Services level but on the Raven DB level this value will be mapped into etag metadata field.

public interface IEtag
{
Guid? Etag { get; set; }
}

[MetadataTypeAttribute(typeof(PersonMetadata))]
public partial class Person : IEtag
{
[JsonIgnore] // do not store value of this property
public Guid? Etag { get; set; }

public class PersonMetadata
{
[Key]
public object Id { get; set; }

[RoundtripOriginal] // send back value of this property to client
public object Etag { get; set; }
}
}

[MetadataTypeAttribute(typeof(MovieMetadata))]
public partial class Movie : IEtag
{
[JsonIgnore] // do not store value of this property
public Guid? Etag { get; set; }

public class MovieMetadata
{
[Key]
public object Id { get; set; }

[Display(Name="nazwa")]
[Required]
[StringLength(200)]
public object Name { get; set; }

[Range(1900, 2012)]
public object ReleaseYear { get; set; }

[RoundtripOriginal] // send back value of this property to client
public object Etag { get; set; }
}
}

Attributes like Key, Display, Required, Range, RoundtripOriginal and MetadataType are a part of the standard .net mechanism called data annotations. RIA Services similar to other frameworks like ASP.MVC or Dynamic Data uses them to define validation rules or UI controls in layout in declarative way. Key attributes tell RIA Services which property identifies the entity object, RoundtripOriginal attributes tell that the value of the property is changed on the server side so the new value should be sent back to the client after calling server method. JsonIgnore is specific to Raven DB and it means that this property should be ignore during serialization and deserialization process. IEtag interface was introduced as a marker interface so all entities implementing this interface are treated specially by listener code. Listeners are a part of Raven DB client API and we can think of them as of a client side triggers executed in various situations.For instance during JSON document conversion process (IDocumentConversionListener) or before and after document is stored inside Raven DB (IDocumentStoreListener).

public class Global : System.Web.HttpApplication
{
private static IDocumentStore _documentStore;

protected void Application_Start(object sender, EventArgs e)
{
var etagSetterListener = new EtagSetterListener();

_documentStore = new DocumentStore() { Url = "http://localhost:8080", }
.RegisterListener(etagSetterListener as IDocumentConversionListener)
.RegisterListener(etagSetterListener as IDocumentStoreListener)
.Initialize();

DomainService.Factory = new RavenDomainServiceFactory(_documentStore, DomainService.Factory);
}
}

public class EtagSetterListener : IDocumentConversionListener, IDocumentStoreListener
{
#region IDocumentConversionListener

public void EntityToDocument(object entity, RavenJObject document, RavenJObject metadata)
{
}

public void DocumentToEntity(object entity, RavenJObject document, RavenJObject metadata)
{
var etag = entity as IEtag;
if (etag != null)
{
etag.Etag = Guid.Parse(metadata["@etag"].ToString());
}
}
#endregion


#region IDocumentStoreListener

public bool BeforeStore(string key, object entityInstance, RavenJObject metadata)
{
return false;
}

public void AfterStore(string key, object entityInstance, RavenJObject metadata)
{
var etag = entityInstance as IEtag;
if (etag != null)
{
etag.Etag = Guid.Parse(metadata["@etag"].ToString());
}
}

#endregion
}


public class RavenDomainServiceFactory : IDomainServiceFactory
{
private readonly IDocumentStore _documentStore;
private readonly IDomainServiceFactory _domainServiceFactory;

public RavenDomainServiceFactory(IDocumentStore documentStore, IDomainServiceFactory domainServiceFactory)
{
_documentStore = documentStore;
_domainServiceFactory = domainServiceFactory;
}

public DomainService CreateDomainService(Type domainServiceType, DomainServiceContext context)
{
var service = _domainServiceFactory.CreateDomainService(domainServiceType, context);

if (service is RavenDomainService)
{
((RavenDomainService)service).DoumentStore = _documentStore;
}

return service;
}

public void ReleaseDomainService(DomainService domainService)
{
_domainServiceFactory.ReleaseDomainService(domainService);
}
}

IDomainServiceFactory interface allows us extend the process of creation and destruction of domain services instance on the server side. Finally let’s see how the RavenDomainService class has been implemented:

public abstract class RavenDomainService : DomainService
{
public IDocumentStore DoumentStore { get; set; }

private IDocumentSession _session;
protected IDocumentSession Session
{
get
{
if (_session == null && DoumentStore != null)
{
_session = CreateSession();
}
return _session;
}
}

private IDocumentSession _refreshSession;
private IDocumentSession RefreshSession
{
get
{
if (_refreshSession == null && DoumentStore != null)
{
_refreshSession = CreateSession();
}
return _refreshSession;
}
}

protected virtual IDocumentSession CreateSession()
{
var session = DoumentStore.OpenSession();
return session;
}

protected override void Dispose(bool disposing)
{
if (disposing)
{
if (_session != null)
{
_session.Dispose();
_session = null;
}
if (_refreshSession!= null)
{
_refreshSession.Dispose();
_refreshSession = null;
}
}
base.Dispose(disposing);
}

protected override int Count<T>(IQueryable<T> query)
{
return query.Count();
}

protected override bool PersistChangeSet()
{
try
{
this.Session.SaveChanges();
}
catch (ConcurrencyException exception)
{
var items = base.ChangeSet.ChangeSetEntries
.Where(changeSet =>
{
var etag = changeSet.Entity as IEtag;
if (etag == null)
return false;
return etag.Etag == exception.ExpectedETag;
})
.ToArray();

foreach (var item in items)
{
var o = ChangeSet.GetChangeOperation(item.Entity);
if(o == ChangeOperation.Delete)
{
item.IsDeleteConflict = true;
}
else
{
item.ConflictMembers = new[] { "unknown" };
var id = Session.Advanced.GetDocumentId(item.Entity);
item.StoreEntity = RefreshSession.Load<object>(id);
}
}

this.OnError(new DomainServiceErrorInfo(exception));

if (!base.ChangeSet.HasError)
{
throw;
}

return false;
}

return true;
}
}

I know, I know, I hear you saying: "What did you do all this for ??? Raven DB gives you a wonderful client API for Silverlight". I totally agree with you. I am not trying to say that you should use RIA Services instead of Raven DB directly via its Silverlight API. Those two frameworks are slightly different things. They have a different functionalities, they are designed to resolve a bit different kinds of problems. Raven DB is a database system so it is focused mainly on data storage. RIA Services solves many common problems for a business n-tier application like server and client side validation, easy integration with DataGrid or DataForm UI controls, sharing common code between client and server, marshaling exceptions and so on. For me, those two frameworks can work great together and such cooperation seems to be very reasonable. I was searching google for any samples showing how to integrate them but I couldn’t find it. So I did my own !:) I hope you find it useful.

Download

Friday, February 10, 2012

RIA Services (and Raven DB) presentation

This presentation is an introduction to RIA Services framework. Most tutorials show how to integrate RIA Services with relational database using DAL frameworks such as LINQ to SQL, LINQ to Entities or NHibernate. This presentation shows how to integrate RIA Services with document database Raven DB. Source code and slides can be found here.

Thursday, September 15, 2011

Design-time support for Caliburn.Micro

Recently me together with Bartek Pampuch have been wondering if it’s possible to fire Caliburn.Micro’s binding process at design-time. Caliburn.Micro is a really great framework (just take a look at some of our multitouch apps based on this framework and BFSharp). You just name the controls appropriately and all magic happens automatically. Controls are bound to the View Model’s properties and methods for you. The problem is that it is all happing at run time not at design time. When you are preparing the form in the Visual Studio or Expression Blend you aren’t able to see how the form will look like with bound data. In some cases such feature would very useful, especially when we want to use the sample data generated by Expression Blend or Visual Studio designers. In this post we will show you how to run Caliburn.Micro’s conventions based binding mechanism at design time.

Firstly, we discovered that we can change the objects tree representing the screen and that change is not persisted back to xaml file. You can read more about this feature in my previous post. Secondly, we tried to run Caliburn.Micro’s binding process to see what happens. What was really amazing is that after first try all just started working smoothly! The framework itself is implemented so well :)

One of the samples provided with Caliburn.Micro release is a very simple application called GameLibrary. AddGameView.xaml file defines the screen that looks like this:

image

After setting up sample data generated by Visual Studio and setting attached property 'DesignTime.Enable=”True” the same form looks like this:

clip_image001

In the simplest scenario all we need to run Caliburn.Micro binding process at design time is setting design time data context and enabling binding via Enable attached property. In sample above we are setting sample data generated by Visual Studio because the AddGameViewModel class doesn’t provide default constructor. Of course we could always add a default constructor in code and define view model instance in xaml file.

I chose this particular form intentionally because it contains the control that hasn’t defined custom binding convention by default. The Rating control is responsible for displaying Rating property value using stars. Rating property value is 0.8 so 4 stars should be displayed. In such cases where we are using controls with custom binding convention we need to register those conventions at design time. Any application using Caliburn.Micro has a type inherited from Bootstrapper type. This type is a starting point of our application and the instance of that type very often is defined as a resource inside App.xaml file. In fact Caliburn.Micro framework is already prepared for design time scenarios because Bootstrapper type contains virtual method called StartDesignTime. Let’s override this method and register appropriate convention:


public class Bootstrapper
{
public Bootstrapper()
{
if (Execute.InDesignMode) StartDesignTime(); else StartRuntime();
}
}

public class Bootstrapper<TRootModel> : Bootstrapper
{
}

public class AppBootstrapper : Bootstrapper<IShell>
{
private static bool isInitializedInDesignTime = false;

protected override void StartDesignTime()
{
base.StartDesignTime();

if (isInitializedInDesignTime)
return;
isInitializedInDesignTime = true;

ConfigureConventions();
}


void ConfigureConventions()
{
ConventionManager.AddElementConvention<Rating>(Rating.ValueProperty, "Value", "ValueChanged");
}
}

The instance of AppBootstapper type can be created many times at design time so boolean flag ensures that the conventions will be registered only once. When we compile the project and reopen the form we will see something like this:

image

This form represents a very simple scenario where the view model type has only properties of simple types like: string, double. boolean. It doesn’t contain any property of collection type or other view model type. In such scenarios Caliburn.Micro can find the appropriate type of view (control type) based on the type of view model. Let’s see another form called ResultsView.xaml to demonstrate this case:

image

This is a typical problem when we work with Caliburn.Micro and we cannot see anything at design time because the control is entirely collapsed :). Let’s see what happen after connecting sample data.

image

The list contains some elements but the font color is white so we aren’t able to read it because of the default Visual Studio background color. We can try to open this form inside Expression Blend where the background is dark or we can use custom design time attributes presented in the previous post.

image

Now we can see that Caliburn.Micro is not able to find view for view model type representing list item. It’s because we are using sample data mechanism from Visual Studio which generates dynamic type _.di0.GameLibrary.ViewModels.IndividualResultViewModel with the shape of the original view model type GameLibrary.ViewModels.IndividualResultViewModel. We need to change the way Caliburn.Micro is searching for view type based on specified view model type.

protected override void StartDesignTime()
{
base.StartDesignTime();

if (isInitializedInDesignTime)
return;
isInitializedInDesignTime = true;

ConfigureConventions();

AssemblySource.Instance.AddRange(new[] { typeof(App).Assembly });

var originalLocateTypeForModelType = ViewLocator.LocateTypeForModelType;
Func<Type, bool> isDesignTimeType = type => type.Assembly.IsDynamic;
ViewLocator.LocateTypeForModelType = (modelType, displayLocation, context) =>
{
var type = originalLocateTypeForModelType(modelType, displayLocation, context);
if (type == null && isDesignTimeType(modelType))
{
if (modelType.Name == "IndividualResultViewModel")
{
type = typeof(IndividualResultView);
}
}
return type;
};

IoC.GetInstance = base.GetInstance;
IoC.GetAllInstances = base.GetAllInstances;
}

Finally the list of items and generated sample data look like this:

image

Attached property is not the most convenient way of extending the designer functionality because we need to write xaml code. I tried to rewrite attached property to custom behavior so we could use dra&drop instead of writing the code. The problem is that behaviors code is not executed at design time. But I have good news. If you are creating UI with Expression Blend you can use our attached property on the property grid like a normal property.

image

It was possible thanks to the usage of attribute called AttachedPropertyBrowsableForTypeAttribute decorating attached property. Everything that was presented so far works both in Silverlight and WPF environments. It’s time to reveal how the magic works.



public static class DesignTime
{
public static DependencyProperty EnableProperty =
DependencyProperty.RegisterAttached(
"Enable",
typeof(bool),
typeof(DesignTime),
new PropertyMetadata(new PropertyChangedCallback(EnableChanged)));

#if !SILVERLIGHT && !WP7
[AttachedPropertyBrowsableForTypeAttribute(typeof(DependencyObject))]
#endif
public static bool GetEnable(DependencyObject dependencyObject)
{
return (bool)dependencyObject.GetValue(EnableProperty);
}


public static void SetEnable(DependencyObject dependencyObject, bool value)
{
dependencyObject.SetValue(EnableProperty, value);
}

static void EnableChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
{
if (!Execute.InDesignMode)
return;

BindingOperations.SetBinding(d, DataContextProperty, (bool)e.NewValue ? new Binding() : null);
}

private static readonly DependencyProperty DataContextProperty =
DependencyProperty.RegisterAttached(
"DataContext",
typeof(object),
typeof(DesignTime),
new PropertyMetadata(new PropertyChangedCallback(DataContextChanged))
);

private static void DataContextChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
{
if (!Execute.InDesignMode)
return;

object enable = d.GetValue(EnableProperty);
if (enable == null || ((bool)enable) == false || e.NewValue == null)
return;

var fe = d as FrameworkElement;
if (fe == null)
return;

ViewModelBinder.Bind(e.NewValue, d, string.IsNullOrEmpty(fe.Name) ? fe.GetHashCode().ToString() : fe.Name);
}
}

I hope you find this solution useful.

Download

Custom design-time attributes in Silverlight and WPF designer

We all know the standard design-time attributes in the Silverlight designer like d:DataContext, d:DesignWidth or d:DesignHeight. Let’s see how easy we can add some new attributes.

image
using System.ComponentModel;
using System.Windows;

namespace DesignTimeProperties
{
public static class d
{
static bool? inDesignMode;

/// <summary>
/// Indicates whether or not the framework is in design-time mode. (Caliburn.Micro implementation)
/// </summary>
private static bool InDesignMode
{
get
{
if (inDesignMode == null)
{
var prop = DesignerProperties.IsInDesignModeProperty;
inDesignMode = (bool)DependencyPropertyDescriptor.FromProperty(prop, typeof(FrameworkElement)).Metadata.DefaultValue;

if (!inDesignMode.GetValueOrDefault(false) && System.Diagnostics.Process.GetCurrentProcess()
.ProcessName.StartsWith("devenv", System.StringComparison.Ordinal))
inDesignMode = true;
}

return inDesignMode.GetValueOrDefault(false);
}
}

public static DependencyProperty BackgroundProperty = DependencyProperty.RegisterAttached(
"Background", typeof(System.Windows.Media.Brush), typeof(d),
new PropertyMetadata(new PropertyChangedCallback(BackgroundChanged)));

public static System.Windows.Media.Brush GetBackground(DependencyObject dependencyObject)
{
return (System.Windows.Media.Brush)dependencyObject.GetValue(BackgroundProperty);
}
public static void SetBackground(DependencyObject dependencyObject, System.Windows.Media.Brush value)
{
dependencyObject.SetValue(BackgroundProperty, value);
}
private static void BackgroundChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
{
if (!InDesignMode)
return;

d.GetType().GetProperty("Background").SetValue(d, e.NewValue, null);
}
}
}

Great, but what about many others very useful properties ? Do we need to write them all manually? No, we don’t. I have prepared T4 template that generates code for all properties of all controls available in WPF and Silverlight assemblies. When you look carefully at the code above you will find that I am using the reflection to set the value of the property. It’s because some properties like Background are defined many times in many different controls so instead of checking the actual control type I assume that the control contains appropriate property. This assumption simplifies the implementation very much. And of course the property can be set only in design time when we are editing the form inside Expression Blend or Visual Studio. T4 template code analyzes all properties from all controls and properties with the same names but different types are skipped during generation process.

Download

Monday, October 25, 2010

Debugging Reactive Framework (RxDebugger) and Linq to objects (LinqDebugger)

[New version (2011.05.06)]

[Project download has been upgraded to the newest version of Rx (Build v1.0.2787.0) and VS2010 ] download
(Changes: projects converted to VS2010, sample project for RxDebugger added)

In this post I will present two projects LinqDebugger and RxDebugger. Few months ago after reading Bart De Smet’s great post about tracing execution of the Linq to objects queries I was wondering if it was be possible to implement the same concept but in more general way. If we want to trace the execution of all of the Linq operators using described approach we would have to implement many extension methods, one for each Linq operator. How to avoid this ? LinqDebugger is the the answer ;) . If you are wondering what the lazy evaluation is and how to debug Linq to object queries this project can be very useful. Let’s see a very simple query :

var q =
    from i in Enumerable.Range(0, 12)
    where i > 5 && i % 2 == 0
    select i.ToString("C");

foreach (var s in q)
    Console.WriteLine(s);

Now let’s debug that query:

var q =
    from i in Enumerable.Range(0, 12).AsDebuggable()
    where i > 5 && i % 2 == 0
    select i.ToString("C");

foreach (var s in q)
    Console.WriteLine(s); 

After running this code you will find the following text on the console:

Select creation
Select begin
Where creation
Where begin
 predicate i => ((i > 5) && ((i % 2) = 0)) : (0) => False
 predicate i => ((i > 5) && ((i % 2) = 0)) : (1) => False
 predicate i => ((i > 5) && ((i % 2) = 0)) : (2) => False
 predicate i => ((i > 5) && ((i % 2) = 0)) : (3) => False
 predicate i => ((i > 5) && ((i % 2) = 0)) : (4) => False
 predicate i => ((i > 5) && ((i % 2) = 0)) : (5) => False
 predicate i => ((i > 5) && ((i % 2) = 0)) : (6) => True
Where end 6
 selector i => i.ToString("C") : (6) => 6,00 zł
Select end 6,00 zł
6,00 zł
Select begin
Where begin
 predicate i => ((i > 5) && ((i % 2) = 0)) : (7) => False
 predicate i => ((i > 5) && ((i % 2) = 0)) : (8) => True
Where end 8
 selector i => i.ToString("C") : (8) => 8,00 zł
Select end 8,00 zł
8,00 zł
Select begin
Where begin
 predicate i => ((i > 5) && ((i % 2) = 0)) : (9) => False
 predicate i => ((i > 5) && ((i % 2) = 0)) : (10) => True
Where end 10
 selector i => i.ToString("C") : (10) => 10,00 zł
Select end 10,00 zł
10,00 zł
Select begin
Where begin
 predicate i => ((i > 5) && ((i % 2) = 0)) : (11) => False
Where end (no result)
Select end (no result)

This text shows how the query has been executed. There can find there information about enumerators object creation, about data passing from one enumerator to another and execution of all functions used inside the query. One thing worth mentioning is that everything is presented in the same order as it was executed. Having this information we can easily figure out for example in which iteration the exception has been thrown and what were the values processing by the query at that moment. If such text representation is hard to read for you I also provide the VS visualizer for ExecutionPlan type presenting the same information on the tree control (copy LinqDebugger.dll and LinqDebugger.Visualizer.dll files to C:\Program Files\Microsoft Visual Studio 9.0\Common7\Packages\Debugger\Visualizers folder to make it available).

var executionPlan = new ExecutionPlan();

var q =
    from i in Enumerable.Range(0, 12).AsDebuggable(executionPlan)
    where i > 5 && i % 2 == 0
    select i.ToString("C");

foreach (var s in q)
    Console.WriteLine(s);

image

I am not going to describe here in details how LinqDebugger has been implemented, you can download the code and check this out yourself. As a hint I will just show you the signature of AsDebuggable method. Please notice what else you can pass to that method. We can choose which operators we want to trace using LinqOperators enum type or pass TextWriter object (Console.Out is set as a default TextWriter).

public static class LinqDebuggerExtensions
{
    public static IQueryable<T> AsDebuggable<T>(this IEnumerable<T> source, ExecutionPlan executionPlan,
        LinqOperators linqOperators, TextWriter textWriter)
    { ... }
}

[Flags]
public enum LinqOperators : long
{
    None = 0,
    Aggregate = 1,
    All = 2,
    Any = 4,
    AsQueryable = 8,
    Average = 16,
    Cast = 32,
    Concat = 64,
    Contains = 128,
    ...
}

[Serializable]
public sealed class ExecutionPlan
{
    public Expression Query { get; internal set; }
    public List<Record> Records { get; }
    public void Reset();
}
    
[Serializable]
public class Record
{
    public RecordType RecordType { get; internal set; }
    public MethodCallExpression OperatorCallExpression { get; internal set; }
    public object Result { get; internal set; }
    public LambdaExpression FuncCallExpression { get; internal set; }
    public string FuncCallName { get; internal set; }
    public object[] Arguments { get; internal set; }
}

Tracing similar information during Rx queries execution is even more useful because in most cases such queries are executed asynchronously so debugging them is really hard. Let’s debug Rx version of previous query using RxDebugger:

var q =
    from i in Enumerable.Range(0, 12).ToObservable().AsDebuggable(
        new DebugSettings {SourceName = "range", Logger = DebugSettings.ConsoleLogger})
    where i > 5 && i % 2 == 0
    select i.ToString("C");

q.Run(Console.WriteLine);


range.Where.Select.Subscribe()
range.Where.Subscribe()
range.Subscribe()
range.OnNext(0)
range.OnNext(1)
range.OnNext(2)
range.OnNext(3)
range.OnNext(4)
range.OnNext(5)
6,00 zł
range.OnNext(6)
range.Where.OnNext(6)
range.Where.Select.OnNext(6,00 zł)
range.OnNext(7)
8,00 zł
range.OnNext(8)
range.Where.OnNext(8)
range.Where.Select.OnNext(8,00 zł)
range.OnNext(9)
10,00 zł
range.OnNext(10)
range.Where.OnNext(10)
range.Where.Select.OnNext(10,00 zł)
range.OnNext(11)
range.OnCompleted()
range.Where.OnCompleted()
range.Where.Select.OnCompleted()
range.Where.Select.Dispose()
range.Where.Dispose()
range.Dispose()

As you can see this time a quite deferent information displayed on the screen and there is no VS visualizer available. It’s because the implementation of RxDebugger is totally different from LinqDebugger. But there are some additional features too. To understand how RxDebugger works I will show you the Debug method which gives us ability to trace single observable object instead of the whole Rx query.

var q =
    from i in Enumerable.Range(0, 12).ToObservable().Debug(
        new DebugSettings {SourceName = "range", Logger = DebugSettings.ConsoleLogger})
    where i > 5 && i % 2 == 0
    select i.ToString("C");

q.Run(Console.WriteLine);

range.Subscribe()
range.OnNext(0)
range.OnNext(1)
range.OnNext(2)
range.OnNext(3)
range.OnNext(4)
range.OnNext(5)
6,00 zł
range.OnNext(6)
range.OnNext(7)
8,00 zł
range.OnNext(8)
range.OnNext(9)
10,00 zł
range.OnNext(10)
range.OnNext(11)
range.OnCompleted()
range.Dispose()

 

public static IObservable<T> Debug<T>(this IObservable<T> source, DebugSettings settings, Func<T, object> valueSelector)
{
    Action<T> onNext = v => { };
    if ((settings.NotificationFilter & NotificationFilter.OnNext) == NotificationFilter.OnNext)
        onNext = v => 
        { 
            if (settings.Logger != null) 
                settings.LoggerScheduler.Schedule(() => settings.Logger(DebugEntry.Create(settings, NotificationFilter.OnNext, valueSelector(v)))); 
        };
        
    Action<Exception> onError = ... ;
    Action onCompleted = ... ;
    Action subscribe = ... ;
    Action dispose = ... ;
    
    return Observable.CreateWithDisposable<T>(o =>
    {
        var newObserver = Observer.Create<T>
        (
            v => { onNext(v); o.OnNext(v); },
            e => { onError(e); o.OnError(e); },
            () => { onCompleted(); o.OnCompleted(); }
        );

        subscribe();
        var disposable = source.Subscribe(newObserver);

        return Disposable.Create(() =>
        {
            dispose();
            disposable.Dispose();
        });
    });
}

public sealed class DebugSettings
{
    // defaults 
    public static Action<DebugEntry> DefaultLogger { get; set; }
    public static IScheduler DefaultLoggerSchduler { get; set; }       
    public static string DefaultMessageFormat { get; set; }

    //loggers
    public static Action<DebugEntry> ConsoleLogger { get; private set; }
    public static Action<DebugEntry> DebugLogger { get; private set; }
   
    public string MessageFormat { get; set; }
    public Action<DebugEntry> Logger { get; set; }
    public IScheduler LoggerScheduler { get; set; }
    public string SourceName { get; set; }
    public NotificationFilter NotificationFilter { get; set; }
    public OperatorFilter OperatorFilter { get; set; }

    static DebugSettings()
    {
        ConsoleLogger = n => Console.WriteLine(n.FormattedMessage);
        DebugLogger = n => Debug.WriteLine(n.FormattedMessage);

        DefaultLogger = DebugLogger;
        DefaultLoggerSchduler = Scheduler.CurrentThread;
        DefaultMessageFormat = "{0}.{1}({2})";
    }

    public DebugSettings()
    {
        SourceName = "";
        NotificationFilter = NotificationFilter.All;
        OperatorFilter = OperatorFilter.AllOperators;

        Logger = DefaultLogger;
        LoggerScheduler = DefaultLoggerSchduler;
        MessageFormat = DefaultMessageFormat;
    }
}

public class DebugEntry
{
    public string SourceName { get; set; }
    public string FormattedMessage { get; set; }
    public NotificationFilter Kind { get; set; }
    public Exception Exception { get; set; }
    public object Value { get; set; }
}

Debug method creates a new observable object on the top of given observable sources. Each observer passed to this observable source is wrapped into a new observer tracing information about calling Subscribe, Dispose methods at the IObservable level and OnNext, OnError, OnCompleted methods at the IObserver level. We can provide filter on Rx operators (OperatorFilter enum type) or logged information (NotificationFilter enum type). In LinqDebugger project TextWiter class has been used to log information. Here we have much more flexible solution because we can pass delegate type responsible for storing logged information. RxDebugger provides standard loggers such as DebugSettings.ConsoleLogger or DebugSettings.DebugLogger but we can also set our own delegate type or even merge many different logger delegates. Such a scenario will be presented in further part of the post. Once we know how Debug method works let’s reveal the secret behind the AsDebuggable method.

public interface IDebuggedObservable<T> : IObservable<T>
{
    DebugSettings Settings { get; }
}
    
public static partial class RxDebuggerExtensions
{    
    public static IDebuggedObservable<T> AsDebuggable<T>(this IObservable<T> source, DebugSettings settings)
    {
        return new DebuggedObservable<T>(source.Debug(settings), settings);
    }
        
    public static IDebuggedObservable<TSource> Where<TSource>(this IDebuggedObservable<TSource> source , Func<TSource,bool> predicate)
    {
        var settings = source.Settings.Copy();
        settings.SourceName = settings.SourceName +  ".Where";
        return new DebuggedObservable<TSource>((source as IObservable<TSource>).Where<TSource>(predicate).Debug(settings), settings);
    }
    public static IDebuggedObservable<TResult> Select<TSource,TResult>(this IDebuggedObservable<TSource> source , Func<TSource,TResult> selector)
    {
        var settings = source.Settings.Copy();
        settings.SourceName = settings.SourceName +  ".Select";
        return new DebuggedObservable<TResult>((source as IObservable<TSource>).Select<TSource,TResult>(selector).Debug(settings), settings);
    } 
    ... 
    
    private class DebuggedObservable<T> : IDebuggedObservable<T>
    {
        private readonly IObservable<T> _source;
        private readonly DebugSettings _settings;
        public DebugSettings Settings { get { return _settings; } }

        public DebuggedObservable(IObservable<T> source, DebugSettings settings)
        {
            _source = source;
            _settings = settings;
        }
        public IDisposable Subscribe(IObserver<T> observer)
        {
            return _source.Subscribe(observer);
        }            
    }
}

Of course I didn’t implement extension methods for all Rx operator manually, I wrote T4 template generating appropriate extension methods (currently 127 methods in .Net version and 125 methods in Silverlight version :) ). The best way to use RxDebugger in your projects is to add T4 template to the project you are working on and run template every time you change the version of Rx. It allows you to always be synchronized with Rx dlls. Finally let’s see how to use RxDebugger in Silverlight application.

<UserControl x:Class="Blog.SL.Post016.RxDebuggerTest"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008" 
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
    mc:Ignorable="d" d:DesignWidth="640" d:DesignHeight="480">
    <StackPanel>
        <TextBox x:Name="input"/>
        <TextBox x:Name="output"/>
        <ItemsControl x:Name="log"/>
    </StackPanel>
</UserControl>

 

public partial class RxDebuggerTest : UserControl
{
    public RxDebuggerTest()
    {
        InitializeComponent();

        var entries = new ObservableCollection<DebugEntry>();
        log.ItemsSource = entries;

        var q = input
            .GetObservableTextChanged()
            .Select(e => ((TextBox)e.Sender).Text)
            .AsDebuggable(new DebugSettings
                              {
                                  SourceName = "textChanged", 
                                  LoggerScheduler = Scheduler.Dispatcher, 
                                  Logger = DebugSettings.DebugLogger + entries.Add,                                      
                              })
            .Throttle(TimeSpan.FromSeconds(2))
            .Select(t => new string(t.Reverse().ToArray()));

        q.ObserveOnDispatcher().Subscribe(t => output.Text = t);
    }
}

image

And that’s it for the post. I encourage you to download and play with the two simple tools I provided here. They can be very helpful in debugging LINQ quires or learning about the internals of LINQ and Rx.
downlaod (Rx versions: .Net3.5 v1.0.2698.0 and SL3 v1.0.2698.0) Always check for newest version at the beginning of the post.

Thursday, November 12, 2009

Linq to ICollectionView

Currently I'm working on the project written in Silverlight and I have encountered an interface called ICollectionView and its standard implementation PagedCollectionView. In short, this components allow us to create the view of collection of items with filtering, grouping and sorting functionality. The idea behind this interface is very similar to DataView/DataTable mechanism. DataTable is responsible for storing data and DataView is just an appropriately configured proxy (only filtering and sorting in this case) which can be bound to UI controls. Let's look at a very simple example:

public class Number
{
    public int Value { get; set; }
    public int Random { get; set; }

    public static Number[] GetAll()
    {
        var random = new Random();
        return
            (from n in Enumerable.Range(1,5)
             from m in Enumerable.Repeat(n, n)
             select new Number {Value = n, Random = random.Next(10)}).ToArray();            
    }
}

// Silverlight
public class LinqToICollectionView : UserControl
{
    public LinqToICollectionView()
    {
        Number[] numbers = Number.GetAll();

        Content = new StackPanel
        {
            Orientation = Orientation.Horizontal,
            Children =
            {
                new DataGrid { ItemsSource = new PagedCollectionView(numbers).SetConfiguration()},
            }
        };
    }
}

public static class Configurator
{
    public static ICollectionView SetConfiguration(this ICollectionView view)
    {
        // filtering
        view.Filter = (object o) => ((Number)o).Value < 5;
        // grouping
        view.GroupDescriptions.Add(new PropertyGroupDescription("Value"));
        // sorting
        view.SortDescriptions.Add(new SortDescription("Value", ListSortDirection.Descending));
        view.SortDescriptions.Add(new SortDescription("Random", ListSortDirection.Ascending));
        return view;
    }     
}

image

But wait a minute, we said ... filtering, ordering, grouping ? Let's use LINQ query to configure ICollectionView:

public static class Configurator
{
    public static ICollectionView SetConfigurationWithLinq(this ICollectionView view)
    {
        var q =
            from n in new View<Number>()
            where n.Value < 5
            orderby n.Value descending, n.Random
            group n by n.Value;
        q.Apply(view);
        return view;
    }        
}

The whole implementation consists of 3 simple classes: View<T>, OrderedView<T> and GroupedView<T>.

public class View<T>
{
    public IEnumerable<GroupDescription> GroupDescriptions { get { ... } }
    public IEnumerable<SortDescription> SortDescriptions { get { ... } }
    public Func<T,bool> Filter { get { ... } }
    
    public View<T> Where(Func<T, bool> func) { ... }
    public SortedView<T> OrderBy<T2>(Expression<Func<T, T2>> func) { ... }
    public SortedView<T> OrderByDescending<T2>(Expression<Func<T, T2>> func) { ... }
    public GroupedView<T> GroupBy<T2>(Expression<Func<T, T2>> func) { ... }
    
    public void Apply(ICollectionView collectionView) { ... }
}
public sealed class SortedView<T> : View<T>
{    
    public SortedView<T> ThenBy<T2>(Expression<Func<T, T2>> func) { ... }
    public SortedView<T> ThenByDescending<T2>(Expression<Func<T, T2>> func)  { ... }
}
public sealed class GroupedView<T> : View<T>
{
    public GroupedView<T> ThenBy<T2>(Expression<Func<T, T2>> func) { ... }
}

Methods for sorting and grouping which take an expression tree as a parameter analyze the tree looking for indicated members (fields or properties) and collect appropriate SortDescription and GroupDescription objects. Where method takes a delegate type which is combined via logical and operator with existing filter delegate set previously (in case when Where method is called many times). Of course the same mechanism work also in WPF.

Number[] numbers = Number.GetAll();

// WPF
new Window
{
    Content = new StackPanel
    {
        Orientation = Orientation.Horizontal,
        Children =
        {
            CreateListView(new CollectionViewSource { Source = numbers }.View.SetConfiguration()),
            CreateListView(new CollectionViewSource { Source = numbers }.View.SetConfigurationWithLinq())
        }
    }
}
.ShowDialog();


private static ListView CreateListView(ICollectionView view)
{
    return new ListView
    {
        GroupStyle = { GroupStyle.Default },
        View = new GridView
        {
            Columns =
            {
                new GridViewColumn { Header = "Value", DisplayMemberBinding = new Binding("Value") },
                new GridViewColumn { Header = "Random", DisplayMemberBinding = new Binding("Random") },
            }
        },
        ItemsSource = view
    };
}

At the end I'd like to mention one interesting thing. We only support filtering, sorting and grouping and don't support for instance projection, joining and so on. That's way this code should compile:

var v1 = // only filtering specified
    from n in new View<Number>() 
    where n.Value < 5 
    select n;

var v2 = // grouping (last grouping definition overrides previous ones)
    from n in new View<Number>()
    group n by n.Random into s 
    group s by s.Value;

but this will not:

var v3 = // joining is not supported
    from p in new View<Number>()
    join pp in new[] { 1, 2, 3, 4 } on p.Random equals pp
    select p;

var v4 = // projection is not supported
    from p in new View<Number>()
    where p.Value > 5
    select p.Random;

var v5 = // at least one filtering, grouping or sorting definition must be specified
    from p in new View<Number>()
    select p;

var v6 = // Numer type is the only valid type of grouped element
    from p in new View<Number>()
    group p.Random by p.Value;

As a homework I leave you a question: why it works this way ? :)

Sources