Few weeks ago I promised recording of my presentation given during MTS 2010 conference, here it is. Enjoy!
Thursday, December 9, 2010
Sunday, November 7, 2010
Currying in C#
During reviewing samples for Async Ctp I have found such code:
var cts = new CancellationTokenSource(); btnCancel.Click += cts.EventHandler; // !!!! public static class Extensions { public static void EventHandler(this CancellationTokenSource cts, object sender, EventArgs e) { cts.Cancel(); } }
Did you know it was possible ?
Wednesday, October 27, 2010
After Microsoft Technology Summit 2010
This year again I had the opportunity to speak during MTS 2010 conference in Warsaw. Last time I have been talking about WF4 and this time the title of my talk was “Introduction to F#”. I would like to thank all attendees for coming! All resources presented during the session are available for download here, here and here. In the near future I’ll publish recording from this session.
Monday, October 25, 2010
Debugging Reactive Framework (RxDebugger) and Linq to objects (LinqDebugger)
[New version (2011.05.06)]
[Project download has been upgraded to the newest version of Rx (Build v1.0.2787.0) and VS2010 ] download
(Changes: projects converted to VS2010, sample project for RxDebugger added)
In this post I will present two projects LinqDebugger and RxDebugger. Few months ago after reading Bart De Smet’s great post about tracing execution of the Linq to objects queries I was wondering if it was be possible to implement the same concept but in more general way. If we want to trace the execution of all of the Linq operators using described approach we would have to implement many extension methods, one for each Linq operator. How to avoid this ? LinqDebugger is the the answer ;) . If you are wondering what the lazy evaluation is and how to debug Linq to object queries this project can be very useful. Let’s see a very simple query :
var q = from i in Enumerable.Range(0, 12) where i > 5 && i % 2 == 0 select i.ToString("C"); foreach (var s in q) Console.WriteLine(s);
Now let’s debug that query:
var q = from i in Enumerable.Range(0, 12).AsDebuggable() where i > 5 && i % 2 == 0 select i.ToString("C"); foreach (var s in q) Console.WriteLine(s);
After running this code you will find the following text on the console:
Select creation Select begin Where creation Where begin predicate i => ((i > 5) && ((i % 2) = 0)) : (0) => False predicate i => ((i > 5) && ((i % 2) = 0)) : (1) => False predicate i => ((i > 5) && ((i % 2) = 0)) : (2) => False predicate i => ((i > 5) && ((i % 2) = 0)) : (3) => False predicate i => ((i > 5) && ((i % 2) = 0)) : (4) => False predicate i => ((i > 5) && ((i % 2) = 0)) : (5) => False predicate i => ((i > 5) && ((i % 2) = 0)) : (6) => True Where end 6 selector i => i.ToString("C") : (6) => 6,00 zł Select end 6,00 zł 6,00 zł Select begin Where begin predicate i => ((i > 5) && ((i % 2) = 0)) : (7) => False predicate i => ((i > 5) && ((i % 2) = 0)) : (8) => True Where end 8 selector i => i.ToString("C") : (8) => 8,00 zł Select end 8,00 zł 8,00 zł Select begin Where begin predicate i => ((i > 5) && ((i % 2) = 0)) : (9) => False predicate i => ((i > 5) && ((i % 2) = 0)) : (10) => True Where end 10 selector i => i.ToString("C") : (10) => 10,00 zł Select end 10,00 zł 10,00 zł Select begin Where begin predicate i => ((i > 5) && ((i % 2) = 0)) : (11) => False Where end (no result) Select end (no result)
This text shows how the query has been executed. There can find there information about enumerators object creation, about data passing from one enumerator to another and execution of all functions used inside the query. One thing worth mentioning is that everything is presented in the same order as it was executed. Having this information we can easily figure out for example in which iteration the exception has been thrown and what were the values processing by the query at that moment. If such text representation is hard to read for you I also provide the VS visualizer for ExecutionPlan type presenting the same information on the tree control (copy LinqDebugger.dll and LinqDebugger.Visualizer.dll files to C:\Program Files\Microsoft Visual Studio 9.0\Common7\Packages\Debugger\Visualizers folder to make it available).
var executionPlan = new ExecutionPlan(); var q = from i in Enumerable.Range(0, 12).AsDebuggable(executionPlan) where i > 5 && i % 2 == 0 select i.ToString("C"); foreach (var s in q) Console.WriteLine(s);
I am not going to describe here in details how LinqDebugger has been implemented, you can download the code and check this out yourself. As a hint I will just show you the signature of AsDebuggable method. Please notice what else you can pass to that method. We can choose which operators we want to trace using LinqOperators enum type or pass TextWriter object (Console.Out is set as a default TextWriter).
public static class LinqDebuggerExtensions { public static IQueryable<T> AsDebuggable<T>(this IEnumerable<T> source, ExecutionPlan executionPlan, LinqOperators linqOperators, TextWriter textWriter) { ... } } [Flags] public enum LinqOperators : long { None = 0, Aggregate = 1, All = 2, Any = 4, AsQueryable = 8, Average = 16, Cast = 32, Concat = 64, Contains = 128, ... } [Serializable] public sealed class ExecutionPlan { public Expression Query { get; internal set; } public List<Record> Records { get; } public void Reset(); } [Serializable] public class Record { public RecordType RecordType { get; internal set; } public MethodCallExpression OperatorCallExpression { get; internal set; } public object Result { get; internal set; } public LambdaExpression FuncCallExpression { get; internal set; } public string FuncCallName { get; internal set; } public object[] Arguments { get; internal set; } }
Tracing similar information during Rx queries execution is even more useful because in most cases such queries are executed asynchronously so debugging them is really hard. Let’s debug Rx version of previous query using RxDebugger:
var q = from i in Enumerable.Range(0, 12).ToObservable().AsDebuggable( new DebugSettings {SourceName = "range", Logger = DebugSettings.ConsoleLogger}) where i > 5 && i % 2 == 0 select i.ToString("C"); q.Run(Console.WriteLine); range.Where.Select.Subscribe() range.Where.Subscribe() range.Subscribe() range.OnNext(0) range.OnNext(1) range.OnNext(2) range.OnNext(3) range.OnNext(4) range.OnNext(5) 6,00 zł range.OnNext(6) range.Where.OnNext(6) range.Where.Select.OnNext(6,00 zł) range.OnNext(7) 8,00 zł range.OnNext(8) range.Where.OnNext(8) range.Where.Select.OnNext(8,00 zł) range.OnNext(9) 10,00 zł range.OnNext(10) range.Where.OnNext(10) range.Where.Select.OnNext(10,00 zł) range.OnNext(11) range.OnCompleted() range.Where.OnCompleted() range.Where.Select.OnCompleted() range.Where.Select.Dispose() range.Where.Dispose() range.Dispose()
As you can see this time a quite deferent information displayed on the screen and there is no VS visualizer available. It’s because the implementation of RxDebugger is totally different from LinqDebugger. But there are some additional features too. To understand how RxDebugger works I will show you the Debug method which gives us ability to trace single observable object instead of the whole Rx query.
var q = from i in Enumerable.Range(0, 12).ToObservable().Debug( new DebugSettings {SourceName = "range", Logger = DebugSettings.ConsoleLogger}) where i > 5 && i % 2 == 0 select i.ToString("C"); q.Run(Console.WriteLine); range.Subscribe() range.OnNext(0) range.OnNext(1) range.OnNext(2) range.OnNext(3) range.OnNext(4) range.OnNext(5) 6,00 zł range.OnNext(6) range.OnNext(7) 8,00 zł range.OnNext(8) range.OnNext(9) 10,00 zł range.OnNext(10) range.OnNext(11) range.OnCompleted() range.Dispose()
public static IObservable<T> Debug<T>(this IObservable<T> source, DebugSettings settings, Func<T, object> valueSelector) { Action<T> onNext = v => { }; if ((settings.NotificationFilter & NotificationFilter.OnNext) == NotificationFilter.OnNext) onNext = v => { if (settings.Logger != null) settings.LoggerScheduler.Schedule(() => settings.Logger(DebugEntry.Create(settings, NotificationFilter.OnNext, valueSelector(v)))); }; Action<Exception> onError = ... ; Action onCompleted = ... ; Action subscribe = ... ; Action dispose = ... ; return Observable.CreateWithDisposable<T>(o => { var newObserver = Observer.Create<T> ( v => { onNext(v); o.OnNext(v); }, e => { onError(e); o.OnError(e); }, () => { onCompleted(); o.OnCompleted(); } ); subscribe(); var disposable = source.Subscribe(newObserver); return Disposable.Create(() => { dispose(); disposable.Dispose(); }); }); } public sealed class DebugSettings { // defaults public static Action<DebugEntry> DefaultLogger { get; set; } public static IScheduler DefaultLoggerSchduler { get; set; } public static string DefaultMessageFormat { get; set; } //loggers public static Action<DebugEntry> ConsoleLogger { get; private set; } public static Action<DebugEntry> DebugLogger { get; private set; } public string MessageFormat { get; set; } public Action<DebugEntry> Logger { get; set; } public IScheduler LoggerScheduler { get; set; } public string SourceName { get; set; } public NotificationFilter NotificationFilter { get; set; } public OperatorFilter OperatorFilter { get; set; } static DebugSettings() { ConsoleLogger = n => Console.WriteLine(n.FormattedMessage); DebugLogger = n => Debug.WriteLine(n.FormattedMessage); DefaultLogger = DebugLogger; DefaultLoggerSchduler = Scheduler.CurrentThread; DefaultMessageFormat = "{0}.{1}({2})"; } public DebugSettings() { SourceName = ""; NotificationFilter = NotificationFilter.All; OperatorFilter = OperatorFilter.AllOperators; Logger = DefaultLogger; LoggerScheduler = DefaultLoggerSchduler; MessageFormat = DefaultMessageFormat; } } public class DebugEntry { public string SourceName { get; set; } public string FormattedMessage { get; set; } public NotificationFilter Kind { get; set; } public Exception Exception { get; set; } public object Value { get; set; } }
Debug method creates a new observable object on the top of given observable sources. Each observer passed to this observable source is wrapped into a new observer tracing information about calling Subscribe, Dispose methods at the IObservable level and OnNext, OnError, OnCompleted methods at the IObserver level. We can provide filter on Rx operators (OperatorFilter enum type) or logged information (NotificationFilter enum type). In LinqDebugger project TextWiter class has been used to log information. Here we have much more flexible solution because we can pass delegate type responsible for storing logged information. RxDebugger provides standard loggers such as DebugSettings.ConsoleLogger or DebugSettings.DebugLogger but we can also set our own delegate type or even merge many different logger delegates. Such a scenario will be presented in further part of the post. Once we know how Debug method works let’s reveal the secret behind the AsDebuggable method.
public interface IDebuggedObservable<T> : IObservable<T> { DebugSettings Settings { get; } } public static partial class RxDebuggerExtensions { public static IDebuggedObservable<T> AsDebuggable<T>(this IObservable<T> source, DebugSettings settings) { return new DebuggedObservable<T>(source.Debug(settings), settings); } public static IDebuggedObservable<TSource> Where<TSource>(this IDebuggedObservable<TSource> source , Func<TSource,bool> predicate) { var settings = source.Settings.Copy(); settings.SourceName = settings.SourceName + ".Where"; return new DebuggedObservable<TSource>((source as IObservable<TSource>).Where<TSource>(predicate).Debug(settings), settings); } public static IDebuggedObservable<TResult> Select<TSource,TResult>(this IDebuggedObservable<TSource> source , Func<TSource,TResult> selector) { var settings = source.Settings.Copy(); settings.SourceName = settings.SourceName + ".Select"; return new DebuggedObservable<TResult>((source as IObservable<TSource>).Select<TSource,TResult>(selector).Debug(settings), settings); } ... private class DebuggedObservable<T> : IDebuggedObservable<T> { private readonly IObservable<T> _source; private readonly DebugSettings _settings; public DebugSettings Settings { get { return _settings; } } public DebuggedObservable(IObservable<T> source, DebugSettings settings) { _source = source; _settings = settings; } public IDisposable Subscribe(IObserver<T> observer) { return _source.Subscribe(observer); } } }
Of course I didn’t implement extension methods for all Rx operator manually, I wrote T4 template generating appropriate extension methods (currently 127 methods in .Net version and 125 methods in Silverlight version :) ). The best way to use RxDebugger in your projects is to add T4 template to the project you are working on and run template every time you change the version of Rx. It allows you to always be synchronized with Rx dlls. Finally let’s see how to use RxDebugger in Silverlight application.
<UserControl x:Class="Blog.SL.Post016.RxDebuggerTest" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:d="http://schemas.microsoft.com/expression/blend/2008" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" mc:Ignorable="d" d:DesignWidth="640" d:DesignHeight="480"> <StackPanel> <TextBox x:Name="input"/> <TextBox x:Name="output"/> <ItemsControl x:Name="log"/> </StackPanel> </UserControl>
public partial class RxDebuggerTest : UserControl { public RxDebuggerTest() { InitializeComponent(); var entries = new ObservableCollection<DebugEntry>(); log.ItemsSource = entries; var q = input .GetObservableTextChanged() .Select(e => ((TextBox)e.Sender).Text) .AsDebuggable(new DebugSettings { SourceName = "textChanged", LoggerScheduler = Scheduler.Dispatcher, Logger = DebugSettings.DebugLogger + entries.Add, }) .Throttle(TimeSpan.FromSeconds(2)) .Select(t => new string(t.Reverse().ToArray())); q.ObserveOnDispatcher().Subscribe(t => output.Text = t); } }
And that’s it for the post. I encourage you to download and play with the two simple tools I provided here. They can be very helpful in debugging LINQ quires or learning about the internals of LINQ and Rx.
downlaod (Rx versions: .Net3.5 v1.0.2698.0 and SL3 v1.0.2698.0) Always check for newest version at the beginning of the post.
Monday, July 5, 2010
After Virtual Study Conference 2010
VirtualStudy Conference 2010 was the first edition of the virtual conference where all attendees as well as speakers have been remotely connected together via Live Meeting platform. This time I have been talking about reactive programming as a general way of writing code. Here is the overview of my presentation:
Reactive Programming - a new paradigm of programming
Nowadays more and more systems created by us work in the cloud. Client while connecting using WebService is performing certain operations. Remote calls have longer time of duration than local methods, so often we are forced to perform them asynchronously. .NET Framework provides mechanisms, such as threads, APM pattern (Asynchronous Programming Model) or EAP (Event-based Asynchronous Pattern.) But the problem arises, when we want to coordinate a number of simultaneous asynchronous requests. What will happen in a situation where one of the operations will be canceled, or there is an unexpected error? Code that supports such a scenario becomes unreadable, and thus - difficult in maintenance and testing. During this session we will be presenting several approaches to reactive programming, including Reactive Framework, TPL, asynchronous workflows in F # and AsyncEnumerator project.
As usually code samples and slides are available here and you can watch the presentation here. Enjoy!
Tuesday, March 30, 2010
After Rx Road show
Last week I have finished Rx Road show. I have been giving presentations about Reactive Framework on four Polish .Net Users Groups: in Wroclaw, Lodz, Krakow and Chorzow. Thank you all for coming to my session and filling the evaluations, I hope you have learned something useful about Rx during the session. I have also good news for those of you who were interested in my presentation but for some reason couldn’t come, one of the presentation has been recorded and it is now available for download or watching live online here. Slides and all code samples can be download from here. Enjoy!
Monday, March 8, 2010
RxSandbox V1
[New version (2011.05.06)]
[RxSandbox downlaod has been upgraded to the newest version of Rx (Build 1.0.2677.0 08/27/2010).] download
[RxSandbox downlaod has been upgraded to the newest version of Rx (Build 1.0.2617.0 07/15/2010).] download
(Changes: solution converted to VS2010, Net 4.0; 56 operators, grouping operators on the tree control; zoom)
[RxSandbox downlaod has been upgraded to the newest version of Rx (Build 1.0.2441.0 04/14/2010).] download
[RxSandbox downlaod has been upgraded to the newest version of Rx (Build 1.0.2350.0 03/15/2010).] download
I am really happy to announce that the new version of RxSandbox has been just released. Last time I have been writing about the concept and the main ideas behind the project. That was just a small prototype, the proof of concept. This release is much more mature, the user can test more standard Rx operators and the API has been changed a little bit, however the concept stated the same.
These are the main new features:
- Marble diagrams
- New powerful API
- Extensibility mechanism
- Description of Rx the operators taken from Rx documentation
Lets start from the the end-user which is not necessary interested in writing any code. He just wants to experiment with Rx operators, check how they behave in specific scenarios. When we start RxSandbox application we will see the list of standard operators on the left hand side. Lets assume that we don’t know how Merge operator works. After double click on the Merge node the new tab will be displayed. We can find the short description of Merge operator taken from documentation provided by Rx installer, we can also find the code sample that can be tested interactively trough UI. Each input argument is presented as a simple automatically generated UI with one textbox where we can write input value, tree buttons and the history of the source. The output of the expression can be presented in two ways: by a simple list displaying the results (“Output” tab) or the marble diagrams drawn live during testing (‘Marble diagram – Live” tab). There is also one tab called “Marble diagram - Definition” showing the operator definition.
Now lets see what the developer can do with RxSandbox. The most important class in Rx API is the ExpressionDefinition. It holds all necessary information describing tested expression such as name, description and sample code.
When we look inside RxSandbox project we will see that the definitions of all standard operators are very similar, for example the Merge operator is defined like this:
public static ExpressionDefinition Merge() { Expression<Func<IObservable<string>, IObservable<string>, IObservable<string>, IObservable<string>>> expression = (a, b, c) => Observable.Merge(a, b, c); return ExpressionDefinition.Create(expression); }
This is all we need to write. Other things like operator’s name, description and the text of expression can be inferred from the Linq expression. Of course all these information can be set manually using appropriate Create method overload and ExpressionSettings class. All .Net types are supported as observable type, not only the System.String type like in this example. The only requirement is there must exist a TypeConverter for that type. Later in this post I’ll how implement TypeConverter for custom type and how to create ExpressionDefinition without using Linq expression but using the whole method with many statements. The second very important class in RxSandbox API is an ExpressionInstance class which is very useful in scenarios where we want to use some RxSandbox functionalities directly from code without any UI experience (for example during writing Unit Tests or recording marble diagram).
Expression<Func<IObservable<string>, IObservable<string>, IObservable<string>, IObservable<string>>> expression = (a, b, c) => Observable.Merge(a, b, c); ExpressionDefinition definition = ExpressionDefinition.Create(expression); using (var instance = ExpressionInstance.Create(definition)) { ExpressionInstance instance = ExpressionInstance.Create(definition); // using non-generic type 'ObservableSource' ObservableSource output1 = instance.Output; output1.ObservableStr.Subscribe(Console.WriteLine); // using generic type 'ObservableSource<T>' ObservableOutput<string> output2 = instance.Output as ObservableOutput<string>; output2.Observable.Subscribe(Console.WriteLine); instance["a"].OnNext("one"); // using non-generic type 'ObservableInput' (instance["a"] as ObservableInput<string>).OnNext("two"); // using generic type instance["b"].OnNext("tree"); instance["a"].OnCompleted(); instance["b"].OnCompleted(); instance["c"].OnCompleted(); }
When we add delay before sending each input signal we can very easily record sample marble diagram.
internal static class Extensions { internal static void OnNext2(this ObservableInput input, string value) { input.OnNext(value); Thread.Sleep(100); } internal static void OnError2(this ObservableInput input) { input.OnError(new Exception()); Thread.Sleep(100); } internal static void OnCompleted2(this ObservableInput input) { input.OnCompleted(); Thread.Sleep(100); } }
Marble diagrams are described in a very simple object model.
This object model can be serialized to Xml format, for instance the code above creates fallowing marble diagram:
<Diagram> <Input Name="a"> <Marble Value="one" Order="0" /> <Marble Value="two" Order="1" /> <Marble Kind="OnCompleted" Order="3" /> </Input> <Input Name="b"> <Marble Value="tree" Order="2" /> <Marble Kind="OnCompleted" Order="4" /> </Input> <Input Name="c"> <Marble Kind="OnCompleted" Order="5" /> </Input> <Output> <Marble Value="one" Order="0" /> <Marble Value="two" Order="1" /> <Marble Value="tree" Order="2" /> <Marble Kind="OnCompleted" Order="5" /> </Output> </Diagram>
As we can see single marble diagram have a very simple Xml representation and diagrams for all standard operators from RxSandbox are stored in Diagrams.xml file (this file path can be changed in the configuration files).
Short description added to all standard operators extracted from Rx documentation Xml file is a next new feature of current release. Of course not all tested reactive expression are related to one particular operator so the description can be set manually (ExpressionSettings.Description property). When we want write a very complicated Linq expression or the expression is passed as delegate type to the ExpressionDefinition.Create method and we also want to provide description from particular Rx operator at the same time, we can do it indicating operator thought MethodInfo type (ExpressionSettings.Operator property).
The last but not least new feature is the extensibility mechanism. When we want to write our custom reactive expression without changing anything inside RxSadbox project we can do it by implementing IExpressionProvider interface directly or by inheriting from an abstract class ExpressionAttributeBasedProvider and setting our assembly name in the configuration file (ExtensionsAssembly element). During startup process RxSandbox loads that assembly, analyzes it and finds all expression providers. Few weeks ago I have been writing about Incremental Find implemented using Rx, lets see how such a query can tested via RxSandbox.
[AttributeUsage(AttributeTargets.Method,AllowMultiple = false, Inherited = true)] public class ExpressionAttribute : Attribute { } public interface IExpressionProvider { IEnumerable<ExpressionDefinition> GetExpressions(); } public abstract class ExpressionAttributeBasedProvider : IExpressionProvider { public IEnumerable<ExpressionDefinition> GetExpressions() { var q = from m in this.GetType().GetMethods() let attr = Attribute.GetCustomAttribute(m, typeof(ExpressionAttribute)) as ExpressionAttribute where attr != null select m.Invoke(null, null) as ExpressionDefinition; return q.ToList(); } } public class CustomExpressions : ExpressionAttributeBasedProvider { [Expression] public static ExpressionDefinition IncrementalSearch() { Func<IObservable<string>, IObservable<Person>, IObservable<Person>> expression = (codeChanged, webServiceCall) => { var q = from code in codeChanged from x in Observable.Return(new Unit()) .Delay(TimeSpan.FromSeconds(4)).TakeUntil(codeChanged) from result in webServiceCall.TakeUntil(codeChanged) select result; return q; }; return ExpressionDefinition.Create(expression, new ExpressionSettings { Name = "Incremental find", Description = @"Send the code of the person you are looking for, " + "after four seconds (if you don't send new code again) web service " + "will be called. The result won't be returned if new code is provided " + "in the meantime.", }); } } [TypeConverter(typeof(PersonConverter))] public class Person { public string Code { get; set; } public string Name { get; set; } } public class PersonConverter : TypeConverter { public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType) { if (sourceType == typeof (string)) return true; return base.CanConvertFrom(context, sourceType); } public override object ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, object value) { if (value is string) { string[] v = ((string)value).Split(new[] { ',' }); return new Person {Code = v[0], Name = v[1]}; } return base.ConvertFrom(context, culture, value); } public override object ConvertTo(ITypeDescriptorContext context, CultureInfo culture, object value, Type destinationType) { if (destinationType == typeof (string)) { var person = value as Person; return person.Code + "," + person.Name; } return base.ConvertTo(context, culture, value, destinationType); } }
So that’s it for this release of RxSandbox. I encourage you to play with it a little bit and let me know what you think.
And that’s not the end of the project. In upcoming releases I plan to:
- create Silverlight version with ability to write reactive expression directly in the web browser
- add integration with MEF
- add better look and feel experience
Here you can download sources and binaries