Today I gave my old (from 2009) presentation about the LINQ. The video can be found here, slides and all code samples presented during this talk can be downloaded from here. I hope you find it useful.
Friday, December 23, 2011
Tuesday, September 20, 2011
When the Reactive Framework meets F# 3.0
Few days ago I have been writing about asynchronous sequences in C#. This time we will see how easy it is to implement the same web crawler using Reactive Framework and a new F# 3.0 feature called Query Expressions. The original implementation of web crawler created by Tomas Petricek can be found here and my implementation is surprisingly similar:
let rec randomCrawl url =
let visited = new System.Collections.Generic.HashSet<_>()
let rec loop url = obs {
if visited.Add(url) then
let! doc = (downloadDocument url) |> fromAsync
match doc with
| Some doc ->
yield url, getTitle doc
for link in extractLinks doc do
yield! loop link
| _ -> () }
loop url
rxquery {
for (url, title) in randomCrawl "http://news.bing.com" do
where (url.Contains("bing.com") |> not)
select title
take 10 into gr
iter (printfn "%s" gr)
}
|> ObservableExtensions.Subscribe |> ignore
There are two interesting things inside the code above. The first one is “obs { … } ” code construction. This is custom implementation of Computation Expression builder. obs is just a normal variable of a type that contains a special set of methods like: Bind, Delay, For, Combine, … and so on. The F# compiler translates the code inside the curly brackets into code calling those methods. In my case the whole expression returns the implementation of IObservable<T> type. This allows us to write imperative code representing observable source where each “yield” call returns next value (OnNext of subscribed observer is being called) and the “let!” keyword causes the program to wait for the values from the other observable source (let! keyword works like await keyword in C#). See the implementation of the builder class below:
type ObsBuiler() =
member this.Zero () = Observable.Empty(Scheduler.CurrentThread)
member this.Yield v = Observable.Return(v, Scheduler.CurrentThread)
member this.Delay (f: _ -> IObservable<_>) = Observable.Defer(fun _ -> f())
member this.Combine (o1,o2) = Observable.Concat(o1,o2)
member this.For (s:seq<_>, body : _ -> IObservable<_>) = Observable.Concat(s.Select(body))
member this.YieldFrom a : IObservable<_> = a
member this.Bind ((o : IObservable<_>),(f : _ -> IObservable<_>)) = o.SelectMany(f)
member this.TryFinally((o : IObservable<_>),f : unit -> unit ) = Observable.Finally(o,fun _ -> f())
member this.TryWith((o : IObservable<_>),f : Exception -> IObservable<_> ) = Observable.Catch(o,fun ex -> f(ex))
member this.While (p,body) = Observable.While((fun () -> p()), body)
member this.Using (dis,body) = Observable.Using( (fun () -> dis), fun d -> body(d))
let obs = new ObsBuiler()
The second interesting part is “rxquery { … }” code construction. This time we are using a feature called Query Expressions introduced in F# 3.0 which was released few day ago during Build conference together with Visual Studio 11 and Windows 8. We can write queries very similar to LINQ queries and the F# compiler translate them into code calling methods like: where, select, groupBy, take and on so on. So it works like LINQ queries in C# but here we can extend the set of available methods arbitrarily !!! Look at Zip and ForkJoin methods below which are not available by default with QueryBuilder implementation working with IEnumerable<T> type. Let’s see the implementation of query builder class:
type RxQueryBuiler() =
member this.For (s:IObservable<_>, body : _ -> IObservable<_>) = s.SelectMany(body)
[<CustomOperation("select", AllowIntoPattern=true)>]
member this.Select (s:IObservable<_>, [<ProjectionParameter>] selector : _ -> _) = s.Select(selector)
[<CustomOperation("where", MaintainsVariableSpace=true, AllowIntoPattern=true)>]
member this.Where (s:IObservable<_>, [<ProjectionParameter>] predicate : _ -> bool ) = s.Where(predicate)
[<CustomOperation("takeWhile", MaintainsVariableSpace=true, AllowIntoPattern=true)>]
member this.TakeWhile (s:IObservable<_>, [<ProjectionParameter>] predicate : _ -> bool ) = s.TakeWhile(predicate)
[<CustomOperation("take", MaintainsVariableSpace=true, AllowIntoPattern=true)>]
member this.Take (s:IObservable<_>, count) = s.Take(count)
[<CustomOperation("skipWhile", MaintainsVariableSpace=true, AllowIntoPattern=true)>]
member this.SkipWhile (s:IObservable<_>, [<ProjectionParameter>] predicate : _ -> bool ) = s.SkipWhile(predicate)
[<CustomOperation("skip", MaintainsVariableSpace=true, AllowIntoPattern=true)>]
member this.Skip (s:IObservable<_>, count) = s.Skip(count)
member this.Zero () = Observable.Empty(Scheduler.CurrentThread)
member this.Yield (value) = Observable.Return(value)
[<CustomOperation("count")>]
member this.Count (s:IObservable<_>) = Observable.Count(s)
[<CustomOperation("all")>]
member this.All (s:IObservable<_>, [<ProjectionParameter>] predicate : _ -> bool ) = s.All(new Func<_,bool>(predicate))
[<CustomOperation("contains")>]
member this.Contains (s:IObservable<_>, key) = s.Contains(key)
[<CustomOperation("distinct", MaintainsVariableSpace=true, AllowIntoPattern=true)>]
member this.Distinct (s:IObservable<_>) = s.Distinct()
[<CustomOperation("exactlyOne")>]
member this.ExactlyOne (s:IObservable<_>) = s.Single()
[<CustomOperation("exactlyOneOrDefault")>]
member this.ExactlyOneOrDefault (s:IObservable<_>) = s.SingleOrDefault()
[<CustomOperation("find")>]
member this.Find (s:IObservable<_>, [<ProjectionParameter>] predicate : _ -> bool) = s.First(new Func<_,bool>(predicate))
[<CustomOperation("head")>]
member this.Head (s:IObservable<_>) = s.First()
[<CustomOperation("headOrDefault")>]
member this.HeadOrDefault (s:IObservable<_>) = s.FirstOrDefault()
[<CustomOperation("last")>]
member this.Last (s:IObservable<_>) = s.Last()
[<CustomOperation("lastOrDefault")>]
member this.LastOrDefault (s:IObservable<_>) = s.LastOrDefault()
[<CustomOperation("maxBy")>]
member this.MaxBy (s:IObservable<'a>, [<ProjectionParameter>] valueSelector : 'a -> 'b) = s.MaxBy(new Func<'a,'b>(valueSelector))
[<CustomOperation("minBy")>]
member this.MinBy (s:IObservable<'a>, [<ProjectionParameter>] valueSelector : 'a -> 'b) = s.MinBy(new Func<'a,'b>(valueSelector))
[<CustomOperation("nth")>]
member this.Nth (s:IObservable<'a>, index ) = s.ElementAt(index)
[<CustomOperation("sumBy")>]
member inline this.SumBy (s:IObservable<_>,[<ProjectionParameter>] valueSelector : _ -> _) = s.Select(valueSelector).Aggregate(Unchecked.defaultof<_>, new Func<_,_,_>( fun a b -> a + b))
[<CustomOperation("groupBy", AllowIntoPattern=true)>]
member this.GroupBy (s:IObservable<_>,[<ProjectionParameter>] keySelector : _ -> _) = s.GroupBy(new Func<_,_>(keySelector))
[<CustomOperation("groupValBy", AllowIntoPattern=true)>]
member this.GroupValBy (s:IObservable<_>,[<ProjectionParameter>] resultSelector : _ -> _,[<ProjectionParameter>] keySelector : _ -> _) = s.GroupBy(new Func<_,_>(keySelector),new Func<_,_>(resultSelector))
[<CustomOperation("join", IsLikeJoin=true)>]
member this.Join (s1:IObservable<_>,s2:IObservable<_>, [<ProjectionParameter>] s1KeySelector : _ -> _,[<ProjectionParameter>] s2KeySelector : _ -> _,[<ProjectionParameter>] resultSelector : _ -> _) = s1.Join(s2,new Func<_,_>(s1KeySelector),new Func<_,_>(s2KeySelector),new Func<_,_,_>(resultSelector))
[<CustomOperation("groupJoin", AllowIntoPattern=true)>]
member this.GroupJoin (s1:IObservable<_>,s2:IObservable<_>, [<ProjectionParameter>] s1KeySelector : _ -> _,[<ProjectionParameter>] s2KeySelector : _ -> _,[<ProjectionParameter>] resultSelector : _ -> _) = s1.GroupJoin(s2,new Func<_,_>(s1KeySelector),new Func<_,_>(s2KeySelector),new Func<_,_,_>(resultSelector))
[<CustomOperation("zip", IsLikeZip=true)>]
member this.Zip (s1:IObservable<_>,s2:IObservable<_>,[<ProjectionParameter>] resultSelector : _ -> _) = s1.Zip(s2,new Func<_,_,_>(resultSelector))
[<CustomOperation("forkJoin", IsLikeZip=true)>]
member this.ForkJoin (s1:IObservable<_>,s2:IObservable<_>,[<ProjectionParameter>] resultSelector : _ -> _) = s1.ForkJoin(s2,new Func<_,_,_>(resultSelector))
[<CustomOperation("iter")>]
member this.Iter(s:IObservable<_>, [<ProjectionParameter>] selector : _ -> _) = s.Do(selector)
let rxquery = new RxQueryBuiler()
Thursday, September 15, 2011
Design-time support for Caliburn.Micro
Recently me together with Bartek Pampuch have been wondering if it’s possible to fire Caliburn.Micro’s binding process at design-time. Caliburn.Micro is a really great framework (just take a look at some of our multitouch apps based on this framework and BFSharp). You just name the controls appropriately and all magic happens automatically. Controls are bound to the View Model’s properties and methods for you. The problem is that it is all happing at run time not at design time. When you are preparing the form in the Visual Studio or Expression Blend you aren’t able to see how the form will look like with bound data. In some cases such feature would very useful, especially when we want to use the sample data generated by Expression Blend or Visual Studio designers. In this post we will show you how to run Caliburn.Micro’s conventions based binding mechanism at design time.
Firstly, we discovered that we can change the objects tree representing the screen and that change is not persisted back to xaml file. You can read more about this feature in my previous post. Secondly, we tried to run Caliburn.Micro’s binding process to see what happens. What was really amazing is that after first try all just started working smoothly! The framework itself is implemented so well :)
One of the samples provided with Caliburn.Micro release is a very simple application called GameLibrary. AddGameView.xaml file defines the screen that looks like this:
After setting up sample data generated by Visual Studio and setting attached property 'DesignTime.Enable=”True” the same form looks like this:
In the simplest scenario all we need to run Caliburn.Micro binding process at design time is setting design time data context and enabling binding via Enable attached property. In sample above we are setting sample data generated by Visual Studio because the AddGameViewModel class doesn’t provide default constructor. Of course we could always add a default constructor in code and define view model instance in xaml file.
I chose this particular form intentionally because it contains the control that hasn’t defined custom binding convention by default. The Rating control is responsible for displaying Rating property value using stars. Rating property value is 0.8 so 4 stars should be displayed. In such cases where we are using controls with custom binding convention we need to register those conventions at design time. Any application using Caliburn.Micro has a type inherited from Bootstrapper type. This type is a starting point of our application and the instance of that type very often is defined as a resource inside App.xaml file. In fact Caliburn.Micro framework is already prepared for design time scenarios because Bootstrapper type contains virtual method called StartDesignTime. Let’s override this method and register appropriate convention:
public class Bootstrapper
{
public Bootstrapper()
{
if (Execute.InDesignMode) StartDesignTime(); else StartRuntime();
}
}
public class Bootstrapper<TRootModel> : Bootstrapper
{
}
public class AppBootstrapper : Bootstrapper<IShell>
{
private static bool isInitializedInDesignTime = false;
protected override void StartDesignTime()
{
base.StartDesignTime();
if (isInitializedInDesignTime)
return;
isInitializedInDesignTime = true;
ConfigureConventions();
}
void ConfigureConventions()
{
ConventionManager.AddElementConvention<Rating>(Rating.ValueProperty, "Value", "ValueChanged");
}
}
The instance of AppBootstapper type can be created many times at design time so boolean flag ensures that the conventions will be registered only once. When we compile the project and reopen the form we will see something like this:
This form represents a very simple scenario where the view model type has only properties of simple types like: string, double. boolean. It doesn’t contain any property of collection type or other view model type. In such scenarios Caliburn.Micro can find the appropriate type of view (control type) based on the type of view model. Let’s see another form called ResultsView.xaml to demonstrate this case:
This is a typical problem when we work with Caliburn.Micro and we cannot see anything at design time because the control is entirely collapsed :). Let’s see what happen after connecting sample data.
The list contains some elements but the font color is white so we aren’t able to read it because of the default Visual Studio background color. We can try to open this form inside Expression Blend where the background is dark or we can use custom design time attributes presented in the previous post.
Now we can see that Caliburn.Micro is not able to find view for view model type representing list item. It’s because we are using sample data mechanism from Visual Studio which generates dynamic type _.di0.GameLibrary.ViewModels.IndividualResultViewModel with the shape of the original view model type GameLibrary.ViewModels.IndividualResultViewModel. We need to change the way Caliburn.Micro is searching for view type based on specified view model type.
protected override void StartDesignTime()
{
base.StartDesignTime();
if (isInitializedInDesignTime)
return;
isInitializedInDesignTime = true;
ConfigureConventions();
AssemblySource.Instance.AddRange(new[] { typeof(App).Assembly });
var originalLocateTypeForModelType = ViewLocator.LocateTypeForModelType;
Func<Type, bool> isDesignTimeType = type => type.Assembly.IsDynamic;
ViewLocator.LocateTypeForModelType = (modelType, displayLocation, context) =>
{
var type = originalLocateTypeForModelType(modelType, displayLocation, context);
if (type == null && isDesignTimeType(modelType))
{
if (modelType.Name == "IndividualResultViewModel")
{
type = typeof(IndividualResultView);
}
}
return type;
};
IoC.GetInstance = base.GetInstance;
IoC.GetAllInstances = base.GetAllInstances;
}
Finally the list of items and generated sample data look like this:
Attached property is not the most convenient way of extending the designer functionality because we need to write xaml code. I tried to rewrite attached property to custom behavior so we could use dra&drop instead of writing the code. The problem is that behaviors code is not executed at design time. But I have good news. If you are creating UI with Expression Blend you can use our attached property on the property grid like a normal property.
It was possible thanks to the usage of attribute called AttachedPropertyBrowsableForTypeAttribute decorating attached property. Everything that was presented so far works both in Silverlight and WPF environments. It’s time to reveal how the magic works.
public static class DesignTime
{
public static DependencyProperty EnableProperty =
DependencyProperty.RegisterAttached(
"Enable",
typeof(bool),
typeof(DesignTime),
new PropertyMetadata(new PropertyChangedCallback(EnableChanged)));
#if !SILVERLIGHT && !WP7
[AttachedPropertyBrowsableForTypeAttribute(typeof(DependencyObject))]
#endif
public static bool GetEnable(DependencyObject dependencyObject)
{
return (bool)dependencyObject.GetValue(EnableProperty);
}
public static void SetEnable(DependencyObject dependencyObject, bool value)
{
dependencyObject.SetValue(EnableProperty, value);
}
static void EnableChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
{
if (!Execute.InDesignMode)
return;
BindingOperations.SetBinding(d, DataContextProperty, (bool)e.NewValue ? new Binding() : null);
}
private static readonly DependencyProperty DataContextProperty =
DependencyProperty.RegisterAttached(
"DataContext",
typeof(object),
typeof(DesignTime),
new PropertyMetadata(new PropertyChangedCallback(DataContextChanged))
);
private static void DataContextChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
{
if (!Execute.InDesignMode)
return;
object enable = d.GetValue(EnableProperty);
if (enable == null || ((bool)enable) == false || e.NewValue == null)
return;
var fe = d as FrameworkElement;
if (fe == null)
return;
ViewModelBinder.Bind(e.NewValue, d, string.IsNullOrEmpty(fe.Name) ? fe.GetHashCode().ToString() : fe.Name);
}
}
I hope you find this solution useful.
Custom design-time attributes in Silverlight and WPF designer
We all know the standard design-time attributes in the Silverlight designer like d:DataContext, d:DesignWidth or d:DesignHeight. Let’s see how easy we can add some new attributes.
using System.ComponentModel;
using System.Windows;
namespace DesignTimeProperties
{
public static class d
{
static bool? inDesignMode;
/// <summary>
/// Indicates whether or not the framework is in design-time mode. (Caliburn.Micro implementation)
/// </summary>
private static bool InDesignMode
{
get
{
if (inDesignMode == null)
{
var prop = DesignerProperties.IsInDesignModeProperty;
inDesignMode = (bool)DependencyPropertyDescriptor.FromProperty(prop, typeof(FrameworkElement)).Metadata.DefaultValue;
if (!inDesignMode.GetValueOrDefault(false) && System.Diagnostics.Process.GetCurrentProcess()
.ProcessName.StartsWith("devenv", System.StringComparison.Ordinal))
inDesignMode = true;
}
return inDesignMode.GetValueOrDefault(false);
}
}
public static DependencyProperty BackgroundProperty = DependencyProperty.RegisterAttached(
"Background", typeof(System.Windows.Media.Brush), typeof(d),
new PropertyMetadata(new PropertyChangedCallback(BackgroundChanged)));
public static System.Windows.Media.Brush GetBackground(DependencyObject dependencyObject)
{
return (System.Windows.Media.Brush)dependencyObject.GetValue(BackgroundProperty);
}
public static void SetBackground(DependencyObject dependencyObject, System.Windows.Media.Brush value)
{
dependencyObject.SetValue(BackgroundProperty, value);
}
private static void BackgroundChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
{
if (!InDesignMode)
return;
d.GetType().GetProperty("Background").SetValue(d, e.NewValue, null);
}
}
}
Great, but what about many others very useful properties ? Do we need to write them all manually? No, we don’t. I have prepared T4 template that generates code for all properties of all controls available in WPF and Silverlight assemblies. When you look carefully at the code above you will find that I am using the reflection to set the value of the property. It’s because some properties like Background are defined many times in many different controls so instead of checking the actual control type I assume that the control contains appropriate property. This assumption simplifies the implementation very much. And of course the property can be set only in design time when we are editing the form inside Expression Blend or Visual Studio. T4 template code analyzes all properties from all controls and properties with the same names but different types are skipped during generation process.
Sunday, September 4, 2011
Programming with C# asynchronous sequences
Tomas Petricek in his last blog post titled “Programming with F# asynchronous sequences” presents F# implementation of something called asynchronous sequences. In this post I will show you how the same concept can be implemented in C#. Let’s look at the sample code below to better understand what the asynchronous sequence is:
IEnumerable<...> AsyncSeq()
{
yield return "Hello";
await TaskEx.Delay(100);
yield return "world!";
}
Asynchronous sequences is a code that produces the sequence of values generated on demand (this is how the IEnumerable interface can be interpreted) but additionally does some asynchronous work during the evaluation process (await keyword). Every time the client of asynchronous sequence calls MoveNext method, next value is being evaluated. The key feature here is that the client decides when to produce next value and when to stop the processing.
There are two problems with such an implementation of asynchronous sequence. Sequences in .Net world are represented with IEnumerable interface, but the interface allows only synchronous processing. Since the MoveNext method returns bool value in the interface implementation, we need to immediately decide whether the next value can be produced or not. In the asynchronous processing it can take a few minutes or even hours to provide such information. The second problem is that so far we cannot mix together await keyword (Async Ctp) with yield return/yield break keywords inside the same method body. My solution resolves those two problems and the above sequence can be implemented the fallowing way:
IEnumerable<AsyncSeqItem<string>> AsyncSeq()
{
yield return "Hello";
yield return TaskEx.Delay(100);
yield return "world!";
}
public enum AsyncSeqItemMode
{
Value, Task, Sequence
}
public class AsyncSeqItem<T>
{
public AsyncSeqItemMode Mode { get; private set; }
public T Value { get; private set; }
public Task Task { get; private set; }
public IEnumerable<AsyncSeqItem<T>> Seq { get; private set; }
public AsyncSeqItem(T value)
{
Value = value;
Mode = AsyncSeqItemMode.Value;
}
public AsyncSeqItem(Task task)
{
Task = task;
Mode = AsyncSeqItemMode.Task;
}
public AsyncSeqItem(IEnumerable<AsyncSeqItem<T>> seq)
{
Seq = seq;
Mode = AsyncSeqItemMode.Sequence;
}
public static implicit operator AsyncSeqItem<T>(T value)
{
return new AsyncSeqItem<T>(value);
}
public static implicit operator AsyncSeqItem<T>(Task task)
{
return new AsyncSeqItem<T>(task);
}
}
AsyncSeqItem represents one of three following values:
- Value – next value generated by the sequence
- Task – some asynchronous work that needs to be awaited for before going forward
- Sequence – it’s used with recursive calls and it means that we want to use tail recursion
There are two ways of consuming such sequence in the client:
public static class AsyncSeqExtensions
{
public static IEnumerable<Task<Option<T>>> ToTaskEnumerable<T>(this IEnumerable<AsyncSeqItem<T>> seq, bool continueOnCapturedContext = true)
{ ... }
public static IAsyncEnumerable<T> ToAsyncEnumerable<T>(this IEnumerable<AsyncSeqItem<T>> seq, bool continueOnCapturedContext = true)
{ ... }
}
public class Option<T>
{
public T Value { get; private set; }
public bool HasValue { get; private set; }
public Option()
{
HasValue = false;
}
public Option(T value)
{
Value = value;
HasValue = true;
}
public static implicit operator Option<T>(T value)
{
return new Option<T>(value);
}
}
In the first approach we are calling ToAsyncEnumerable extension method returning the sequence of tasks. Each task wraps special type called Option<T> which can be used similarly to Nullable<T> type except that it works with value and reference types. Returning task with option object without the value means that we reached the end of the sequence. I also provide few standard LINQ operators built on the top of such a sequence semantic:
public static class AsyncSeqExtensions
{
async private static Task ForEachTaskImpl<T>(this IEnumerable<Task<Option<T>>> seq, Action<Task<Option<T>>> action)
{
foreach (var task in seq)
{
await task;
action(task);
}
}
public static Task ForEachTask<T>(this IEnumerable<Task<Option<T>>> seq, Action<Task<Option<T>>> action)
{
return ForEachTaskImpl(seq, action);
}
public static Task ForEach<T>(this IEnumerable<Task<Option<T>>> seq, Action<T> action)
{
return seq.ForEachTask(task =>
{
if(task.Result.HasValue)
action(task.Result.Value);
});
}
async private static Task<T[]> ToArrayImpl<T>(IEnumerable<Task<Option<T>>> seq)
{
var list = new List<T>();
await seq.ForEach(v => list.Add(v));
return list.ToArray();
}
public static Task<T[]> ToArray<T>(this IEnumerable<Task<Option<T>>> seq)
{
return ToArrayImpl(seq);
}
public static IEnumerable<Task<Option<TResult>>> Select<T, TResult>(this IEnumerable<Task<Option<T>>> source,
Func<T,TResult> selector) { ... }
public static IEnumerable<Task<Option<T>>> Where<T>(this IEnumerable<Task<Option<T>>> source,
Func<T, bool> predicate) { ... }
public static IEnumerable<Task<Option<T>>> Take<T>(this IEnumerable<Task<Option<T>>> source,
int count) { ... }
...
}
Returning additional task object at the end of a sequence with special value allows us to use standard IEnumerable<T> interface but it’s a little bit inconvenient. In the second approach we use the IAsyncEnumerable interface from the Reactive Framework library released some time ago.
public interface IAsyncEnumerable<out T>
{
IAsyncEnumerator<T> GetEnumerator();
}
public interface IAsyncEnumerator<out T> : IDisposable
{
Task<bool> MoveNext();
T Current { get; }
}
public static class AsyncEnumerable
{
public static IAsyncEnumerable<TResult> Select<TSource, TResult>(IAsyncEnumerable<TSource> source,
Func<TSource, TResult> selector) { ... }
public static IAsyncEnumerable<TSource> Where<TSource>(IAsyncEnumerable<TSource> source,
Func<TSource, bool> predicate) { ... }
public static IAsyncEnumerable<TSource> Take<TSource>(IAsyncEnumerable<TSource> source,
int n) { ... }
}
This interface perfectly represents the semantic of asynchronous sequence. Rx library also provides many standard LINQ operations like: Where, Select, Take, Sum, First and so on. This allows us to write almost any LINQ query executing on the top of asynchronous sequence.
Now let’s summarize what we achieved so far. We can write imperative code implementing asynchronous sequence. We can use extension method to create one of two asynchronous sequence representations. Finally we can iterate through all items in such a sequence or we can build a new sequence object using LINQ operators.
The C# version of the web crawler presented in Tomas Petricek’s blog post could look like this:
public static class AsyncSeqSample
{
async public static Task CrawlBingUsingAsyncEnumerable()
{
await RandomCrawl("http://news.bing.com")
.ToAsyncEnumerable()
.Where(t => !t.Item1.Contains("bing.com"))
.Select(t => t.Item2)
.Take(10)
.ForEach(Console.WriteLine);
Console.WriteLine("the end...");
}
async public static Task CrawlBingUsingTaskEnumerable()
{
await RandomCrawl("http://news.bing.com")
.ToTaskEnumerable()
.Where(t => !t.Item1.Contains("bing.com"))
.Select(t => t.Item2)
.Take(10)
.ForEach(Console.WriteLine);
Console.WriteLine("the end...");
}
public static IEnumerable<AsyncSeqItem<Tuple<string, string>>> RandomCrawl(string url)
{
return RandomCrawlLoop(url, new HashSet<string>());
}
private static IEnumerable<AsyncSeqItem<Tuple<string,string>>> RandomCrawlLoop(string url, HashSet<string> visited)
{
if (visited.Add(url))
{
var downloadTask = DownloadDocument(url);
yield return downloadTask;
if (downloadTask.Result.HasValue)
{
var doc = downloadTask.Result.Value;
yield return Tuple.Create(url, GetTitle(doc));
foreach (var link in ExtractLinks(doc))
{
foreach (var l in RandomCrawlLoop(link, visited))
{
yield return l;
}
}
}
}
}
private static string[] ExtractLinks(HtmlDocument doc)
{
try
{
var q = from a in doc.DocumentNode.SelectNodes("//a")
where a.Attributes.Contains("href")
let href = a.Attributes["href"].Value
where href.StartsWith("http://")
let endl = href.IndexOf('?')
select endl > 0 ? href.Substring(0, endl) : href;
return q.ToArray();
}
catch
{
return new string[0];
}
}
async private static Task<Option<HtmlDocument>> DownloadDocument(string url)
{
try
{
var client = new WebClient();
var html = await client.DownloadStringTaskAsync(url);
var doc = new HtmlDocument();
doc.LoadHtml(html);
return new Option<HtmlDocument>(doc);
}
catch (Exception)
{
return new Option<HtmlDocument>();
}
}
private static string GetTitle(HtmlDocument doc)
{
var title = doc.DocumentNode.SelectSingleNode("//title");
return title != null ? title.InnerText.Trim() : "Untitled";
}
}
Now let’s see how ToAsyncEnumerable and ToTaskEnumerable methods have been implemented:
public static class AsyncSeqExtensions
{
public static IAsyncEnumerable<T> ToAsyncEnumerable<T>(this IEnumerable<AsyncSeqItem<T>> seq, bool continueOnCapturedContext = true)
{
if (seq == null) throw new ArgumentNullException("seq");
return new AnonymousAsyncEnumerable<T>(() =>
{
var enumerator = seq.ToTaskEnumerable(continueOnCapturedContext).GetEnumerator();
seq = null; // holding reference to seq parameter introduces memory leaks when asynchronous sequence uses recursive calls
TaskCompletionSource<bool> currentTcs = null;
Task<Option<T>> currentTask = null;
return new AnonymousAsyncEnumerator<T>(
() =>
{
currentTcs = new TaskCompletionSource<bool>();
if (CheckEndOfSeq(currentTask) == false)
{
currentTcs.SetResult(false);
return currentTcs.Task;
}
enumerator.MoveNext();
enumerator.Current.ContinueWith(t =>
{
if (t.IsFaulted)
{
currentTcs.SetException(t.Exception);
}
else
{
if (!t.Result.HasValue)
{
currentTcs.SetResult(false);
}
else
{
currentTask = t;
currentTcs.SetResult(true);
}
}
});
return currentTcs.Task;
},
() => currentTask.Result.Value
);
});
}
public static IEnumerable<Task<Option<T>>> ToTaskEnumerable<T>(this IEnumerable<AsyncSeqItem<T>> seq, bool continueOnCapturedContext = true)
{
if (seq == null) throw new ArgumentNullException("seq");
return new AnonymousEnumerable<Task<Option<T>>>(() =>
{
var synchronizationContext = continueOnCapturedContext ? SynchronizationContext.Current : null;
var enumerator = seq.GetEnumerator();
seq = null; // holding reference to seq parameter introduces memory leaks when asynchronous sequence uses recursive calls
TaskCompletionSource<Option<T>> currentTcs = null;
return new AnonymousEnumerator<Task<Option<T>>>(
() =>
{
if (CheckEndOfSeq(currentTcs) == false)
return false;
currentTcs = new TaskCompletionSource<Option<T>>();
Action moveNext = null;
moveNext = () =>
{
Start:
bool b;
try
{
b = enumerator.MoveNext();
}
catch (Exception exception)
{
currentTcs.SetException(exception);
return;
}
if (b == false)
{
currentTcs.SetResult(new Option<T>());
}
else
{
var c = enumerator.Current;
if (c.Mode == AsyncSeqItemMode.Value)
{
currentTcs.SetResult(c.Value);
}
else if (c.Mode == AsyncSeqItemMode.Task)
{
if (synchronizationContext != null)
c.Task.ContinueWith(_ => synchronizationContext.Post(s => ((Action)s)(), moveNext));
else
c.Task.ContinueWith(_ => moveNext());
}
else if (c.Mode == AsyncSeqItemMode.Sequence)
{
enumerator = c.Seq.GetEnumerator();
goto Start;
}
}
};
moveNext();
return true;
},
() => currentTcs.Task
);
});
}
}
As you can see the implementation is really simple but the whole concept of asynchronous sequence is very powerful.
Friday, May 6, 2011
Rx projects update
In the past I have been writing about my projects related to Reactive Framework. Subsequent versions of Rx are published every one or two months and of course I have been updating my projects for my own purpose but they have not been available publicly. Last release of Rx (RC0) distinguishes two versions “Stable” and “Experimental” so I think we can expect that Rx API is really close to the final version. I thought that it is a great opportunity to publish updated versions of all my projects:
Basically the content of the first two projects stays the same but the RxSandbox has changed a little bit. Few new operators have been added and there are currently 69 total, additionally operators that are still in experimental state (not stable) are highlighted. I am considering moving RxSandbox application from WPF to Silverlight, that will simplify the process of deployment and the end users will be always up to date with the newest version of Rx. What do you think?
Thursday, May 5, 2011
Simplifying XNA game development using Async CTP or F# asynchronous workflow (GameAwaiter)
Last time I have been writing about launching the Async CTP on WP7 platform. The Async CTP Refresh has been released during MIX 2011 and finally WP7 platform is supported, so my last post is quite obsolete right now. But I hope that, after reading it, you better understand how Async CTP works underneath.
This time we will go one step forward. Once we have Tasks on WP7 and two new keywords await and async in C# we can use all that to change the way we write games in XNA.
If you know something about TPL and XNA framework it may seem a bit strange how I am going to mix them both since at first look TPL is all about building multithreaded applications and XNA application uses one single thread in most cases.
I will not not write too much about XNA framework itself (especially because I am a beginner in game development :) ) but if you try writing even a very simple 2D game you will find out that code for XNA is really hard to understand and maintain. Probably you will have many fields storing state of the game items, many boolean flags and a lot of if-then-else statements changing that state. You can say that the game is actually one big state machine. That’s what I’ve found out after writing my fist game in XNA.
Async CTP can change the structure of the code dramatically so we can easily discover the flow of the game logic and what is most interesting everything is running on single thread.
In this post I will show you three implementations of a very simple 2D game called Smiley. All of those implementations use XNA framework but the first one is a typical XNA approach and the last two are totally different. They are using Async CTP and the F# asynchronous workflow.
I took the idea for the game from the Tomas Petricek’s master thesis and it can be presented on the three screens:
We tap the screen to start the game, then we have 20 seconds to hit moving Smiley image as many times as we can and at the end the number of hits is displayed for 4 seconds. The first screen appears again. The Smiley position is changing in 2 seconds periods or directly after tapping on it.
Originally, that game was implemented in F# using Joinads. A few months ago I have implemented that game also in F# using Reactive Framework and treating the IObservable<T> type as the Monad type. With that I built my own workflow builder. I will try to describe that approach in the next blog post but now let’s come back to the subject of today’s post.
As you can see in the class diagram above, all three implementations have a common base class called SmileyGameBase. That class inherits from Game class which is the main XNA component representing the whole game. And all we need to do when writing an XNA game is to override two methods: Update and Draw. Update method is responsible for recalculating the game state and the Draw method draws all game items like texts, textures and so on. Both methods are called by XNA environment (Update before Draw) many times during each second when the game is running (about 30 times per second on WP7). In our game the Draw method looks the same in all three cases but the Update is specific for each of them.
public abstract class SmileyGameBase : Game { // ... protected override void Draw(GameTime gameTime) { GraphicsDevice.Clear(Color.Black); _spriteBatch.Begin(); if (_isRunning) { float fontScale = 2; _spriteBatch.DrawString(_spriteFont, "time : " + _time, new Vector2(10, 5), Color.White, 0, Vector2.Zero, fontScale, SpriteEffects.None, 0); _spriteBatch.DrawString(_spriteFont, "score : " + _score, new Vector2(500, 5), Color.White, 0, Vector2.Zero, fontScale, SpriteEffects.None, 0); _spriteBatch.Draw(_smileyTexture, _smileyPosition, Color.White); } else { float fontScale = 3; _spriteBatch.DrawString(_spriteFont, _message, new Vector2(10, 100), Color.Red, 0, Vector2.Zero, fontScale, SpriteEffects.None, 0); } _spriteBatch.End(); base.Draw(gameTime); } }
The code is pretty simple, we have few fields representing the game state and the Draw method draws Smiley image and prints some text messages. Now let’s look at the XNA and Async CTP implementations to compare them together.
This is the Async CTP implementation.
using System.Threading.Tasks; using Microsoft.Xna.Framework.Input.Touch; namespace SmileyGame.Common { using AsyncXnaIntegration; public abstract class AsyncCtpGame : SmileyGameBase { private GameAwaiter _gameAwaiter; protected AsyncCtpGame() { _gameAwaiter = new GameAwaiter(this); Components.Add(_gameAwaiter); } protected override void LoadContent() { base.LoadContent(); StartMenu(); } async private void StartMenu() { while (true) { _message = "tap to start the game ... "; await _gameAwaiter.Gesture(GestureType.Tap); _isRunning = true; await StartGame(); _isRunning = false; _message = "your score : " + _score; await _gameAwaiter.Delay(4000); } } async private Task StartGame() { _score = 0; var timer = StartTimer(GameDuration); while (true) { var match = await _gameAwaiter.WhenAny(timer, _gameAwaiter.Delay(ChangePositionPeriod), _gameAwaiter.Gesture(IsSmileyClicked)); switch (match.Index) { case 0: return; case 1: _smileyPosition = GetRandomPostion(); break; case 2: _smileyPosition = GetRandomPostion(); _score++; break; default: break; } } } async private Task StartTimer(int n) { while (n >= 0) { _time = n; await _gameAwaiter.Delay(TimerPeriod); n--; } } } }
And here is the pure XNA implementation.
using System; using System.Collections.Generic; using System.Linq; using Microsoft.Xna.Framework; using Microsoft.Xna.Framework.Input.Touch; namespace SmileyGame.Common { public abstract class XnaGame : SmileyGameBase { protected XnaGame() { _message = "tap to start the game ... "; } private TimeSpan _nextTimerTime; private TimeSpan _nextChangePostionTime; private TimeSpan? _nextDisplayFinalScoreTime; protected override void Update(GameTime gameTime) { base.Update(gameTime); var gestures = GetGestures(); if (!_isRunning) { if (_nextDisplayFinalScoreTime.HasValue) { if (gameTime.TotalGameTime > _nextDisplayFinalScoreTime.Value) { _nextDisplayFinalScoreTime = null; _message = "tap to start the game ... "; } } else if (gestures.Any(g => IsGestureType(g, GestureType.Tap))) { _isRunning = true; _score = 0; _time = GameDuration; _nextTimerTime = gameTime.TotalGameTime + TimeSpan.FromMilliseconds(TimerPeriod); _nextChangePostionTime = gameTime.TotalGameTime + TimeSpan.FromMilliseconds(ChangePositionPeriod); } } else { if (gameTime.TotalGameTime > _nextTimerTime ) { _time--; _nextTimerTime = gameTime.TotalGameTime + TimeSpan.FromMilliseconds(TimerPeriod); if (_time < 0) { _isRunning = false; _nextDisplayFinalScoreTime = gameTime.TotalGameTime + TimeSpan.FromMilliseconds(4000); _message = "your score : " + _score; } } else if (gameTime.TotalGameTime > _nextChangePostionTime ) { _smileyPosition = GetRandomPostion(); _nextChangePostionTime = gameTime.TotalGameTime + TimeSpan.FromMilliseconds(ChangePositionPeriod); } else if (gestures.Any(IsSmileyClicked)) { _smileyPosition = GetRandomPostion(); _nextChangePostionTime = gameTime.TotalGameTime + TimeSpan.FromMilliseconds(ChangePositionPeriod); _score++; } } } private static bool IsGestureType(GestureSample gestureSample, GestureType gestureType) { return (gestureSample.GestureType & gestureType) == gestureType; } private static List<GestureSample> GetGestures() { var gestures = new List<GestureSample>(); while (TouchPanel.IsGestureAvailable) gestures.Add(TouchPanel.ReadGesture()); return gestures; } } }
When we read first implementation we can easily discover the flow of the program and the order of particular pieces of code look natural. In the pure XNA approach it is really hard to understand how the game works internally and how to extend them. Let’s image that we would like to display sequentially “3” “2” “1” “start” just after first tap in 1 second periods. Think how to do it in each of these two approaches ?
So now lets go to F# implementation. F# has something called asynchronous workflow which was available for F# developers long before Async CTP (and in fact it was the inspiration for Async CTP authors) the F# implementation is very similar to Async CTP one.
namespace SmileyGame.FSharp open SmileyGame.Common open AsyncXnaIntegration open SmileyGame.FSharp.Extensions open Microsoft.Xna.Framework.Input.Touch type FSharpGame() as this = inherit SmileyGameBase() let _gameAwaiter = new GameAwaiter(this) do this.Components.Add(_gameAwaiter) // access to protected members member private this.set_smileyPosition p = this._smileyPosition <- p member private this.set_time t = this._time <- t member private this.set_score s = this._score <- s member private this.set_message m = this._message <- m member private this.set_isRunning r = this._isRunning <- r member private this.inc_score() = this._score <- this._score + 1 member private this.get_score = this._score member private this.get_TimerPeriod = SmileyGameBase.TimerPeriod member private this.get_GameDuration = SmileyGameBase.GameDuration member private this.get_ChangePositionPeriod = SmileyGameBase.ChangePositionPeriod member private this.IsSmileyClicked g = base.IsSmileyClicked g member private this.GetRandomPostion() = base.GetRandomPostion() member private this.StartTimer n = let n = ref n async { while !n >= 0 do this.set_time !n do! _gameAwaiter.Delay(this.get_TimerPeriod) |> Async.AwaitTask n := !n - 1 } member private this.StartGame() = async { do this.set_score 0 let timer = this.StartTimer(this.get_GameDuration) |> Async.ToTask let c = ref true while !c do let! m = _gameAwaiter.WhenAny(timer,_gameAwaiter.Delay(this.get_ChangePositionPeriod), _gameAwaiter.Gesture(this.IsSmileyClicked)) |> Async.AwaitTask match m.Index with | 0 -> c := false | 1 -> this.set_smileyPosition(this.GetRandomPostion()) | 2 -> this.set_smileyPosition(this.GetRandomPostion()) ; this.inc_score() | _ -> () } member private this.StartMenu() = async { while true do this.set_message "tap to start the game ... " let! _ = _gameAwaiter.Gesture(GestureType.Tap) |> Async.AwaitTask this.set_isRunning true do! this.StartGame() this.set_isRunning false this.set_message ("your score : " + (this.get_score).ToString() ) do! _gameAwaiter.Delay(float 4000) |> Async.AwaitTask } override this.LoadContent() = base.LoadContent() this.StartMenu() |> Async.StartImmediate
Fine, but how does it work? All I had to do was to implement class GameAwaiter deriving from GameComponent which means that we can add it to the components stored inside Game class and its Update method will be called automatically when Game’s Update method is being called. GameAwaiter gives us basically a few public methods such as Gesture, Delay and WhenAny. All of them return Task object which means that at some point in the future the result will come. Let’s see the details:
using System; using System.Collections.Generic; using System.Linq; using System.Threading.Tasks; using Microsoft.Xna.Framework; using Microsoft.Xna.Framework.Input.Touch; namespace AsyncXnaIntegration { using AsyncXnaIntegration; public class GameAwaiter : GameComponent { private readonly List<Job> _jobs = new List<Job>(); public GameAwaiter(Game game) : base(game) { } public Task Delay(TimeSpan interval) { return RegisterJob(new TimerJob { Interval = interval }); } public Task Delay() { return RegisterJob(new TimerJob()); } public Task While(Func<bool> condition) { return RegisterJob(new WhileJob { Condition = condition }); } public Task<GestureSample[]> Gesture(Func<GestureSample, bool> predicate) { return RegisterJob(new GestureJob { GestureCondition = predicate }); } public void TryDisposeTask(Task tt) { var job = _jobs.FirstOrDefault(j => j.TaskO == tt); if (job != null) job.IsDisposed = true; } public override void Update(GameTime gametime) { var state = new State { GameTime = gametime, TouchState = TouchPanel.GetState(), }; while (TouchPanel.IsGestureAvailable) state.Gestures.Add(TouchPanel.ReadGesture()); foreach (var job in _jobs.ToArray()) { if (job.IsDisposed || job.Update(state)) _jobs.Remove(job); } } private Task<T> RegisterJob<T>(Job<T> job) { _jobs.Add(job); return job.Task; } #region inner types private class State { public GameTime GameTime; public TouchCollection TouchState; public readonly List<GestureSample> Gestures = new List<GestureSample>(); } private abstract class Job { public bool IsDisposed { get; set; } public abstract Task TaskO { get; } public abstract bool Update(State state); } private abstract class Job<T> : Job { protected TaskCompletionSource<T> _source = new TaskCompletionSource<T>(); public Task<T> Task { get { return _source.Task; } } public override Task TaskO { get { return Task; } } } private class TimerJob : Job<object> { private TimeSpan? StartTime; public TimeSpan? Interval; public override bool Update(State state) { if (!Interval.HasValue) { _source.TrySetResult(null); return true; } if (!StartTime.HasValue) StartTime = state.GameTime.TotalGameTime; if (state.GameTime.TotalGameTime - StartTime >= Interval) { _source.TrySetResult(null); return true; } return false; } } private class WhileJob : Job<object> { public Func<bool> Condition; public override bool Update(State state) { if (Condition()) { _source.TrySetResult(null); return true; } return false; } } private class GestureJob : Job<GestureSample[]> { public Func<GestureSample, bool> GestureCondition; public override bool Update(State state) { if (state.Gestures.Count > 0) { if (GestureCondition == null) { _source.TrySetResult(state.Gestures.ToArray()); return true; } var gestures = state.Gestures.Where(GestureCondition).ToArray(); if (gestures.Length > 0) { _source.TrySetResult(gestures); return true; } } return false; } } #endregion } public static class GameAwaiterExtensions { public static Task<Branch> WhenAny(this GameAwaiter gameAwaiter, params Task[] tasks) { return GameAwaiterExtensions.WhenAny(tasks) .ContinueWith(t => { try { foreach (var tt in tasks) gameAwaiter.TryDisposeTask(tt); return t.Result; } catch (Exception exception) { Debugger.Break(); return t.Result; } }, TaskContinuationOptions.ExecuteSynchronously); } public static Task<Branch> WhenAny(params Task[] tasks) { return TaskEx .WhenAny(tasks) .ContinueWith(t => { var task = t.Result; var taskType = task.GetType(); object value = null; // we cannot read nonpublic types via reflection in silverlight if (taskType.IsGenericType && taskType.GetGenericArguments()[0].IsPublic) value = task.GetType().GetProperty("Result").GetValue(task, null); return new Branch { Index = Array.IndexOf(tasks, task), Value = value }; }, TaskContinuationOptions.ExecuteSynchronously); } public static Task Delay(this GameAwaiter gameAwaiter, double milliseconds) { return gameAwaiter.Delay(TimeSpan.FromMilliseconds(milliseconds)); } public static Task<GestureSample[]> Gesture(this GameAwaiter gameAwaiter, GestureType gestureType) { return gameAwaiter.Gesture(g => (g.GestureType & gestureType) == gestureType); } public static Task<GestureSample[]> Gesture(this GameAwaiter gameAwaiter) { return gameAwaiter.Gesture(g => true); } } public class Branch { public int Index; public object Value; } }
The last thing I would like to mention is a really tricky stuff :) What was really important for me when writing GameAwaiter, was the linear execution of the code. I didn’t want to introduce any new threads or use the SynchronizationContext object underneath. XNA framework calls Update and Draw methods one by one in the single thread many times per second and next iteration can start after the previous one has finished, so it’s really easy to debug such a single-threaded application. The problem is that, by default, Async CTP uses SynchronizationContext‘s Post method when the continuation delegate passed to TaskAwaiter object is called. Of course that happens if any context exists and in XNA case the “SynchronizationContext.Current” property returns the instance of the context. The latest version of Async CTP gives us the ability to configure that behavior but we would need to call “task.ConfigureAwait(false)” in every place where we use await keyword. That would be pretty inconvenient. So I have implemented my own extension method for Task object, that creates appropriately configured awaiter object.
namespace AsyncXnaIntegration { public static class Extensions { public static ConfiguredTaskAwaitable.ConfiguredTaskAwaiter GetAwaiter(this Task task) { return task.ConfigureAwait(false).GetAwaiter(); } public static ConfiguredTaskAwaitable<T>.ConfiguredTaskAwaiter GetAwaiter<T>(this Task<T> task) { return task.ConfigureAwait(false).GetAwaiter(); } } }
Now we need to find a way to force C# compiler to use our extension method instead of that provided in AsyncCtpLibrary_Phone.dll library. We can do it for example by placing “using AsyncXnaIntegration;” statement inside the namespace declaration in all files where we are using Async CTP. Thanks to that little trick our method will be “closer” than all others defined outside the namespace declaration.
Of course GameAwaiter class can be extended (and it most likely will be) by many other useful methods simplifying use of phone’s Keyboard, Geo-Position or Accelerometer APIs, but this is just a proof of concept. As always I encourage you to download the source code and definitely play the Smiley game :) Thanks for reading this post and I hope it will open your eyes for many new very interesting scenarios where the Async CTP can change XNA game development.
Subscribe to:
Posts (Atom)
Posts
