Comparison: DbC and TDD – Part 3

This blog post is a continuation of the comparison between DbC and TDD that started with a dedicated look at code specification and covered other aspects in part 1 and part 2. It takes a look at further points and shows the characteristics (commonalities and differences) of TDD and DbC towards these aspects.

Universality

One of the most important differences between DbC and TDD is their different universality, because it influences expressiveness as well as verifiability of correctness.

A significant limitation (or better characteristic) of TDD is that tests are example-driven. With a normal manually written unit test you provide exemplaric input data and check for certain expected output values after the test run. Thus you have concrete pairs of input/output values to express a certain feature/behavior that you want to test. Since these values are only examples there has to be a process of finding relevant examples which can be very difficult. One step towards correctness is high code coverage. You should find examples that examine all paths of your code components. That can be a mess since the amount of possible code paths grows exponentially with the count of branches (if, switch/case, throw, …). But even if you find examples that give you 100% code coverage you cannot be sure that the component acts right for all possible input values. On the one side there could be hardware-dependent behavior (e.g. with arithmetic calculations) that leads to differences or exceptions (division-by-zero, underflow/overflow etc.). On the other side perhaps you get 100% code coverage for your components during the TDD process, but this doesn’t help much on integration with other code components or external systems that could act in complex ways. This problem goes beyond TDD. Thus unit tests in a TDD fashion are nice by their flexibility and simplicity, but they cannot give you full confidence. Solutions like Pex can help you here, but this is part of another story (that I hope to come up with in the near future).

DbC on the other side introduces universal contracts that must hold for every value that an object as part of a contract can take. That’s an important aspect to ensure correct behavior in all possible cases. Thus contracts have a higher value in terms of universality than tests (but they fall back in other terms).

Expressiveness

The expressiveness of a component’s behavior and qualities as part of its specification is an important aspect since you want to be able to express arbitrary properties in a flexible and easy way. Universality is one part of expressiveness and has just been discussed. Now let’s look at expressiveness on a broader scope.

TDD has a high value on expressiveness besides the exemplaric nature of tests. You are free to define tests which express any desired behavior of a component that can be written in code. With TDD you have full flexibility, but you are also responsible to get this flexibility under your control (clear processes should be what you need). One aspect that goes beyond the scope of TDD is interaction/collaborative testing and integration testing. The TDD process implies the design of code components in isolation, but it doesn’t guide you in testing the interactions between components (and specification of behavior which relies on those interactions). TDD is about unit testing, but there is a universe beyond that.

Their universal nature makes contracts in terms of DbC a valuable tool. And moreover by extending the definition of a code element they improve the expressiveness of these code elements, what’s great e.g. for the role of interfaces and for intention revealing. But they have downsides as well. First they are tightly bound to a certain code element and are not able to express behavior that spans several components (e.g. workflows and classes interacting together). And second they have a lack of what I would call contentual expressiveness. With contracts you are able to define expectations with arbitrary boolean expressions and that’s a great thing. But it’s also a limitation. For example if you have an algorithm or complex business operation then it’s difficult to impossible to define all expected outcomes of this code as universal boolean postcondition. In fact this would lead to full functional specification which implies a duplication of the algorithm logic itself (in imperative programming) and this would make no sense! On the other side if you use example-driven tests you would not have a problem since you should know what values to expect on a certain input. Furthermore no side effects are allowed inside of a contract. Hence if you use a method to define a more complex expectation this method must be pure (= free of visible side effects). The background of this constraint is that contracts mustn’t influence the behavior of the core logic itself. It would be a mess if there would be a different behavior depending on the activation state (enabled/disabled) of contracts (e.g. for different build configurations). This has a limitation if you want to define certain qualitites with contracts like x=pop(push(x)) for a stack implementation, but it has advantages as well, since it leads to the enforcement of command-query separation by contracts.

Checking correctness

Of course you want to be able to express as many behaviors as possible to improve the specification of your code components. But expressiveness is not leading anywhere when it’s impossible to figure out if your components follow the defined specification. You must be able to stress your components against their specification in a reproducible way to ensure correct behavior!

With TDD correctness of the defined behavior (= tests) can be checked by actually running the tests and validating expected values against the actual values as outcome of a test run. The system-under-test is seen as blackbox and behavior correctness is tested by writing values to the input channels of the blackbox and observe the output channels for correct values. This could include techniques like stubbing or mocking for handling an object in isolation and for ensuring reproducible and verifiable state and behavior. This testability and reproducibility in conjunction with a well implemented test harness is important for continuously checking correctness of your code through regression testing. It’s invaluable when performing continuous integration and when code is changed, but this aspect will be covered in the next blog post. However the exemplaric nature of tests is a limitation for checking correctness which has been shown above when discussing universality.

At first DbC as principle doesn’t help you in terms of checking correctness. It introduces a fail-fast strategy (if a contract is not met, it’s a bug – so fail fast and hard, because the developer has to fix the bug), but how can you verify the correctness of your implementation? With DbC bugs should be found in the debugging step when developing code, thus by actually executing the code. And common solutions like Code Contracts for .NET come with a runtime checking component that checks the satisfaction of the defined contracts when running the code. But this solution has a serious shortcoming: It relies on the current execution context of your code and hence it takes the current values for checking the contracts. With this you get the same problems as with tests. Your contracts are stressed by example and even worse you have no possibility to reproducible check your contracts! Thus dynamic checking without the usage of tests makes no sense. However contracts are a great complement of tests. They specify the conditions that must apply in general and thus a test as client of a component can satisfy the preconditions and then validate postconditions or custom test behavior. Another interesting possibility to verify contracts is static checking. Code Contracts come with a static checker as well that verifies the defined contracts at compile-time without executing the code. On the contrary it actually inspects the code (gathers facts about it) and matches the facts as abstraction of the implementation against the defined contracts. This form of whitebox code inspection is done by the abstract interpretation algorithm (there are other solutions like Spec# that do real formal program verification). The advantage of static checking is that it’s able to find all possible contract violations, independent of any current values. But static checking is hard. On the one side it’s hard for the CPU to gather information about the code and to verify contracts. Hence static checking is very time-consumptive which is intolerable especially for bigger code bases. On the other side it’s hard for a developer to satisfy the static checker. To work properly static checking needs the existence of the right (!) contracts on all used components (e.g. external libraries) which is often not the case. And looking at the static checker from Code Contracts it seems to be too limited at the current development state. Many contracts cannot be verified or it’s too impractical to define contracts that satisfy the checker. Thus static checkers are a great idea to universally verify contracts, but especially in the .NET world the limitations of the checkers make them impractical for most projects. Hence a valid strategy for today is to write tests in conjunction with contracts and to validate postconditions and invariants with appropriate tests.

[To be continued…]

kick it on DotNetKicks.com

Comparison: DbC and TDD – Part 2

This blog post is a continuation of the comparison and differentiation between DbC and TDD. Please take a look at part 1 which covers the design aspect (and shortly specification again which has been discussed in more detail here). Today’s post takes a look at documentation and code coupling and shows commonalities and differences of TDD and DbC towards these aspects.

Documentation

One important thing for other developers to understand your code is documentation. Code comments are a first necessity here to basically explain what a code component is intended to do. But comments have the unpleasant habit to run out-of-sync with the real code if you are not carefully and consequently adapting them. Most of all if you write obligations or benefits into comments there will be no check if those requirements hold or if the real implementation matches them.

DbC with its contracts is a much better way to document those specification aspects. Contracts are coupled to the code and e.g. with Code Contracts you get static and runtime checking for them. This checked documentation makes DbC really powerful (if properly used) and avoids the asynchronicity between code and documentation. It shows developers how a component should be used, what requirements the client  has to fulfill and what he can expect in return. The client can rely on these specified qualities which improves the reusability of contracted code components. Furthermore there are possibilities to integrate Code Contracts into the Sandcastle documentation and for Visual Studio 2010 there will be an add-in that immediately shows defined contracts on a component as you develop against it. With that you get great MSDN-like documentation that contains the defined contracts as well as support for your development process when you use contracted code.

Tests in terms of TDD add another aspect of documentation. Due to their exemplary nature and their specification of an element’s behavior those tests are great to show a developer the intent of a code component and give him a guideline to its usage. Since tests can be run and validated in a reproducible way developers are able to rely on defined behaviors as well.

Together with documentation comes the aspect of intention revealing. And both TDD and DbC add some value here. Both express the developer’s intention with a certain component and show far beyond code comments and naming conventions what behavior a client can expect. Developers can use the component in these specified ways and don’t have to manually investigate the component’s implementation.

Code coupling

One drawback of TDD is the locational gap between the code implementation and the tests as specification and documentation source. For sure this has an advantage as well: the code logic isn’t polluted with the specification and thus it’s kept clean. But the disadvantages weigh heavier for me. If developers want to show how a component behaves they have to cross the gap and investigate the tests. This is difficult for developers who are not very familiar with TDD. Furthermore tests don’t give any support for client usage of a component. Of course they give usage examples, but developers can use and especially misuse a component in arbitrary ways. This can lead to serious problems (e.g. inconsistent states) if there are no other mechanisms to prohibit misusage.

DbC on the other side sets contracts directly on the implementation of a code component in place. Thus they have a declarative nature and extend the definition of a code component. Some realizations of DbC like Code Contracts in .NET have drawbacks since they set contracts imperatively into the code, but rewrite the code after compilation to set the contracts in the „right“ places. Thus Code Contracts break the uniformity principle (different semantics should be expressed through different syntax) and pollute the code logic in some way. Other realizations like the Eiffel language have contracts as keywords built into the language which makes a better choice in my opinion. Anyway contracts at the same place as the implementation avoids the drawbacks of a locational gap. And moreover DbC prevents misusage of a component. Contracts are dynamically checked at runtime or statically at compile time and fail early if requirements are not satisfied. That’s a very important concept because it expresses a clear behavior if something goes wrong (existence of a bug) and gives a clear contract for obligations and benefits that hold and are checked in client/supplier communication.

[To be continued…]

kick it on DotNetKicks.com

Comparison: DbC and TDD – Part 1

Let’s come to another blog post in preparation of elaborating the synergy of DbC and TDD. My first blog post on this topic covered an initial discussion on specification of code elements. Thereby it has shown different characteristics of DbC and TDD in terms of code specification. Since specification is one really important aspect in comparison of DbC and TDD, it’s of course not the only one. Hence today’s blog post is a starting point for a more general comparison of DbC and TDD with some other important aspects. There are 3 more blog posts that I will come up with during the next weeks, forming a 4-part comparison series.

To say first, Test-Driven Development (TDD) and Design by Contract (DbC) have very similar aims. Both concepts want to improve software quality by providing a set of conceptual building blocks that allow you to prevent bugs before making them. Both have impact on the design or development process of software elements. But both have their own characteristics and are different in achieving the purpose of better software. It’s necessary to understand commonalities and differences in order to take advantage of both principles in conjunction.

Specification again…

The last blog post has already given a dedicated look at the specification aspect and how DbC and TDD can add some value. To summarize both principles extend the code-based specification in their own ways. TDD let’s you specify the expected behavior of a code element in an example-driven and reproducible way. It’s easy to use and allows the expression of any expected behavior. DbC on the other side sets universal contracts in place that extend the definition of a code element and are tightly coupled to it. It’s a great concept for narrowing the definition of a code element by defining additional physical constraints as preconditions, postconditions and invariants. By defining contracts on interfaces DbC strengthens the role of interfaces and enforces identical constraints/behavior over all implementations of an interface. However not every behavior can be expressed by contracts and they’re bound to a single code element. Thus they don’t lessen the position of tests, but can be seen as great complement.

Design aspect

Both DbC and TDD have impact on the design of an API. Well, there is a slight but important difference as the names imply: it’s Design by Contract, but Test-Driven Development. Thus with DbC (contract-first) on the one side you are encouraged to write your contracts as you design your components (at design phase), which perfectly fits with the idea of contracts as extension of a component’s definition. With TDD and the test-first principle on the other side you write a test which maps to a new feature and afterwards you directly implement the code to get the test to green state. Hence TDD is tightly coupled to the development phase in contrast to DbC, which seems to come first. In addition personally I wouldn’t fight a religious war on this naming. If you think of DbC as „Contract-First Development“ or „Development by Contract“ you would have the contract-first principle coupled to the development phase as well. The more important thing is to find a way to effectively use contracts in the development cycle. If you are an advocate of up-front design you would perhaps want to set your contracts at design phase. But if you intensively use TDD it would be difficult to go down this design phase road. However you would set your contracts at development phase in conjunction with the test-first principle. This leads to the question of an effective development model with TDD and DbC and that’s another important story…

But for now let’s come back to the impact of DbC and TDD to the design of an API. With TDD you write a new test for each new feature and then you bring this test to green by implementing some piece of logic. This is some form of Client-Driven Development. Your test is the client of your API and you call your methods from the client’s perspective (as a client would do). If the current API doesn’t fit your needs, you extend or modify it. Thus the resulting API is very focussed on the client’s needs and furthermore doesn’t contain code for unnecessary features, which is a great thing in terms of YAGNI. Another impact of TDD is that it leads to loosely coupled components. Tests in form of unit tests are very distinct and should be coupled to the tested component, but not far beyond that (other components that are called, e.g. data access). Thus there is a certain demand for loose coupling e.g. by DI.

With DbC and contracts on your components you have other impacts. Contracts clarify the definition and intent of your components. When you come up with contracts you strengthen your opinion about a component by setting contractual obligations and benefits. This leads to much cleaner components. Moreover your components will have fewer responsibilities (and thus a better cohesion) since it would be painful to write contracts for components with many different responsibilities. Thus DbC is great in terms of supporting the SRP and SoC. Another impact comes from the „limitation“ of contracts to support only pure methods as part of a contract. If you want to use class methods in contracts (e.g. invariants) of this class you have to keep those methods pure. This leads to the enforcement of command-query separation by contracts, which very often is a good thing in terms of comprehensibility and maintainability.

[To be continued…]

kick it on DotNetKicks.com

Specification: By Code, Tests and Contracts

Currently I’m taking further investments in thinking about the synergy of Design by Contract (DbC) and Test-Driven Development (TDD). This process is partially driven by my interests in Code Contracts in .NET 4.0 and other current developments at Microsoft (e.g. Pex). The basic idea I’ve got in mind is to combine DbC and TDD to take best advantage of both principles in order to improve software quality on the one side and to create an efficient development process on the other side. My thoughts on this topic are not too strong at the moment, so feel free to start a discussion and tell me what you think.

Before I come to discuss the synergy of DbC and TDD it’s absolutely necessary in my opinion to understand the characteristics, commonalities and differences of both concepts. This first blog post drives in this direction by looking at a very important aspect of both concepts: the formal and functional specification of code. It’s a starting point for further discussions on this topic (a 4part comparison series follows). So let’s start…

Code-based specification

In essence, DbC and TDD are about specification of code elements, thus expressing the shape and behavior of components. Thereby they are extending the specification possibilities that are given by a programming language like C#. Of course such basic code-based specification is very important, since it allows you to express the overall behavior and constraints of your program. You write interfaces, classes with methods and properties, provide visibility and access constraints. Furthermore the programming language gives you possibilities like automatic boxing/unboxing, datatype safety, co-/contravariance and more, depending on the language that you use. Moreover the language compiler acts as safety net for you and ensures correct syntax.

Since such basic specification is necessary to define the code that should be executed, it has a lack of expressiveness regarding the intent of a developer and there is no way to verify correct semantics of a program with it. As an example interfaces define the basic contract in terms of method operations and data, but looking at C# a client of this interface does not see what the intent of a method is or which obligations he has to fulfill when calling a method or what state he can assume on return of a method. Furthermore interfaces can not guarantee uniform behavior across their implementations. TDD and DbC are there to overcome or at least decrease this lack of expressiveness at some points and to guarantee correct semantics.

Test-based specification

Let’s come to test-based specification using TDD (as well as BDD as „evolutional step“ of TDD). This is inevitably integrated in the TDD cycle. Every test written by a developer maps to a new feature he wants to implement. This kind of specification is functional and example-driven, since a developer defines by exemplary input/output pairs what output he expects as result of the test run under a certain input. With well-known techniques (stubs, mocks) he is able to run his tests in isolation, get reproducible test results and perform state and behavior verification.

Compared to code-based specification, test-based specification in a TDD manner is very valuable when it comes to expression of the behavior of a code element. It gives a set of tests that could at its extreme span the whole behavior of a code element. With their reproducibility the tests are indispensable when it comes to continuous integration to ensure correct behavior of modified code elements. Furthermore tests are a great source for other developers to show the intent of a developer for a method’s behavior and to give demonstration of the usage of a code element. There are other benefits and characteristics of TDD that will be discussed when comparing TDD and DbC altogether in subsequent blog posts.

An important aspect of test-based specification using TDD is that it’s done at a very granular level. For each new feature a test is written and if the present API doesn’t fit the needs, it will be extended or refactored. Thus TDD drives the API design as you go with your tests.

Contract-based specification

Another possibility for specifying code is contract-based specification in terms of DbC, thus defining preconditions and postconditions as method contract and invariants as class contract. With contracts you are able to define the basic obligations that a client must fulfill when calling a method as well as the benefits he gets in return. Furthermore invariants can be used to define basic constraints that ensure the consistency of a class. Thus with DbC a developer is able to define a formal contract for code that makes a clear statement of obligations and benefits in client/supplier communication (aka caller/callee). If a contract fails the behavior is clear as well: by failing fast the developer can be sure that there is a problem (bug) with his code and he has to fix it.

On a technical level there are several possibilities to define contracts. In Eiffel contracts are part of the programming language itself in contrast to Code Contracts that become part of the .NET framework. In any case contracts are directly bound to the code that they are specifying and express additional qualities of the code. Those qualities go beyond the code-based specification. In general contracts allow arbitrary boolean expressions what makes them a very powerful and flexible specification source. Nonetheless contracts only allow partial functional specification of a component. It can be very difficult or even impossible to define the complex behavior of a method (which methods is it calling, what business value does it have, what’s the concrete result for a concrete input, …) or to ensure certain qualities (e.g. infrastructure-related questions like „is an e-mail really sent?“) with contracts. Furthermore it’s impossible to use impure functions in contracts which would be necessary to express certain qualities of code (like expressing inverse functions: x = f-1(f(x)), if f or f-1 are impure). Test-based specification could be used here instead.

But let’s come back to the technical aspect: Contracts are wonderful animals in terms of extending the code-based specification by narrowing the definition of a code element and becoming part of the element’s signature. They can be used to define general conditions, e.g. physical constraints on parameters and return types. Thereby (for example with Code Contracts) the contracts can be inherited to sub-classes while respecting the Liskov substitution principle (LSP). Moreover contracts can be defined on interfaces and thus they are a valuable tool for expressing constraints and qualities of an interface that must be respected by every implementation. With that contracts are wonderfully strengthening the role and expressiveness of interfaces and complementing code-based specification at whole.

With the tight coupling to the code, contracts give immediate support for other developers how an API should be used. They express the developer’s intent (intention revealing) for his code elements, which leads to easier comprehension and usage of those components. They give some kind of checked documentation, which again greatly complements code-based specification and leads to fewer misusage of contracted components.

In contrast to test-based specification, contracts are employed at level of whole components by employing invariants to classes and pre-/postconditions to class methods. Moreover contracts lead to components with very few dependencies (promoting the SRP) since it’s difficult to write contracts for components with many responsibilities.

First conclusion

This blog post is intended to be a first preview of what I want to come up in the next time. It has given an overview of possibilities for specification of code elements with focus on test-based and contract-based specification in terms of TDD and DbC. It has shown some commonalities and qualities of both concepts. In conclusion TDD and DbC are both valuable in terms of the specification of code elements and in revealing the developer’s intent.

In my opinion it’s by no means a „one or the other“ choice. TDD and DbC act on their own terrains with their own benefits and drawbacks. There are overlaps, but in general they can naturally complement each other. It’s time to think about ways to leverage a conjunction of both concepts, isn’t it?

[To be continued…]

kick it on DotNetKicks.com

Code Contracts: Video on the static checker

Just a short hint for today: Francesco Logozzo from the Code Contracts team has made a little introductory video on the static checker. In this video he introduces the static checker and basically describes its fundamental algorithm: abstract interpretation.

Take a look at this video and please keep giving feedback to the Code Contracts team. Those are great guys who are very open for every suggestion!

Currently I’m making my mind up on DbC and the synergy with TDD and other paradigms again, so stay tuned for further blog posts on this topic.

Merry christmas to all of you!

Rx: Event composition – single-valued

In a first blog post on the topic „event composition with Rx“ I presented operators that take several event input streams and create a single output streams, whose elements contain values from each input event („multi-valued“ event composition operators). This second blog post instead treats “single-valued” event composition. Multiple streams go into those composition operators and one single stream comes out, where each element of the output stream matches exactly one event/value from any input stream!

Example solution, PPT slides and general information

In general, all introductory information on composition, description of the „multi-valued“ event composition operators and more bits-and-pieces can be found in my first blog post.

I’ve created a SL3 demo project where you can experiment with each composition operator. You can find the sourcecode (SL3 VS2008 project) here and the live demo on Windows Azure here.

The PPT slides that contain all diagrams of the composition operators can be downloaded here.
I would be glad if you could leave my name and/or the blog URL (www.minddriven.de) on every diagram.

Merge()

The operator Merge does serialization of all input event streams by firing an event everytime one of the input streams fires. The event/value of the output stream will be the same as the event/value on the input stream that has fired.

The following code shows the simple usage of Merge:

    var composition = 
        Observable.Merge(stream1, stream2, stream3);

And the following example flow diagram shows the behavior of the operator under this code. Note how the incoming events are written to the output stream in exactly the same order as they come in:
Rx Event Composition - Merge

Amb()

Amb comes from ambiguous and has the following semantics. If you combine several input event streams with Amb, the output stream will only contain the events from the input stream that fires first. The other input streams are not taken into account afterwards. That’s a kind of winner-takes-all-strategy and could be valuable in some situations.

Amb can be used as follows:

    var composition = 
       stream1.Amb(stream2).Amb(stream3);

And the following diagram shows an example composition situation. Note that stream 1 fires first and consequently Amb only forwards events from this stream:
Rx Event Composition - Amb

Concat()

Concat does (the name says it all) concatenation of the input event streams. But wait: how can this work? In fact Concat only works when you have finite event streams! On infinite streams, Concat watches the very first stream forever and the other streams don’t come to their turn. Thus use Concat on finite event streams, when serialization of the input streams in the right order is crucial.

The following code shows the use of Concat on three finite input streams:

    var composition = 
        stream1.Concat(stream2).Concat(stream3);

And here’s the concatenation done by this code in an example situation (the square brackets illustrate finite streams):
Rx Event Composition - Concat

Until()

The operator name says it all: Until forwards events from an input stream to the output stream until an event on a second stream occurs.

The following code shows the usage:

    var composition = 
        stream1.Until(stream2);

And the following diagram shows the behavior of Until on 2 streams based on this code:
Rx Event Composition - Until

WaitUntil()

Again the operator name speaks for itself: WaitUntil does not forward events from an input stream to an output stream until an event on a second stream occurs. From then on every event of the first input stream will be forwarded.

Here’s some code that shows how WaitUntil works:

    var composition = 
        stream1.WaitUntil(stream2);

And the following diagram shows WaitUntil’s behavior based on this code in an example situation:

Rx Event Composition - WaitUntil

That’s it again. I hope you finde this information useful. As in the blog post on multi-valued event composition, I want to encourage you to give me active feedback and to improve the examples and descriptions of the composition operators above. Thanks!

kick it on DotNetKicks.com

Rx: Event composition – multi-valued

In preparation of some presentation on the Reactive Extensions (Rx) last week I’ve investigated the composition methods for several event streams on the Observable class. What methods are existing and what are they doing? Let’s take a look…

This first blog post treats „multi-valued“ event composition in terms of multiple streams going in and one stream comes out, but each element of the output stream is composed by sub-elements from each input stream! There is a second blog post that treats „single-valued“ event composition as well.

Example solution

I’ve written a little SL3 application where you can discover the behavior of those composition operators. It builds on example code that first has been published by Paul Batum some months ago and extends his code with some more possibilities. I’ve hosted the SL3 on Windows Azure where you can test it directly:

Rx Composition Demo

You can download the sourcecode of the Visual Studio 2008 solution as well: [Event Composition Sourcecode (ZIP)]
All you need to run the solution is the latest Rx for SL3 build, Visual Studio 2008 and the Silverlight 3 SDK.

Powerpoint Sources

If you want to use my diagrams in your presentations or somewhere else, here is the source PPT file: [Rx Event Composition PPT]
I would be glad if you could leave my name and/or the blog URL (www.minddriven.de) on every diagram.

How to read the diagrams and code

The general visualization concept is taken from Eric Meijer’s PDC session about Rx. Thanks Eric for your great work!

Rx Event Composition - General Visualization

The visualization shows an Rx composing operator in the middle. On the left there are event input streams, on the right is the composed output stream. The order of the events on a timeline is from right to left: the most right event goes first into the operator and comes first out in composed form. This is visualized by the t1, t2, t3 and t1‚, t2‚, t3‚ points.

The sourcecode snippets below are kind of abstract. They only show how the operators are used. They are based on 3 streams stream1, stream2 and stream3. Those streams can be seen as general abstract event streams, no matter which datatype they’re really using.

Now let’s come to our multi-valued composition operators…

SelectMany()

The SelectMany operator should be known from LINQ on IEnumerable or others and is reflected by the from keyword in LINQ queries as well. It projects a foreach loop or several cascaded foreach loops, if you execute multiple SelectMany calls after another. Thus the order in which SelectMany is executed is relevant!

The same is true for SelectMany on IObservable. Take the following code:

var composition = 
    from element1 in stream1
    from element2 in stream2
    from element3 in stream3
    select new[] { element1, element2, element3 };

This is equal to the following direct use of SelectMany:

var composition = 
    stream1.SelectMany(
        element1 => stream2.SelectMany(
        element2 => stream3.SelectMany(
        element3 => Observable.Return(new[] { element1, element2, element3 }))));

With this code an example composition scenario could be:

Rx Event Composition - SelectMany

Note that the event order is crucial. The first event on every input stream is in the right order, thus yielding to a first composed event. The second event on input stream 2 first fires when the third event on stream 3 is fired. The resulting event in this case contains the first event from stream 1, since it retains the right event order.

CombineLatest()

CombineLatest is a very useful operator. It fires everytime when one input stream fires. Then the corresponding output stream contains the latest values/events from each input stream.

Here’s the code how to use the operator:

var composition = 
    stream1.CombineLatest(
        stream2, (element1, element2) => new[] { element1, element2 })
    .CombineLatest(
        stream3, (elements12, element3) => new[] { elements12[0], elements12[1], element3 }));

And an example composition follows:

Rx Event Composition - CombineLatest

Zip()

In comparison to CombineLatest and others the Zip operator doesn’t use any results from an event stream multiple times. However it builds a queue of the events on every input stream. When every input stream queue contains at least one event, the Zip composition operator fires. Thus it synchronizes events, whereas every event on an input stream is waiting for an event on the other input streams.

The following code shows the usage of Zip():

var composition = 
    stream1
        .Zip(stream2, (element1, element2) => new[] { element1, element2 })
        .Zip(stream3, (elements12, element3) => new[] { elements12[0], elements12[1], element3 });

And the following diagram shows an example composition. Note that no event is used twice in the output stream and compare this with CombineLatest and the other operators:

Rx Event Composition - Zip

Note that the last event on stream 3 has no „partner“ on the other streams and thus it doesn’t appear on the output stream in this example.

Also notice that Zip() is equal to Join() if used with a conjunction of all input event streams as parameter:

var composition = 
    Observable.Join(
        stream1.And(stream2).And(stream3)
            .Then((element1, element2, element3) => new[] { element1, element2, element3 }));

Join() – 2 out of 3

The Join operator can be used in many cases and is very flexible. The method Join() can take several event compositions as parameters. Everytime one composition fires, the whole operator fires. That said, if you create a conjunction of all your event streams, then the composed output stream will be equal to Zip().

But Join gives you more power and flexibility. One example is a „2 out of 3“-situation. Imagine you have 3 input streams and want Join to fire everytime when 2 input streams contain an event. This can be done by passing Join the appropriate partial compositions as shown in the following code:

var composition = 
    Observable.Join(
        stream1.And(stream2)
            .Then((element1, element2) => new[] { element1, element2 }),
        stream1.And(stream3)
            .Then((element1, element3) => new[] { element1, element3 }),
        stream2.And(stream3)
            .Then((element2, element3) => new[] { element2, element3 }));

This yields to the following resulting composition for example input streams:

Rx Event Composition - Join 2 out of 3

Again in this example, the last event on stream 3 has no partner and doesn’t appear on the output stream (due to an odd amount of events over all input event streams).

ForkJoin()

ForkJoin can be used if you want a composition to fire exactly once. When all input event streams contain at least one event/value, the ForkJoin operator fires with the first event on every stream. Afterwards it doesn’t react to any changes/event on the input streams again.

Here’s the code how ForkJoin can be used:

var composition = 
    Observable.ForkJoin(stream1, stream2, stream3);

And here’s the diagram of a composition example:

Rx Event Composition - ForkJoin

That’s it for now, guys. I hope it’s valuable for you. Since these are just my initial ideas on this topic, I’m reliant on your feedback. Together we could make all those descriptions even more useful.

kick it on DotNetKicks.com

Rx: Bing Translate Example

In his Rx session at PDC09 Eric Meijer gave a nice example named Bing Translate. It was connecting to Bing via the Bing Web Service API and translated some term given by the user. It was quite interesting because Eric used the Reactive Framework for handling the translation and did some composition of the incoming results. He took just the first 2 translations and discarded the others. This didn’t make much sense, but it was ok for demonstration purposes. As followup Somasegar described the example in his blog.

But there was a lack of the whole sourcecode (btw: Eric Meijer posted it now on the Rx forum, but not as VS solution) and so I implemented it on my own with some modifications and now I want to share the whole Visual Studio solution with you.

Live demo

I’ve put my Bing Translate example as SL3 application on Windows Azure at http://bingtranslate.cloudapp.net, where you can directly test it:

Rx Bing Translate Example - Reactive Extensions - Reactive Framework

Sourcecode requirements

Since the Bing Translate example uses the Bing Web Service API, you need to get a free AppId here. Additional information on the Bing API can be found on the Bing Developer Center. When you’ve obtained your AppId, insert it into the BingService.cs file (field AppId of class BingService) of the sourcecode below. Now you should be able to run the solution.

What you furthermore need to open and run the solution:

  • Visual Studio 2008
  • Silverlight 3 SDK
  • Reactive Extensions for SL3 (latest release)

Now give me the code!

Ok, you want it, you get it 😉

[Visual Studio Solution for Silverlight 3 (ZIP)]

Notes on the code:

  • Contrary to Eric’s solution, mine will not synchronize the translation events when using the SetUpTranslationService() method in the MainPage.xaml.cs file. Use the SetUpTranslationServiceWithJoin() method if you want to have some synchronization. The main advantage of Rx is the composition of event streams, so this synchronization shows how simple things could be.
  • The code’s style is not expressing my real coding skills. Since this is just an example project, I did not make it stylish and bullet-proof.

That’s it! Have fun and please feel free to give me feedback.

kick it on DotNetKicks.com