Bus of Doom

In a previous Bacon blog post a way to structure Bacon application was outlined. It introduces Buses as a central way to glue components with each other. I'm in a very strong disagreement with the proposed style. Why?

Quoting Erik Meijer

Subjects are the "mutable variables" of the Rx world and in most cases you do not need them.

In Bacon parlance that would be

Buses are the "mutable variables" of the Bacon world and in most cases you do not need them.

Now, that needs an explanation. We can split the statement to two parts and treat each individually. "Why Buses (and mutable variables) are bad", then "why you don't usually need them".

Problems with Bus

There was a time when data structures in our programs were built by mutating those data structures. In case you have entered this field only recently you may be a lucky one and haven't seen that madness in full glory (== spaghetti).

var cart = ShoppingCart()
var view = ShoppingCartView()
view.cart = cart

This was a bad idea as it creates temporal dependencies all over the program, making it difficult to locally understand how a piece of code works. Instead, a global view on a program is required. Who mutates what and when. It also created many bugs as components of a system are from time to time in an invalid state. Most common invalid state being at a construction phase where fields are initialized to nulls. A whole slew of bugs were eliminated and sanity regained by moving to immutable data.

var cart = ShoppingCart()
var view = ShoppingCartView(cart)

Ok, what does all that have to do with Buses? Well, Buses introduce similar temporal dependencies to your program. Is that component ready to be used? I don't know, did you plug its Buses already with this and that?

var shoppingCartBus = new Bacon.Bus()
$.ajax('/api/cart').done(cart => shoppingCartBus.push(cart))
...
shoppingCartBus.onValue(cart => renderCart(cart))

Here's a recent bug (simplified from a real world app) found in our company's internal chat. Can you spot it?

There's a chance that the ajax call on line 2 returns before line 4 is executed, thus the event is completely missed. It is temporal dependencies like that which are nearly impossible to understand in a bigger context. And what's worse, these bugs are difficult to reproduce as we are programming in a setting where stuff is nondeterministic (timers, delays, network calls etc.). I'm sure that many Bus fetished programs contain subtle bugs like above.

How to avoid Buses

I'll give examples of techniques avoiding Buses by refactoring the example in the previous blog post.

The first one is simple and obvious. Turn inputs of a component to be input arguments of the component.

Before:

function ShoppingCart(initialContents) {
  var addBus = new Bacon.Bus()
  var removeBus = new Bacon.Bus()
  var contentsProperty = Bacon.update(initialContents,
    addBus, function(contents, newItem) { return contents.concat(newItem) },
    removeBus, function(contents, removedItem) { return _.remove(contents, removedItem) }
  )
  return {
    addBus: addBus,
    removeBus: removeBus,
    contentsProperty: contentsProperty
  }
}

After:

function ShoppingCart(initialContents, addItem, removeItem) {
  return Bacon.update(initialContents,
    addItem, function(contents, newItem) { return contents.concat(newItem) },
    removeItem, function(contents, removedItem) { return _.remove(contents, removedItem) }
  )
}

I'm pretty sure everyone agrees that the refactored version is simpler.

The next refactoring has to do with remove links. Each shopping cart item will have a link and clicking a link will remove the item from a cart. The refactored version of the ShoppingCart needs removeItemclick stream as a function argument, but the individual items are created dynamically as the user selects items. This can be solved by event delegation.

$('#shopping-cart').asEventStream('click', '.remove-item')

You can state just once and for all: here's a stream of clicks of every .remove-item link in the shopping cart, and of all the future .remove-item links that will appear in the shopping cart. That is fantastic. It's like, you put it there and there it is. Event delegation is such a god sent tool and my heart fills with joy every time I have a chance to use it. After that the click events must be associated with items. A canonical way to do it is with data attributes.

Now the sample program is Bus-free.

function ShoppingCart(initialContents, addItem, removeItem) {
  return Bacon.update(initialContents,
    addItem, function(contents, newItem) { return contents.concat(newItem) },
    removeItem, function(contents, removedItem) { return _.remove(contents, removedItem) }
  )
}

var removeItemStream = $('#shopping-cart').asEventStream('click', '.remove-item')
  .map(function(e) { return $(e.currentTarget).data('id') })
var newItemView = NewItemView()
var cart = ShoppingCart([], newItemView.newItemStream, removeItemStream)
var cartView = ShoppingCartView(cart)

All done? Not yet. removeItemStream is hanging there while it probably should be part ofShoppingCartView.

function ShoppingCartView(cart) {
  return {
    cartView: ...
    removeItemStream: $('#shopping-cart').asEventStream('click', '.remove-item')
      .map(function(e) { return $(e.currentTarget).data('id') })
}

Whoops, now we introduced a cyclic dependency between ShoppingCart and ShoppingCartView.

var cart = ShoppingCart(initialContents, addItem, removeItemStream)
var {removeItemStream} = ShoppingCartView(cart)

Cyclic dependency is often given as an example where Buses are needed. After all the hard work should we now reintroduce Buses?

Here a Bus can be used to break the cyclic dependency, just as a mutable variable would do if you will. But we have other options too. Why don't we factor the components so that the cyclic dependency completely disappears.

function RemoveItems(container) {
  return {
    view: ...
    removeItemStream: container.asEventStream('click', '.remove-item')
      .map(function(e) { return $(e.currentTarget).data('id') })
}

var viewContainer = $('#shopping-cart')
var removeItems = RemoveItems(viewContainer)
var cart = ShoppingCart(initialContents, addItem, removeItems.removeItemStream)
ShoppingCartView(viewContainer, cart, removeItems)

Similar factorings can be almost always used to break cyclic dependencies.

Conclusion

Avoid Buses. View those as mutable variables and you will understand the kinds of problems they create. By relating Buses to mutable variables gives you an intuition on how to avoid those in a first place.

Structuring Real-Life Applications

The Internet is full of smart peanut-size examples of how to solve X with "FRP" and Bacon.js. But how to organize a real-world size application? That's been asked once in a while and indeed I have an answer up in my sleeve. Don't take though that I'm saying this is the The Definitive Answer. I'm sure your own way is as good or better. Tell me about it!

I think there are some principles that you should apply to the design of any application though, like Single Reponsibility Principle and Separation of Concerns. Given that, your application should consist of components that are fairly independent of each others implementation details. I'd also like the components to communicate using some explicit signals instead of shared mutable state (nudge nudge Angular). For this purpose, I find the Bacon.js EventStreams andProperties quite handy.

So if a component needs to act when a triggering event occurs, why not give it an EventStream representing that event in its constructor. The EventStream is an abstraction for the event source, so the implementation of your component is does not depend on where the event originates from, meaning you can use a WebSocket message as well as a mouse click as the actual event source. Similarly, if you component needs to display or act on the state of something, why not give it a Property in its constructor.

When it comes to the outputs of a component, those can exposed as EventStreams and Properties in the component's public interface. In some cases it also makes sense to publish a Bus to allow plugging in event sources after component creation.

For example, a ShoppingCart model component might look like this.

function ShoppingCart(initialContents) {
  var addBus = new Bacon.Bus()
  var removeBus = new Bacon.Bus()
  var contentsProperty = Bacon.update(initialContents,
    addBus, function(contents, newItem) { return contents.concat(newItem) },
    removeBus, function(contents, removedItem) { return _.remove(contents, removedItem) }
  )
  return {
    addBus: addBus,
    removeBus: removeBus,
    contentsProperty: contentsProperty
  }    
}

Internally, the ShoppingCart contents are composed from an initial status and addBus and removeBus streams usingBacon.update.

The external interface of this component exposes the addBus and removeBus buses where you can plug external streams for adding and removing items. It also exposes the current contents of the cart as a Property.

Now you may define a view component that shows cart contents, using your favorite DOM manipulation technology, like virtual-dom:

function ShoppingCartView(contentsProperty) {
  function updateContentView(newContents) { /* omitted */ }
  contentsProperty.onValue(updateContentView)
}

And a component that can be used for adding stuff to your cart:

function NewItemView() {
  var $button, $nameField // JQuery objects
  var newItemProperty = Bacon.$.textFieldValue($nameField) // property containing the item being added
  var newItemClick = $button.asEventStream("click") // clicks on the "add to cart" button
  var newItemStream = newItemProperty.sampledBy(newItemClick)
  return {
    newItemStream: newItemStream
  }
}

And you can plug these guys together as follows.

var cart = ShoppingCart([])
var cartView = ShoppingCartView(cart.contentsProperty)
var newItemView = NewItemView()
cart.addBus.plug(newItemView.newItemStream)

So there you go!

Beware of Lazy Evaluation

A funny thing happened to me when I was presenting Bacon.js at the great Oredev conference a few weeks ago. You could say my own dogs bit me. What happened was that I used map to get the current value of a text field to be associated with a Bacon event.

The thing is, methods such as map and the combine use lazy evaluation to avoid evaluating values that aren't actually needed. This can be generally considered a Good Thing, but it has it's pitfalls.

If you pass a function that referentially transparent, you'll be fine. This means that your function should return the same value regardless of when it's called.

On the other hand, if you pass a function that returns a value depending on time, you may have problems. Consider a property contents that's derived from events like below.

var items = clicks.map(getCurrentValueFromUI).toProperty()
var submittedItems = items.sampledBy(submitClick)

Now the submittedItems stream will produce the current value of the items property when an event occurs in the submitClick stream. Or so you'd think. In fact, the value of submittedItems is evaluated at the time of the event in the submitClick stream, which means that it will actually produce the value of getCurrentValueFromUI at that time, instead of at the time of the originalclick event.

To force evaluation at the time of original event, you can just use flatMap instead of map. As in here.

var items = clicks.flatMap(getCurrentValueFromUI).toProperty()

So there I was, on the stage, looking stupid because of lazy evaluation. 

Now, looking stupid in places is what I do every day but, if it lazy evaluation fools me (the guy who wrote it), maybe it's not reasonable to expect new Bacon.js users to do well with lazy eval.

So, I opened an issue on Github where I kinda suggest that we should remove lazy evaluation in Bacon.js 0.8. What do you think?

Oh, and should I go to Minsk to the Rolling Scopes conference next year?

Bacon Blog is Back!

Been a while since my last post on this blog. And it's not that stuff hasn't been happening around Bacon.js. It's more that I haven't had time to write and/or have kept the bar a bit high. But now it's time for a change.

I'm gonna throw a post here weekly from now on, with short posts that take less than 30 minutes to write. Let's see what happens.

I just made 2 releases of Bacon.js today. And in total, there's been 32 minor releases since 0.7.0 that my previous post covered. A lot of bugs have been fixed and performance improvements have been made. So if you're for some reason using Bacon.js version < 0.7.32, you should update! In case you're interested in a detailed list of changes, have a look at RELEASE-NOTES.md!

Anyway, let me briefly introduce the API changes since 0.7.0.

Error handling improvements (#310)

Added Bacon.retry for retrying a background request a few times before giving up and producing an Error event. Here's @mileskin's original example with JQuery Ajax.

Bacon.retry
  source: -> Bacon.fromPromise($.ajax(...))
  retries: 4
  interval: ({retriesDone}) -> (1 + retriesDone) * 5000
  isRetryable: (response) -> response.status isnt 401

Also added flatMapError for flatMapping errors.

flatMapWithConcurrencyLimit, flatMapConcat, bufferingThrottle, holdWhen (#324)

The bufferingThrottle method allows you to throttle events with a buffer. So, when a regular throttle discards excess events, this one will buffer them.

The flatMapWithConcurrencyLimit method is a variant of flatMap that limits the number of concurrently active child streams, so that you can, for instance limit the number of concurrent AJAX requests to a certain maximum by automatically queuing the excess requests.

The flatMapConcat method is like the previous, but with concurrency limit of 1.

The holdWhen method can be used like a valve, to buffer events when a Property holds the value true, and release all buffered events when the value changes to false.

Support "on/off" events in Bacon.fromEventTarget (#461)

Some libs use the "on" and "off" methods for registering/removing event listeners. Now it's easy to integrate into those libs too. So, for instance, if you use CodeMirror and want a stream of code changes in the editor, you can now

    Bacon.fromEventTarget(codeMirror, "change")

That's it for now. Will come back next week. Suggestions for topics?

Bacon.js 0.7.0 is out!

This is the stuff I've been slowly working on in my spare time. I'm so glad I finally got this version released, since it's been in a kind of a 99% ready state for quite a long time, and I absolutely hate to have any "almost ready" code rotting in a codebase. Gotta ship it! 

And the timing is of course perfect too, so you can consider this my $holiday_occurring_in_late_december present to you.

Glitch-free FRP #272

The main goal of the 0.7.0 release is to make FRP with Bacon.js essentially glitch-free. And this is the most invasive change into Bacon.js internals for quite a while. Also something that I many times thought impossible to solve with the brain capacity available to me.

Let's start with the problem that I'm trying to solve here. A glitch in FRP is a temporary violation of an invariant that, with bacon.js at least, occurs when simultaneous events are involved. And just to be precise, my definition for simultaneous events is events that have the same root event. So, for instance, a stream a and its descendant a.map((x) -> x * 2) produce simultaneous events by this definition, while a = Bacon.later(1) and b = Bacon.later(1) do not.

The exec summary of this improvement is that 0.7.0 is more reliable.

But if you're intrested in the details, read on! For instance, if you use Bacon.js 0.6.x and run this

a = Bacon.sequentially(10, [1, 2])
b = a.map((x) -> x * 2)
c = Bacon.combineAsArray([a, b])
c.log()

You'll get

[ 1, 2 ]
[ 2, 2 ]
[ 2, 4 ]

where the pair [2,2] is a glitch that occurs because c is updated first by the changed value of a and the the changed value ofb. In glitch-free FRP you should get just

[ 1, 2 ]
[ 2, 4 ]

Since 0.4.0, Bacon.js has featured "atomic updates", but that has beed quite limited, because they only apply to a graph of Properties. The above example fails in pre-0.7.0 because of this limitation. In this example, the problem could be worked-around by making a a property as in

a = Bacon.sequentially(10, [1, 2]).toProperty()

But there are more complex cases that are harder to work-around. Like if you have a complex graph of Properties that you combine using combineWith(a, b, f), the pre-0.7.0 system doesn't guarantee that your combinator function f won't be called for some invalid intermediate pairs of values when the sources a and b occur simultaneously.

And then there are takeUntil and skipUntil. Previously the functioning of these combinators has depended on subscription order. For example, the correct functioning of a.takeUntil(b) in case of simultaneous events from sources a and b has depended on that the event from b gets in first. And it did in most cases but not always. There's a new test case (look for the word "evil") that fails for pre-0.7.0 versions.

How did I solve this? Simply using a dependency-graph. Now all Observables have a list of "deps". For instance, if you run

Bacon = require("../src/Bacon")

a = Bacon.constant("a")
b = Bacon.constant("b")
c = Bacon.combineAsArray([a, b])

console.log("deps of c:", c.deps().map((s) -> s.toString()))

You'll get

deps of c: [ 'Bacon.constant(a)', 'Bacon.constant(b)' ]

So now, using this new dependency information, it's possible to hold the output of any combined Observable until all of its deps are ready. And that's how it works. Well, except when you add a new subscriber while bacon.js is currently flushing events. To make that work flawlessly too, I had to shave another yak. But it's shaved now and all is well.

Observable.toString #265

As you might have spotted above, there's now a rather nice toString method for all Observables. The output of this method resembles very closely to the code that was used to create the Observable. For instance

Bacon.combineTemplate({name:Bacon.constant("Bacon"), version: Bacon.constant("0.7")}).toString()

will output

Bacon.combineTemplate({name:Bacon.constant(Bacon),version:Bacon.constant(0.7)})'

Also, I've added a name method as @rassie suggested in #273 and implemented .inspect() as an alias to .toString() so that Node.js will show your Observables in a nice manner in the console.

So now you can do this in Node.js.

> B.once("1").name("one")
one
> B.once(1).name("stuff").take(1)
stuff.take(1)

Support for analysis/debugging/profiling tools #273

These are some experimental features that I included in 0.7.0 for preview.

I'm talking about improvements that will enable the implementation of development tools that can, for instance, visualize the event network and event flow going on. The improvements include

observable.deps()

Returns an array of dependencies that the Observable has. For instance, for a.map(function() {}), the deps would return [a]. This method returns the "visible" dependencies only, skipping internal details. This method is thus suitable for visualization tools.

Internally, many combinator functions depend on other combinators to create intermediate Observables that the result will actually depend on. The deps method will skip these internal dependencies.

observable.toString()

Returns a nice textual presentation of the stream. Covered in #265.

observable.internalDeps()

Returns the true dependencies of the observable, including the intermediate "hidden" Observables. This method is for Bacon.js internal purposes but could be useful for debugging/analysis tools as well.

observable.desc()

Returns a structured version of what toString returns. The structured description is an object that contains the fields context,method and args. For example, for Bacon.fromArray([1,2,3]).desc() you'd get

{ context: Bacon, method: "fromArray", args: [[1,2,3]] }

Bacon.spy(f)

Adds your function as a "spy" that will get notified on all new Observables. This will allow a visualization/analytis tool to spy on all Bacon activity. Current implementation just calls your function, but maybe, it should allow you to wrap all created Observables in your own wrapper to make spying even easier? This would enable AOFRP (aspect oriented functional reactive programming) lol.

All of this stuff is included in 0.7.0 but only as experimental features subject to change later. The problem is that spy alone is not enough for visualizing the event network. To observe the lifecycle and events passing through all the Observables, we need to add a way to observe without forcing a subscription. Currently, all you can do is to call subscribe to watch all the events, but that will also prevent the stream from being disposed when it has no "real" subscribers. Nasty details to be be found in #273.

So, even though my intention was to "officially" include #273 into 0.7.0 too, it proved a bit more involved than it seemed at first, so we'll get back to it later.

Anyways, enjoy the new version of Bacon.js and please let me know what you think!

Cheers!

@raimohanska