Mocking clojurescript code written with core.async



When I write tests for the code in clojurescript and core.async I feel little pain — with-redefs doesn’t work correctly with go-blocks. For example I have a function:

(defn get-subtitles
  [sources limit]
  (go (-> (http/get (get-url sources limit))
          <!
          :body
          format-dates)))

And the test for it:

(deftest ^:async test-get-subtitles
  (with-redefs [http/get (constantly fixture)]
    (go (is (= (<! (get-subtitles const/all 100)) expected))
        (done))))

It didn’t work, it actually tries to make http request. Ok, I can try to put with-redefs inside go-block:

(deftest ^:async test-get-subtitles
  (go (with-redefs [http/get (constantly fixture)]
        (is (= (<! (get-subtitles const/all 100)) expected)))
      (with-redefs [http/get (constantly blank-result)]
        (is (= (<! (get-subtitles const/addicted 100)) [])))
      (done)))

The first assertion works, but in the second assertion I have previously redefined http/get and it’s incorrect and the assertion fails — with-redefs permanently changes var when applied in the go-block.

So I’ve developed a little macro which works like with-redefs and can be used inside of go-block and with code without core.async:

(defmacro with-reset
  [bindings & body]
  (let [names (take-nth 2 bindings)
        vals (take-nth 2 (drop 1 bindings))
        current-vals (map #(list 'identity %) names)
        tempnames (map (comp gensym name) names)
        binds (map vector names vals)
        resets (reverse (map vector names tempnames))
        bind-value (fn [[k v]] (list 'set! k v))]
    `(let [~@(interleave tempnames current-vals)]
       (try
         ~@(map bind-value binds)
         ~@body
         (finally
           ~@(map bind-value resets))))))

And usage:

(deftest ^:async test-get-subtitles
  (go (with-reset [http/get (constantly fixture)]
        (is (= (<! (get-subtitles const/all 100)) expected)))
      (with-reset [http/get (constantly blank-result)]
        (is (= (<! (get-subtitles const/addicted 100)) [])))
      (done)))

I put this macro in clj-di for avoiding copying it between projects.

Tests for this macro on github.

UPD: example updated.

Destructuring arguments in CoffeeScript



CoffeeScript has a good and fairly undocumented feature — destructuring arguments. It’s very similar to binding forms from clojure and a bit like pattern matching like in scala and erlang. It has syntax similar to destructuring assignment but works with functions (and methods) arguments.

Simple examples:

getUppercaseName = ({name}) -> name.toUpperCase()
getUppercaseName name: 'test'
# 'TEST'

head = ([x, ...]) -> x
head [1, 2, 3, 4]
# 1

tail = ([_, xs...]) -> xs
tail [1, 2, 3, 4]
# [ 2, 3, 4 ]

prettifyUser = ({groups: {names: groupNames}, username}) ->
    "#{username}: #{groupNames.join(', ')}"
prettifyUser 
    groups: {names: ['a', 'b', 'c']}
    username: 'user'
# 'user: a, b, c'

Looks useful? But will be more if we combine it with underscore.js. Assume we need to transform result of $.serializeArray() like:

[
    {"name": "title", "value": "Test Title"},
    {"name": "text", "value": ""},
    {"name": "tags", "value": "Python"},
    {"name": "tags", "value": "CoffeeScript"},
    {"name": "tags", "value": "Clojure"},
    {"name": "csrfmiddlewaretoken", "value": "token"}
]

To something like (fields with blank value and service fields should be removed, values of fields with same name should be presented as a list):

{
    "title": "Test Title",
    "tags": ["Python", "CoffeeScript", "Clojure"]
}

Let’s start with imperative solution:

transform = (serializedForm) ->
    groupByName = ->
        result = {}
        for field in serializedForm
            if field.value != '' and field.name != 'csrfmiddlewaretoken'
                if result[field.name]
                    result[field.name].push field.value
                else
                    result[field.name] = [field.value]
        result
    
    flattenFieldsWithSingleValue = ->
        result = {}
        for name, value in groupByName()
            if value.length == 1
                result[name] = value[0]
            else
                result[name] = value
        result
    
    flattenFieldsWithSingleValue()

It works, but looks ugly and complex.

Now look to implementation with underscore.js and destructuring arguments:

transform = (serializedForm) ->
    _.chain serializedForm
     .reject ({name, value}) ->
         value == '' or name == 'csrfmiddlewaretoken'
     .groupBy 'name'
     .map (vals, name) ->
         vals = _.map(vals, ({value}) -> value)
         [name, if vals.length > 1 then vals else vals[0]]
     .object()
     .value()

A lot better I think. Less lines of code, each transformation step can be simple separated from each other. And all this because of fluent interface presented by underscore.js _.chain and destructuring arguments.

Simple application for Google Glass written in scala



About a month ago I received Google Glass and only last weekend I found time for developing something for it. I don’t have a lot of experience in developing software for android, so I’ve started developing simple TODO application.

As a language I selected scala (but I think I should try to write similar application in clojure) and gradle as a build system (because it works out of the box with Android Studio).

For integrating scala and gradle I’ve used gradle-android-scala-plugin which works perfectly and requires adding only ~10 lines to build.gradle for configuration.

And I used scaloid — it’s a very great library, it makes work with android API less painful and makes code a bit more Scala-style.

Developing in scala for android is fun, but I stumbled over one not straightforward problem (perhaps it’s straightforward for mature android developers) — compilation fails with:

[info] Generating classes.dex
[warn] 
[warn] trouble writing output: Too many method references: 181232; max is 65536.
[warn] You may try using --multi-dex option.
[warn] References by package:
...

And the simple solution — change my proguard config to:

-dontoptimize
-dontobfuscate
-dontpreverify
-dontwarn scala.**
-keep class scala.collection.SeqLike {
    public protected *;
}
-ignorewarnings

Another minus it’s compile time. My simple application compiles around 1.5 minutes, tests — around 1 minute. But I found that incremental compilation will be available soon in gradle-android-scala-plugin, so this minus will disappear.

Source code of result, video of result:

Why I like writing proper docstrings while coding in python



Everyone knows that having proper docstrings in code is good, and most straightforward reason why — it’s in-place documentation for classes, functions and methods. So that’s another three reasons:

Sphinx and Read the Docs

For example, we have simple function with docstring:

def as_chan(create_chan):
    """Decorator which creates channel and coroutine. Passes channel as a
    first value to coroutine and returns that channel.

    Usage:

    .. code-block:: python

        @as_chan
        def thermo(chan, unit):
            while True:
                yield chan.put(convert(thermo_get(), unit))

        @coroutine
        def main():
            thermo_chan = thermo('C')
            while True:
                print((yield thermo_chan.get()))  # prints current temperature

    :param create_chan: Type of channel.
    :type create_chan: type[Channel]
    :returns: Created coroutine.

    """
    def decorator(fn):
        def wrapped(*args, **kwargs):
            chan = create_chan()
            coroutine(fn)(chan, *args, **kwargs)
            return chan
        return wrapped
    return decorator

And using sphinx-apidoc and Read the Docs we can generate cool documentation like this. And it’ll be automatically generated after each push to github.

Quick help in IDE

I’ve been using pycharm, but I think there’s something similar in other IDEs.

Suppose we have function (full source code):

def get_servo(num):
    """Creates write and read channels for servo. Should be used
    only in coroutine.

    Usage:

    .. code-block:: python
    
        servo_set, servo_get = get_servo(1)
        yield servo_set.put(90)  # set servo to 90 degrees
        yield servo_get.get()  # prints current servo degree

    :param num: Number of servo.
    :type num: int
    :returns: Write and read channels.
    :rtype: (Channel, SlidingChannel)

    """

It’s not simple to guess what it returns and how work with it, but if we just press ctrl+Q:

screenshot

Warnings on wrong type and better autocomplete with type hinting

Assume we started writing simple function:

def get_user_groups(user):
    """
    :type user: User 
    :rtype: list[Group]
    """

With type hinting we can use proper autocomplete inside it:

screenshot

And for result of the function:

screenshot

And we can see warnings when pass value of wrong type to the function:

screenshot

More about type hinting.

Steven F. Lott: Mastering Object-oriented Python



book cover Few days ago I finished reading “Mastering Object-oriented Python” by Steven F. Lott, and I think it’s nearly a good book. Maybe it’s longer than need (~600 pages) and sometimes I used to think that content repeats. Some code samples are controversial and not always follows PEP8.

But I think this book worth reading, they contains good explanation of magic methods, good examples of using abc and collections.abc, interesting chapter about logging and good manual for documenting python code.

FRP* on pyboard



When almost a year ago I watched interesting course Principles of Reactive Programming, I found that many examples of FRP are about circuits and wires. I tried to do something similar to course examples on arduino, but nothing good came, mostly because low expressiveness of Wiring.

And now I trying to do it with pyboard and my microasync library.

First of all, I wrote simple actor (not a really actor, but something similar) for bicolor LED:

from microasync.async import as_chan, do_all
from microasync.device import get_output_pin


@as_chan(Channel)  # now this function returns channel
def get_bicolor_led(chan, left, right):  # `channel` is this actor "mailbox"
    left_pin = get_output_pin(left)  # returns channel for a pin, when we put 1 in that 
    right_pin = get_output_pin(right)  # channel then voltage will be set to 3.3V; when we put 0 - 0V
    while True:
        msg = yield chan.get()  # receive message from a channel
        if msg == 'red':
            yield do_all(left_pin.put(1),  # do commands sequentially
                         right_pin.put(0))
        elif msg == 'green':
            yield do_all(left_pin.put(0),
                         right_pin.put(1))
        elif msg == 'yellow':
            yield do_all(left_pin.put(1),
                         right_pin.put(1))
        elif msg == 'none':
            yield do_all(left_pin.put(0),
                         right_pin.put(0))

This actor is simple to use, for changing color of LED we just need to send (put in channel) red, green, yellow or none:

from microasync.async import coroutine


@coroutine
def test_bicolor_led():
    led = get_bicolar_led('X1', 'X2')
    yield led.put('red')  # switch LED color to red
    yield led.put('green')  # switch LED color to green
    yield led.put('yellow')  # switch LED color to yellow (red and green together)
    yield led.put('none')  # turn off LED

Then I created simple filter for channel, which can be toggled by button on pyboard:

from microasync.async import as_chan, select
from microasync.device import get_switch


@as_chan(Channel)
def switchable_filter(chan, orig_chan, fn):
    # Returns channel from which we can get values from switch and this actor
    # "mailbox", works almost like `select` from golang:
    select_ch = select(get_switch(), chan)
    enabled = False
    while True:
        result_ch, val = yield select_ch.get()
        if result_ch == chan:
            if not enabled or fn(val):  # apply filter if filter enabled
                yield orig_chan.put(val)
        else:
            enabled = not enabled  # toggle filter state

Now I created simple coroutine, which sends red, green, yellow and none to two LEDs sequentially in loop. And when we click button on pyboard we toggle filter, which passess all messages except red to the first LED and only red to the second LED:

from microasync.async import coroutine


@coroutine
def main():
    first_led = switchable_filter(get_bicolor_led('X1', 'X2'),  # creates bicolor led and applies filter
                                  lambda msg: msg != 'red') 
    second_led = switchable_filter(get_bicolor_led('X3', 'X4'),
                                   lambda msg: msg == 'red')
    while True:
        for led in (first_led, second_led):
            for mode in ('red', 'green', 'yellow', 'none'):
                yield led.put(mode)  # sends red, green, yellow, none 

Full source code of example available on github. And video with result:

* not a really FRP, but almost follows The Reactive Manifesto

clj-di: dependency injection for clojure and clojurescript



I was surprised when i found that no one wrote library for dependency injection which works on clojure and clojurescript. I found component, but it’s too complex and not working with clojurescript.

So i developed little library – clj-di. I wrote it using cljx, so library can work with clojure and clojurescript. Library uses atom with hash map for storing dependencies. Dependency can be registered and received using keyword.

Library has two ways for registering dependencies. Permanently with register! (i use it in lein-ring :init for creating db connections):

(register! :db (Database.)
           :logger (get-logger))

And for code block with with-registered (i use it in tests for registering mocks):

(with-registered [:db (Database.)
                  :logger (get-logger)]
  ...)

And two ways for receiving dependency. With get-dep function (useful for receiving single dependency):

(get-dep :db)

And with let-deps macro (useful for receiving more than one dependency):

(let-deps [conn :db
           log :logger]
  ..)

You can see more on github and in documentation.

CSP on pyboard



Lately i worked a lot with clojure and core.async, and i was very impressed with CSP. Mostly because it saves me from js callback hell. For example, js code with sequential http requests (heh, with promises it’s less ugly):

$http.get('/users/').then(function(data){
    return $http.post('/user-data/', data.items);
}).then(function(){
    return $http.get('/posts/');
}).then(function(posts){
    console.log(posts);
});

With clojurescript and core.async it will be:

(go (let [users (<! (http/get "/users/"))]
       (<! (http/post "/user-data/" (:items users)))
       (js/console.log (<! (http/get "/posts/")))))

Clojurescript code looks more readable and simple.

And i developed CSP for micropython in my microasync library.

Example application — servo should rotate to angle which equal to X angle of accelerometer, and user can start/stop app with button:

from microasync.device import get_servo, get_accel, get_switch
from microasync.async import coroutine, loop, Delay, select


@coroutine
def servo_coroutine():
    servo, _ = get_servo(1)  # get_servo returns set and get channels
    accel = get_accel()
    switch = get_switch()
    on = False
    x = 0
    accel_or_switch = select(switch, accel)  # like select from go and like clojure core.async alts!
    while True:
        chan, val = yield accel_or_switch.get()  # like clojure (<! (accel_or_switch))
        if chan == accel:
            x, *_ = val  # we don't need y and z
        elif chan == switch:
            on = not on

        if on:
            yield servo.put(x)  # like clojure (>! servo x)
        yield Delay(0)  # like clojure (<! (timeout 0))


servo_coroutine()
loop()

And recorded on google glass (yep, i bought it few days ago) video of result:

Fixing StackOverflowError on travis-ci when running clojurescript tests



Recently i received strange error when running clojurescript tests on travis-ci:

$ lein2 cljsbuild test
...
Compiling "target/cljs-test.js" failed.
Exception in thread "main" java.lang.StackOverflowError, compiling:(/tmp/form-init311799534829133165.clj:1:89)
	at clojure.lang.Compiler.load(Compiler.java:7142)
	at clojure.lang.Compiler.loadFile(Compiler.java:7086)
	at clojure.main$load_script.invoke(main.clj:274)
	at clojure.main$init_opt.invoke(main.clj:279)
	at clojure.main$initialize.invoke(main.clj:307)
	at clojure.main$null_opt.invoke(main.clj:342)
	at clojure.main$main.doInvoke(main.clj:420)
	at clojure.lang.RestFn.invoke(RestFn.java:421)
	at clojure.lang.Var.invoke(Var.java:383)
	at clojure.lang.AFn.applyToHelper(AFn.java:156)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.main.main(main.java:37)
Caused by: java.lang.StackOverflowError
...

But locally all worked ok. And i found simple solution – increase thread stack size up to 16mb by adding this line to my project.clj:

:jvm-opts ["-Xss16m"]

As a result travis-ci build succeeded.

Switching windows with Leap Motion



After i abandoned my project on Clojure Cup i received little free time on this weekend. And i decided to play with Leap Motion and create little app for switching windows using it.

Result — leaptab. By default app uses win + w for switching windows, but with --use-alt-tab it can use alt + tab.

For opening window switcher (win + w or alt + tab) and switching to window you need to use circle gesture. For selecting window – swipe gesture.

Video with usage of leaptab:

Leaptab tested with ubuntu, but should work on all major platforms, but probably only with --use-alt-tab argument.