Scala and Kotlin

I've been spending the last three years writing code in Scala. In general I've been pretty happy with the language. It's much more compact than Java, and its higher-level features, such as closures (which have been around forever but which were popularized, I think, by Ruby), enable me to discard several of the Java tools and frameworks that I've had to drag around. Scala also comes with a very powerful collection library that I use on just about every line of code.

Lately I've been following Kotlin, which looks very promising. I'm old enough to realize that this may just be a case of the grass looking greener on the other side of the fence. Every programming language has its flaws, and Scala is no exception. There's always the sense that maybe the next language will be The One. The One that does everything perfectly. The One that incorporates all that is good about Java and Scala and C# and C and C++ (well okay, maybe not C++), introduces exciting new concepts, yet has no egregious flaws (like green threads, or no reflection, or different-just-for-the-sake-of-being-different keywords, etc.), and for that matter, no minor ones either. A language the bridges the divide between OO and functional, and between OO and relational, and between functional and relational, and between relational and non-relational, and... Well, you get the idea.

Back to Kotlin. I'm hoping that Kotlin will give us most of the benefits of Scala, but without some of Scala's more unusual features.

Consider operators. C++ offered operator overloading, which meant that every math geek could now write "m1 * m2" to multiply two matrices instead of writing (in C) "matrixMultiply(m1, m2)". It also meant that the C++ designers could create a completely new I/O library, in which you don't write data or read it but rather shift it into or out of streams. You can even shift formatting functions into a stream to change its state! Unfortunately they declined to give this new I/O library the obvious name, Shift Happens. But what can I say, I wasn't on the team.

Anyway, with C++, you were limited to overloading a subset of the operators supported by C++. You could overload "+" and "-" and even "," (the comma operator, which most folks don't even realize is an operator), but not "." (the member selection operator) and "::" (scoping operator). There were lots of advantages to sticking with the existing operators. No changes were required to the parser, which must have been a major consideration for Bjarne Stroustrup, the guy who had to write the damn thing. Nor did Stroustrup have to worry about how random new operators might fit into the precedence table. The existing operators also had certain meanings, and so when used judiciously, could actually improve code readability. While "matrixMultiply(m1, m2)" has a fairly obvious meaning, "m1 * m2" is better, and shorter too.

Of course people often did not use operators judiciously, and James Gosling decided to leave the whole business out of Java, and that's where things stayed for about 10 years.

The along came Scala, which eliminates the distinction between methods and operators entirely. Method names can have any character in them. Literally any character. If you want to call your method ";" (normally the statement delimiter in Scala), you can do it:

def `;` { println("Don't try this at home") }

You have to enclose it in backquotes every time, so that Scala know you're calling your stupidly-named ";" method and not trying to end a statement. But it's possible.

Naturally this leads to some questions about how exactly these strangely-named functions should work.

If the programmer writes "a*b," how does Scala know that's "(a) * (b)" and not a method called "a*b"?

The method name can't be completely arbitrary. It must follow a prescribed format. If it starts with a letter, then it cannot include a non-alphanumeric character other than "_" unless those characters follow a "_". So a method called "a*b" is not allowed, but "a_*" is okay. If a method name starts with a non-alphanumeric character, then all of the characters must be non-alphanumeric. So "*" and "**" are both okay but "*a" is not. (However, I should note that these rules only apply to method names not enclosed in backquotes. If you are willing to use backquotes, then anything goes.)

How to the precedence rules work for unusual operators?

Precedence is determined by the first character of the method name. Except when it isn't; Scala requires some special rules so that precedence works as expected. If the operator name ends in "=" then it gets the precedence of "=". Unless it's "<=" or ">=" or "!=" or also starts with an "=" (like "==").

If I define an method called "-", is that the infix subtraction operator or the prefix negation operator?

Infix. To define the prefix negation operator, you have to create a method called "unary_-". Also, the only prefix operators that are allowed are the usual suspects: "+", "-", "!", and "~".

If I define a method called "++", is the pre-increment operator or the post-increment operator?

Okay, actually this is a trick question. Scala doesn't recognize the "++" and "--" operators as such. You can define a "++" method, but it won't do what you expect, that is, assign back to the variable on which it's called. To increment you must use "x += 1".

Method names ending in "=" are special. If you write "a += 3" and there is no method called "+=" on "a" then Scala will try to interpret the expression as "a = a + 3". This only works if the method name starts with a non-alphanumeric character that's not "=".

By far the strangest aspect of Scala's method-operator unification is that if the name of the method/operator ends in ":" (a colon), Scala invokes the method on the object to the right of the operator rather than the left. So while "a*b" means "a.*(b)", "a*:b" means "b.*:(a)". As far as I can figure, this was done solely so that the "::" operator could be used to build singly-linked lists.

Now I have to admit, when I first read the Scala language specification, I thought that method/operator unification was pretty cool, even with the quirky bits described above. But after a few years, I'm not so sure.

Obviously, method and operators cannot be completely unified, at least not in a language in the C family tree. People want precedence to work the way it does in C, more or less, and also want certain conventions retained (like += performing an assignment), so Scala has to recognize a lot of special cases. And Scala had to do away with the increment and decrement operators, which just didn't fit into the generalized model.

But the biggest problem with Scala's strategy is that non-standard operators have no implicit meaning. When we see "+" in the code, we read that as "plus," and we expect it to do act something like addition, or concatenation, or union. We would be very surprised if it, in fact, performed multiplication, or closed a file. That's why operators work: they are symbols that identify specific concepts or specific operations.

But what does the "/:" operator do? Would you have guessed "fold left?" That is what it does, but of course only if the object on the right is a Scala collection. If it's something else, then who knows, it's whatever that object defined. Anyone who's writing in Scala is probably using the collections constantly and has probably reconciled with "/:".

But what if you're writing something with Dispatch, an HTTP client library for Scala? Dispatch implements a typical Scala "domain-specific language," or DSL. In the context of Scala, a DSL is a block of code that looks nothing like Scala and is nearly impossible to read, yet somehow conforms to Scala's syntax rules and produces some desired effect. Let's say you wanted to parse the headers from the HTTP response. You might go looking for a method called parseHeaders or getHeaders or something like that, but you'd be out of luck. The method that introduces a header handler is called ">:+". How do you even pronounce that?

Getting back to Kotlin again. Kotlin takes a much more pragmatic approach to operator overloading. When designing Scala, Martin Odersky recognized that making "==" be the equality method wasn't going to work, because Java had already established that the equality method was named "equals." So in Scala, the "==" operator actually invokes the "equals" method (and cannot be redefined). Kotlin just extends this concept to the other operators as well. The "+" operator invokes the "plus" method, "%" invokes "mod," etc. Notably, the comparison operators "<", ">", etc. invoke compareTo, in keeping with Java's Comparable interface.

In taking this approach, Kotlin loses some flexibility. You can't define a ">:+" operator in Kotlin. But that isn't such a loss, and in exchange, you get some potentially more-useful features that Scala can't support. For example, Kotlin maps ".." to the "rangeTo" method, and maps "[]" to the methods "get" or "set" depending on context, and supports the increment and decrement operators. I think that Kotlin's approach will prove more useful practical over the long run.