A variation on Scala's Either type

I’ve been using Scala’s Either type a bit recently, and found a couple of minor annoyances with it. So as a learning exercise I tried to make a variation of Either, and this is what I came up with. I also wanted to experiment with syntax a little. See what you think.

Either is a disjunctive type with two possible values: a left and a right. It places no meaning at all on left or right, but 99% of its uses in practice involve putting normal, expected values in right, and errors or other “exceptional” values in left. This means that almost all the time you end up getting the RightProjection, e.g.:

for { v1 <- foo(…).right v2 <- bar(…).right v3 <- baz(…).right } yield …

Whereas I’ve never used .left even once.

The second minor issue is that Either has some limitations with patterns in for expressions, because of how it is forced to implement the filter method. See this Stack Overflow question for the details.

So I created a type that is explicitly designed to handle the possibility of “exception” conditions. Because the word “exception” already has a well known meaning on the JVM, I called this type Anomaly instead. Actually, I ended up calling the type |: because I thought it worked nicely syntactically. Here’s a comparison with Either:

val e1: Either[String, Int] = Right(30) val e2: Either[String, (Int, Int)] = Right((10, 2)) val e3 = for { i <- e1.right j <- e2.right } yield i + j._1 + j._2 val e4: Either[String, Int] = Left("error")

vs “Anomaly”:

val a1: String |: Int = Expected(30) val a2: String |: (Int, Int) = Expected((10, 2)) val a3 = for { i <- a1 (j, k) <- a2 } yield i + j + k val a4: String |: Int = Anomaly("error")

The |: operator is right-associative, which helps for the occasional cases where you have an Anomaly within an Anomaly, eg. String |: String |: Int is equivalent to Either[String, Either[String, Int]]. Of course, depending on your taste you could reverse the order of type arguments, using an operator like :|, or use a name instead of an operator.

There’s a cost to making the for expression pattern matching work like this, however. In addition to the Anomaly and Expected cases, I had to introduce a NoValue case object, which is what is returned when a filter does not match. This is annoying, because now folding or pattern matching must account for three cases instead of two. And unless you have explicit filters or patterns matching particular values, the NoValue outcome won’t come up, so it’s a pain to have to check for it. What I ended up doing was providing an alternate two-parameter fold method that calls error in case of a NoValue. You just have to be careful about only calling it when you know it’s safe.

All-in-all I’m still wondering whether this experiment is worthwhile, but here’s the code in case anyone is interested. There are other utility methods that could be added:

sealed abstract class |:[+A, +E] { def anomaly: A def expected: E def isExpected: Boolean = false def isAnomaly: Boolean = false def isNoValue: Boolean = false  def map[X](f: E => X): A |: X  def flatMap[X, AA >: A](f: E => AA |: X): AA |: X  def withFilter(f: E => Boolean): A |: E = this final def filter(f: E => Boolean): A |: E = withFilter(f)  def foreach(f: E => Unit): Unit = {}  final def fold[X](fExpected: E => X, fAnomaly: A => X, fNoValue: => X): X = this match { case Anomaly(a) => fAnomaly(a) case Expected(e) => fExpected(e) case NoValue => fNoValue }  final def fold[X](fExpected: E => X, fAnomaly: A => X): X = this match { case Anomaly(a) => fAnomaly(a) case Expected(e) => fExpected(e) case NoValue => error("Attempt to fold NoValue case") }  final def join[X, AA >: A](implicit ev: E <:< (AA |: X)): AA |: X = this match { case Anomaly(a) => Anomaly(a) case Expected(e) => e case NoValue => NoValue } }  final case class Expected[+A, +E](expected: E) extends (A |: E) { override def isExpected = true def anomaly: A = Predef.error("Not an anomaly")  override def withFilter(f: E => Boolean) = if (f(expected)) this else NoValue  def flatMap[X, AA >: A](f: E => AA |: X) = f(expected)  def map[X](f: E => X) = Expected[A, X](f(expected))  override def foreach(f: E => Unit) { f(expected) } }  object Expected { def cond[A, E](test: Boolean, expected: => E, anomaly: => A): A |: E = if (test) Expected(expected) else Anomaly(anomaly) }  final case class Anomaly[+A, +E](anomaly: A) extends (A |: E) { override def isAnomaly = true def expected: E = Predef.error("Not an expected value")  def flatMap[X, AA >: A](f: E => AA |: X) = Anomaly[AA, X](anomaly)  def map[X](f: E => X) = Anomaly[A, X](anomaly) }  object Anomaly { def cond[A, E](test: Boolean, anomaly: => A, expected: => E): A |: E = if (test) Anomaly(anomaly) else Expected(expected) }  case object NoValue extends (Nothing |: Nothing) { override def isNoValue = true def anomaly = Predef.error("Not an anomaly") def expected = Predef.error("No value")  def flatMap[X, AA >: Nothing](f: Nothing => AA |: X) = this  def map[X](f: Nothing => X) = this }

Posted via email from lockster's posterous

Ceylon: Interesting for the Wrong Reasons

A couple of days ago, Gavin King announced that he is leading a team at RedHat that’s creating a new JVM language called Ceylon. See here for most of what is known so far. The stated motivation, in a nutshell is that they like Java very much, but it has deficiencies that are preventing progress in key areas, and they are generally feeling “frustrated”. They don’t have a complete compiler yet, but they’re working on it, and I get the impression RedHat is pretty serious about it.

Gavin King’s slides provide a fascinating insight into the kind of thinking that remains so dominant in the Java community. Gavin King is well known in the Java world as the inventor/leader of Hibernate and various other JBoss projects. He certainly knows Java inside out, and his list of “frustrations” is astute, if incomplete. While I haven’t dug into all the details so far revealed about Ceylon, but it seems likely that they’ll succeed in their goal of creating “a better Java”. If given the choice between using Java and Ceylon, I could certainly see myself choosing Ceylon. But Ceylon is a terrible idea. The problem is that they are treating the symptoms, not the disease; and they’re repeating exactly the same mistakes that resulted in them finally hitting a dead-end with Java.

Possibly the best outcome of Ceylon is that it emphatically refutes the argument that “Java is all we need”. Few have put more hard labour into trying to make Java feasible than Gavin King and the JBoss folks. They and a few others have spent the last decade or so producing a staggering number of bloated tools, libraries and frameworks in order to keep Java going. Don’t get me wrong, there’s a lot of extremely useful Java libraries out there, but there’s also a huge amount of guff that exists solely to compensate for the impracticality of the Java language. Ceylon is the “main-stream” admitting, finally, that not all problems can be solved the Java way: more frameworks, more libraries, more tools, more code.

So Mr King and friends have finally hit the wall with Java. They’re a bit late to the party, but we should welcome them none the less. So why do I say Ceylon is a terrible idea? The short answer is: Scala. The long answer is, well, longer. My biased summary of the situation is this: after years of having to build more and more layers of increasingly baroque frameworks in order to make progress, they realise that they can no longer keep up with their competitors (cough_C#cough_) because Java the language is an unavoidable “bottleneck”. They identify certain essential abstractions that are needed to solve their immediate problems, and proceed to design a new language that provides those essential abstractions, while remaining as “familiar” as possible.

In other words, Ceylon is all about making “a better Java”. There is certainly support for a evolutionary approach like this. Some argue that Java itself was (in language terms) “a better C++”, so why not follow that successful approach now? Well, if that worked so well for Java, why are even its greatest fans now trying to replace it? I’m reminded of Linus Torvalds' reaction to “Subversion is a better CVS”. The problem with Subversion isn’t that is poorly designed or built; it isn’t. The problem with Subversion is that the goal itself just isn’t very useful.

This is how it’s going to go: Ceylon will be built, they’ll make a SDK library, then there will be Hibernate-Ceylon, Spring-Ceylon, WebFrameworks-Ceylon 1 through 63,854, Logging4Ceylon, Ceylon.util.logging, commons-logging-ceylon and Simple Logging Facade 4 Ceylon. All these will be an improvement over their Java equivalents. Then, in another five or so years they’ll hit some new pain points and realise that some of those other “academic” abstractions are actually useful in the real world after all. For example, Ceylon supports higher-order functions (but, inexplicably, not lambda expressions). If we could go back in time 5 years and ask Gavin King “how important do you think higher-order functions are?”, what would he say? My guess is “they’re not essential for Real Work™”.

But it’s still an improvement, so why not embrace it? Because we have an already have a much better option that’s already doing the job in the real world: Scala. Or look at the interesting stuff going on in .NET land. Ceylon’s goals look positively narrow and uninspiring by comparison. Of course, Gavin King was immediately asked “why not Scala?”. It is my sincere hope that this question will continue to be asked of him. His answer consists of the same misconceptions that Scala advocates have been trying to dispel since forever. Take this statement from Mr King, from an interview in InfoQ:

One important force that helped make Java so successful was the things Java left out. But you need to be very smart here: Java’s decision to leave out higher-order function support was enormously harmful. Now, obviously the Scala team have gone down a very different path here and have created a very feature-rich type system. As a general rule I would prefer to solve my problems with fewer, more general constructs, than with more, less powerful constructs.

This is the exact inverse of the truth. No one who has made a serious effort to understand Scala could make such a statement. “Fewer, more general constructs” is exactly what Scala is all about! My summary of Scala’s mission statement would be as follows:

• Scala aims to maximise the potential for useful abstractions, whilst:
  • giving assurance that your abstractions are correct (static typing)
  • allowing you to do what is necessary to get the performance you need
  • maintaining seamless interoperability with Java

Scala aims to be compatible with Java, but it’s about much more than just “a better Java”. Ironically, it seems as though Ceylon will not interoperate with Java as seamlessly as Scala does (e.g. Scala keeps null and type erasure). It’ll be interesting to see how that works out. To get an idea of how serious the Scala team is about powerful but practical abstractions, look at how they handle the tensions that crop up between abstraction and performance. Great examples of this are how Scala handles the unpleasantness of arrays on the JVM, and also the @specialized annotation.

Scala is by no means perfect. The trade-offs it makes will not suit everyone fully. But I cannot help but scratch my head over statements like this (Gavin King, again):

But I guess I should mention that the number one technical problem that we simply can’t solve to our satisfaction in Java – or in any other existing JVM language – is the problem of defining user interfaces and structured data using a typesafe, hierarchical syntax.

I could very well be missing something, but this would seem like a problem that is right in Scala’s wheelhouse. Perhaps it may not be solved satisfactorily today, but it could done with far less effort than creating a whole new language (and new standard library, and new tool chain…). It is depressing to think of what these guys could achieve working with Scala instead of pouring resources into Ceylon.

You may wonder, “if you think Scala is so good, then use it and be happy, why worry about Ceylon?” Well, I worry because I think Ceylon is worse than Scala, but it could win anyway. That actually seems to be the more common outcome in these situations. I would much prefer a world with Scala jobs in demand than one with Ceylon jobs in demand. So, yes, it’s all about me being selfish. I would say to all Scala fans: don’t be afraid to be a little selfish and evangelise for the better outcome.

Posted via email from lockster's posterous

Creationism and climate change denial

I’ve noticed this more and more: people lumping “climate change denial” and “evolution denial” (creationism) into the same category. The argument is that both of these are the result of the same religious conservative “anti-science” movement. In a very narrow sense, this is true; creationists generally do not seem to believe in anthropogenic global warming (AGW), for the same bogus reasons they don’t believe in evolution. But the reverse is not true: many (most I would say) AGW skeptics are not creationists. Lumping these two things together allows the environmentalists to dismiss any criticism of the established climate change narrative as the ravings of anti-scientific religious nuts. But there are some very significant scientific and economic arguments around climate change that are most definitely not in the same class as creationist arguments. Time for some de-lumping.

One major difference between climate change and evolution is how the evidence is understood and presented. It is now a familiar experience to see someone on TV telling us we need to immediately reduce emissions to avoid climatic catastrophe. In the rare event that such a person is asked to provide evidence of a human cause, the most common answer I’ve observed is along the lines of “ice caps and glaciers are melting, and the scientific consensus is that human CO₂ emissions are to blame.” Now, when was the last time you heard someone asked for evidence supporting evolution to respond with “life forms adapt to their environment and the scientific consensus is that Darwinian evolution is the cause.” This is not a small point. The fact that “consensus” is so regularly rolled out to support AGW, but is never used to support evolution shows that the levels of scientific understanding and debate are completely different.

No one talks about the “scientific consensus” for evolution because there’s no need to: evolution has a mountain of cold hard facts on its side, all confirmed by independent observation countless times. Climate research is very tough by comparison: our observational capabilities generally aren’t good enough to falsify competing theories. Instead, we have the IPCC saying that since we can’t establish the facts by direct observation, we’ll establish them by consensus. You can’t take the IPCC publications, repeat the experiments and confirm their results, because at the end of the day it’s just opinion.

My point here is not to show that the IPCC is wrong, but simply to show that the scientific process that it follows is fundamentally and necessarily different to that of evolutionary biology, or any other hard science for that matter. Questioning the consensus-derived “facts” of climate research is not the same thing as questioning the empirically verified facts of evolution.

Another difference I think is worth pointing out is how politics is so commonly mixed with climate science, in contrast to evolution science. Tell me if this scenario sounds familiar: well credentialed scientist goes on TV and says “my research shows that climate change is likely to significantly damage coral reefs…” and barely pausing for breath continues “so it’s clear we must act to cut CO₂ emissions.” This casual connection of minor scientific research to sweeping reorganization of the world’s energy production is now so common place that people don’t find it remarkable. And if you accept the climate change problem but doubt the solution you’ll likely still receive the “denier” pejorative. Evolutionary biology simply isn’t political in this way.

Skepticism regarding climate change is not comparable to the denial of the established facts of evolution. Climate research has a long, long way to go before it achieves the level of confidence we have in evolution. It is very wrong to think that all opposition to the climate change narrative is driven by Biblical literalism.

Posted via email from lockster's posterous

WikiLeaks

Attack and Counter-Attack

I must confess, no story in recent months has captured my attention like the still-developing WikiLeaks saga. The ongoing dump of diplomatic cables combined with Julian Assange being charged with some sort of sex crime in Sweden means this is going to be in our face for a while yet.

The response to WikiLeaks has been loud, angry, hyperbolic attacks from “tough on terror” types in the U.S., and a corresponding thoughtless, reflexive defence from many “progressive” types. Amongst the noise are a few precious thoughtful critiques.

Clearly, WikiLeaks is not engaged in terrorism, and Julian Assange is not remotely comparable to the most junior jihadist. Sarah Palin’s pronouncements on the subject are pure demagoguery (by now I think we can fairly declare that an axiom with regard to any statement she makes). And Senator Lieberman needs to seriously deflate his sense of his own importance, and recognise that maintaining the government’s operational security isn’t actually in his job description.

I believe all these clumsy attacks against WikiLeaks will fail in the end, and they deserve to fail. In fact, I think they are completely counter-productive: sympathy for WikiLeaks is growing as people become concerned about the government trying to restrict speech.

The defenders of WikiLeaks are themselves hugely overreacting, however. The “attacks” against WikiLeaks and Mr Assange so far consist of: a few random nuts-with-a-mike threatening violence, some companies cutting off service, and a number of people expressing the opinion that U.S. law has been broken. Taken together, all this doesn’t come close to being a serious attempt to censor WikiLeaks. In the event that Mr Assange is charged by the DOJ, which is unlikely in my opinion, he’ll have his day in court. In the even more unlikely event that he’s actually convicted of espionage, then it should be the law you’re complaining about rather than the enforcement of it. And as Christopher Hitchens points out, those who choose to engage in civil disobedience must be prepared to face the consequences.

(The reason I say charges are unlikely is that I can’t see how the DOJ could credibly prosecute Mr Assange without also prosecuting the editor of the New York Times, which would be sure to end in disaster. But, who knows?)

At a time when publishing a cartoon depicting the Prophet Mohammed means you run the very real risk of being murdered, I can read people earnestly telling me that Amazon and Paypal cutting off WikiLeaks represents the greatest free-speech fight of our generation. Perspective is sorely lacking here.

Transparency

Overall, I’m far more concerned about the way WikiLeaks is being lauded and defended than the way it’s being attacked. I’ve seen a call for Julian Assange to be awarded the Nobel Peace Prize and another for him to be named Australian of the Year. This is madness.

The word most often used by WikiLeaks fans is transparency. More transparency is good, and WikiLeaks provides more transparency, therefore WikiLeaks is good. Mr Assange certainly understands his fan base very well, as this is the line he takes in his deceitful, self-serving piece in The Australian. Well, I need to make a few points about transparency.

First, transparency is a means, not an end in itself. The ends are, essentially, ensuring that government office holders are acting within the law and in accordance with the will of the people. Of course, this is both a good and necessary thing. In principle, I’m all in favour of leaking government secrets when there’s a clear public interest in doing so. This has served us well in the past, and will do so again in the future.

But it must be recognised that transparency may also lead to negative ends. This is why responsible journalists weigh, as best they can, the public interest against the potential downsides before publishing leaked information. This article provides a good description of the negative ends that may result from the “transparency” that WikiLeaks has forced on us.

Now we must delve into the theories and ideology of Julian Assange. He has explained these in detail, but I shall do my best to summarise them fairly here. What the defenders of WikiLeaks don’t seem to have realised is that Mr Assange actually has no interest in improving transparency, or holding governments to account. His goal is essentially to conduct information warfare against what he views as “conspiratorial authoritarian regimes”. This is why, in sharp contrast to all the talk of transparency, WikiLeaks does everything it can to conceal its own activities. It’s an organisation built for conducting information warfare.

So, the objective of indiscriminately releasing all the diplomatic cables was not to inform the public, the objective was to impede the ability of the “regimes” to conduct diplomacy in the future. This is a goal that may very well have been achieved. Julian Assange is not trying to improve the system, he’s trying to smash it. In a nutshell, he’s an anarchist using information as his weapon. And after he has smashed every institution of liberal democracy, what then? Like any good anarchist, he has nothing more to say.

The fact that so many of the leaks have been “against” the U.S. government leads to the unavoidable conclusion that Mr Assange considers that government to be a “conspiratorial authoritarian regime”. Judging by his rhetoric, I strongly suspect he views all governments that way. Now we come to the really critical point: if Julian Assange is right about this, then his actions make perfect sense and should be supported. It should be no surprise by now that I think he is completely wrong.

It is necessary to follow the implications of accepting Mr Assange’s thesis. He is claiming that our governments that are ostensibly democratic systems working for the people, are in fact authoritarian regimes that are actively conspiring against the people. Your nation’s constitution is a lie. Our traditions of liberal democracy are actually myths. Debate is pointless. Your vote, irrelevant. Breaking the machine with information warfare is the only way to truly affect change.

Now if I try to muster every last reserve of cynicism and despair at my disposal, I cannot bring myself close to believing such a thing. Am I a naive, dewy-eyed, babe in the woods?

Returning the subject of transparency, I would argue that in historical terms we’re actually doing reasonably well on the transparency front these days. Let us consider some of the government activities that the WikiLeaks cheerleaders are generally most concerned about: detention without trial, “enhanced interrogation”, handing prisoners over to foreign authorities known to practice torture, extraordinary rendition, support for the Karzai regime/racket, I could go on. Many of these issues concern me too. But we knew about these policies before the valiant Mr Assange rode onto the scene. The leaks have, at most, underscored a few things. These policies are no more likely to be changed after the leaks than they were before.

Opponents of these harsh policies need to face up to a conclusion that appears most unwelcome to them: conspiracy and lack of transparency are not what keeps the policies alive. The real reason they persist is much more prosaic: the electorate, broadly speaking, is ok with them. What’s called for is less vandalism and more old-fashioned criticism. Make your arguments and try to convince your fellow citizens. Don’t be afraid to be boring about it (I certainly have no such fear – ha!). Or forget all that and go with cynical ranting about a public stupefied by mass media and consumerism. Either way.

In conclusion, I do not want to see WikiLeaks silenced or Julian Assange in jail for espionage (I’m not going to comment on the Swedish charges, thus far they seem completely tangential). However, those of us who believe that liberal democracy has some life in it yet need to criticise the undemocratic and destructive ideology that drives WikiLeaks. This is a secretive, unelected, unaccountable group that releases whatever information it can get its hands on. It indiscriminately and purposefully harms the ability of our government office holders to do the work we elected them to do. Our oft-derided citizenry at least have the pragmatic sense to recognise that those to whom we have delegated power can better achieve our common good when some secrecy is maintained. Julian Assange should respect that democratically determined outcome, and give up this reckless test of his pet conspiracy theories.

Posted via email from lockster's posterous

Some more thoughts on Apple

A few more points that occurred to me, following up on my last post.

Whither the mouse?

Since its inception in 1984, the Mac user interface has obviously been designed for desktop computers with a mouse and a keyboard. When notebook computers came along and we wanted to be able to use them without carrying a mouse everywhere, or without even a flat, stable surface to put the mouse on, they had to emulate the mouse as best they could. This may now be reversing.

In the announcement of Mac OS X Lion, there was great emphasis placed on the use of multitouch gestures as a means of control. I think Lion will be the first OS designed for notebooks with multitouch trackpads, with the desktop Macs left to emulate the MacBooks as best they can. If you consider that a sizeable majority of the hardware Apple sells today is mobile and primarily operated via some form of multitouch instead of a mouse, this move seems sensible. And all new desktop Macs ship with the multitouch-capable Magic Mouse as standard now.

The Costs of Supporting Java

With regard to Java, there's been a few comments along the lines of "all the hardware Apple sells to Java developers must more than cover the cost of the handful of engineers needed to keep maintaining Java". I think this underestimates the costs involved. Supporting Java means, in addition to the engineers, that:

• Java must be accounted for in development plans
• Java must be accounted for in QA plans
• Customer support needs to deal with it
• Security vulnerabilities need to be investigated and fixed

The true cost isn't so much the money, it's that it unavoidably diverts some of the company's attention from things it would rather focus on. While continuing with Java support wouldn't be hugely onerous for Apple, eliminating distractions that aren't bringing much benefit can really help competitiveness.

Also, as the vast majority of Java work (since the switch to Intel) has been on the GUI, I'd expect the Java team knows the guts of the Mac GUI pretty well by now. Engineering talent isn't exactly a fungible resource. There's probably a development manager somewhere in Apple salivating at the thought of having the Java people on her team.

Apple and the Server Side

Folks have pointed out that Apple is a heavy user of server-side Java in their web services. I can think of many ways they can deal with that, and have no idea which option they'll take. But I think we can be sure they've got a plan and aren't leaving it to chance. And despite everyone (including me) getting very animated over "Apple dropping Java", it's worth remembering that their Java team isn't going to be shutting down tomorrow.

More generally, Apple's plans for the cloud are completely opaque to me right now. We're all assuming they have big plans for cloud services: they built that huge new data centre, and their hardware strategy seems to imply more reliance on cloud services in the future. The iPad's greatest weakness is that so much of its functionality, including just initialising it, requires USB synchronisation with a PC or Mac. MobileMe syncing is great, but it only works for a few things.

The new MacBook Air models were just announced, including one with an 11" screen, which I think is a first for Apple notebooks. When making the announcement, Steve Jobs said something like "we think this is the future of MacBooks". With their emphasis on mobile, wireless use and limited storage, these too are crying out for more cloud services.

And I think this could be Apple's greatest weakness. If I were Steve Jobs, the thing that would worry me most about Android isn't its "openness" (however you care to define it), or the temporary situation of shelf-space in U.S. phone outlets, but the fact it is backed up by the world's best cloud services. And they're only going to get better. Obstacles remain, but the movement towards doing more and more in the cloud is inexorable. Where Google has no rival, Apple's track record is not that great. The iTunes store is a huge success, obviously, but it's not really all that "cloudy", as you need a Mac with enough hard drive to hold all your content. MobileMe is expensive, and often unreliable and slow; it really only survives due to the first class client support on Mac and iOS. The iWork service is not bad, but lacks Google's killer feature of collaborative editing. Apple has its work cut out for it when it comes to the cloud.

Flash, Core Animation and Multitouch

In my last post I claimed Flash didn't support Core Animation or Multitouch. Well, @johnyanarella set me right on that. Flash does support both of these. I haven't used that many Flash apps, so now I'm wondering to what extent the multitouch support is actually used in apps.

Java and the Mac

So Apple has quietly announced that it is deprecating its support for Java on the Mac. The wording was (not uncharacteristically for situations like this) a bit vague on the details of what this means. I think two things are definite: 1) Apple will continue to support Java 6 on Snow Leopard until it reaches end of life; which I think happens when the OS after Lion is publicly released. 2) Apple will not port Java 7+ to the Mac. The main unresolved question is whether Java 6 will be supported on Lion. My guess is that it will be offered as an optional download, similar to Rosetta in Snow Leopard. Apple won't make any promises though, as they will want to have the option to drop it if it's too much effort.

As a Mac user who has been working on the Java platform for many years, this is a fairly big deal for me. I think it's an interesting event that provides a chance to examine both Java's position and Apple's plans for the Mac. People in the Mac Java community are speculating about the deeper meaning behind this move, especially when combined with Apple effectively banning Java platform applications from the new Mac App Store. So time for me to add my speculation to the noise.

Is Apple trying to force the use of their platform and tools?

No. True, Apple has made it crystal clear on numerous occasions (the Mac App store requirements are just the latest case) that they want developers targeting their OS platforms to use their development platform (Objective-C, Cocoa, Xcode). But I don't think they are deprecating Java in an attempt to force the issue. After all, Lion will come with Perl, Python and Ruby all pre-installed.

Does Apple want to kill cross-platform desktop apps?

No. I think it's true that, strategically speaking, Apple has no interest at all in cross-platform apps (outside of the browser). But along with announcing the deprecation, the latest Apple Java release also includes brand-new support for installing multiple versions of the Java platform, explicitly mentioning support for third-party versions. It's clear that Apple has no problem at all with Java running on Macs. I think they want Oracle to take it over, but I doubt they bothered to make any sort of deal with Oracle before making this move. Update: as I was drafting this, news came through that Steve Jobs has replied to a question about this, which basically confirms that he thinks Oracle should be responsible for Java on the Mac.

Then why did Apple do this?

The reason isn't that complicated: Apple no longer needs Java. If you make a list of what Steve Jobs sees as the critical objectives for Apple, it becomes immediately obvious that maintaining a Mac port of Java is not helping to advance any of them. Of course, neither does maintaining, say, Apple's port of Python. But Python takes very little effort to port and maintain. The Java port requires a team of engineers permanently dedicated to it. Also, the huge success of iOS has given Apple the confidence that their approach to working with third-party developers is working out great for everyone. The prospect of Java developers and applications abandoning the Mac is no longer remotely scary for them. Apple have decided they'd rather pay the costs of dropping Java than keep maintaining it.

But what are Apple's critical objectives?

The main theme of the "Back to the Mac" event on October 20 was the idea of moving iPad software and hardware features into the Mac, in a way that made sense for the Mac. My theory is that Apple has now oriented its entire strategy around the iPad. I think Steve Jobs probably regards the iPad has the best thing Apple has ever made. This is because the iPad provides the "purest" user experience: you hold it in your hands, rotate it, move it, directly manipulate the information you're working with via touch, and it dedicates itself to the task at hand. Of course, the iPhone has these characteristics too, but if will forever be limited by the (obviously worthwhile) trade-off of fitting in your pocket. The larger screen size of the iPad allows the touch interface to reach its full potential.

Slight diversion: in the recent financial call, Jobs said that 7" was a lousy form-factor for a tablet. You had better believe that he meant it. I will bet that Apple will never release an iPad with a different form factor. Going smaller puts to many constraints on the options for touch UI, so you end up with a bulky iPhone. Going larger would be nice, but it rapidly becomes unwieldy (the fact the iPad can be easily held and moved around is a critical part of the "experience"). The dimensions were chosen to balance the needs of video with those of e-books. Believe me, Apple chose the form-factor it did very carefully.

And yes, something else Jobs likes about the iPad is that Apple has more control over it. While many ascribe this to cynical motives, I believe Jobs honestly sees this as being good for the customer, and part of the reason for the success of iOS devices. Whether this view is correct will, in the end, be decided in the marketplace.

What is the future for the Mac?

So Apple sees the iPad as the future. But while it might be a "pure" experience, there's obviously a large number of important things that the iPad simply cannot do¹. So Macs aren't going anywhere, but Jobs wants to make them more iPad-like. In the Mac event, Jobs called out what this meant: curated App Store, full screen apps, app "Launchpad", autosave/resume. What is equally interesting is what this implies they are leaving behind. During Craig Federighi's demo of Lion on the 20th, he showed the use of swipe to switch between a) full screen apps b) dashboard c) "the desktop". Perhaps I'm reading too much into this, but what today constitutes "the Mac Desktop experience" seems, in Lion, to be just another full-screen app you can switch to. A Rosetta-like box to run "the legacy stuff". I think Lion could be Apple's first step in really redesigning the Mac from first principles.

This iPad-like approach also reinforces other trends already underway. The traditional document-centric Mac app is basically dead. Most apps will be one-window, "content-centric", and abstract the filesystem completely. And they'll be made with Apple's tools. In other words, iPhoto '11 is the model. It's going to be fascinating to see how far Apple can take this. There have been people saying, literally for decades, that the fundamental desktop concept of movable, resizable, overlapping windows was the worst idea in the history of user interfaces. Will they be proven right? I have no idea. The best part is, as others have pointed out, Windows cannot easily copy what Mac OS X is doing. Greater distinction between the keyboard/pointer OS platforms will be a great thing, in my opinion.

Where does Java fit in all this?

It doesn't. Regardless of whether you think Apple's plans for the Mac are great or terrible, it's clear that heavy-weight "cross-platform platforms" like Java and Flash, which are controlled by others, could hardly be more useless to Apple. They don't interfere with Apple's plans, but they don't help them either. Java and Flash apps don't use compelling OS X technologies like Core Animation. They're completely clueless when it comes to multitouch. Apple only cares about one "cross-platform platform": the light-weight and mobile-friendly HTML5, which they can also influence.

Will Oracle Provide Java on Mac?

I'm not sure, but I think yes. The answer to this question will also answer many questions about what Oracle really has planned for Java and JavaFX. In my opinion, JavaFX has already lost the cross-platform war to HTML5 (I like JavaFX a lot, and would love Oracle to prove me wrong here). Oracle claims to be committed to developing and promoting JavaFX, so it apparently believes it can turn things around. But there's one thing I'm certain about: JavaFX cannot succeed as a credible platform without first-class support on the Mac. If Oracle is actually serious about JavaFX (which is client-side Java now) then it needs to very soon announce that it will be adding Mac as a fully supported platform, by Java 8 at the latest. Bringing Mac support online in a Java 7 update would be better. If Oracle lets Java on the Mac rot, then the death of client-side Java will be certain and final.

Where did it all go wrong for Java?

A few people have pointed out that just 10 short years ago Steve Jobs was proudly proclaiming that the Mac would be the premier platform for Java, and now after years of disinterest Apple is finally giving up on Java altogether. How times have changed. In 2000, Apple desperately needed apps on the Mac - any apps, and of course developers. Now, it's Sun/Oracle that really needs Java on the Mac, and Apple can afford to be indifferent.

Even if Oracle produces a brilliant implementation of JavaFX for the Mac, things do not look great for Java on the client. I won't make this post even longer by discussing the reasons for client Java's failure, but it's not because Apple didn't give Java a fair chance. Speaking as someone who really believed in and supported the "write once, run anywhere" ideal of Java, the fact has to be faced: the dream is well and truly finished. Where Java has failed, HTML5 has succeeded, primarily by keeping its goals modest and being ruthlessly pragmatic.

¹ But I think that we haven't even scratched the surface of what the iPad is capable of. Many of its limitations are "a simple matter of software". I think iOS 4.2 is going to give a big boost to the iPad's utility, as will iOS 5 and beyond. Having said that, there'll always be some jobs that just need a mouse pointer and a keyboard.

Scala's Missing Monad

I've been writing more and more pure functional code in both Scala and Java recently. An issue I found myself running into quite often is this: say you have a function that sometimes returns no results (e.g. looking up a key in a map). A common way to deal with that in Scala is to return an Option:

Simple enough. But then a typical use of such a function is to map it over all members of a collection:

What I find I often want to do in a situation like this is to only bother with the results if all the applications of foo returned a result. In other words, I want to turn the strings list above into an Option[List[String]], which is Some(List(foo(1).get, foo(2).get, foo(3).get)) if all those applications of foo give values, and None otherwise. I couldn't see any method in the Scala standard library that does what I want.

As a bit of a functional programming n00b, one approach I've found to be useful is to work out the Haskell type signature of the function I want, feed that into Hoogle and see what I get, if anything. In this case, the Haskell type is:

The top Hoogle result for that is the sequence function:

And look, the sequence function works on any monad, not just Maybe/Option. For example, if my foo function had been returning Either[SomeErrorType, String], then the sequence function would give the first error in the list if there was an error, or the list of results if there was no error. So this is a useful function, but as I said, I can't find it in the standard Scala library.

When I thought about how to implement sequence in Scala, I immediately ran into trouble. It is often said that "Scala has monads", and we know that the flatMap method is doing a monadic bind. But this support isn't much more than a convention for method names and type signatures, combined with nice syntax in the form of for expressions. There is no monad type declared anywhere in the standard library. Without a monad type, the sequence function would have to be re-implemented over and over: Option.sequence, Either.sequence, List.sequence, etc.

The good news is that Scala's type system can express the monad type. And the better news is that some really smart people have already done all the work for us in a library called Scalaz. So I had a look, and lo and behold there is the sequence function:

That may look nothing like the Haskell function, but that is mostly a result of how typeclasses are expressed in Scala. Trust me, this is the same function. Also, this Scalaz function is more general than the Haskell function above. It turns out that sequence works for any "traversable" thing (not just lists) containing anything that is an applicative functor (not just monads). Haskell also has the equivalent fully generalised sequence function in the Data.Traversable library.

So what did I learn from this exercise? First that the lack of explicit types for monad, functor and friends in Scala's standard library is a greater problem than I'd expected. Second, if you're writing pure functional Scala code, you should be using Scalaz. Since all Scala programmers should be writing pure functional code, it follows that all Scala programmers should be using Scalaz!