Navigating in Scala Land

In a previous post titled Toward Polyglot Programming on the Java Virtual Machine (JVM), I described my preliminary  exploration of the other languages and frameworks on the JVM including Groovy, Gradle, Grails, Scala, Akka, Clojure, and the Play Framework. I made the switch from Maven to Gradle, a Groovy-based build language that combines the best of Ant and Maven. I was seduced by the scaffolding capabilities of Grails, but decided to make the jump to Scala and functional programming. So I have been navigating in Scala land recently. In this post, I described my journey.

I am still learning, but I can tell you that I never had so much fun learning a new programming language. It probably has something to do with the use of pure mathematical functions in Scala. I did spend a year studying pure mathematics at the University of Abomey-Calavi after graduating from high school. My first exposure to functional programming was with XSLT and XQuery and I very much enjoyed programming without side effects when using those languages. XSLT 3.0 is a fully-fledged functional programming language with support for functions as first class values and high-order functions. XQuery 3.0 is a typed functional language for processing and querying XML data.


Scala is a complex and ambitious language as it supports both object-oriented and functional programming. When I started to learn Scala, I took the wrong directions several times and had to make several U-turns. So the following steps have been effective for me:

• The Coursera class Functional Programming Principles in Scala taught by Martin Odersky (the designer of Scala) is a good place to start. It explains the motivations behind Scala and emphasizes its mathematical and functional nature without trying to map pre-existing knowledge (of Java or Python, or any other language) to Scala. This is a refreshing approach because Scala is a different language although it can interoperate with Java. A good companion to this course is the book Programming in Scala: A Comprehensive Step-by-Step Guide, 2nd Edition by Martin Odersky, Lex Spoon and Bill Venners.


• If you're a Java programmer moving to Scala, then the book Scala for the Impatient by Cay S. Horstmann would be a good reference. 


• If you're interested in building a web application in Scala, then I would recommend the book Play for Scala by Peter Hilton, Erik Bakker and Francisco Canedo. Scala and Play come with their own ecosystem of tools. This includes a Scala-based build system called sbt (simple build tool), testing tools (like Spec2 and ScalaTest), IDEs (Eclipse-based Scala IDE and IntelliJ), database drivers (like ReactiveMongo and Slick), and authentication/authorization (SecureSocial and Deadbolt). Play has first-class support for JSON and REST, supports asynchronous responses (based on the concepts of "Future" and "Promise"), reactive programming with Akka, caching, iteratees (for processing large streams of data), and real-time push-based technologies like WebSockets and Server-Sent Events.


• A this point, if you decide to dive into the deep waters of Scala, you might want to consider learning Reactive Programming and purely functional data structures. 


• There is a second Scala course at Coursera titled Principles of Reactive Programming taught by Martin Odersky, Erik Meijer, and Roland Kuhn. The Reactive Manifesto makes the case for a new breed of applications called "Reactive Applications". According to the manifesto, the Reactive Application architecture allows developers to build "systems that are event-driven, scalable, resilient, and responsive."  In Scala land, there are currently two leading frameworks that support Reactive Programming: Akka and RxJava. The latter is a library for composing asynchronous and event-based programs using observable sequences. RxJava is a Java port (with a Scala adaptor) of the original Rx (Reactive Extensions) for .NET created by Erik Meijer. Based on the Actor Model, Akka is a framework for building highly concurrent, asynchronous, distributed, and fault tolerant event-driven applications on the JVM (it supports both Java and Scala).


• For purely functional data structures, there is a Scala-based library called Scalaz. The book Functional Programming in Scala by Paul Chiusano and Rúnar Bjarnason is a good resource for exploring Scalaz.


• Readers of this blog probably know that I am a proponent of Domain Driven Design (DDD) in building complex software systems. So I have been investigated how DDD principles can be implemented with a functional and reactive approach. Vaughn Vernon recently presented a podcast on Reactive DDD with Scala and Akka. In a post titled Functional Patterns in Domain Modeling - Anemic Models and Compositional Domain Behaviors, Debasish Ghosh provides an interesting perspective on the subject of anemic domain models in DDD done within a functional programming language as opposed to an object-oriented one.


• For me, Big Data is real, not just a buzzword. I believe in analyzing humongous amounts of data to find hidden patterns and obtain insight for solving complex problems. Dean Wampler called copious data, the killer app for functional programming. Scalding by Twitter can be used for writing MapReduce jobs in Scala. Apache Spark which is written in Scala can run Machine Learning programs up to 100x faster than Hadoop MapReduce in memory. A talk titled Why Spark is the Next Top Compute Model by Dean Wampler explains why Spark has emerged as the most likely replacement for MapReduce in Hadoop applications.

Toward Polyglot Programming on the JVM

In my previous post titled Treating Javascript as a first class language, I wrote about how the Java Virtual Machine (JVM) is evolving with new languages and frameworks like Groovy, Grails, Scala, Akka, and the Play Framework. In this post, I report on my experience in learning and evaluating these emerging technologies and their roles in the Java ecosystem.

A KangaRoo on the JVM

On a previous project, I used Spring Roo to jumpstart the software development process. Spring Roo was created by Ben Alex, an Australian engineer who is also the creator of Spring Security. Spring Roo was a big productivity boost and generated a significant amount of code and configuration based on the specification of the domain model. Spring Roo automatically generated the following:

• The domain entities with support for JPA annotations.
• Repository and service layers. In addition to JPA, Spring Roo also supports NoSQL persistence for MongoDB based on the Spring Data repository abstraction.
• A web layer with Spring MVC controllers and JSP views with support for Tiles-based layout, theming, and localization. The JSP views were subsequently replaced with a combination of Thymeleaf (a next generation server-side HTML5 template engine) and Twitter Boostrap to support a Responsive Web Design (RWD) approach. Roo also supports GWT and JSF.
• REST and JSON remoting for all domain types.
• Basic configuration for Spring Security, Spring Web Flow, Spring Integration, JMS, Email, and Apache Solr.
• Entity mocking, automatic generation of test data ("Data on Demand"),  in-container integration testing, and end-to-end Selenium integration tests.
• A Maven build file for the project and full integration with Spring STS.
• Deployment to Cloud Foundry.

Roo also supports other features such as database reverse engineering and Ajax . Another benefit of using Roo is that it helped enforce Spring best practices and other architectural concerns such as proper application layering.

For my future projects, I am looking forward to taking developer's productivity and innovation to the next level. There are several criteria in my mind:

• Being able to do more with less. This means being able to write code that is concise, expressive, requires less configuration and boilerplate coding, and is easier to understand and maintain (particularly for difficult concerns like concurrency which is a key factor in scalability).
• Interoperability with the Java language and being able to run on the JVM, so that I can take advantage of the larger and rich Java ecosystem of tools and frameworks.
• Lastly, my interest in responsive, massively scalable, and fault-tolerant systems has picked up recently.

Getting Groovy

Maven has been a very powerful build system for several projects that I have worked on. My goal now is to support continuous delivery pipelines as a pattern for achieving high quality software. Large open source projects like Hibernate, Spring, and Android have already moved to Gradle. Gradle builds are written in a Groovy DSL and are more concise than Maven POM files which are based on a more verbose XML syntax. Gradle supports Java, Groovy, and Scala out-of-the box. It also has other benefits like incremental builds, multi-project builds, and plugins for other essential development tools like Eclipse, Jenkins, SonarQube, Ivy, and Artifactory.

Grails is a full-stack framework based on Groovy, leveraging its concise syntax (which includes Closures), dynamic language programming, metaprogramming, and DSL support. The core principle of Grails is "convention over configuration". Grails also integrates well with existing and popular Java projects like Spring Security, Hibernate, and Sitemesh. Roo generates code at development time and makes use of AOP. Grails on the other hand generates code at run-time, allowing the developer to do more with less code. The scaffolding mechanism is very similar in Roo and Grails.

Grails has its own view technology called Groovy Server Pages (GSP) and its own ORM implementation called Grails Object Relational Mapping (GORM) which uses Hibernate under the hood. There is also decent support for REST/JSON and URL routing to controller actions. This makes it easy to use Grails together with Javascript MVC frameworks like AngularJS in creating more responsive user experiences based on the Single Page Application (SPA) architectural pattern.

There are many factors that can influence the decision to use Roo vs. Grails (e.g., the learning curve associated with Groovy and Grails for a traditional Java team). There is also a new high-productivity framework called Spring Boot that is emerging as part of the soon to be released Spring Framework 4.0.

Becoming Reactive

I am also interested in massively scalable and fault-tolerant systems. This is no longer a requirement solely for big internet players like Google, Twitter, Yahoo, and LinkedIn that need to scale to millions of users. These requirements (including response time and up time) are also essential in mission-critical applications such as healthcare.

The recently published "Reactive Manifesto" makes the case for a new breed of applications called "Reactive Applications". According to the manifesto, the Reactive Application architecture allows developers to build "systems that are event-driven, scalable, resilient, and responsive." That is the premise of the other two prominent languages on the JVM: Scala and Clojure. They are based on a different programming paradigm (than traditional OOP) called Functional Programming that is becoming very popular in the multi-core era.

Twitter uses Scala and has open-sourced some of their internal Scala resources like "Effective Scala" and "Scala School". One interesting framework based on Scala is Akka, a concurrency framework built on the Actor Model.

The Play Framework 2 is a full-stack web application framework based on Scala which is currently used by LinkedIn (which has over 225 millions registered users worldwide). In addition to its elegant design, Play's unique benefits include:

• An embedded Java NIO (New I/O) non-blocking server based on JBoss Netty, providing the ability to call collaborating services asynchronously without relying on thread pools to handle I/O. This new breed of servers is called "Evented Servers" (NodeJS is another implementation) as opposed to the old "Threaded Servers". Older frameworks like Spring MVC use a threaded and synchronous approach which is more difficult to scale.
• The ability to make changes to the source code and just refresh the browser page to see the changes (this is called hot reload).
• Type-safe Scala templates (errors are displayed in the browser during development).
• Integrated support for Akka which provides (among other benefits) fault-tolerance, the ability to quickly recover from failure.
• Asynchronous responses (based on the concepts of "Future" and "Promise" also found in AngularJS), caching, iteratees (for processing large streams of data), and support for real-time push-based technologies like WebSockets and Server-Sent Events.

The biggest challenge in moving to Scala is that the move to Functional Programming can be a significant learning curve for developers with a traditional OOP background in Java. Functional Programming is not new. Languages like Lisp and Haskell are functional programming languages. More recently, XML processing languages like XSLT and XQuery have adopted functional programming ideas.

Bringing Clojure to the JVM

Clojure is a dialect of LISP and a dynamically-type functional programming language which compiles to JVM bytecode. Clojure supports multithreaded programming and immutable data structures. One interesting application of Clojure is Incanter, a statistical computing and data visualization environment enabling big data analysis on the JVM.