Saturday, May 27, 2006

Functional Programming Under the Hoods - Developers Call it LINQ

The May CTP of C# 3.0 introduces LINQ amongst some other additional features of the language like implicitly typed local variables, extension methods, collection and object initializers etc. All of these features have been extensively critiqued here, here and here. But undoubtedly the features on which most blogs have burnt are the lambda expressions and Language INtegrated Query (LINQ). Lambda expressions in C# form the core of LINQ and brings us back the memories of functional programming of Lisp genre. With the current generation afflicted with the perils of Java schools, it will definitely require a change in mindset, a fresh relook at the old pages of Paul Graham to appreciate the underpinnings of the design of LINQ in the new release of C# 3.0 and Visual Basic 9. Think collections as monads, query operators as monad primitives, queries as monad comprehensions and what you get in LINQ is today's incarnation of Lisp, Haskell or ML. As Erik Meijer has mentioned correctly,
Functional programming has finally reached the masses, except that it is called Visual Basic instead of Lisp, ML, or Haskell.

Code-as-Data: Unveiling Lambdas in C# 3.0

Like Lisp, the lambdas of C# 3.0 support the code-as-data paradigm. However, in Lisp or Scheme, it is upto the developer to decide whether to use the code or data through the application of quote and quasi quote. C# 3.0 and VB 9 compiler automatically decides on the code or data for the lambda expression depending on the static type of the context in which the lambda occurs.

Expression<Func<Customer,bool>> predicate = c => c.City == "London";
Func<Customer,bool> d = predicate.Compile();

In the first statement, Expression<T> is a distinguished type, which preserves the lambda in the form of expression trees instead of generating traditional IL-based method body. Thus the lambda is data, instead of code - lispers reminisce of the quoted expression to achieve the same effect. In Lisp you use eval to translate the data into some executable - C# 3.0 offers Compile() over the expression tree, as in the second line above, which makes the compiler emit the IL.

The Grand Unification

The primary goal of LINQ is to unify programming against relational data, objects, and XML. Typically in today's applications, developers employ three disparate models - SQL for relational database programming, XQuery, XSLT etc. for XML processing and OOP for business logic. The LINQ framework presents a unified model using the technology of functional programming under the hoods and exploiting the algebraic nature of collections and operations on collections.

The design of LINQ consists of generic operations on collections abstracted as base patterns of query operators and domain specific extensions for XML (XLINQ) and relational database programming (DLINQ). Some of the experts have expressed concerns that LINQ may once again prove to be a bloat in the language with efforts towards ORM implementations. I have a completely different opinion on this. I have worked on enterprise scale Java EE projects with a relational database at the backend. I have personally burnt my head over performance issues in these applications and have come to the hard realization that you need to write performant SQL queries to have good transaction throughput. And typically developers write these SQL queries as untyped string constants within the Java codebase. Efforts towards using popular ORM frameworks like Hibernate simplify things a little bit, but still you cannot avoid having typeless SQL statements buried within your Java code, so long u have the impedance mismatch of the relational database at the backend. This is where LINQ and DLINQ rocks with the typed lambdas - the developer can write typed queries that work seamlessly over collections of objects, relational tables and XML documents. The ultimate aim of Microsoft is to use the LINQ framework to provide a unified approach to programming data and offering DSL support over multiple tuple-based data sources, resulting in a simpler programming model.

Polymorphic Queries in LINQ

Another exciting new feature that LINQ brings is the IQueryable interface, which allows writing true polymorphic queries that can be executed on an immediate or deferred mode in any target environment. The following example is from The Wayward WebLog :

public int CustomersInLondon(IQueryable<CUSTOMER> customers) {
  int count = (from c in customers
        where c.City == "London"
        select c).Count();
  return count;

In case you want to execute the query over a collection of objects ..

List<Customer> customers = ...;

And if the query is executed against a remote DLINQ collection, the underlying engine translates it to the appropriate SQL for the underlying database. Actually what goes on underneath is what the functional community calls comprehension. In case of the above query, the comprehension syntax is translated by the compiler into standard query operators.

All the new features in C# 3.0 and Visual Basic 9 bring back strong memories of Lisp and Haskell and rediscovers the joy of declarative programming. It only took a couple of decades to establish them as part of a mainstream language. It remains to be seen if other languages like Java follow suit.

Saturday, May 20, 2006

The Show is Over!

Day 4 at the Moscone Center began with the Sun raazmataaz where all real time applications of Java were at display. Mobile devices with rich Swing UIs, racing cars with Java inside, Java enabled robots etc. lighted up the Moscone Center for the last time in 2006. All of these high tech stuff was being conducted by James Gosling, the father himself, with a smile of satisfaction, that only goes along with an invention. Of course before the tech extravaganza, we had Scott McNealy giving away the Lance Armstrong tshirt to the winner of ride-your-bike competition in this year's JavaOne. Boy, what a shift in the paradigm - Scott was the CEO a month back when possibly he had prepared his keynote for JavaOne 2006. And now he got a 20 minute slot on the last day of the Conference for giving away tshirts!

There were some very good sessions lined up for the last day. I started with the one on EJB 3.0 delivered by Craig McClanahan, Linda DeMichiel and Gavin King. The session took us through the POJO slogan which EJB 3.0 could ultimately get out of its door. Linda described all the major architectural features of EJB 3.0, Craig took us through the JSF integration part, while Gavin King demonstrated some of the advanced applications of transaction handling using the architecture. EJB 3.0 is a much better incarnation than its predecessor, EJB 2.0, which was almost unusable. But still the capabilities of 3.0 are only a subset of what we have in Spring - experts are very sceptical and some of them has procrastinated EJB 3.0 as inferior and a big step backward from Spring. For more details, go through what Rick Hightower has to say in this JDJ article.

On popular demand, an introductory session of Ajax was repeated on this day - the one by Ben Galbraith. I decided to walk into it - at the end of it, I do not regret my decision. It was a very well delivered session with lots of information (mostly introductory though) on Ajax. The guy showed lots of real life examples of Ajax implementations including google maps, google suggest and some cool programming tricks and debugging how-tos.

On the whole, JavaOne 06 demonstrated to the world the awesome community that has been built around this technology platform. It has truly transcended all barriers of life and touched upon all walks of the social fabric. All said and done, the users want more from the technology which promises to be the ubiquitous platform of computing. A more viable licensing policy for Java, more enhanced JVM with support for functional languages, continuations, tail call, higher order functions etc. etc. .. the list goes on. I am always hopeful of a brighter tomorrow ..

And oh, BTW, the bookstore at the Conference was a smashing hit - I myself picked up five books for my collection at a coool discount of 20% ..

JavaOne Day 3 - The Rampdown Begins

Day 3 began with Erich Gamma taking centrestage with mission Eclipse. In the presentation titled "Java in the Trenches - Reflections Along the Eclipse Way", Gamma and John Wiegand took us through the development cycle of the Eclipse platform, which has delivered quality software with impeccable precision on delivery dates. The visuals which they demonstrated for the Eclipse version of agile methodologies reflected upon the enormous experience and process maturity that the team has imbibed over the years.

Gilad - The Saint

Gilad Bracha has a unique way of presenting things. He does not have the aggression of Josh Bloch, rather a very saintly way of taking the hall through the journey. His session entitled "Dynamic Languages on the Java Platform" had a full house. Though there was nothing very new from the content point of view apart from what he has already blogged on this subject, it was a fairly informative session. Typical responses expected as Q&A like tail call support in the JVM were dealt with in a saintly way (of course in the negative) by Gilad. Later after the talk, he has blogged his disappointment at the crowd not asking for Continuations support in the JVM - he had the response ready though. Support of continuations in the JVM has been discussed in many forums and Gilad has categorically turned all of them down citing reasons like cost of implementation and incompatibility with the current Java SE security model. Hence, I guess, people have given up all hope of finding continuation support in a JVM in the near future. This is what Gilad has to say in his blog on this issue
... continuation based web servers are very nice. Seaside is the nicest one I've seen. But ultimately they are just a phase, and we can already see how we will outgrow that phase. Since continuation-based servers won't be all that significant in the long term, and given the huge lead times and costs of adding continuations to the JVM, it makes little sense to support them.

Web servers aside, continuations, though admittedly difficult to implement, has many other uses. The comments section of Gilad's blog entry has ample testimony towards that.

Ajax Ajax

In day 3, also there were lots of sessions on Ajax, RIA, Dojo and Web 2.0. Not all of them were of high quality, but, while the lights are on, go follow the bandwagon. The new kid on the Ajax and SOA combo, JackBe, had Deepak Alur and Dan Malks (both of J2EE patterns fame) moderate a session on "Java Technology, Ajax, Web 2.0 and SOA", which had Craig McClanahan and Rod Johnson (among others) on the panel. I attended the session, which I felt was quite crappy, since none of Craig or Rod ever found the space to get going. And when Ajax is there in the panel discussion topic and u have a full house, nothing, but chaos can only reign!

In another session titled "Using the DOJO Toolkit to Develop AJAX enabled Java EE Web Applications", the speakers did a fairly good job of taking the (once again) full house through the process of using frameworks in an Ajax application. And DOJO seems like a leader in this space.

Apart from the above, I attended some of the other sessions where the main themes were once again GlassFish, NetBeans and how a combination can give us an easier programming model. Motorola, in their general session once again emphasized how they are transforming the mobile world with Java technology. The evening sessions on Day 3 saw a remarkable fall in attendance, since the sessions had to compete with free beer available in the After Dark Bash - no wonder we had some more crappy stuff scheduled during these hours.

Tomorrow I plan to attend a couple of sessions on EJB 3.0, which has been released after it passed the community process. If u consider the pre-Spring era, EJB 3.0 looks to be a solid progress from its demon predecessor towards a scalable component model. But hey, we have been through Spring and the best practices sermons of Rod Johnson - compared to that EJB 3.0 falls far below expectations. However, that's for another day, folks .. for now it is DG signing off from Moscone Center, San Francisco.

Friday, May 19, 2006

JavaOne Day 2 - Josh Rocks, But It's Ajax All the Way

Every year, JavaOne throws up the theme of the show. This year it's Ajax all the way. Every tech session on Ajax is booked, the most notable among them being the flagship session on "An Introduction to AJAX" by the founders of, Ben Galbraith and Dion Almaer. As has been mentioned in the conference report by John O' Conner
Using simple but effective examples, the two made a compelling argument for using Asynchronous JavaScript with XML (AJAX) to create dynamic, responsive applications that are easy to deploy and that require nothing more than a common web browser.

Besides this flagship from the creators, there were many other sessions which focussed on Ajax, RIA, SWT and the likes which promise to bring heaven on the client browser.

Josh Rocks!

I attended three sessions by Joshua Bloch, one of them the famous Puzzlers Session with Neal Grafter. Their book "Java Puzzlers - Traps, Pitfalls and Corner Cases" has been doing great in this year's JavaOne, but in their tech session the two wizards set the floor on fire with new additions to the puzzlers collection. Great show and I enjoyed every bit of it. This was preceded by a solo from Josh where he discussed some of the advanced Java patterns and idioms, which he promised to load in the next edition of Effective Java. Cool stuff! The icing on the cake was the BOF on "Collection Connection", which happened to be the eleventh edition of the show. Eleven long years - yet the Collections Framework is evolving, a definite sign of a living language (or perhaps Dynamic Language, as Gilad would like to mention it). The BOF was brightened up by the lumious anecdotes which Josh mentioned while recollecting his struggle with Binary Search in his CMU days. Overall the BOF was extremely enlightening with good indications of what to expect from the Collections Framework in Mustang and Dolphin.

Don't Ignore the Scripters

Two very interesting sessions which I attended on the second day of the conference included the ones on Groovy and JRuby. The Groovy session, led by Rod Cope had entertainments written all over. Some of the crazy things which he demonstrated included opening up new excel worksheets, filling up with data, formatting stuff, generating charts and ultimately putting the entire control within a Swing panel with buttons and callbacks - all with 20 lines of Groovy code! This is developer's productivity, at its best.

In the session on JRuby, the creators, Charles Nutter and Thomas Enebo, were there discussing passionately their experience trying to bring up the most powerful scripting language on planet into the Java Virtual Machine.

Can Java Studio Creator Eclipse the IDE World ?

Sun has come out strong with the message to use the Sun Java Studio Creator. Based on its much hyped NetBeans development environment, the creator has lots to offer as a rapid web application visual development tool. The Creator comes bundled with the Application Server, a database named Derby, data binding facilities through drag and drop, Ajax enabled JSF components, portlets as a project type, bunch of goodies for writing Web Services, themes and what not!. It will be really interesting to find out how the Big Blue responds with all the happenings in its Eclipse world. Tomorrow, I will find out what IBM has to say in the General Session to be addressed by none other than Erich Gamma.

In today's general sessions, both Oracle and BEA, led with their efforts towards contributing stuff to the Java space. BEA mentioned about their "blended strategy" of fusing commercial software and open source software towards development of enterpise tools that will make programming an enjoyable developer experience. The director of developer relations at BEA reiterated that BEA has been a key player in the open-source space. The company is a board member of the Eclipse Foundation and sponsored such projects as Apache Beehive, Apache XMLBeans, Eclipse AspectJ, Eclipse WTP, and more, besides being the first application server to provide certified support for the Spring framework.

Overall, day 2 at JavaOne had the all pervasive theme of Rich Internet Experience with Ajax leading all the way. Amidst all the industry frantics and open source hullahs, Gilad Bracha's session on "Superpackages: Development Modules in Dolphin" went almost unnoticed, with half of the hall empty. It had some important points though, which promise to make modular developments much easier in Java. But, at least for today, Ajax rules !

Thursday, May 18, 2006

JavaOne Rumblings - Day 1

More community partcipation, more Open Source, crying for more compatibility - these sum up the clairvoyance of the chief Java evangelists of Sun Microsystems, as the first General Session of JavaOne 2006 unfolded to a live band playing below the multiplex of gigantic screens in Moscone Center of San Francisco. I am here for the first time to attend the pinnacle of all Java summits and celebrate the technology that Jonathan Schwartz says has become the key ingredient of the digital experience for hundreds of millions of people across the planet. Thank you Anshinsoft for sponsoring me for this trip.

I attended one of the Java University courses titled "How Java EE Patterns Help in Architecting and Designing Robust Java EE Applications". The instructor was very energetic and covered lots of material within the stipulated time frame. But at the end of the day, it was all same old stuff regarding Session Facade, Business Delegate and Transfer Objects. I expected something new related to the upcoming EJB 3.0 stuff, patterns for designing generic frameworks using EJB 3.0 interceptor technology, patterns for exploiting Java 5 annotations etc. With those old patterns which Deepak Alur had documented 5 years ago, the course was a run down of yet another UML based development, the old way, when people used to develop tons of models (read diagrams) only to discover that they have become outdated before the programmers could start writing a single line of code. Give me a break! The waterfall model of producing use-cases, refining use-cases, generating class diagrams, generating sequence diagrams etc. always work in demos and tutorials with a handful of classes, but they never scale in real life applications.

Open Source! Open Source!

All projects in has been open sourced and this was repeated in almost all of the sessions by all leading Java evangelists. Jeff Jackson, SVP, Java Enterprise Platforms and Developer Products, emphasized on more community involvement to support all Java EE initiatives. He called for the following Calls to Action for the audience:

  • Join JCP

  • Download NetBeans and join the NetBeans community

  • Join, and join the projects now open sourced, such as Sun Java Studio Creator 2, GlassFish, Mustang, WSIT (whiz-it)

  • Get the free GlassFish Open Source App Server

  • Visit The Aquarium, a Virtual water cooler for WSIT and GlassFish

  • Contribute to the BPEL Engine into Open ESB

  • Use the Beta of Java SE 6

What does the above imply ? As a developer, do I really care whether Java becomes open source from tomorrow - I have access to all source codes of the Java platform, except the JVM. And 99% of all developers worldwide do not care about the inner workings of the Java Virtual Machine. I fully agree with Cedric, that the one thing that Sun needs to figure out is the licensing and redistribution policies of Java. There are companies like BEA, IBM and Oracle, with bigger stake in Java's success waiting ringside to drive forward with developments in the core platform. It cannot be denied that much of the progress that we have seen on the Java front has been due to community participation - the progress on the core side (i.e. JVM and JRE) has been impeded by Sun itself. Compare it with Microsoft's progress with the .NET platform - there has been a flurry of developments in the recent past with all sorts of creative features being incorporated within the core platform.

Showcasing NetBeans

The first day at JavaOne was dominated by the showcasing of NetBeans - at least three sessions demonstrated how a sample Java EE application can be developed using NetBeans 5.5 in less than 15 minutes. Boy, if application development was really this easy! However, on the whole, Sun has been able to make its presence felt in the IDE space with ample demonstrations of the capabilities of NetBeans and its integration with the entire GlassFish suite.

Back to the POJOs

EJB 3.0 is here! In the technical session on EJB 3.0, it was obvious that the creators admitted the cumbersome programming model of earlier versions of EJB and brought out the POJO based architecture of EJB 3.0. The Java Persistence API of EJB 3.0 has been contributed by BEA through its KODO implementation, which will also form the backbone of the next Weblogic Application Server.

On the whole the main theme of Java EE 5 as evident from the first day's session in JavaOne is to come out with a more user-friendly programming model focussing on the ease of development. I think with all collaborations and community building exercises, this will make life easier for programmers at large and architects in general.

Tomorrow, the second day of JavaOne promises great stuff from Joshua Bloch. I will be back with some more news and fresh perspectives in the next edition of this blog. Till then .. Peace.

Monday, May 08, 2006

Don't Generate Code - Generate Model Implementations Instead

There has been lots of confusions regarding my views on automated code generation, since I wrote my last post. I guess it is the following rumbling that spread across the bad word and insinuated me as one of the crusadors of tool based code generation:
I have never been a fan of automatic code generators, which always tend to produce tons of boilerplates, which could otherwise be refactored much more elegantly using OO frameworks.

To some extent, the lack of expressiveness of my intent has been responsible for this - possibly my ardent readers have missed out the last segment of the above statement which qualifies my disliking to only those code generators which produce tons of boilerplate codes violating the ageold principles of abstraction and encapsulation. That was exactly my point in the last post, where the DAO generator generated lots of Java code which could otherwise be abstracted in an OO framework.

In today's world code generators have a significant role to play in the areas of Model Driven Architecture (MDA). Until recent times, UML had acquired the ignominy of being the unfriendly companion to the project manager - to the extent that we had started talking about Death by UML Fever, where designers produced tons of model documents which became obsolete even before the developers could install the proper plugins of Eclipse. UML 2.0 promises support for MDA - the new MDA-based features allow creation of various architectural modeling styles supporting reusable design patterns.

EMF, the Eclipse Modeling Framework offers MDA capabilities with UML as one of the importers of the model. The model which you design using UML can be directly imported into EMF and Java code for the implementation can be generated directly out of it. The model can be as rich with all sorts of relationships like aggregation, composition, inheritance and association - the EMF code generation module takes care of every iota of the richness and generates Java code as the implementation. Apart from UML, EMF also supports XMI documents, XML Schema and annotated Java as source of model artifact. e.g. We can define an annotated Java interface as follows:

 * @model
public interface Book {
   * @model
  String getTitle();

   * @model
  int getPages();

EMF code generator will spit out the following as implementation of the above model, a Java interface along with the corresponding implementation class:

public interface Book extends EObject {
  String getTitle();
  void setTitle(String value);

  int getPages();
  void setPages(int value);

public class BookImpl extends EObjectImpl implements Book {
  protected static final int PAGES_EDEFAULT = 0;
  protected int pages = PAGES_EDEFAULT;

  public int getPages() {
    return pages;

  public void setPages(int newPages) {
    int oldPages = pages;
    pages = newPages;
    if (eNotificationRequired())
      eNotify(new ENotificationImpl(this, Notification.SET, ..., oldPages, pages));


In the above code fragment (adopted from this series in developerWorks), the generator takes care of complicated semantics like enabling the observer pattern for the set method by notifying any observers of the abstraction through the eNotify() method. Also note the optimization guards implemented by the call to eNotificationRequired(), which, in the degenerate case (no observers) amounts to nothing more than an efficient null pointer check, which is inlined when using a JIT compiler.

The code generator component of EMF, called EMF Codegen, also supports customizable code generation through JSP like templates, JET(Java Emitter Templates) and easy regeneration and merging.

How is the code generator of EMF different from the run of the mill ones which generate boilerplates ?

The heart of the EMF is the ECore Metamodel, which has mapping for every UML construct. Hence when the UML model is translated into Java code, we have a lossless translation which preserves all the richness of the original artifact. An excellent illustration of these capabilities has recently been discussed in a TSS article by Patrik Nordwall. Hence EMF Codegen generates model implementations, as opposed to boilerplates.

So, next time when you think about rolling out your code generator, double check if you are producing code that can be better abstracted through an OO framework. And use your framework model and a lightweight DSL to generate model implementations, rather than boilerplates.