Notes from my first real Dart Web App

I mostly build interfaces for IE. Well they have to be compatible for IE7+ so like most web developers I build them on Chrome and then fix the IE issues. Since Dart or dart2js only works with IE9+ I didn't have any projects that I could use for it. Finally I have a contact database I needed to build that I only needed to support HTML5 browsers! I figured I would give Dart a real shake down. I didn't want to build everything from scratch which I was luckily that DWT (Dart Web Toolkit) existed.

Dart

The language type system is a little Java ishy but that makes it really trivial to learn if you know Java. I find this mainly annoying with Generics but it is just one of those things that is here to stay I suppose. It is dynamic enough to do things like for(var x in [foo, bar, bat]) {x.show();}. After writing Python for almost a decade I have a hard time living without a level of flexibility. I like the Javascript bits since it make for first pass coding really fast and then it can be statically typed on future passes. I love being able to organize the code in files that make sense and not worry about some complex build process since dart2js will optimize it for me (dependency management is so much better done by machines).

DWT

I spent a couple years writing GWT code and it has its ups and downs of which I have written about extensively. I have also written a good mountain of Javascript and more recently Coffeescript. DWT give that rich API from GWT, I have found the static typing makes it easy to read libraries and consistant coding. Fast development combined with a great widget library and code that is not a complete mess to look at. Honestly I have been amazed at how easy it is to write DWT. I have found it faster to write DWT that it does GWT mainly because Dart is not the pig Java is. It has been relatively easy to style with the existing application using bootstrap.

var's, typedef's and dynamic's oh my!

Oh how I love var. Even though I write static typed code all the time I love being able to just type var when I am in the zone. Save from having to look up a type or to create it and loose the flow. Many times I am not sure exactly what the type is going to be so better to just have a place holder and then during a code cleanup pass fix the var types.
dynamic is the type that json.parse returns. I have found this extremely useful for two purposes. I like being able to configure using it like I do with the DataTable in dwt_lhj. JSON from the backend is another thing I often don't think needs a formally design data structure.
typedef is how to define a function. Say I have a function that take to int and I need to call it once the server responds. For example the Paginator that I wrote in the dwt_lhj tables code:
typedef SelectedPage(int page, int start);
When the user selects a different page then the paginator will call this function. Compile time failure if it doesn't work woot!

dwt_lhj

This are a handful of things I wrote that seem to be missing. I am sure eventually there will be better implementations however I figured I would share them in case someone else had simular needs.
UL and LI elements in HTML tags are missing in GWT as well in DWT. Since I use them a lot for navigation (bootstrap) and for making list of things! I created pretty simple wrappers. It is very self explanatory and easy to use with very little code.
Displaying tabular data is another thing that is currently not implemented in DWT. GWT calls them cell widgets. These are pretty large undertaking from a code perspective to implement for DWT. I have used Cell Widgets a lot in GWT. I find them very tedious to code. I have been using Datatables.net which is pretty nice Javascript library but I have found the API to be pretty complex for the functionality I am looking for (maybe that is just the docs). I was hoping to build something in the middle that I could quickly build but would have a lot of future flexibility that I could change the behavior in. So I built one. I also built a simple paginator that I use with the bootstrap paginator css classes. I am hot real happy with the code but it works and I look forward to improving it in the future.
Wrote a regex routing library because I couldn't get the existing https://github.com/dart-lang/route library to work. Based on the tickets they seem to exclude the features I needed. I decided to write a simple one that uses the DWT History package however I think it could quickly be improved and migrated to be a more general purpose library. When I get some better docs and examples up and someone finds it useful I will publish it in the pub.dartlang.org repository.

Mopping up

So far development in dart has been pretty fantastic. There are a couple bumps in the road with bugs in DWT or old examples to dart code online. I think in a year most of this will get sorted out. There seems to be some amount of effort going into coding Dart for the backend. In general I don't find much advantage to this. I find languages like Python and Go to be much better language for solving those types of problems. As a "unicorn" (I code both front end and backend) I find certain languages have strengths that appeal to the types of front end or backend problems (Dart craptastic support for concurrency IMO is one reason I wouldn't want to use it for backend coding).   For building large code base of widgets Dart seems to be fitting the bill nicely. This app will hopefully go into production in June and I will follow up with another post then.

My first live Dart + Go web app (lhj.me)

Last year I wrote a url shortener using Go and Riak. Trying to eat my own dog food since I wrote a REST API for Riak in Go. It used jQuery and just so I could get this up and running as quickly as possible. I put this up on lhj.me. Partly to test my libraries goriak and httphacks that I wrote in Go. It was a fun project! I was amazed how much I used goroutines in such a small app.I used this app just for personal use and experimentation. Basically just a dev instance and it crashed and died a week ago or so. So by by database and all.
This motivated me to bring her back up again instead of ec2 I thought I would try rackspace. Replace the javascript with Dart! Well technically javascript generated by Dart. I have been writing a lot of Coffeescript lately which I don't mind but I really want to find something with DEPENDENCY MANAGEMENT! So Dart it was to see what it is like to code in.
First I made some Golang code updates to make hosting the static content a bit easier in the future. Then it was writing Dart. I tried the Chromium that is distributed with Dart to make the coding faster. I wasn't able to get it working quickly. Since it was a simple ajax request I just compiled Dart to Javascript from the command line using Dart2js tool. The Dart code: http://lhj.me/dart/ushort.dart 
I have already written about some of my complaints about Dart for a intro presentation I did last summer. The presentation and code are here. The presentation was more of an intro to the language as a general purpose language. Major complaints I had was about concurrency and some confusing OO features but mostly I found it an improvement on Javascript. Since I complain about... well ... everything it was a pretty short list of complaints.
A language that requires or is a pain to use without an IDE like Java I tend to shy away from. I mean it is hard enough to write code but to be fighting complex builds systems or tools that make coding extremely slow makes me want to run away. Vim (my editor of choice) has a bundle for syntax highlights for Dart which I installed. The indention is not so good but I assume it will get better as Dart becomes more popular. If I build anything more complex than a page of code I plan to use the IDE. The most surprising thing to me was when I went to compile it with type checking it failed to import the html package! ERROR: "Do not know how to load 'dart:html'". I guess this makes sense but I was pretty shocked thinking the types I was using would get checked. The type checks are a bit sad (dart --enable_type_checks) since it is a long command line option you pass to the dart vm. Takes digging for in the documentation to find (it will not show up if you type dart --help).
Since I know both Java and Javascript the syntax seems very natural. The API could use some help. I found the HttpRequest documentation much to be desired. It took me a lot of trial and error to figure out how to do a POST with JSON. They added some useful things like a HttpRequest.get static method. I was surprised to find out there was no HttpRequest.post static method. Giving an intro screencast on how to build a Dart web app in 10 minutes 'get' save a lot of code however for the web apps I build I use POST as much if not more than GET.
From languages like Python, Ruby, Go or Coffeescript there seems to be a lot of typing. Compared to Java on the flip side it is a very terse language. About the same as Javascript maybe a small amount more if I want to compile with types.

Coffeescript + requirejs keeping web app development bearable

I have been building web applications since the turn of the century (and I have the grey hair in my beard to prove it). Back in my college days M$ was basically the only UI. Legacy Unix apps had X support and Linux seemed like it might be give no Windows GUI's a comeback until 2000 tech crash right when I graduated. Luckily my network programming and database programming allowed me to write web applications aka the only other interface game other than Windows. At the time Javascript was a dirty word. It was a gimmicky Netscape (pre Mozilla pre Firefox) feature that didn't work on IE (90% of the market). IE6 came out and Firefox which had good Javascript support but maintaining 2 code bases for each browser to handle the inconsistencies seemed expensive. More importantly I developed on Linux and IE was 90% of the platform so I would have to develop on gasp XP?
Sea change game when Gmail was released combined with some snazzy demos with sprinkling of Javascript and AJAX! Autocomplete was what converted me over. Autocomplete user experience was worth the extra work. I was dipping my toe in for a while from 2004 to 2008 when I realized the change in Javascript AJAX request was going to also mean a change in backend server. I blogged back in 2008 about this: "The new web seems to be lot of little request which means more small requests and it is going to require a lot more static resources." from then on I would start spending 50% of my development in Javascript GUI land.
I basically went though three phases before settling on what I am coding in now.

Phase 1

Generated Javascript by the backend was my first approach. All the apps I was building in the early days where based on web pages with some limited interactivity. Wanting to minimize browser computability by building backend widgets that would generate the HTML and Javascript so once a widget was made compatible with all the browsers it could be added to any web page with some level of confidence. This worked great for legacy apps that could be improved greatly with some simple show/hide or single ajax widget like autcomplete. This fell apart as the Javascript became the majority of the code being written.

Phase 2

The second generation was using jquery, YUI and other Javascript frameworks. Writing mountains of Javascript (10-100K) for a page. Selenium test made browser compatibility possible. The first advantage was being able to decouple the Javascript code base from the backend (Python/Java). Quickly found that Javascript lack of basic tooling like dependency management, packaging and performance optimization really stood out. Reinventing the wheels was tedious and ineffective.

Phase 3

GWT solved all of these problems! The biggest problem was learning Java which I had been coding since the 90's in, so no big deal right. Wrong! Swapped out one build mess for another. A lot of developer seem to love Maven however I found it as time consuming if not more so than Makefiles or ant. Development was slow! Java may be fast at number crunching but it has always been a resource hog. 8Gigs of ram dual core 64bit solid state drive. Sit back and wait for you changes to get rebuilt before you can view them. Another trade of writing software is slow but managing it and running it is quick.

Current Phase

Enter Requirejs and Coffeescript! All tool choices are compromises. Making code changes and seeing those changes immediately makes me more productive, keeps the momentum (critical in any project) and is more fun (look forward to return back to coding after a day off)! Coffeescript allows for immediate generation of Javascript so I don't have to wait for a compile step. It provides a sane for loops and other syntactical sugar greatly reducing lines of code. Requirejs make dependency management in Javascript bearable. As a backend developer I find manual dependency management feels like the dark ages (bring out your dead) of coding. Especially after the automagic of dependency management of a language like Go. I still waste a lot of time thinking about what file I should use to make sure the page loads faster and trying to manually consolidate code. There is a way to generate single Javascript files however after GWT all these techniques are pretty lame. Now that I embed my html templates in the Coffeescript source files and I render things in a way that lazy loads I am not loosing to much performance (famos last words about unmeasured performance).

Future Phase?

Dart seems to be a pretty good future option that solves some of the problems. Still has manual dependency management but it won't compile if that is wrong. Since I need to support 3-4 versions of IE back I am hoping I can fully test it out once IE12 comes out (not holding my breath on that release date). Until then I hope they fix the dependency management stuff using some type of url import statement. Until then I am just thankful for the great work of the Requirejs and Coffeescript teams.

Presentation / Discussion of Dart

This blog post is about two months late since I gave the presentation / hosted a discussion about Dart over two months ago. This is the first talk I did which was all code. I figured it was an intro to a language so everything should be in that language. I organized it via a github repo: https://github.com/lateefj/dart-intro I had all the slides opened in a vim tab and just when though each one demoing the code and discussing it.

There where three types of developers at the meeting (technically it was a Python/Django group but other came). These where some observations I had from the different groups.
Python/Django - They generally write backend software that uses mostly static web pages. Or generators like Rails, Django admin stuff. By and large they enjoy backend programming (Python / Ruby) and find Javascript for some snazzy UI stuff annoying to write in. Dart is not very appealing to them because it is a more taxing syntax "Javaish".
Java/GWT - By and large they see that Google is moving its mindshare to Dart from GWT and came wondering what all the hoopla is about. As a long time Java developer it is frustrating that the Java community has turned into a monolanguage culture. Since there is a library somewhere to do just about anything in Java it make most Java programmers look at everything with blinder. Dart looks like Java, smells like Java and specifically for GWT folks it generate Javascript so what is the point?
Javascript - The latest rage (or has it already died out). Anyway Javascript developers don't really see why an alternative is necessary.

In general I think everyone in the discussion found it interesting even if most couldn't figure out why? My motivations was to learn mostly by discussing it with other developers. Bottom line for the types of large web applications that I have been developing I have tried man approaches (GWT, Coffeescript ect) but it all comes down to Javascript was design and built in the 90's and it is very difficult to over come its limitations. Will leave my thoughts about Dart for the next post....

Part 2 Dart vs Go vs Python (and PyPy) Performance

UPDATE: In the comments Will pointed out I had a bug in the Python code so updated 22:00 EST sorry.
UPDATE 2: Anonymous posted a more comparable version of the Go (http://pastie.org/3772555) so I updated the numbers 04-12-2012 09:00 EST (sorry). Please let me know if there are any other bug fixes!

The previous post I wrote Surprising performance from Dart inspired me to do some more test runs with Dart, Go, Python and PyPy. For round two of some simple performance comparisons between  Dart, Go, Python and PyPy I decided to do some basic file reading and processing. Since the vast majority of data that I process is JSON I decided to generate some random JSON data and store it into a file. Using each language read the entire file and for each line parse the JSON and aggregate it. Assuming I wrote the code right (big assumption) then I can get an idea which language is best for processing large sums of JSON.

The code basically is broken down into a module (package) that contains an Aggregator class (type) and the main file which reads from the data file. The hosted repository is on bitbucket.
Dart code:
I was able to code this using async style programming which in theory would maximize for IO performance if Dart supported async IO. I was a bit shaky at first with the terse syntax of the onLine callback but I am getting a bit ok with it.  The duplicate use of StringInputStream seems a bit wordy, probably could have just said "var stream" instead of "StringInputStream stream" to increase readability (Go got this right with the := syntax). Otherwise the code is very straight forward.

Since Dart is design for the browser and it is a preview the IO would be very slow but I expected it to be able to process JSON very fast since that is a large part of what a modern web application does. This also means that the StringInputStream should be pretty fast since it will be the main source of data for the application. Go would probably outperform Python and not really sure about Python vs PyPy. I figured PyPy would use more memory but be faster for doing aggregation.

Runtime comparison:

This is only 10,000 JSON documents in a single file. I have no idea how it does it but Python just kills it. Not that surprising that it is 18x faster than Dart but 2x (roughly) faster than Go is pretty amazing I knew the Python 2.x IO was fast but I didn't realize it was this fast. I think there could be a faster way to process the JSON in Go it just didn't jump out at me to do it without using the Unamarshal.

Memory comparison:

I expected Go to have used the least memory but didn't expect it to be so tiny (2,076, after update Python is 4,692). Dart was unusable but that is expected in a preview release. PyPy is not much better I guess I need to send them a note and ask them what they think. Python again rock star here.

I have chosen to kick the tires at a pretty high level but I think it is a realistic situation since web apps these days are processing a lot of JSON. Dart faired as expected for a preview release. Python rock star status as usual. PyPy we will see what happens when I contact them about it. Finally Go was very disappointing how much runtime it took to process the JSON, however it did recover some dignity by it exceptional use of memory.

Surprising performance from Dart

About every two years I try to learn a new programming language. I find the new language opens up a new set of applications and I can use it to improve existing applications. Dart this year mainly because I have been writing a lot of GWT (Java -> Javascript) and Javascript over the last couple years. They both have a lot going for them and against them. From what I have read about Dart it takes the best parts from both.
 GWT Complaints:

• Costly project setup and maintenance
• Java development is slow
• Language and library incompatibilities

 Trying to setup and maintain a GWT application is a project in itself. From what I have used of Maven it seems to save little time if any. Then there are a lot of choices of design vs performance and cross browser support that are pretty loose / loose.
Java development is slow. Yep I said it and many say it isn't but it is. The JVM is extremely fast at doing a lot of things unfortunately it seems all development type things like IDE (GUI), compilation and building use excessive amounts of CPU and memory in the JVM. It is jaw dropping to see a hello world REST service take up the better part of 100MB (jar dependencies mostly from maven). Reloading a page using GWT (any servlet for that matter) requires a recompile step that is slow and bottom line a momentum killer.
GWT has support for calling into other Javascript libraries but it is not trivial. A lot of productivity is lost not being able to add a jquery plugin to a page in 10 seconds.

Where GWT kills it is deployment. The end product (assuming you can get it built) is a fully optimized cross platform wonderfulness that would take massive amounts of time to get equivalent Javascript app built.

Javascript Complaints:

• Hard to manage large code bases
• Browser incompatibilities

Trying to manage a large Javascript code base extremely painful. After a couple files it gets messy and slow. Keeping related code in separate files is really nice however it has an impact that those files have to get loaded into the browser unless you are using some build system. Then there is a lot of decorating of pages that only need X amount of code vs another page which needs a different set. There are ways but namespace seems like something that should be built into the language like almost every other programming language.

Debugging IE is what my generation of developers will be telling war stories over someday I suppose. However I feel like it is time to move on. Over a decade of doing it is time to innovate beyond these kind of shenanigans.  

The great part about Javascript is it is extremely fast to develop. I don't even have to download the framework I can just add a script tag to the page and reload the page. This is really great for getting products built quickly or adding features quickly. 

Dart:

Static type safe compilation as an option for those really large code bases (libraries anyone). However also the ability to just reload the browser for fast development. New language but with very simular syntax as Javascript that compiles to Javascript to support legacy browsers? Maybe the best of both worlds plus a bit more ontop?

With most programming languages I play with I start with Fibonacci number sequence as the first program. I am a fan of Fibonacci and it is a very simple algorithm to get the basics of a language. So lets take a look at the code:

Dart fib code:

fib(int n) {
  if (n == 1) {
    return 1;
  } else if (n == 0) {
    return 0;
  } else {
    return fib(n-1) + fib(n-2);
  }
}

  main() {
    var length = 35;
    for (int i = 0; i < length; i++) {
      print(fib(i));
    }
  }

I decided to benchmark compare Dart with Python, PyPy and Go run (go run fib.go)  which compiles and runs the the program and then compiled it to see it fully optimized.

Then I compared memory utilization:

Fat disclaimer about the benchmark is that this is basically a number crunching thing that I ran a couple times to take these numbers for what they are (primitive). PyPy being way faster than Python is not a surprise at all. Lets take Python out because it is is skewing the chart.

The first really big surprise was the Dart was faster than PyPy. Another big surprise was how much memory PyPy was using like 5X (JIT maybe that is why Java is such a memory hog). Dart had amazing performance and memory utilization for a preview release of a language. The compiled version of Go had amazingly low memory utilization which lines up with my experience running a web app written in Go.  This is very primitive experiment so I think I will try something next time with a little more complexity to see if CPU and memory hold up with a little bit of pounding. This is exciting news because I think Dart can be not only a front end web language but also a backend language. Gives me ideas!

Update: Part 2

Added Secondary Index support to goriak

Have been pretty heads down on writing business software and was getting a little burnt. So to revive I hacked a little on the driver I have been playing with for Go (golang) to use Riak. There are basically 3 motivations other than I need to do a little fun hacking to recharge the batteries.

1. Go weekly is getting close to version 1 so they added packaging using the "go command" which I cover in a previous blog.
2. Riak add secondary index which is so much more developer friendly than before.
3. The types of problems I really enjoy solving are ones that Go and Riak would be good tools for.

Quick recap is that I had the basic support for key/value store with a hint to the type to structure to marshal it would do the conversion of struct to json and back. Get, Put and Delete basically was all that was implemented at the time. 

Secondary Indexes provide a binary (string) and integer types for indexing. I implemented exact match and range queries. So method signatures:

SearchBin(bucketName, index, value string) ([]string, error) 

SearchBinRange(bucketName, index string, start, end string) ([]string, error) 

SearchInt(bucketName, index string, value int) ([]string, error) 

SearchIntRange(bucketName, index string, start, end int) ([]string, error) 

Now there is a complete example included with the source code:

Time to billed that kill web service! Source code is on bitbucket: https://bitbucket.org/lateefj/goriak/