Goriak uploaded code to the public

Goriak a Go (golang) library for the Riak database. 

Got enough time to learn enough Go to code up a basic key/value interface to Riak. It is still missing a couple of the REST API calls which I plan to implement over the next couple weeks. I figured it had enough work on it to share with the world and hopefully some one else can review my code and contribute to it. I originally looked to just contribute to another project but the only project I could find was riak.go and it didn't have as complete an implementation as I had. I was able to use a couple lines of riak.go and it really helped me implementing the Put function so I am really glad it is there. Also the author is really nice he is just going off to work on something else so I decided to create a new repository for the project.

The current API has three functions:

GetBucket - Retrieves a Bucket type which contains all kinds of information about the bucket

Get - Marshals the JSON that is returned by the request into a the interface that is passed in. 

GetCAP - Same as get but support an option for the number of nodes that need to agree before returning

Put - Unmashals the type passed in and uploads it to the server

PutCAP - Same as Put but can specify the number of nodes that need to write the json

Delete - Removes the object from the cluster

DeleteCAP - Same as above but can specify number of nodes to delete it from 

I still need to implement MapReduce, LinkWalk, Ping and ServerStatus. These should come over the next couple weeks as I get time to spend on them. MapReduce being the only non trivial one.

I did start to experiment with some concurrent access patterns. The first I call BackgroundWait which can make Get or Put which return a request id. The responses are stuffed in a channel until the code calls Wait(requestId). Instead of waiting for every single request it can do many requests at the same time and just wait when required. This reduces the number of choke points. Assuming web applications do something like this:

1. Check Authentication 
2. Application configuration / setup that where user is not required
3. Based on user / permissions additional data is requested
4. Possible a second round of data based on application data
5. Response returned
6. Logging 

As the use of key/value and document databases increase I think this will be more important to maximize backend performance. Riak ability to scale up both for reading and writing allows for this type of access pattern similarly to memcached. I don't know of any RDMBS (maybe VoltDB?) that could handle a large number of concurrent requests since they have very constrained IO or process (thread) resources. It is a lot of fun to think about ways that increase IO distribution could mean potential new access patterns for low latency data access. 

Would be happy for any help, code review and contribution much appreciated.

Concurrently golang programming

I would guess like most developer my first exposure to concurrency was in my parallel programming class in college. MPI and PVM where rather painful libraries to C/C++ experiences to cute my teeth on but it was a very exciting and rewarding experience. When I started writing Java thread programs there was an uneasy sense that parallel programming couldn't be this easy (after all the locks and synchronous calls there wasn't many operations running in parallel) but it sure was a lot of fun. In Python I found threads to be crazy simple but only useful for IO wait and loving miltiprocess programming. Parallel Python was the only thing that actually provided the same true parallelism as the original MPI and PVM but was also amazingly simple. Goroutines (coroutines) make me never want to see thread programming again. The killer combination of channels (kinda like a queue in the thread world) and coroutines are really enjoyable to code compared to threads and queues.

Currently I am working on creating a library for golang to talk to riak. If multiple request need to be made then they can be done using goroutines. Thus reducing round trip wait time. Depending on IO limitations this could increase the speed of web pages that need multiple resource for a specific url. For things like memecache and very fast datastores this would work great. The downside is that potential overloaded IO situations would be made much worse by the increase number of connections. This would probably limit to nonblocking IO datasources which most new datasources can handle a large number of connections.

First Impressions of golang

Another language? 
Yep golang or just go depending if you want to find anything from a search result. For me I write mainly in Python, Java, Javascript. Python as my GOTO (pun intended) language because I can write thing extremely quickly and sometimes even cleanly. Java for embedded platforms and GWT. Javascript the language of the web so that mean almost all developers use it (or should be). It is inevitable that a new language will replace older ones but I think there are two things that might have delayed the creation of golang and ultimately what I intend to use it for.
The first and largest is if Java continued to evolve not just its libraries but significant syntactical changes. Java from a completeness of implementation is great and the toolset is wonderful. Java requires way to much code volume. Code generators help however the code that is not generated has to access a majority of the generated code and no matter what there is a feeling of bloat. All the performance improvements in Java have yet to make development feel faster.
The second thing is Objective-C lack of popularity. Maybe because like Darwin it has a kinda of open source aspect to it but really the only significant implementation is proprietary implementation by Apple. Even though the IPhone has help its popularity it has reduced it openness and reduced its ability to become a general purpose language.
Golang fate could be similar because Google could be the Sun of Java (even more bad punning) or the Apple of Objective-C. Golang is not currently on that projection but I would not rule it out as a possibility.
Golang's Niche
I would like golang to replace everything I use Java and Objective-C. There is a lot of Python code that I think should be in a statically typed language but for various reasons mainly time I will note rewrite but golang would be great. Even though golang is very productive to develop in for a static language, on occasion my face slams into STATIC TYPE. For real stable API's that could have a real a large amount of dependent code I can see it will fit nicely. Until I get proficient with it I am going to try to keep working with it on simple projects.
Tasty Features
Terse Syntax
With exceptions of languages like Perl less typing is better. Golang shines in this manner. It is the most terse static typed language I have ever programmed in. Refreshing from a Java perspective and surprisingly close to Python.
Type System
I must say I really loved C when I first learned about stucts and header files to define functions. There is something really solid about the type system that I haven't felt since a C header file except more terse. Make me never want to see another Java interface in my life.
Channel
I am a bit of a server performance geek and well what can I say channels are just awesome. I have yet to write any code with channels but it is one of the things I am most excited about. I plan to use it when I write a golang driver for Riak. I would like to write a WSGI inspired web server in golang that I could access from Python and that is where I look forward to testing the mettle of channels.
Testing
When it come to code it is all about "hitting the ground testing". The development tool gotest makes this fast and easy.

Walkscore Widget with Ads!

So my adventure into mobile development continues. The new version of Walkscore Widgets has ads!

The widget didn't change much. The app got a lot of features besides Ads (going to need to support my habit of writing code somehow). Progress bar and better messaging to explain why a lookup failed. The progress bar was difficult until I figured out that threads are really needed for good UI development. Over all I am pretty happy about it. As always I figured out more features I would like to add and I had a bug fix 24 hours after the release but in general I am much happier. Still could use some graphic design help but maybe the next app...

It is good to be a geek (me)!

As I sit in a franchise restaurant waiting for an Android developer group meeting I realize that there are people envious of my coding skills. I can't believe I take for granted the decade I have spent learning Java, Python, C, SQL, Javascript, Linux, OS X, Android ect. Makes my current frustration trying to get an Android progress bar seem ridiculous. Going to Google I/O and spending time in coffee shops where everyone is using modern technology has made me complacent. I think I just saw someone using OUTLOOK! I mean who uses Outlook never mind exchange any more? Ok back to progress bar land with a lot less frustration and complacency.

GWT is great except for that pesky little Java problem.

First Java is really the only language option in the current language options. During the Google I/O keynote GWT actually mentions a spring tool that generates boiler plate code. This is ultimately my problem with Java it is not expressive enough. With some exceptions (low level languages like C) this is a FAIL mode for any language. Just can't beat the GWT UX though from performance to maintainability.

Cloud application (AppEngine) or server (Rackspace, amazon)?

Flexibility vs Productivity vs Creativity

Starting out building an application the thing we don't know is the path the application is going to go. Like the CAP theorom that states pick two of the three options (Constancy, Availability, Partition Tolerance) we have to pick two of Flexibility, Productivity and Creativity. 

Take Hardware Off The Table

We have enough to worry about with all the laptops, monitors and cell phones but add rack of servers and we are really not happy. These type of resources are also very cost prohibitive especially for a startup or poorly funded project. Physical servers, switches, battery backups, facility (with special sprinkler system), security systems on physical access to equipment, power redundancy, network redundancy and capacity are all things that are enjoyable engineering tasks but a complete distraction from focusing on the project.

Hosted Server Overhead

Amazon user experience for managing servers is horrible so I am only going to talk about what I think the viable option is something like Rackspace. Come down to 3 things:

• Setup - still need to manage configuration and setup of all the services that are running database, web servers ect. Even a small deployments can get complex quickly and require lots of resources.
• Deployment - getting code actually onto the servers (this is harder than it sounds especial with web servers, databases, message queues ect)
• Scaling - If your application is at all useful then people will want to use it and so you will need to figure out how to add more web servers, databases (RDBMS big piles of pain and expensive consultants licking there lips)

Hosted Application Constraints

AppEngine since that is the most popular and I have and am currently using I will point out some of the issues. 

•  Development and testing can only use a very limited sample data set because of the way the development environment is. 
• Simple and easy crojob that taskqueue is not. (probably going to end up using cron hitting a url anyway)
• It is very expensive when the traffic hits!
• Limited on which software libraries can use!

How about Hybrid

What constraints that AppEngine places will limit creativity and flexibility however is very productive. Rackspace allows for maximizing flexibility and creativity on tools to solve the problem but puts a large burden on productivity. A hybrid approach might be exactly what the doctor ordered. Starting out with AppEngine and migrating to a solution like Rackspace as the needs arise to be more flexible and creative.