Project Automation Experience

CodeSherpas is looking for new talent

David Bock — Tue, 19 Jun 2012 07:44:00 CDT

Just over 5 years ago we started CodeSherpas with a vision - the nature of software development is changing, and we wanted to be on the forefront of that change. Our jobs are many and varied - sometimes training and mentoring, sometimes augmenting existing staff, sometimes the project is ours and ours alone. Sometimes we work at home, sometimes on client-site. We bring the deep experience we have developed to bear on our client's projects, and have a strong reputation doing so.

The time has come for us to broaden ourselves; we are looking for a couple of passionate, self-motivated software engineers who want the opportunity to take their expertise to the next level. Are you interested in working for a company where your skillset is the product, and the company invests in it? Are you interested in working someplace where even the owners write code, and there are no 'pointy-haired bosses'? Are you interested in solving a variety of interesting technical problems? Have you been dabbling in recently emerged technologies and want the opportunity to use them full-time in mission-critical production applications?

We currently spend most of our time in a web-centric development stack. To us that means: Ruby, Java, Javascript, CSS, HTML5, JQuery, NodeJS, Backbone, MySQL, Redis, Linux, OSX, and so on. We also occasionally dabble in Objective C and Clojure, and see those technologies as crucial to the future. When we run a project we prefer 2 week iterations, constant feature delivery, and reflection on the work to improve ourselves. We know and love git. We like to publish, speak at conferences, record screencasts, and promote the local user group communities. If you think you can contribute to a team doing these things, we want to talk to you.

As consultants, our jobs tend to take us around the Northern Virginia and Washington DC area. Sometimes we work at home, sometimes in the Reston area, and sometimes in DC. We know commuting in the DC metro area is tough, we do it ourselves. Despite this, we strive to maintain a healthy work-life balance.

We're not looking for rock stars... we're looking for orchestra members. We're not looking for ninjas... we're looking for Sherpas. We do the technical heavy-lifting.

Does that sound interesting? Lets talk.

Goetz's First Law

David Bock — Mon, 18 Jun 2012 14:44:00 CDT

"Its never any one feature that causes problems; it is the interaction of otherwise reasonable-seeming features."

In a recent conversation with a number of NFJS speakers, David Sletten (via his brother Brian) brought up a tricky Java puzzler - a piece of code that is seemingly simple, but doesn't do quite what you'd expect. Consider:


Arrays.asList(new Integer[] {1, 2, 3}).size();
Arrays.asList(1, 2, 3).size();
Arrays.asList(new int[] {1, 2, 3}).size();

To make a long story short, any reasonable first-reading of this code would expect all three examples to return '3', yet the bottom one returns '1'. Why?

Brian Goetz, who has the esteemed title of "Architect for the Java Language and Libraries" gave a clue that leads to the answer when he said "Hint: asList is a generic method. What type is being inferred for T?"

This led to a conversation that only the nerdiest of nerds lubricated with the finest of scotches would dare sit through. Brian eventually concludes with the statement, "The bipartite nature of the type system is the root cause here; int is not an Object, but int[] is. Mix that with varargs' attempt to conflate T... with T[], and...bam, astonishment. Its never any one feature that causes problems; it is the interaction of otherwise reasonable-seeming features."

That last sentence was quickly seized upon as a profound truth in software development and dubbed "Goetz's first law".

The code example above isn't nearly as important as the sentiment that inspired the law. Since he said that, I've seen virtually every problem I'm encountering as a variation of this.

A Recurring Billing System with Stripe

Brian Sam-Bodden — Mon, 11 Jun 2012 11:31:00 CDT

These notes are meant to answer some basic questions about using Stripe for billing, and also to answer some questions about implementation which are not covered in the Stripe documentation and forums. This is not a comprehensive overview by any means. If you find errors, please let me know and I will fix them. The examples will use the stripe-ruby gem (v1.7.0). A list of basic operations can be found in Stripe's documentation.

Stripe recommends downloading the gem over SSL from the mirror at https://code.stripe.com.

sudo gem install --source https://code.stripe.com stripe

Here is a list of resources which I have found useful:

Creating Subscription Plans

Plans can be created through the Stripe Dashboard or the API. Since plans will not change often, it’s easy to create them through the Dashboard and retrieve them from the application via the API.

Stripe::Plan.create(
  :amount => 999,
  :interval => 'month',
  :name => 'My Basic Plan',
  :currency => 'usd',
  :id => 'basic'
)

Create Token with Stripe.js

Stripe provides a javascript library which sends encrypted credit card and cardholder data to Stripe’s servers from the browser. Card info is sent (along with the accounts public key) directly to Stripe and is never handled by host application.

// https://gist.github.com/1750368
Stripe.setPublishableKey('pk_YOUR_PUBLISHABLE_KEY');

Stripe.createToken({
  number: $('.card-number').val(),
  cvc: $('.card-cvc').val(),
  exp_month: $('.card-expiry-month').val(),
  exp_year: $('.card-expiry-year').val()
}, stripeResponseHandler);

The response includes the ID of the created card token if successful, and relevant error messages otherwise. The token represents a valid credit card stored by Stripe’s servers. Card tokens do not represent charges of any kind.

Token Object

A The host server can retrieve information about the card by retrieving the token referenced by the token ID. The response will contain basic card details including the last four digits of the card, instead of the entire number.

Stripe::Token.retrieve("tok_00000000000000")

Create Customer

The card token ID can then be submitted to the host server. Card tokens can be used to create a one-time charge or to create a customer with a subscription. The relationship between a customer and a plan is described by a subscription. Subscriptions can be updated or canceled through the customer object.

Stripe::Customer.create(
  :description => "Customer for jim@example.com",
  :plan => "basic",
  :card => "tok_00000000000000" # obtained with Stripe.js
)

Many applications will offer a free plan. Stripe will allow plans to have an amount of 0 (zero). Customers can be subscribed to a free plan without providing credit card information. Subscribing free users to a free plan could make the upgrade process simpler, since a customer record will already exist. Providing a card token is optional when creating a customer with a subscription to a free account.

Stripe::Customer.create(
  :description => "Customer for jim@example.com",
  :plan => "my_free_plan_id"
)

Updating Customer Subscription

It’s likely that a host application will allow users to change to a different plan. In this case, the relevant customer object can be retrieved and updated. Updates can optionally be prorated.

c = Stripe::Customer.retrieve("cus_00000000000000")
c.update_subscription(:plan => "enterprise", :prorate => true)

Customers can only be updated from a free plan to a paid plan if a valid card token is provided with the request.

c = Stripe::Customer.retrieve("cus_00000000000000")
c.update_subscription(:plan => "pro", :card => "tok_00000000000000" ,:prorate => true)

Customers can be updated from a paid plan to a free plan without providing a card token. In this case Stripe will still have the user’s active card on file; so if the user switches back to a paid plan, it is optional to include a card token with the update request.

At this time, Stripe does not offer a way to remove an active card from a customer which is subscribed to a free plan.

Cancel Customer Subscription

When a user deletes their account or their customer record is no longer needed, the customer object can be retrieved and deleted.

c = Stripe::Customer.retrieve("cus_00000000000000")
c.delete

Webhook Authentication

It is critical to properly authenticate webhooks. There are two methods of authentication recommended by Stripe. The first (and simplest) method is to retrieve the event using the id parameter of the webhook request.

# in a Rails controller
def my_webhook_action
  # this will raise InvalidRequestError (status 404) if the event does not exist
  event = Stripe::Event.retrieve(params[:id])
  # safe to use the returned event object
  # ...
rescue Stripe::InvalidRequestError => e
  logger.error "Unable to authenticate webhook request: #{e.message}"
  # 401 Unauthorized response
end

The second method for authenticating webhooks is to use HTTP basic authentication. When setting the webhook url in the Stripe Dashboard, provide a username and password. See Railscasts episode 82 for details on implementing HTTP basic authentication in a Rails app.

Testing

If you are implementing your system in Ruby, I highly recommend the VCR gem. For a good overview of VCR and it capabilities, watch Railscasts episode 291 *.

It is important not to commit your secret api key to the repo. Use the filter_sensitive_data configuration option to redact your private key from the recordings.

VCR.configure do |c|
  # ...
  c.filter_sensitive_data('[secret_key]') { Stripe.api_key }
end

Appendix

* Please note that some of the configuration syntax has changed in newer versions of the VCR gem:

# VCR.config do |c| # outdated
VCR.configure do |c| # new
  # ...
  # c.stub_with :fakeweb # outdated
  c.hook_into :fakeweb # new
end

Go beyond Java with CI server and Artifactory

Baruch Sadogursky — Thu, 31 May 2012 02:36:00 CDT

During the last couple of years, continuous integration (CI) and automated release management methodologies have become much stronger in non-Java builds.

Number of familiar tools are used for these methodologies, like the version control system to manages your sources, your build tool to actually build your software from sources and your build server, which builds your software continuously using the build tool. But is there something else missing? Let’s look at a simple diagram, which describes your CI process:

Developers fetch needed dependencies (3rd party libraries, other modules).
Developers write new code.
Code is committed to VCS.
Build server polls VCS for changes.
Once discovered, the build server builds the software: compiles it, runs tests, and assembles artifacts.
Built artifacts are published to QA, staging, or even directly to the end users.

As you can see, everything except stages 1 and 6 is well known and familiar. But what about those two stages? You need a dependency management mechanism for them.

In this post, we’ll illustrate the options for automated builds with dependency management in non-Java builds or when not using .NET with NuGet (use it, if you can! Here’s our take about it). The proposed solution works for any kind of build, be it C, C++, C#, iOS and Objective C, Python, or whatever.

So, let’s see how you can implement CI with dependency management for non-Java projects. Here are the options with their pros and cons:

#	Solution	Pros	Cons
1	Dependencies in VCS	Use proven build tools Simple setup – you use VCS anyhow	VCS won’t fit for binary dependencies. You can read why here.
2	Build tools with built-in dependency management like Maven and Gradle	Build tool with dependency management	New tool to learn. Both Maven 2 and Gradle have a pretty steep learning curve. “Adapted” from Java. While there are ways to build “native” applications with them (easier with Gradle, harder with Maven) it still feels, let’s say, artificial, imposing their new conventions.
3	Declare dependencies on builds in your build server	Use proven build tools	Only works for inter-project dependencies (which were built by the same build server) Not flexible enough Includes/excludes Layout changes
4	Use shared dependencies storage, a.k.a. “repository” (FTP, file server, etc.)	Use proven build tools	Manual repository populating Managing the repository

As you can see, none of the solutions are without the cons.

Here comes Artifactory

Artifactory is a binary repository – built with binaries management in mind in terms of versioning, management, security, and build servers integration.

As such, using Artifactory completely eliminates the cons of the fourth solution. Your build can populate the repository using powerful REST API. Management is easy and streamlined.

But there’s more. By using Artifactory CI integration (for Jenkins, TeamCity and Bamboo), you can also avoid the cons of the third solution listed above. First, you can specify which artifacts you are willing to publish. They will be uploaded to Artifactory in the end of the build. Second, you can specify which dependencies your build needs. They will be downloaded from Artifactory during the run. All this simply by using build server UI.

The Artifactory Jenkins plugin, for example, defines as part of the Generic build support, a simple pattern-based language that allows you to express what artifacts from the build are going to be deployed to the repository. This includes attaching dynamic searchable properties to these artifacts upon deployment.

First, enable Generic-Artifactory Integration in the Build Environment:

Next, configure your custom deployment and resolution rules. Let’s start with deployment (publishing):

In the above example, we configured the following rules for deployment. All the artifacts will be deployed from the working directory into the ‘libs-release-local’ repository (configured above), maintaining the path for each file:

Pattern	Meaning
*/x64/.dll=>x64Win	Deploys all DLLs to the ‘x64Win’ directory
*/.zip=>winFiles	Deploys all zip files to the winFiles directory
unix/*.tgz	Deploys all tgz files under the unix directory to the root directory of the target repository

The plugin also controls which artifacts to resolve from the repository before the build starts. The resolution allows you to specify smart search patterns:

In the above example, we configured the following rules for dependencies resolution

Pattern	Meaning
libs-release-local:x64Wi/*;compatibilityLevel =medium,high	Resolves the files from the x64Wi directory of the libs-release-local repository to the root of the workspace, but only if the ‘compatibilityLevel’ property is set to be above medium
libs-snapshot-local:*.zip=>winFiles	Resolves all zip files from libs-snapshot-local repository to winFiles directory under the root of the workspace
libs-snapshot-local:unix/distro.tgz=>linuxFiles	Resolves the distro.tgz file from unix directory in libs-snapshot-local to linuxFiles directory under the root of the workspace
libs-release-local:*/@winx64_build#released	This example shows dependency to artifacts produced during an earlier build that has been marked with a “released” status

From a configuration standpoint, this support for automatic dependency declaration and publishing makes it very easy to support a release flow where multiple artifacts are collected towards building the final release.

You are more than welcome to give it a try. The publishing functionality described above is available in the free open source version of Artifactory. If you want to configure dependencies resolution, go ahead and download the Pro version evaluation.

Dependency management is critical for continuous integration and rapid release. By adding Artifactory to your CI stack, you can easily overcome the lack of such functionality in your build tool, leaving the dependency resolution and artifacts deployment to the binary repository and your build server.

Resolves the distro.tgz file from unix directory in libs-snapshot-local to linuxFiles directory under the root of the workspace

Filed under: Artifactory, JFrog Tagged: .net, artifactory, best practices, build, c, continuous integration, iOS, jenkins, native, objective C, Pyhton

The Key Tenets - 7 Years after The Internet Services Disruption

Ken Sipe — Mon, 28 May 2012 11:49:32 CDT

October 28 2005 Ray Ozzie, having recently joined Microsoft (msft) at the time, put together an interesting manifesto with the subject title of "The Internet Services Disruption", with the purpose of "get all of us roughly on the same page". For currently undisclosed reasons, I had reason to review this document and found it completely fascinating. Ray's foresight and ability to forecast the future is just amazing. Although his writing was made public (good for us all), it was intended to inspire and position MSFT for it's next great phase. Let's look at each of the key tenets Ray outlined and see who in our industry has the upper hand.

1. The power of the advertising-supported model
Is MSFT winning here... definitely not. Winners in this category are Google, making money off google.com. Facebook making money on ads based again on high traffic visitations. While these two companies are making good money with this concept, the company to watch is Apple. Why? Apple is actually empowering developers with this ad-support model through iAd. Like ads or not... Apple is enabling developers to provide solutions free to consumers and are paid through ad hits. There has also been a resent uptick in the number of ads off services like Hulu and youtube. So there is no question that Ray nailed this tenet.

2. The effectiveness of a new delivery and adoption model
While Ray describes this in a try before you buy, and a readership and review model, while being valid it is ubiquitous, so it is hard to give any one company brownie points for this. Most of the good companies have their share of fan boys, VIP programs and evangelist (in the which MSFT does a stellar good at), I would like to take this concept a little deeper. New delivery... and delivery of what. Considering operating systems and software, everyone is going disc-less. Delivery of updates are internet provided. But lets talk content... There is a new delivery mechanism for content... how do you watch your movies, how to you read your books, how do you get your music? Where is MSFT in this space? No where to be found. Who owns this space? Well... depending it seems divided between Amazon, NetFlix AND the big one? Apple. Apple hits gold stars in this category up and down their product like. In fact one could say that the adoption of an iPod or iPad is a gateway into the full Apple stack.

3. The demand of compelling, integrated user experiences that "just work"
While I could go into detail and beat around the bush... this statement screams Apple. While apple costs more in some cases, they are uncompromising and focus on the user experience. They have all the advantages in this category as well. they own the full tech stack... they don't have to make it work on a billion devices... they have theirs.

I consider myself a technologist and I don't "belong" to a specific tech group. I see value in a number of camps. The take away for me... is how insightful Ray was almost 7 years ago. He nailed it. It is too bad that MSFT doesn't seem to be paying attention to him.

The Relaunch

Matt Stine — Thu, 24 May 2012 19:45:00 CDT

I have rebooted this blog many times over the last several years. If you’ve been a reader of my blog in the past, you will have noticed significant changes. If you’re new here, welcome!

This reboot has been in the works for several months now, even though I’ve probably spent far less than 24 active hours working on it. Life as an “itinerant” consultant and conference speaker is extremely busy compared to what I was doing on May 16, 2012 (the date of my last blog posting). At that time I was in the middle of a transition from 3+ years as a front-line manager of a software development team into a lead software architect role. Since that time I’ve changed jobs (careers?) twice:

June 2011 - February 2012: During this time I was titled as a “Technical Architect” at AutoZone’s corporate offices in Memphis, TN. My focus was the modernization of their retail store management system architecture and technical stack. While there I also did a fair amount of agile coaching.
February 2012 - Present: In January, VMware came calling, wanting to know if I would join the vFabric Cloud Application Platform PSO as a Senior Consultant. After a few weeks of interviews and careful consideration, I made the jump. I’m now in the midst of helping a Fortune 50 company build a brand new “private cloud” hosting platform for their next generation of application development.

During that time I also significantly increased my participation on the No Fluff Just Stuff tour. In other words, I’ve BEEN BUSY.

At any rate, I have for quite awhile now wanted to get back into the practice of writing. However, I’ve wanted to do so outside the constraints of a hosted platform like Blogger or WordPress.com. Those systems place far too many constraints on how your blog works, and they also happen to be the only place that your data is housed. Sure, I could write my posts in Vim and then copy/paste them into the platform, but that’s annoying. I wanted to manage my writing using a plain text editor, mark it up using something lightweight like Markdown, check my writing into version control using Git, generate my site using a tool like Jekyll, and then push my site to the cloud. What was the easiest way to achieve all of this goodness? Octopress seemed to fit the bill.

So, what you now see is the result of my labor in migrating from WordPress.com to Octopress, hosted at Heroku. One day (perhaps) I’ll write up the process that I used. It was a bit convoluted and involved gigantic piles of yak hair, so I’d rather not consider that piece right now.

What I will be focusing on quite a bit is the idea of simplicity. For more than a decade I have wandered through the barren wasteland that is accidental complexity in software. A few months ago I viewed Rich Hickey’s keynote session at StrangeLoop 2011 entitled “Simple Made Easy”. It is Rich’s definition of simplicity in software that has inspired the brand new title of this blog, wherein I will chronicle my continuing quest to seek out simplicity in software structure.

Alas, it is time to return to billable work. I shall endeavor to post again in far less than a year. So, as we say in the south, “Ya’ll come back now, you hear?”

On Using the Right Tool for the Job

Baruch Sadogursky — Mon, 21 May 2012 09:57:00 CDT

Disclaimer: The code samples in the post below are for brain teasing only. Do not, I repeat, do not ever do such things for any other purpose!

A friend of mine is a big fan of puzzlers. Any puzzlers, including programming ones. Here’s his latest and greatest:

Write some code in the static initializer to make the assert pass:

public class Test {
 static {
//Write some code here
 }

public static void main(String[] args) {
 Integer a = 20;
 Integer b = 20;
 assert (a + b == 60);
 }
}

Don’t forget to enable assertions if you want to solve this one (-ea as VM parameter).

If you solved it, or don’t feel like puzzlers, read on for the solution and the punchline.

First, the solution: You need to know refection and two facts about Integers to solve this one:

Integers have cache for all values in range -128 to 127.
Integers use that cache during auto-boxing (as opposite to using Integer’s constructor, for example).

Well, you also need to know the exact implementation of the Integer class to figure out where the cache is and what’s the name, but rt.jar sources are here for the rescue. Guess what? the cache is a private variable inside a private inner class. Joy-joy.

Here’s the plan:

Get Integer’s class object.
Get the list of its inner classes.
Find the right one (what a relief, it only has one. Or not – it is subject to change in future versions since we are digging in highly encapsulated stuff . Duh.)
Get the cache field from it.
It’s private, set it to accessible in order to get the value of the field.
Change the value of “20″ to be “30″

Here’s the code of the static initializer:

 static {
try {
 Class<?>[] declaredClasses = Integer.class.getDeclaredClasses();
 Field cacheField = declaredClasses[0].getDeclaredField("cache");
 cacheField.setAccessible(true);
 ((Integer[]) cacheField.get(null))[20 + 128] = 30;
 } catch (Exception e) { e.printStackTrace(); }
 }

Personally, I hate it. It’s ugly, it messes with very low-level encapuslated stuff that one even doesn’t suppose to know, and it is fragile (e.g. it won’t work if Integer is created with constructor).

And fairly, we just got lucky. If Integers did not have the cache then we would not have any way to change what “+” does!

So, why this simple trick is so difficult to impossible to implement? Simple – Java is not intended for those tricks. Java is static, and it is good! It means that you can rely on it. 20+20 will (almost always) be 40. That is Good.

If you want to do dynamic stuff, like changing the behavior of a “+” operator, you’d better use the right tool for the job. E.g. – Groovy.

Here’s the Groovy version of the puzzler:

//Write some code here
Integer a = 20
Integer b = 20
assert 60 == a + b

Looks almost the same (without the main() mess, some ridiculous semicolons and the need to include -ea in VM parameters), but the difference in the way to achieve the desired is huge. Since Groovy is dynamic it is intended to do stuff like changing the behavior of the plus sign. Here’s the facts about Groovy that you need to know to solve this one:

Groovy operates with wrappers, always.
Groovy implements operators though methods. “+” is implemented in plus() method.
Using Groovy’s MetaClasses you can easily replace method with any closure.

Here’s the plan:

Get Integer’s MetaClass.
Replace plus() method with implementation that returns 60.

That’s all. Really. Here’s the code:

Integer.metaClass.plus = {int i -> 60 }

Nuff said. Now you only have 2 options. Stop doing those ugly tricks, or at least do it with the right tools for the job.

Filed under: Development Tagged: groovy, java, metaClass, reflection

Getting the Spock out of a Gradle War

Ken Sipe — Thu, 05 Apr 2012 13:55:28 CDT

I recent ran into a interesting situation, for which I thought it would be worth sharing. I have a new project with the following build needs: Java, Spring MVC and Spock Testing. The problem is simple... The WAR build in gradle was building a WAR file that included the Groovy libraries. This project has no need for groovy at runtime. Groovy is needed because I'm using Spock for my testing.

Gradle and providedCompile

In older versions of Gradle it was commonly necessary to manage the jars in the lib directory. In the maven world, there is a dependency scope of "provided". This scope basically means that the dependency will be provided on the server classpath, however for the purposes of compiling the code in the project it is necessary. From a WAR perspective it means use this dependency for compile, but do not include it in the lib directory of the WAR. Gradle within the last year has added the same feature as providedCompile or providedRuntime. In my project I have this very need with the servlet-api, shown below.

dependencies {
 compile "org.springframework:spring-webmvc:$springVersion"
compile 'javax.servlet:jstl:1.2'
 providedCompile 'javax.servlet:servlet-api:2.5' 
}

Spock dependencies and Gradle

As Spock is a groovy testing tool, it is no surprise that Spock requires groovy. This requires a couple of well documented configurations in your gradle file. First, you'll have to add the groovy plugin. Second you have to set as a dependency the groovy version. But wait!!! The dependency for groovy doesn't have a scoping ability. This is area where the "model" of gradle breaks down a little bit IMO. Ideally you would be interested in expressing in the model that there are "testing" needs... and those testing needs require groovy and spock dependencies.

apply plugin: 'groovy'

def spockVersion = '0.5-groovy-1.8'
def springVersion = '3.1.0.RELEASE'

dependencies { 
groovy 'org.codehaus.groovy:groovy-all:1.8.3'
 testCompile 'junit:junit:4.8.1'
testCompile "org.spockframework:spock-core:$spockVersion"
testCompile "org.spockframework:spock-spring:$spockVersion"
}

With this groovy dependency any build of WAR will result in a the groovy-all-?.?.?.jar file being added to the the WEB-INF/lib directory. The problem is I have no current interest in having groovy in production for this project. What to do? What to do?

Gradle Doc to the Rescue

I find that a large number of people (usually new to gradle) struggle with discovering how to resolve such a problem. The best starting point for getting to know gradle is the gradle user guide documentation, but the best starting point to use as a reference is the gradle dsl documentation (http://gradle.org/docs/current/dsl). Hopefully it is obvious that what we want to change is the build of the WAR. So lets look at the War task. Here we see all of it's properties... which includes the classpath property. The documentation for this property clearly states that this property affects the WEB-INF/classes and the WEB-INF/lib. Diving deeper into that property we see the default behavior which is:

project.configurations.runtime - project.configurations.providedRuntime

So our logical solution would be to include a war task configuration for the building of the classpath like this:

war {
 classpath = classpath - project.configurations.groovy
}

Happy Coding! and may your builds never fail!

Speak at the 33rd Degree

Ken Sipe — Sun, 18 Mar 2012 12:29:46 CDT

I will be speaking in Poland this month at the 33rd. It looks like a great conference line up and Krakow is wonderful.

QCon 2012 – Perfect as Everything in London Should Be

Baruch Sadogursky — Fri, 16 Mar 2012 02:17:00 CDT

This post was originally posted on ‘From the Frog’s Mouth‘.
You’re welcome to comment here or there.

It was JFrog‘s second QCon London, and it just gets better. Imagine: even the London weather was perfect, not to mention the sessions, booth traffic, show organization and food (what, you say, good English food? Well, great IndoPak food, at least). Due to high demand by sponsors, the exhibition took place on two floors (as opposed to one floor last year). Our JFrog booth was located in the same place as in 2011. We’re getting used to the place and are looking forward to returning next March!

The speaker panel was extremely impressive, featuring gurus like Martin Fowler, Adrian Cockcroft (the man behind Netflix’ cloud), Dwight Merriman (co-creator of MongoDB), Damien Katz (creator of CouchDB) and Rich Hickey (creator of Clojure).

The conference started with two training days – six full-day tutorials each. From my perspective, the most interesting two tutorials were “Cloud Architecture” by Adrian Cockcroft, where he shared the architecture, best practices and decisions behind Netflix’ cloud (which Artifactory is proud to be a part of) and “Continuous Delivery” by ThoughtWorks’ Tom Sulston (that’s as close to the roots of the famous “Continuous Delivery” book as you can get). For me, the most fascinating thing in the Continuous Delivery process as ThoughtWorks sees it, is that its virtues are exactly the same as we based our Artifactory upon back in 2006: DevOps automation and rapid release cycle. We appreciated the validation of our concept.

The remaining three days of the week were all-day sessions. It’s impossible to review such a significant number of talks in one blog post, so I’ll concentrate only on the excellent keynote addresses (with one exception).

Martin’s conference-opening keynote speech was about data. The main feature of modern data is that it is bigger than you think. Polyglot storage in general and various kinds of NoSQL look like the right solution.

My favorite keynotes are usually the evening ones. A beer in your hand makes any amusing talk even more enjoyable. One named “Developers Have a Mental Disorder” I couldn’t miss! Greg Young gave a great show, funny and entertaining, about serious dilemmas in software development that we, the developers, prefer to ignore. The brightest example, of course is the downside of DRY (did you ever think about one?). By removing duplication, we increase coupling, which can be much worse.

Thursday morning’s keynote address was delivered by Rich Hickey. He spoke about the differences between “Simple” and “Easy”. Sounds pretty similar, but in fact they are very far from being the same. Antonyms to the rescue: simple vs. complex, while easy vs. hard. Now it’s clear – we need to strive to prevent and remove complexity (go simple) without being afraid of the hard. Choosing easiness may end up creating complexity. Things which are easy, but complex, include OOP, mutable state, imperative loops and frameworks (especially ORM). Things which are simple, but not necessarily easy (at least not until you get them), are LISP-ish languages, like Clojure.

My session also took place on Thursday, in relatively small room, about 70 people. I was more than happy to see that it was almost packed. I spoke about building trust in your build process by selecting the right tools for the job (of course, we consider Artifactory as one). I also spoke about the problems of DevOps in the word of Continuous Delivery in the cloud and the rapid release cycle of SaaS applications. I stressed the huge timeshare of binaries in ALM process and the importance of using a tool that really understands binaries to deal with them. That’s exactly the reason why we developed Artifactory.

Half of my session was dedicated to live demos, which went smoothly, incredible as it may sound. According to the feedback received, my talk was well accepted, and hopefully will be useful to some of the attendants for building easier and more reliable release processes. The Q&A session continued at our booth, where we repeatedly did live demonstrations and received excellent feedback each time. If you want to get a feeling of my talk, here’s the slide-deck.

Friday was the last day of the conference. It started hard with a highly technical keynote address by John Allspaw with a scary name: “Resilient Response In Complex Systems”. For someone like me, who doesn’t deal on a daily basis with Disaster Recovery, the session was astonishing. Looking behind the curtain of that kitchen reveals a totally different way of thinking and planning. It may be how individuals and teams have to perform during a disaster (e.g. personal heroism is bad even if successful; it sends the wrong message to the team), or simulating disasters on live production systems (I never could even dare to think about that). The most obvious, but still eye-opening advice that John gave is to learn from successes, not only from failures. It can give us a lot of information and happens much more frequently, no? The only organization with which I am familiar that embraces that technique is the Israeli Air Force.

To sum up, the conference was great by every measure: technical sessions, attendance, networking, Artifactory exposure, and after-show quality time. Thank you, InfoQ, for this wonderful event in London. QCon was a great starting shot for JFrog’s “Busy March”. You still can catch Fredric and Yoav giving talks on various events in US and Europe.

Filed under: Artifactory, Confereneces, JFrog Tagged: artifactory, Conferences, jfrog, QCon

Agile: getting the standup right!

Ken Sipe — Wed, 14 Mar 2012 17:51:07 CDT

The standup meeting as part of an agile discipline at first seems like an easy and obvious activity. Yet I keep coming across clients and organizations that seem to struggle with getting it right. It appears that as long as everyone is standing... and we do that everyday... that we can check "standup" off the list. Just standing up doesn't make it a stand up! (Please repeat that... as often as it takes to get it.) On top of that... many organizations seem to think that the stand up is for the scrum master or management to get status. That's not it either! Here are some warning signs:

Standups take 30 mins or longer
People regularly miss the meeting
You are always waiting 10 minutes for that one person
The PM or Scrum Master is asking clarifying questions

The standup meeting is NOT:

Primarily for status (I know that might seem confusing)
About the position you are in during the meeting
About the Scrum Master

Getting the standup right is crucial for a healthy agile team. It is for and about the team. Here is the purpose of the standard up:

Synergy - It is a point for developers to synchronize their efforts and realize when there is about to be some code thrash. It is a point to say, "hey joe, I'm about to do task x which is closely related to your task y, lets get together after the standup".
Transparency - Everyone knows what is being done, which provides a double check that each person is doing the highest priority tasks and is there anything holding us up.
Accountability - This the best place to realize that a team mate is struggling. When you see that Joe has had a 4 hour task for 2 days its time to say "Joe... lets pair on that... lets see if we can knock that out.

These purposes come out of the standard questions most agile developers are aware of: "What did I do?", "What am I going to do?" and "is there anything holding me up?". I know this sounds like status... it's not.

Advice for Scrum Masters

First realize you are not the reason for the standup
Standups are 10-15 mins.. that's it.
Do not ask for follow up during the stand up... if something needs to be followed up, follow the person of interests back to their desk and ask them without tying up everyone else's time.
If it becomes clear that the team needs to meet.. for a design discussion or a big status meeting or... doesn't matter. It is super important to delineate that as a separate meeting. "Hey I would like to discuss x with everyone after the standup. Please stick around.". Then announce "ok, that is the end of the standup, lets discuss x"
The best thing you (Scrum Master) can do during the standup is to facilitate. Do not allow someone to dominate 5 mins of the 10 min standup or elaborate extensively. The challenge is not to be a jerk about it... If it is a challenge, then it is something that needs to be discussed in a retro meeting, asking for the teams opinion and possible solutions.

A big part of getting agile right is to understand why we do certain things! It isn't about standing up. It is about keeping it short and to the point. It is about getting the benefits listed above out of it.

Good Luck

Active Resource Spy

David Bock — Thu, 09 Feb 2012 10:42:00 CST

I'm doing a lot of work lately using ActiveResource. I wanted to get 'down to the wire' and see the traffic flowing over http... so a quick google search, and I found an entry I had written in 2008. I love it when that happens! So I'm reblogging it with a slight change to make it work in Ruby 1.9.

If you're using rails, create an initializer name active_resource_spy.rb and drop in this content:

class ActiveResource::Connection
# Creates new Net::HTTP instance for communication with
# remote service and resources.
def http
    http = Net::HTTP.new(@site.host, @site.port)
    http.use_ssl = @site.is_a?(URI::HTTPS)
    http.verify_mode = OpenSSL::SSL::VERIFY_NONE if http.use_ssl?
    http.read_timeout = @timeout if @timeout

    #Here's the addition that allows you to see the output
    http.set_debug_output $stderr
    return http
end
end

All the data that flows over the http connection will be dumped to $stderr (the terminal you started the app in, unless you redirected it).

Dependency Management with .NET – Doing it Right

Baruch Sadogursky — Mon, 06 Feb 2012 05:39:00 CST

This post was originally posted on ‘From the Frog’s Mouth‘.
You’re welcome to comment here or there.

The problem of dependency management is neither new nor original, it exists in all development platforms, and .NET is no different.

Let’s go through different solutions and see how they perform. I’ll list them here in no particular order.

Keeping dependencies in your source control

That’s a very popular solution, and for a reason. The benefits are obvious. Here are some of them:

No setup. You already have your source control in place (hm, I hope you do!). Add \bin directory, and you are fine.
No learning curve. Developers are used to work with source control.
Shared. The whole team gets changes and updates from the server as they occur.
Enterprisy (in a good way). The software is proven, backed up, DRP is done.

Sounds good, doesn’t it? So, what’s wrong with it? Only one thing – source control systems are designed to control, well, sources. As such they aren’t so great in controlling binaries. These are the shortcomings we all encountered during the years of Version Control System usage for dependencies:

It isn’t a proxy. VCS can’t download the dependency you need from a central repository when you need one. You need to manually download it and add it to VCS. The history starts from there – you just lost the link to the original file. So, you work hard, and on top of it lost information; this repeat itself for each new dependency.
Versioning mismatch. Source files are versioned by their content. VCSs know how to diff them and understand what changed. Binaries, on the other side, usually versioned by their name. From VCS point of view they are different entries, each one without any version history.
Some very popular VCSs (like Subversion) can’t obliterate files. That means – once a file was added, it stay in the repository forever. That’s not a big issue for small source files, but can become quite a pain when it comes to obsolete large binaries.
Source control knows how to search sources. And, of course, the most important type of search is by content. Searching for binaries is different: what matters there is the location, structure of the file name and, in case of archived artifact, the contents of archive.
The permissions scheme of VCSs is tailored for versioning sources (again!). For example, there is no override permission. That’s because overriding sources is something we do all the time (that’s what diff is for in VCS) – it’s the same security level as, let’s say, adding a new source file. With binaries the situation is very different. While adding new binaries is fine, overriding released binary is something that shouldn’t be done, one should have a special permission for it.
Distributed VCSs, awesome by themselves, are particularly unsuited for handling big binary files. When cloning a remote repository to your machine you are bringing all the history of all the files in it. Now just think about all the huge binaries sitting there…

As you see, the conclusion is simple – we can do better. Let’s try something specialized for binaries.

GAC and WebGAC

Global Assembly Cache is, on contradictory to VCS ,tailored for storing binaries. It understands versions, prevents conflicts, and generally does a good job being your local dependencies storage. The main problem with GAC is being local, which means – each and every developer should take the binaries from somewhere and install them in their local GAC. You see the troubles coming in that setup, don’t you? WebGAC to the rescue here. It’s essentially GAC shared by WebDAV and enables clients to fetch dependencies from the server, simplifying dependencies management for a team. Let’s do our pros/cons math. The benefits:

GAC is good, standard and proven solution for binaries management. It deals well with versions.
WebGAC is a central binaries repository for a team. Every team member synchronizes with it.
WebDAV is popular well-known HTTP extension with locking, security management, etc. Working with the Apache WebDAV module is generally straightforward.

Let’s see what won’t work so great with that solution:

It isn’t a proxy. WebGAC can’t download the dependency you need from a central repository when you need one. You need to manually download it, add it to WebGAC and only then it becomes available to the team. Not only must you work for every version of every dependency needed, the link to the original file is lost.
No notion of packages. GAC contains single dlls. You install them one by one. But think about NUnit, as an example. It contains about dozen of dlls along with various xml and configuration files. How can you install it to GAC?
Security is cumbersome. You’ll need to configure Apache Server’s security, and even then it won’t be flexible enough to determine between deployer (a user that can publish private dependencies) and promoter (a user that can move dependencies from a private repository to a public one).
Search is basic. WebDAV by itself only knows about files. It doesn’t care about the structure of the filename, or about the presence of Strong Names.

Looks like we still didn’t find what we are looking for, and then…

Here comes NuGet

This is something else. NuGet designed to be “a developer focused package management system for the .NET platform intent on simplifying the process of incorporating third-party libraries into a .NET application during development”. That’s exactly what we need. Let’s look how great it is:

Manages packages, not dlls.
Provides NuGet Gallery – almost 4.5K (at the time of writing) packages are at your disposal for all your development needs.
Supports binary versioning.
Integrates with Visual Studio.
Integrates with your build.
Integrates with your build server (only TeamCity at the moment of writing).

This tool’s like a dream come true. So, what can I mention as downsides? Most of them are downsides of the NuGet Gallery, not NuGet itself. The Gallery is a young and relatively small project (just for the sake of comparison, Maven Central is 6 years old and contains more than 290K artifacts) and, as such, it has its downsides:

The content of submissions to the Gallery is (almost) unverified. Everyone can register, get the API key and start uploading whatever they like. Scary, isn’t it? (yes, very scary).
Being public, NuGet Gallery can’t be used for inter-team packages exchange. Private Remote Feeds are the recommended solution. Next we’ll see if it is good enough.

Working with NuGet Remote Feeds

Remote Feeds, introduced in NuGet 1.4 are crucial need for any development team. It serves a dual purpose: it allows sharing 3rd party packages that aren’t available on Gallery (or even replaces the Gallery for those who can’t trust it) and it serves as a target for internal deployments – both for team collaboration and for other usages, as making packages available to QA, or even serving them to the customers from the outside world (by using Chocolatey, for example). If that’s so right, what’s wrong? Here’s what:

You saw it coming: It isn’t a proxy. You know the score by now.
It can’t aggregate. The NuGet Remote Feed exposing one monolithic repository: the one you have on your machine. It can’t aggregate NuGet packages from remote repositories, or expose number of local repositories (separated for security reasons, for example).
You can’t attach your own metadata. Let’s say you want to annotate some package with compatibility information (e.g. works with certain browsers). No, can’t do.
The repository is very simplistic. It doesn’t provide any web interface; it browsable and searchable only from a client – be it Visual Studio or the command line interface (pretty basic by itself).
Even the VS search interface is very basic (all you have is arbitrary sorting and free text search). It should be enough for starters but lack of searching inside the packages or by properties (from the previous bullet) will bite you eventually.
The security scheme is even less than simplistic. All that’s required to authenticate a deployment/delete of all the users at once is an API key. What about separation of duties? Some users should only be able to read, others only to annotate with metadata (QA team that tests compatibility in my previous example), and only small subgroup – to deploy. The all-or-nothing scheme is definitely insufficient.
Storage format is suboptimal:
- The packages are stored on the filesystem in a naive simple format. That fine for small repository,but as you grow, you’d expect storage which more optimized for binaries.
- The metadata is not indexed. Again, fine for small repo, troubles are foreseen when it comes to scaling.

So, is there a good alternative to NuGet Remote Feed to be your in-house Gallery for NuGet packages? We, the proud makers of Artifactory, believe there is.

Meet Artifactory

Artifactory is an enterprise-grade Binary Repository that centralizes all aspects of managing software binaries. That means that we tackle all the problems mentioned above. We are developing Artifactory since 2006. Being used by millions of users for storing, sharing and managing binaries, we have gathered great feedback from our users.

That’s what we’ve learned:

Binary packages are different from sources (by being big and binary) and deserve smart storage.
Binary packages are usually archives (be it jars, zips, rpms or nupkgs). They should be browsable and searchable without the need to download them locally to developer’s machine.
Big public repositories exist on the net, they need to be proxied smartly (variations, auditing, managing)
Users come in different flavors. Their permissions should match possible responsibilities (and in the case of binary packages they are different from other cases).
A binary repository holds critical information, it should be rock-solid, backed up, and DRP ready.
Your software ends up being a package. We know how to help you…
- build it in a reproducible manner, integrating with your build tools and your build server.
- stage it to ensure the best quality.
- distribute it to your customers.

You get the gist behind Artifactory by watching this 2.5 minutes video:
If you’re not in the mood for movies (or ran out of popcorn), here’s quick recap:
As you see, instead of working with a number of NuGet Feeds (NuGet Gallery, Orchard Gallery, Remote Feeds from co-workers and from different teams) developers work with exactly one repository. It simplifies setup and daily work and centralizes management and maintenance.
The work is bi-directional, the users resolve their 3rd party dependencies from Artifactory and deploy their created packaged into it.

Now let’s add a build server to the picture (literally):
Yup, with numbers this time. So, here we go:

Developers find and fetch new 3rd party packages from Artifactory in Visual Studio. The packages are downloaded from Artifactory to the developer’s machine. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds. On developer machines packages.config is updated with the list of used packages.
Developers commit their code and packages.config (but not the binaries) to VCS.
The Build server (as I already mentioned, TeamCity now supports NuGet) takes the changes from the VCS.
It builds the solution and packs the produced artifacts as NuGet packages.
During the build it fetches the needed packages from Artifactory. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds.
Once the packages are built they are deployed to Artifactory.

Built packages in Artifactory can be used by other teams (as their 3rd party dependencies), by QA for running tests and even by the end users (Chocolatey FTW), all this with fine-grained permissions and robust promotion procedures (moving a package between repositories with different visibility rules).
You know what? It deserves dedicated how-to blog post. I’ll link it here once published.

Assuming you’ve read up to this point, you’ve gathered that starting from Artifactory version 2.5.0 we are proud to serve the .NET world with full NuGet support. We can proxy any remote NuGet feed (starting with NuGet Gallery, of course), we can host the packages that aren’t found on any remote NuGet feed, we can host the packages you produce and we can aggregate any number of repositories of any kind under single a URL. We provide you with an awesome UI for configuring your repositories, browsing and searching for your packages. We also feature smart storage that enables attaching searchable metadata on top of your binaries. We can do it all on the cloud with our SAAS version.

Hopefully , you’re convinced by now and probably looking for the download link on our site (here’s it, BTW, click on “Evalution”). If not, give it a try by playing with our live demo. Look at the nuget-gallery cache: that’s how we proxy the NuGet Gallery. You’ll find some of the packages saved locally; once you’ve selected a package, you’ll see all kinds of information about it: its name and size, who deployed it to Artifactory, where it came from (from NuGet Gallery, naturally for this is the NuGet Gallery cache) and the operations you can perform on this package (as anonymous the selection is naturally limited). Clicking on the triangle in the tree will open the package and let you dive into its content, including downloading specific files from the archive (click for full-size image):

We, at JFrog, believe that Artifactory is the missing piece of the puzzle for a robust, agile .NET dependency management, which can make the development process easier compared to other alternatives.
We’ll be happy to receive any insights, thoughts and comments on the ideas presented in this blog and/or your experience using Artifactory together with NuGet.

Filed under: Artifactory, JFrog Tagged: artifactory, jfrog, nuget, teamcity

MVC meet JavaScript, JavaScript meet MVC

Brian Sam-Bodden — Tue, 27 Dec 2011 18:00:00 CST

For years the software community has been pushing the MVC architectural pattern to organize and separate the concerns of our applications. So far we seem to have done a decent job of accomplishing that based on the enforcement of the pattern in the most successful web frameworks such as Rails, Grails, JSF, Struts and many others. The last frontier for MVC seems to be the sometimes convoluted world of JavaScript; the client tier of our web applications. Although frameworks like jQuery, Prototype, Scriptaculous, ExtJS, DOJO and others have greatly helped in cleaning up and structuring the client tier, there's still much to be desired. In recent years several micro-frameworks have appeared that aim to put an end to the madness of the JavaScript client tier world. In this article we'll explore the most prominent players and see how their usage impacts modern web development.

Blurting out a solution when so many do not understand the problem would not help. So, let’s start at the beginning… Before the web, but not that long ago, we had client-server applications in which the client was typically a desktop application and the server was a monolithic application that babysat a database. In those days, mastery of programming environments like Delphi, Visual Basic, Powerbuilder and Oracle Forms was in hot demand. Visual component environments allowed, to an extent, the separation of presentation logic from the database. I lived through that period and I remember the feeling that the user interface portions of the application seemed pretty orderly via the use of components but the server side was a disaster.

Then along came the web and, overwhelmed by the new environment, protocols, and stateless nature of the new beast, we somehow forgot everything we’ve learned in the past. The first applications were monolithic messes both in the front-end and the back (remember CGI scripts?). Then eventually we came to our senses and a myriad of web frameworks started to appear. Most of the frameworks implemented a version of the Model-View-Controller (MVC) pattern.

So while things got better at the web server level, the UI remained the wild frontier of web development, where JavaScript was the lingua franca and not many could speak it. JavaScript, an amazingly powerful but misused and misunderstood language, was caught smack in the middle of the so-called browser wars. These wars were centered around the differences and inconsistencies on the browser’s DOM (Document Object Model), which also made for a mess at development time. Slowly, the community reacted by creating frameworks like jQuery, Scriptaculous, Prototype, MooTools, Dojo, Ext JS and others that could hide the warts of the DOM implementations (and also mask some of the ugly parts of JavaScript).

These frameworks have simplified the way we deal with the DOM and JavaScript. They’ve given us the ability to encapsulate visual aspects of the DOM and deal with events in a uniform fashion. But in spite of our best efforts, complex client-side functionality still gets messy very fast. Applications typically end up as a spaghetti mess of callbacks between the DOM, your code and the server-side application. A more structured approach to building rich JavaScript applications is needed.

Model-View-Controller

The MVC pattern made its first appearance in 1979 when Trygve Reenskaug described a new architecture for developing interactive applications while working on Smalltalk at Xerox PARC. MVC breaks a functional slice of an application into three basic components: models, views and controllers.

In the MVC world, the model is responsible for maintaining an aspect of the state of the application. This “state” could be transient, lasting only while dealing with an interaction with the user or persistent and stored outside of the application’s realm. The model is responsible for encapsulating the operations relevant to the entity it represents. This encapsulation carries with it the responsibility of maintaining the integrity of the data that represents the state of the model.

The view encompasses a user interface element that typically reflects the state of the underlying model. Finally, the controller handles input events from the UI and determines a set of operations to be performed upon the model. Changes to the state of the model are then reflected on the view. How coupled the components of the MVC triumvirate are depends on implementation specific decisions.

A Simple Example: A Shopping Cart

To illustrate the problems the new breed of MVC frameworks aim to solve, I will first build a dynamic single page JavaScript application without them; a simple drag-n-drop jQuery based shopping cart.

Figure BSB-1 – jQuery Shopping Cart Project

As shown in Figure BSB-1 the project consists of an HTML page called shopping_cart.html which uses the JavaScript files integrallis.commerce.js and integrallis.jquery.cart.js. These files are an attempt to separate the business objects from the visual aspects of the application. In integrallis.commerce.js I’ve implemented a simple shopping cart as shown in Listing BSB-1 (the full listing for this example can be found at https://github.com/bsbodden/jquery-shopping-cart)

var INTEGRALLIS = {};
INTEGRALLIS.commerce = {};

INTEGRALLIS.commerce.ShoppingCart = function(options) {
	// initializes the cart
	this.items = {};
	
	// callbacks
	var defaults = {
		removeAll: function() {},
		updateTotal: function() {},
       	updateItemQuantity: function() {},
       	removeItem: function() {}
	};
	
	// merge options with defaults
	for (property in defaults) { 
		if (!options.hasOwnProperty(property)) { 
			options[property] = defaults[property]
		} 
	}
	
	this.options = options;
}

// removes all items from the cart
INTEGRALLIS.commerce.ShoppingCart.prototype.clear = function() {
	this.items = {};
	this.options.removeAll();
	this.options.updateTotal();
}

// returns the value of all items in the cart
INTEGRALLIS.commerce.ShoppingCart.prototype.total = function() {
	var sum = 0.0;
	for (var index in this.items) {
		var item = this.items[index];
		sum = sum + (item.price * item.quantity);
	}
	return sum;
}

// returns the number of unique items in the cart
INTEGRALLIS.commerce.ShoppingCart.prototype.itemsCount = function() {
	var size = 0, key;
    for (key in this.items) {
        if (this.items.hasOwnProperty(key)) size++;
    }

	return size;
}

// returns whether the cart is empty or not
INTEGRALLIS.commerce.ShoppingCart.prototype.isEmpty = function() {
	return this.itemsCount() == 0;
}
	
// adds a new item to the cart or increases the quantity of an existing item
INTEGRALLIS.commerce.ShoppingCart.prototype.add = function(id, price, quantity) {
	var is_new;
	quantity = (typeof(quantity) != 'undefined') ? quantity : 1;
	if (this.items.hasOwnProperty(id)) {
		var item = this.items[id];
		item.quantity = item.quantity + quantity;
		this.options.updateItemQuantity(id);
		this.options.updateTotal();
		is_new = false;
	}
	else {
	    this.items[id] = { quantity : quantity, price : price };
	    this.options.updateTotal();
	    is_new = true;
    }
    return is_new;
}
		
// increases the quantity of an item in the cart
INTEGRALLIS.commerce.ShoppingCart.prototype.increase = function(id, quantity) {
	quantity = (typeof(quantity) != 'undefined') ? quantity : 1;
	if (this.items.hasOwnProperty(id)) {
		var item = this.items[id];
		item.quantity = item.quantity + quantity;
		this.options.updateItemQuantity(id);
		this.options.updateTotal();
	}
}
		
// decreases the quantity of an item in the cart
INTEGRALLIS.commerce.ShoppingCart.prototype.decrease = function(id, quantity) {
	quantity = (typeof(quantity) != 'undefined') ? quantity : 1;
	if (this.items.hasOwnProperty(id)) {
		var item = this.items[id];
		if (item.quantity >= quantity) {
		    item.quantity = item.quantity - quantity;
	    }
	    else {
		    item.quantity = 0;
	    }
		this.options.updateItemQuantity(id);
		if (item.quantity == 0) {
			delete this.items[id];
			this.options.removeItem(id);
		}
		this.options.updateTotal();
	}
}
		
// returns the quantity of an item in the cart
INTEGRALLIS.commerce.ShoppingCart.prototype.quantityFor = function(id) {
	return this.items.hasOwnProperty(id) ? this.items[id].quantity : 0;
}

Listing BSB-1

The shopping cart provides the ability to maintain a list of items and their associated quantities as well as their total value. In order to reflect changes to the cart, I’ve created a set of callback functions (initialized to no-op functions) in the Constructor function of the cart. In integrallis.jquery.cart.js shown in Listing BOD-2, I wire the shopping cart object to the shopping_cart.html page. I start by instantiating the cart and passing the shopping cart UI callback functions. The callbacks use jQuery to update elements of the page to reflect changes in the shopping cart.

Interaction with the cart will occur when the user drops an HTML element representing an item into the cart “drop” area. To accomplish the drag-n-drop functionality I’m using jQuery’s draggable and droppable functions to enable the elements with class “product” to be dragged and dropped into the element with id “cart”.

// create a shopping cart instance, set all callbacks
var myShoppingCart = new INTEGRALLIS.commerce.ShoppingCart({ removeItem : removeItem,
	                                                         removeAll : removeAll,
	                                                         updateTotal : updateCartTotal,
	                                                         updateItemQuantity : updateItemQuantity });
	
// shopping cart UI callbacks

function updateItemQuantity(id) {
	$('#cart #' + id)
	    .children('#qty')
	    .text(myShoppingCart.quantityFor(id))
	    .effect("highlight", {}, 1500);		
}

function updateCartTotal() {
	$('#total').text(myShoppingCart.total()).effect("highlight", {}, 1500);
}

function removeItem(id) {
	$('#cart #' + id).effect("puff", {}, "slow", function(){ $(this).remove(); });
}

function removeAll() {
	$('#cart .product').effect("puff", {}, "slow", function(){ $('#cart').empty(); });
}

function decorateForCart(item, id) {
	   item.append(' (1)')
           .append(' +')
           .append(' -');
       item.children('#add').click(function() {
	       myShoppingCart.increase(id);
       });
       item.children('#remove').click(function() {
	       myShoppingCart.decrease(id);
       });
}

$(document).ready(function() {
	// make products draggable
	$(".product").draggable({ helper: 'clone', opacity: "0.5" });
	
	// allow products to be dropped in the cart
	$("#cart").droppable({ accept: '.product', 
	                       drop: function(ev, ui) {
						       var item_dropped = ui.draggable;
						       var id = item_dropped.attr('id');
					           var price = item_dropped.attr('price');	      
						       if (myShoppingCart.add(id, price)) {
							       var item = item_dropped.clone();
							       decorateForCart(item, id);
							       $(this).append(item);
						       }
		                   }
	});
	
	// jquery-fy the 'clear shopping cart' button
	$('#dump').button().click(function() { myShoppingCart.clear() });
});

Listing BSB-2

Finally to put the cart to the test in shopping_cart.html where I provide a div representing the shopping cart and a list of “products” that can be dropped into the cart (Listing BOD-3). The “data” for the products is taken from the attributes of the <li> “product” elements.

<html>
<head>
    <link href='http://fonts.googleapis.com/css?family=Muli' rel='stylesheet' type='text/css'>
    <link type="text/css" href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.16/themes/cupertino/jquery-ui.css" rel="stylesheet" />
    <link type="text/css" href="styles/shopping_cart.css" rel="stylesheet" />
    
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
    <script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.16/jquery-ui.min.js"></script>
    <script src="src/integrallis.commerce.js"></script>
    <script src="src/integrallis.jquery.cart.js"></script>
</head>
<body>
    <h2>jQuery Shopping Cart</h2>

    <h3>Available Products</h3>
    
    <ul class="products" id="products">
        <li class="product ui-state-default" id="item_0" price="100.0">Product 0 - $100</li>
        <li class="product ui-state-default" id="item_1" price="110.0">Product 1 - $110</li>
        <li class="product ui-state-default" id="item_2" price="120.0">Product 2 - $120</li>
    </ul>

    <h3>Your Shopping Cart</h3>

    <div id="cart">
    </div>
    <p>Total: $<span id="total">0.00</span></p>
    <button id="dump">Dump the cart!</button>
</body>
</html>

Listing BSB-3

As the elements are dropped into the cart they are cloned and decorated with a couple of links to increase/decrease the quantity of the particular item in the cart.

In the callbacks we are dealing with dynamically updating the total value of the cart and increasing and decreasing the displayed quantities of an item. Also notice that if we drop an item that already exists in the cart, rather than creating a new item we simply increase the quantity of the existing item. Similarly, when an item’s quantity reaches zero we remove the item from the cart. All these operations are performed with jQuery’s animation capabilities to keep the user aware of changes to the cart.

Figure BSB-2 – jQuery Shopping Cart in Action

Shortcomings of the jQuery Shopping Cart

I am obviously taking some shortcuts for the sake of the example such as assuming that the available products magically appeared in the page (in a real application they would be retrieved from a server). But other than that, at first glance the implementation of the shopping cart seems to work as advertised and even seems well thought-out. That is, until we start asking the hard questions…

From the usability point of view, the first thing that jumps to mind is, what happens when the user clicks the refresh button? Since everything is just being kept in memory the answer is that it simply goes away. The user will end up with an empty shopping cart. The same applies to the back button; both will have the effect of dumping the cart!

One of the features that simple page applications try to achieve is the minimization of round trips to the server. Since the page is not to be refreshed by the user actions on the page, AJAX calls will have to be made to send the state of the cart back for the server for storage.

On the code organization front, even though I have taken steps to separate the business logic from the UI interaction code it is still obvious that for anything more complex we would end up with an unmanageable mess of UI callbacks all of which are currently living in the global JavaScript scope.

Another glaring problem is the coupling of the concepts of the shopping cart and the items in the shopping cart. The items are simple JavaScript objects with no behavior. Their reflection on the UI is controlled by the decorateForCart function as shown in Listing BSB-2. Ideally our cart items would be good object-oriented citizens and encapsulate both data and behavior.

Although it might seem that I’m being nitpicky about this implementation, the small issues above are representative of the types of problems that can rapidly turn a JavaScript application into an unbearable mess.

MVC to the Rescue: Introducing Spine

Spine is one of the many lightweight JavaScript micro MVC framework choices that have popped into the scene.

From the Spine Website:

"The library is written in CoffeeScript, but doesn't necessarily require CoffeeScript to develop applications. You can use CoffeeScript or JavaScript, whichever language you're most familiar with. Spine is tiny, the library comes in at around 500 lines of CoffeeScript, that's about 2K minified & compressed. Being lightweight and simple is fundamental to Spine."

Spine’s take on the MVC pattern might be slightly confusing for those accustomed to more traditional MVC implementations. Spine only provides Models and Controllers.

Figure BSB-3 shows the typical directory structure used for a Spine application. The application consists of 1 model, 2 controllers and a JavaScript file to bootstrap the application.

Figure BSB-3 – jQuery/Spine Shopping Cart Project

Models

In Spine, a model represents an entity, which has state (data) and logic to manipulate said data. A model can be saved, retrieved and even queried for (to an extent). Models can be stored in memory (the default) or use any of the storage modules provided such as Ajax or HTML5 Local Storage. Listing BSB-4 shows the basic Item model that represents an item in our shopping cart application.

// Create the Item model.
var Item = Spine.Model.sub();
Item.configure("Item", "name", "pid", "price", "quantity");

Listing BSB-4

A Spine model “class” is created using the setup method of Spine.Model, which takes the name of the model and an array of properties. To make the model persist between page reloads we extend Item with the Spine.Model.Local module as shown in Listing BSB-5.

// Persist model between page reloads.
Item.extend(Spine.Model.Local);

Listing BSB-5

The extend method adds class properties to the model. We now have an object that can be created, saved and retrieved from the browser local storage.

To add behavior to our model we use the include method which adds instance properties. Listing BSB-6 shows the four methods that we need for our Item model; increase, decrease, total and label.

// Instance methods
Item.include({
    //
    total: function() {
        return (this.price * this.quantity);
    },
    //
    increase: function(quantity) {
        quantity = (typeof(quantity) != 'undefined') ? quantity : 1;
        this.quantity = this.quantity + quantity;
		this.trigger("quantityChanged");
    },
    //
    decrease: function(quantity) {
        quantity = (typeof(quantity) != 'undefined') ? quantity : 1;
        if (this.quantity >= quantity) {
            this.quantity = this.quantity - quantity;
        }
        else {
            this.quantity = 0;
        }
		this.trigger("quantityChanged");
    },
    // 
    label: function() {
        return (this.name + " - $" + this.price);
    }
    
});

Listing BSB-6

To instantiate an Item we use the create method, which takes object literals for the parameters. A new model can be persisted using the save method. When a model is saved it is assigned an identifier that can be retrieved via the id property. All three are demonstrated in Listing BSB-7.

var item = new Item({name: "Product 1", pid: "0001" , price: 100.0, quantity: 1});
item.save();
alert("Just saved Item with id => " + item.id);

Listing BSB-7

Controllers

Controllers in Spine are a combination of a traditional MVC controller and a view. Therefore controllers are in charge of rendering and manipulating one or more models in the context of the controller’s functionality.

Our first Spine controller will deal with the rendering and manipulation of an individual Item. The CartItem controller will deal with user interface events to increase and decrease the quantity of an Item while keeping the user abreast of the changes. Listing BSB-8 shows the CartItem controller contained in the file cart_item.js.

jQuery(function($){
    window.CartItem = Spine.Controller.sub({ 
        
        init: function(){
            var cartItem = this;
            
            this.item.bind("quantityChanged", function() { cartItem.updateQty() });
            
            $('#item_' + this.item.pid + ' .add').live('click', function(e) { 
                cartItem.add(); 
                e.preventDefault(); 
            });
            
            $('#item_' + this.item.pid + ' .remove').live('click', function(e) { 
                cartItem.remove(); 
                e.preventDefault(); 
            });
        },
        
        render: function(){
            this.el = $.mustache($("#cartItem").html(), this.item);
            
            return this;
        },
    
        // event handlers
        add: function(e) {
            this.item.increase();
        },
            
        remove: function(e) {
            this.item.decrease();
        },
    
        // ui methods
        updateQty: function() {
            $('#item_' + this.item.pid + ' #qty')
                .text(this.item.quantity)
                .effect("highlight", {}, 1500);
        }
    });
})

Listing BSB-8

Spine controllers are created using the create method of Spine.Controller, which takes an object literal that sets a wide variety of properties. In Listing BSB-8 we see the first property passed is named proxied and takes an array of Strings. The values are the name of the controller methods that need to be guaranteed to execute in the controller’s context (otherwise you’ll noticed that sometimes the this variable points to something other than the controller).

Events

In the controller’s init method we bind a custom event called "quantityChanged" of the enclosing Item model to the controller’s updateQty method.

this.item.bind("quantityChanged", function() { cartItem.updateQty() });

Listing BSB-9.1

The model’s "quantityChanged" is triggered in both the increase and decrease methods of the Item model:

this.trigger("quantityChanged");

Listing BSB-9.2

Also, in the init method, we wire the ‘add’ and ‘remove’ links of the CartItem to the add and remove methods which invoke the underlying Item model add and remove methods.

Rendering

To render the associated view for the CartItem controller we use the render method. In the render method we use jQuery to locate the element with id cartItem, which is a template, embedded in the HTML page as shown in Listing BSB-10.

render: function(){
            this.el = $.mustache($("#cartItem").html(), this.item);
            
            return this;
        }

Listing BSB-10

The cartItem template is a Mustache.js Template that enables the creation of markup templates containing binding expressions. The mustache method clones the template contents and replaces the binding expressions ({{exp}}) with the values of the object passed, in our case in Listing BSB-11 the controller’s enclosed Item model.

    
    <!-- Mustache :-{)~ Template for CartItem -->
    <script type="text/x-mustache-tmpl" id="cartItem">
        <li class="product ui-state-default" id="item_{{pid}}" price="{{price}}">
            {{label}}
            (<span id="qty">{{quantity}}</span>)
            <a href="#" class="add">+</a>
            <a href="#" class="remove">-</a>
        </li>
    </script>

Listing BSB-11

To test our controller we need to instantiate an Item model and pass it to the controller. We can then render the controller on the DOM of an HTML page as shown in Listing BSB-12.

    
	var item = new Item({name: "Product 1", pid: "0001" , price: 100.0, quantity: 1});
	var view = new CartItem({item: item});
	$("#item").html(view.render().el);

Listing BSB-12

A controller has an el (element) property that reflects the DOM element that will be the target of the rendering. The test page shown in Figure BSB-4 reveals that the CartItem controller accomplishes the functionality required to deal with a single Item model.

Figure BSB-4 – CartItem Controller in Action

Shopping Cart Controller

With the CartItem controller in place we can now concentrate on the ShoppingCart controller (shown in Listing BSB-13), which will manage a collection of Item models.

Internally the ShoppingCart keeps the items dropped in the items property, which is initialized with an object literal in the init method. Also in the init method we set up the drop area for the drag-n-drop operation.

Persistence

In the init method we also query for any existing Item models that were previously saved by using the all method of the model (Item.all()). If any items are found then we simply add them to the cart using the addItem method.

Methods

The cart provides several aptly named methods to fulfill its functionality. The clear method iterates over all the contained Item models and invokes the destroy method which the cart listens to in order to animate the removal of rendered ItemCarts from the cart.

The methods total, isEmpty and itemsCount return the total dollar amount for the cart, whether the cart is empty and the count of all unique items in the cart respectively.

The drop method handles drop events similarly to the pure jQuery shopping cart implementation; it either creates a new item or increases the quantity of an existing item. The main difference being that the drop creates an Item model that is then rendered using the CartItem controller. Concerns separated!

    
jQuery(function($){
    window.ShoppingCart = Spine.Controller.sub({
        el: $("#theCart"),
        
        init: function() {
            var cart = this;
            this.items = {};
            $.each(Item.all(), function(){ cart.addItem(this); });
            this.el.droppable({ accept: '.product', drop: this.proxy(this.drop) });
            $('#dump', this.el).live('click', function() { cart.clear(); });
        },
        
        // removes all items from the cart
        clear: function() {
            $.each(this.items, function(){ this.destroy(); });
            this.items = {};
            this.updateCartTotal(); 
        },
        
        total: function() {
            var sum = 0.0;
            $.each(this.items, function(){ sum += this.total(); });

            return sum;
        },
        
        isEmpty: function() {
            return this.itemsCount() == 0;
        },
        
        itemsCount: function() {
            var size = 0;
            var items = this.items;
            $.each(items, function(){ if (items.hasOwnProperty(this)) size++; });

            return size;
        },
        
        drop: function(ev, ui) {
            var item_dropped = ui.draggable;
            var pid = item_dropped.attr('id');
            var price = item_dropped.attr('price');
            var name = item_dropped.attr('name');

            if (this.items.hasOwnProperty(pid)) {
                this.items[pid].increase();
            }
            else {
                var item = Item.create({name: name, pid: pid, price: price, quantity: 1});
                this.addItem(item);
                $(".items").append(CartItem.init({item: item}).render().el);
            }
        },
        
        render: function() {
            this.el.html($.mustache($("#shoppingCart").html(), {}));
            
            $('#dump').button();

            $.each(this.items, function(){ 
                $(".items").append(CartItem.init({item: this}).render().el);
            });
            
            this.updateCartTotal();
        },
        
        removeItem: function(item) {
            $('#item_' + item.pid).effect("puff", {}, "slow", function(){ $(this).remove(); });
        },
        
        updateCartTotal: function() {
            $('#total').text(this.total()).effect("highlight", {}, 1500);
        },
        
        removeIfQuantityZero: function(item) {
            if (item.quantity == 0) {
                this.removeItem(item);
                delete this.items[item.pid];
                item.destroy();
            }
        },
        
        addItem: function(item) {
            this.items[item.pid] = item;
            item.bind("quantityChanged", this.proxy(this.updateCartTotal));
            item.bind("quantityChanged", this.proxy(this.removeIfQuantityZero));
			item.bind("quantityChanged", function() { item.save() });
            item.bind("destroy", this.proxy(this.removeItem));
            item.save();
            this.updateCartTotal();
        }

    });
})

Listing BSB-13

At the end of Listing BSB-13 we can see that the addItem method sets several listeners to various events on the enclosed Item models. One of the advantages of frameworks like Spine is the ability to deal with standard changes in related models without having to tightly couple their implementation to the controllers. Also, note that in addItem we save the created models so that we can later retrieve them in case of a page refresh.

Tying It All Together

Finally, to kick everything in motion the application.js file begins by fetching any previously stored Item records, making the sample products draggable, creating a cart and rendering it.

  
jQuery(function($){
	Item.fetch();
	$(".product").draggable({ helper: 'clone', opacity: "0.5" });
	var cart = new ShoppingCart();
	cart.render();
});

Listing BSB-14

Conclusions

In this article we’ve covered the basics of single page web applications. These applications are nothing new; we’ve attempted them in the past but now we have frameworks like jQuery to deal with browser disparities and Spine to inject the MVC sense of order.

The new breed of MVC frameworks allows us to create clean, simple, componentized and event-oriented applications. We can separate our applications’ logic into cleanly defined areas of concern, cache data on the client side, minimize AJAX calls and provide many features that we would otherwise have to implement ourselves. We can now build rich state of the art JavaScript applications that can rival their desktop counterparts.

References

Spine: http://spinejs.com/
jQuery: http://jquery.com
jQuery UI: http://jqueryui.com
Mustache Templates Plugin: https://github.com/janl/mustache.js
Example Code:
https://github.com/bsbodden/jquery-shopping-cart
https://github.com/bsbodden/jquery-spine-shopping-cart

About The Author

Brian Sam-Bodden is avid developer, author and recognized international speaker that has spent over fifteen years working with object technologies.

He is the president and chief software architect for Integrallis, where he focuses on building great applications with Ruby, Rails, Groovy, Grails and Java. Brian is the creator of many Ruby open source utilities including the JRuby XML tool “Excemel”, the Component-Oriented Ruby Framework “Trellis” and numerous Ruby on Rails Plugins.

He is also the author of Beginning POJOs: Spring, Hibernate, JBoss and Tapestry and has also co-authored the Apress Java title Enterprise Java Development on a Budget: Leveraging Java Open Source Technologies and contributed to O'reilly's 97 Things Every Project Manager Should Know.

Constant Pain with Non-Constant Constants

Ken Sipe — Tue, 20 Dec 2011 14:55:01 CST

This thought has crossed my mind before... since it is on my mind now I thought I might share. Perhaps it will help someone on their software development career path... or perhaps it is just me venting...

The Art of Coding

One of the greatest challenges of coding is being able to understand it. Skill in coding is being able to understand the problem well enough to create a solution (otherwise known as coding). Great Skill is being able to write a solution so that someone else (including the future you) can read and understand what it is they are looking at. This concept isn't new, but yet we seem to be either challenged with the concept... or distracted. Even back in 1952 (yes 1952!!), in the computer history archive a set of lecture notes which in them states: "It is quite difficult to decipher coded programs even with notes and even if you yourself made the program several months ago." It also states: "I think the answer is simple: to make it easy one must make coding comprehensible". Well said!

The Point of Constants

One area that seems to be a constant :) struggle is constants. There are at least a couple of justifications for using constants I can think of: 1) a point of reference for tools (autocomplete with intellisense in IDEs and refactoring), 2) A common holder of value for which if it changes will be realized across the system. The main point of a constant isn't DRY or reuse... a common point of confusion for some reason.

IMO the stronger argument is point 2. if you were coding in VI or emacs, point 1 may not be of value, but point 2 certainly would be. If you name the reference of a constant the same name as the value of the constant... then you lose any the value of point 2.
It must also be said that constants, although they add value, they make code less readable (see examples below). If your constants have no value, they don't achieve points 1 and 2, then they need to be whacked!

Examples of Losing with Constants

This is real production code... some variables changed to obscure identify.



        // example 1   Really??  where does this end?
 String EMPTY_STRING = "";

        // this isn't readable... it takes minutes (instead of seconds) to understand its value
        // example 2
 String ADVERTISE_PROPERTY_PREFIX = OUR_PROPERTY_PREFIX + "advertise.";
 String ADVERTISE_TRADER_ADDRESS_PROPERTY = ADVERTISE_PROPERTY_PREFIX + "host";
 String ADVERTISE_CONTEXT_PROPERTY = ADVERTISE_PROPERTY_PREFIX + "context";
 String ADVERTISE_TYPE_PROPERTY = ADVERTISE_PROPERTY_PREFIX + "type";

       // example 3
       // here's a common one
static final String TABLE_NAME = "email";

 // Column names for all the fields in the email table
static final String ID = TABLE_NAME + ".email_id";
static final String CUSTOMER_ID = TABLE_NAME + ".customer_id";
static final String ADDRESS = TABLE_NAME + ".address";
        // ...
static final String FIELDS = ID + ", " + CUSTOMER_ID + ", " + ADDRESS + ", " + CREATED_BY + ", " + LAST_MOD_BY
  + ", " + CREATE_DATE + ", " + LAST_MOD_DATE + ", " + TERM_DATE + ", " + WEB_USER_ID;

 // List of fields names with formatting used for select statements
static final String SELECT_FIELDS = ID + ", " + CUSTOMER_ID + ", " + ADDRESS + ", " + CREATED_BY + ", "
  + LAST_MOD_BY + ", " + "TO_CHAR(" + CREATE_DATE + ",'" + DATE_TIME_FORMAT + "')," + "TO_CHAR("
  + LAST_MOD_DATE + ",'" + DATE_TIME_FORMAT + "')," + "TO_CHAR(" + TERM_DATE + ",'" + DATE_TIME_FORMAT
  + "')," + WEB_USER_ID;

 // SQL used to create a new EMAIL record
static final String CREATE_SQL = "INSERT INTO " + TABLE_NAME + " (" + FIELDS
  + ") VALUES (?,?,?,?,?,SYSDATE,SYSDATE,NULL,?)";

 // SQL used to retrieve an email record by id
static final String RETRIEVE_BY_ID_SQL = "SELECT " + SELECT_FIELDS + " FROM " + TABLE_NAME + " WHERE " + TERM_DATE
  + " IS NULL AND " + ID + "=?";

For example 1... where does it end? would you create a constant A = "A"? When can EMPTY_STRING not be an empty string or ""? In this case, we get no value and less readability.

Example 2 is the chaining of constants, commonly found when there are a large number of property values being retrieved from a file or sys env. While not as bad as example 1, it is less readable. Having a prefix constant may be of value, but rarely is the chaining of constants of value. It isn't worth trading the potential for a company name change or something similar for readability. Or put another way, it isn't worth trading something that is unlikely to happen for something else which is more likely to be needed (that being the readability of the code)

Example 3 I see all the time... argghhhhh... please... just stop!

Examples of good constants would include, urls and database connections strings or a project defined standard date format.

May your constants always be final!

Advanced Spock Techniques

Ken Sipe — Wed, 14 Dec 2011 17:49:45 CST

In recent years there have been a couple of tools that stand out when it comes to helping me be productive. One of those is the groovy test framework Spock. It is worthy of an introductory blog post... but that isn't this post. One of the challenges to Spock is the documentation isn't a complete as one would hope. The Spock Basics is fantastic when getting started but it is what it claims to be basics. I've been speaking with NFJS on the subject of Spock for the last half of 2011. On 2 occasions I've had the look our red shirted friend in the picture has when asked about controlling aspects of mocks in spock... meaning I didn't have the answer. In this case, google didn't discover it either... a quick email to the Peter Niederwieser reveals the previously undocumented solution. (at least that's my understanding)... and thanks Peter!

Here we will discuss 2 advanced aspects of mocking with spock. If you need more details on mocking in general hit the Spock site... or wait patiently for a future post:)

Problem Setup:

When mocking with Groovy we get 2 benefits from the verification of the mock. 1) order of execution, 2) failure if another method on the mock was executed but was not setup in the demand configuration.


void testDeposit() {

        Account account = new Account(TEST_ACCOUNT_NO, 100)

        def mock = new MockFor(AccountDao)
        mock.demand.findAccount(TEST_ACCOUNT_NO) { account }
        mock.demand.updateAccount(account) { assertEquals 150, account.balance  }

        def dao = mock.proxyDelegateInstance()
        def service = new AccountServiceImpl(dao)

        service.deposit TEST_ACCOUNT_NO, 50
        mock.verify dao
   }

Spock Cleanup

A similar solution in Spock might look like this:


    def "deposit into account test refactor"() {

        def mock = Mock(AccountDao)
        def service = new AccountServiceImpl(mock)

        when:
        service.deposit nmr, amt

        then:
        mock.findAccount(_) >> account
        1 * mock.updateAccount(_)
        0 * mock.createAccount(_)

        and:
        account.balance == total

        where:
        nmr   | account                 | amt | total
        "101" | new Account("101", 100) | 50  | 150
        "101" | new Account("101", 0)   | 50  | 50
    }

However this solution doesn't guarantee order nor do we want to use the 0 * mock. on all possible methods.

Spock Mock Solution

It turns out that we can repeatedly use the then: dsl to determine the order of events. So we can modify the Spock code above with the following changes.


        then:
        1 * mock.findAccount("1234") >> account

        then:
        _ * mock.updateAccount(_)

In this case we are saying that the findAccount() method will be called exactly once and return the account object and "then" the updateAccount will be call any number of times with any type of argument. The test will fail if the updateAccount is ever called prior to the findAccount being invoked first.

We can use this same technique to solve the strictness concern we have as well by adding one more then: block of code as outlined below:


    then:
        1 * mock.findAccount("1234") >> account

        then:
        _ * mock.updateAccount(_)

        then:
        0 * mock._

In this case the last 0 * mock._ says to expect nothing else. Ahhh.. the beauties of a dynamic language:)

Happy Coding... may your builds never fail and your tests always green bar!

Git and GitHub Support in JetBrains YouTrack

Matthew McCullough — Tue, 22 Nov 2011 22:54:00 CST

I recently had the chance to get a demo of JetBrains products’ Git integration and to meet some of the JetBrains development team in person at Øredev in Malmö, Sweden.

I love seeing things integrate better with the GitHub API. It really is fantastic to see what is possible when a rich Internet application additionally becomes a platform for apps that extend the core value proposition. GitHub has executed on this quite well with their API, now at version 3.0 and with features like gist, repo, user, and even organization management.

I like to keep abreast of just about everything that happens in the Git world, and the JetBrains folks certainly as very active there. The JetBrains issue tracking tool, YouTrack, makes extensive use of the GitHub API and has much in the way of Git love. I’ve been pointing folks at the YouTrack overview video if they want a quick summary and the integration demo, if the GitHub facet is the attraction.

It is always fun to ask a vendor for their view of what’s important and what’s next. The JetBrains team said (paraphrased):

Keyboard-centric approach: All common actions have handy shortcuts.
Smart issue search: Search with queries similar to everyday language aided by completion and highlighting. For example, type for me unresolved to filter down to open issues assigned to me.
Batch modification commands similar to search queries: Select multiple issues and resolve them all by typing fixed assignee Matthew
Report from everywhere: You can report issues via email or any third party application via a REST API.
Full customization: You can define and use new attributes for your bug tracking and create workflows using a YouTrack workflow editor with a domain-specific language.
Integration with VCSs via TeamCity and native integration with GitHub: You can specify an Issue ID and command to be applied to the issue right from commit comment. No opening the bug tracker just to change a bug state.
REST API to perform any action programmatically: Complex actions like administration, issue tracking, and user management all have good treatment in the API which means tools can extend the tool if necessary.
Import from other bug trackers: History can be imported from any issue tracker using the YouTrack Client Python library. There are even some ready to use scripts to import from the most popular trackers like JIRA, FogBugz, Mantis, and Bugzilla.
Constant innovation with transparency: The roadmap is public and agile project management is the next big focus.

More Git-integrating tool reviews are being planned. Stay tuned.

MongoDB Grails and Copying Collections

Ken Sipe — Thu, 13 Oct 2011 09:29:42 CDT

I currently find myself working on a project where Grails and MongoDB are the technology stack... in fact that in part is some of the reason for being interested in the project. I thought I would share 2 aspects of my learning so far:

Grails + Mongo

We spent at least 2 weeks trying to understand which approach we were going to take for our persistent tier. The general concensus was to use the Mongo GORM. We started down that path and ran into a bug and then a limitation on embedding. We got amazing support on the bug followed by the next release of the GORM option which fixed our initial document embedding concerns. We ran into more problems.

We switched to the morphia driver... once again... challenges caused lots of frustration. The challenges included getting other plugins to work with this driver, the fact that the morphia "domain" objects couldn't be in the standard grails domain folder, which further complicated mocking. Frustrations with getting full capabilities of getting spring security to work. It turns out that with spring security there are a number of relational assumptions (along with assumptions of objects being domain classes.. meaning they live in the grails domain package space) which cause problems when you are using NoSQL.

The morphia had another limitation we couldn't live with... we wanted the ability to have different types of objects mapped to the same collection. It appears to me that the Mongo efforts in the Grails space are bolt ons to a rational model. You can get it to work when thinking with a RDBMS mindset. They seem to break down with a NoSQL mindset... this certainly could be attributed to user error (meaning me). I believe we gave it a good try... however the project is under the gun and we didn't have time to "play" with it or wait for responses.

Alas we created our own solution... one of our significant constraints was to be prepared for extremely large scale. This had us duplicating a lot of documents not common in RDBMS solutions... along with the fact we needed to control the number of queries. Ideally we want 1 query for a page... certainly for the main page which is high laden with dynamic content.

Mongo Copying Documents - Collection to Collection

I'm really getting to like Mongo... a lot. One of the downsides is there is limited documentation... including limited forum comments on it. Because of this, I've decided to post interesting solutions to problems as I discover them... time permitting. so for todays tip. I recently needed to duplicate a "record" in mongo speak a document in a collection. There is so much information out there on how not to duplicate records that a google search at this point usually points you in the wrong direction. My situation was such that we have a CMS component to our application, which sets a publish day on content. What I needed was the ability to duplicate a record (so the BSON id would need to change and set a new publish day.

The first thing to recognize for those new to Mongo is that the Mongo console (the command line client to Mongo) works with javascript... so get your javascript fu ready... no.. not your jQuery fu... your plain jane generic javascript fu:)

Here is the solution I came up with:


db.NewsItem.find({newsType: "Blah"}).forEach( function(x){ x._id = new ObjectId(); x.publishDay=283;  db.NewsItem.insert(x)} );

So it turns out that you can pass a lambda expression to the forEach method of the collection. The trick is to assign a new ObjectId() to the _id, make any other assignments and then insert into your collection of choice... in this case back into the same collection. This comes in handy as well when you want to backup a collection real quick. (yea, I know you could use the mongodump and mongorestore, for a quick temp solution this just seems more convenient)


db.NewsItem.find({publishDay : 285}).forEach( function(x){ db.NewsItemBackup.insert(x)} );

Happy Coding!

JavaOne: Rocking the Gradle

Ken Sipe — Wed, 05 Oct 2011 11:06:16 CDT

Presenting Rocking the Gradle at JavaOne tomorrow 12:30pm at the Parc 55... see you there!

Git at the NHJUG

Matthew McCullough — Sat, 23 Jul 2011 22:29:00 CDT

NHJUG

I had the wonderful opportunity of being sponsored by No Fluff, Just Stuff Symposiums to speak at the New Hampshire JUG in Portsmouth on Tuesday. It was a lively crowd of 20 or so persons hosted by the excellent Ted Pennings, Matt Merrill, NHJUG, and Scott Curry.

We discussed Git, GitHub, IntelliJ, and eGit with regards to their importance to the broad JVM ecosystem. Many insightful questions were asked, such as “is the efficiency of the hard links to unchanged blobs maintained on Windows.” The answer was yes, because this is not actually a POSIX hardlink for commits, but rather an implementation like, but not exactly hardlinks inside Git tree objects.

CFMeetup

On Thursday, I had the equally delightful opportunity to speak to the CFMeetup online group, hosted by Charlie Aerhart and facilitated in part by Mike Henke, and supported by Tim Cunningham. It had a turnout of 35-45 persons, and has a significant additional viewership for the recorded sessions at Vimeo.

Resources

I promised some links to Git resources and they are as follows:

Have a Relationship with your Customers

David Bock — Thu, 30 Jun 2011 08:56:00 CDT

I have been a customer of Tivo for a long time; I first saw one at a friend's house in the fall of 1999, and bought one for myself that Christmas. Over the years I have owned somewhere around 6 tivos... Upgrading when the series2, series3, and most recently the Tivo Premiere came out.

Of course at its heart, the Tivo is a computer with a hard drive, and in the past 12 years I've had a few technical issues. This story starts last month when my Tivo Premiere died a horrible hard drive death and needed to be replaced, and ends with a lesson on the importance of having a relationship with your customers.

I wasn't looking forward to replacing the Tivo - besides the call to Comcast (which is an entirely separate blog post), I know from experience that every Tivo is a blank slate... I have to 'teach' it everything about me again... the shows I record, the stuff it should suggest for me, and so on.

Once I got the box set up, I was greeted with a "Welcome to Tivo" message in the Tivo's "messages and settings" that politely told me what I needed to know to use my Tivo, as if I was a first-time user. I was slightly annoyed at this... I thought to myself "Tivo *knows* who I am - this Tivo is linked to a lifetime account. Why not have a welcome message that says 'Here's your replacement Tivo - we hope Rudy at the service center was able to resolve your issue.'?

Of course, Tivo could easily do a lot better than that suggestion. My Tivo is linked to an account where I can schedule shows on the Tivo website; Why not back up my preferences there, and automatically download them to the new tivo? Why do I have to set up, yet again, things like "Record episodes of Phineas and Ferb off of DisneyHD"? I'm not asking Tivo to back up tons of data - not the actual shows, just the 'metadata' about my shows, my thumbs up and down ratings, the premium channels I subscribe to, etc. This can't be more than a few hundred kilobytes. (I would think they would be interested in that data too - wouldn't TV executives PAY to know how many people gave a TV show a thumbs up vs. a thumbs down rating?).

A week after I set up my replacement Tivo, I received a new message that says "You may have noticed extra recordings that you didn't request - these are Tivo Suggestions". This reminded me how annoyed I was... "Yes, I know you suggest shows to me - you've been doing it for ELEVEN YEARS... DON'T YOU KNOW WHO I AM?!?" I began to get the feeling that Tivo, as a company, doesn't care that I'm a loyal customer. They should be sending me a message that says "Its been a week and we see you're recording your preferred shows again. We're happy everything was set up properly with your cable company".

Another week went by and I got another message "Have you ever missed an episode of your favorite program? Learn how to set up a Season Pass". Sigh. Perhaps you didn't notice, but I've already set up, like, 70 of them? Remember me? I was mildly annoyed at having to set everything up - but weeks later, Tivo's naive attempts at customer contact are simply reinforcing the negativity of the whole experience.

Now, I'm not expecting Tivo to greet me by name when I call their service department, or send me cookies at Christmas time, but they can do a lot better than this. As a software engineer, I know the things I'm suggesting are next to trivial to implement. I'm also quite sure the Tivo engineers have a list of user stories like these sitting in an idea tracker somewhere... The only reason Tivo, as a company, would not do these things is because they haven't been made a priority.

Thats right, Tivo, as a company, has not made it a priority to have a relationship with their customers.

As much as I love my Tivo, I think they survive because their biggest competition is from cable-proprietary DVRs... and we all know the track record of customer relationship management cable companies typically have. Tivo should be thankful Apple didn't make the AppleTV a DVR... I don't think they could survive the competition. If Apple made a DVR, I'm sure it would be sending me tweets: "Hey Dave - Neal DeGrasse Tyson is going to be on Letterman tonight - don't worry, I'll record it for you."

You know what would have been the ultimate experience? A message on my tivo 2 months ago that said "We've noticed your Tivo is beginning to have issues with its hard drive. We've backed your schedule data up and scheduled a hardware replacement under warranty. call 1-800-get-tivo to schedule your return and replacement". Upon plugging my new Tivo in, a message like "We've downloaded and rescheduled your shows and noticed you've inserted your cable card. You now need to call 1-800-comcast, and give them this MAC ID and Card Pairing Code to have them reactivate your cable service". Every piece of hardware and data is available to make that possible; all thats missing is software.

Is there a lesson here? Yes! We can trivially use the data we have on our customers to improve their experience, and make it seem like they have a more personal relationship with a big faceless company. How can you improve your relationship with your customers?

Unified (as much as possible) Logging Using SLF4J

Baruch Sadogursky — Wed, 22 Jun 2011 01:40:00 CDT

Integrating, integrating, integrating. That’s what we do in Java enterprise development. Persisting objects with Hibernate wrapped by JPA using C3Po (or JTA?) (or MongoDB over Morphia?), processed with JBMP, created by JAXB (jackson-json?) from JAX-RS scheduled by Quartz … (a few dozen frameworks later) … all this glued with Spring (or Guice?) deployed on Jetty (or Tomcat, JBoss, Resin?) into cluster by Terracotta (or Hadoop, GigaSpaces, JBoss cache, Infinispam?). Ah, and all this built using Maven Gradle with Artifactory on Jenkins. I sure forgot ½ of the frameworks we constantly use.

Generally we don’t mind much about the internals of the frameworks we use (as long as they are good) – the whole encapsulation stuff is the last undoubted good thing. But except for the API (part of which is the configuration) frameworks have another user-facing end – the logging. When we build a system we want it to behave as one system – single configuration from one end, and single log from another (break it to different files, if you wish, but it should still be a unified logging system).

The reality is that there is no standard de-facto for logging. The standard de-jure – JUL, is not very popular because of its lack of functionality (compared to alternatives) and its suboptimal performance. And then there is Log4J, which almost became standard, but did not. And there is logback, which is a Log4J trashover, and there are facades (JCL and SLF4J), which try to unite all this zoo, and some others, which you have probably never heard of, like syslog4j*, logging framework by the Object Guy, jLo, MonoLog, Lumberjack, Houston, JTraceDump, qflog, LN2, TracingClassLoader, SMTPHandler, Log4Ant, Simple Log, Log Bridge, Craftsman Spy, Pencil, JDLabAgent, Trace Log, JDBC Logger, LimpidLog and Microlog.

Let it be, you’d say – why not have many logging tools, which are good and diverse! Well, the problem, as I’ve already mentioned, is that they leak out of the frameworks. Their diverse configuration leaks from one end, while their diverse output from another. Spring uses Log4J over JCL. So does Hibernate. Jetty uses Logback over SLF4J. Some (like Terracotta modules) use plain Log4J, Jersey uses JUL. This means we end up with 5 separate configurations (Log4J, SLF4J, Logback, JCL and JUL) and 3 different types of log files (Log4j, Logback and JUL). What a system!

To make the long story short – How can we achieve the desired consolidation? Clearly, we need a facade. There are two most commonly used – SLF4J and JCL. JCL is known for its classloader hell, SLF4J is newer, better performing, smarter, simplier to use and generally provides better quality for the same buck (well, no buck – both are open source, of course), so we’ll stick to it. SLF4J is an adapter – thin layer of API to and from different logging implementations. Yap, both ways. It means with SLF4J we can use JUL API on top and log using Log4J in the bottom!

First we need to pick an actual logger. Log4j was considered the best choice up until recently (2006) when Ceki Gülcü decided he needed a fresh start and rewrote from scratch a new Java logging framework, just better than log4j, called Logback. We can give it a try as our underlying logging implementation (we can switch in a moment, as we are using good facade, remember?).

So, here’s what we have to do:

Establish our own good logging:
Done here. Now our own brand new code will use top-notch logging.
Now for the tricky part. Let’s make the example stack I listed above taking configuration from one source (our config files) and writing to one target (files, listed in our configuration)

All the tools using SLF4J will just work. That includes dozen of Apache projects, inc. Camel and Mina, some SpringSource projects and many others.
Now let’s start rolling with all the rest. This is how you do it (click to enlarge):

Jakarta Commons Logging:

Remove commons-logging.jar from your classpath. Usually, it is transitive dependency from the framework, so you need to instruct your build tool on how to do it. What a lucky coincidence, I just wrote short and instructive blog post about how to do it!
Add jcl-over-slf4j.jar instead. It contains alternative commons-logging API implementation, so the code will run just fine.

Log4J:

Same goes here! Remove log4j.jar from your classpath (Again, it would usually be a transitive dependency from the framework, look here).
Add log4j-over-slf4j.jar instead. It contains alternative log4j API implementation, so the code will run just fine.

JUL:

Well, you can’t remove JUL from classpath (it’s a part of the JRE, dude). For the same reason SLF4J can’t reimplement JUL’s API.
Add jul-to-slf4j.jar. It will translate java.util.logging.LogRecord objects into their SLF4J equivalent.
Install SLF4JBridgeHandler and LevelChangePropagator.
Expect 20% decrease in performance (so use it wisely).

All done. Now both our code and all the 3rd paries configured from single source and write to single target. Hooray!

* syslog4j claims it is cross-platform. Well, I’ll just quote: “Is Syslog4j cross-platform? Yes! Syslog4j UDP/IP and TCP/IP clients should work in any typical Java JRE environment.”

Filed under: Frameworks Tagged: jcl, jul, log, log4j, logback, slf4j

Banning Transitive Dependencies With Maven2/3, Gradle and Ivy

Baruch Sadogursky — Wed, 22 Jun 2011 01:39:00 CDT

Oh, you are using build tool with dependency management? Good! Be it Maven2/3, Gradle or Ivy, your life as devops or developer is much easier. Until you hit it. The evil transitive dependency. How can it be evil you ask? When the classes in it clash with the classes you really need. Here’s some use-cases:

Same dependency, different jar names, two examples here:

The Jakarta Commons renaming effort: commons-io:commons-io:1.3.2 and org.apache.commons:common-io:1.3.2
The Spring Framework artifacts naming convention alternatives: spring-beans, spring-context, etc in repo1 versus org.springframework.beans, org.springframework.context, etcin
SpringSource EBR.

Different packaging of the sample classes, many examples here:

OSGi repackagings: asm:asm:3.2 and org.objectweb.asm:com.springsource.org.objectweb.asm:3.2.0
Modularization of Spring 2.5.6: as single jar and as spring-whatever multiple modules
Xerces and Xalan are included in JDK since 1.5. They are still present as transitive dependencies in all the tools which support JDK 1.4.
Alternative packagings with and without dependencies: cglib:cglib and cglib:cglib-nodep
Project merges like Google collections, which are now included in Google Guava

Deliberately reimplemented interfaces, for example for bridging legacy APIs to new implementation, such as in SLF4J.
Your patches for 3rd-party tools.

All those may end up with 2 or more classes with the same name in the classpath. Why it is bad? Java class identifier consists of fully-qualified class name and the classloader that loaded it, so if two classes with the same name reside in same classpath JVM considers them to be the same class, and only one of them will be loaded. Which one? The first classloader encounters. Which one will it be? You have no idea.
When the duplicated classes are exactly the same, you will never notice. But if the classes are different, you’ll start getting runtime exceptions, such as NoSuchMethodError, NoClassDefFoundError and friends. That’s because other classes expect for find one API, but encounter another one – wrong class was loaded first. Not fun.

Now, when you know how evil they are, let’s take those bastards down!

Maven 2/3

There is no simple way (Maven’s tagline) to exclude some dependency from all the scopes. I’ll show two cases – manual exclusion and working with IntelliJ IDEA:

Stage 1: exclude all the banned dependencies one by one:

Manually edit Maven’s poms

For each evil dependency:
Find which top-level dependency brings the evil transitive hitcher with it. This is done by using Maven Dependency Plugin:
```
mvn dependency:tree -Dincludes=commons-logging:commons-logging
```

You’ll get something like this:

[INFO] com.mycompany.myproduct:rest-client:1.0
[INFO] \- org.springframework:spring-webmvc:jar:3.0.5.RELEASE:compile
[INFO]    \- org.springframework:spring-core:jar:3.0.5.RELEASE:compile
[INFO]       \- commons-logging:commons-logging:jar:1.1.1:compile

Go to the pom.xml with your dependency management (you use dependency management, don’t you? If you don’t, don’t tell anyone, go and start using it) find spring-webmvc dependency and add an exclusion to it:

1     <dependency>
2     	<groupId>org.springframework</groupId>
3     	<artifactId>spring-webmvc</artifactId>
4     	<version>3.0.5.RELEASE</version>
5         <exclusions>
6             <exclusion>
7                 <artifactId>commons-logging</artifactId>
8                 <groupId>commons-logging</groupId>
9             </exclusion>
10         </exclusions>
11     </dependency>

Working with IntelliJ IDEA:

Open Maven Dependencies Graph.
Filter it by the dependency you are looking for.
Select it and press Shift-Delete.

Good job! Your nailed them down in the current version of your build. But what happens when someone adds a new 3rd party dependency and brings some bad stuff with it as transitives? You need to protect your build from this scenario. So, stage 2: Fail the build if one of the banned dependencies ever added to the build with Maven Enforcer Plugin. Add the plugin to your root project pom:

1 <project>
2   <build>
3     <plugins>
4       <plugin>
5         <groupId>org.apache.maven.plugins</groupId>
6         <artifactId>maven-enforcer-plugin</artifactId>
7         <version>1.0</version>
8         <executions>
9           <execution>
10             <id>enforce-banned-dependencies</id>
11             <goals>
12               <goal>enforce</goal>
13             </goals>
14             <configuration>
15               <rules>
16                 <bannedDependencies>
17                   <excludes>
18                     <exclude>commons-logging</exclude>
19                     <exclude>cglib:cglib</exclude>
20                   </excludes>
21                 </bannedDependencies>
23               </rules>
24               <fail>true</fail>
25             </configuration>
26           </execution>
27         </executions>
28       </plugin>
29     </plugins>
30 </build>
31 </project>

As I mentioned, using the Enforcer plugin won’t exclude the unwanted dependencies, it only will fail the build. Once that happened (and trust me, it will), you need to go and exclude them manually, as described in Stage 1 above.

And we are done with Maven. Not fun? Switch your build tool!

Ivy

Well, comparing to Maven it’s emabrassing how easy is to add global exclusion in Ivy. All you need to do is add exclude tag, and it will do the job for all the transitive dependencies, both in current and future use:

1 <dependencies>
2     <dependency org="org.springframework" name="spring-webmvc"
3 rev="3.0.5.RELEASE" conf="compile->default"/>
4     <exclude org="commons-logging"/>
5 </dependencies>

Done.

Gradle

Since Gradle uses Ivy under the hood, here comes the same ease, but even groovier:

1     configurations {
2         all*.exclude module: 'commons-logging'
3         all*.exclude group: 'cglib', module: 'cglib-nodep'
4     }

That’s all! Now your code is bullet-proof from classloading conflicts and you can do nasty class-replacing stuff, for logging or pleasure.

Filed under: Build Tagged: gradle, ivy, maven2, maven3

Small Issue with the Asset Pipeline

David Bock — Mon, 20 Jun 2011 11:30:00 CDT

This is a small complaint about Rails 3.1, but its one I'm surprised I'm able to make at all.

If you have been following the evolution of Rails, you know that years ago the framework adopted REST as one of its core principals (well, restful concepts - I'll leave it up to others to debate/defend the true purity of the implementation). One of the core ideas behind REST is the idea that a url represents a unique resource, and that the mimetype/filename extension is used to negotiate the format that gets delivered. for instance:


http://www.davebock.com/resume.html

and


http://www.davebock.com/resume.pdf

should be the same underlying *thing*; what gets transferred is a representation of the thing in the format requested... in this case, my resume, as either html or pdf.

Enter the new Rails Asset Pipeline, which is a fantastic piece of work in general. In diving in and playing with it for the first time, I was trying to wrap my head around some of the conventions. I generated a new rails project with v3.1rc4, and I hadn't added any of my own content yet. I was looking at how everything under assets directory seems to be served up under the /assets/ url without much respect for the directory structure underneath it. Out of the box, rails gives us several files that have default contents. For instance:


http://localhost:3000/assets/application.css

and


http://localhost:3000/assets/application.js

Compare that to my first example above. These are the same url, but with different extensions - different "content negotiators"... but they aren't serving up the same thing at all. One could argue that they are two things designed to work together, but I don't think thats enough - the principal is violated - these are not the same thing, just in different formats, these are really much different things - one being code in the Javascript language meant to be used by your web application, and the other being a css file that contains all of the view/layout of your app. the browser isn't saying, "Give me that same information, but in the .js format", the browser really is requesting a different asset.

I like the asset pipeline, and I like the convention of treating all of these things as "first-class citizens" in their own assets directory under /app instead of as "a bunch of stuff" that happens to live under the /public directory - but I don't think this first-class citizenry has gone far enough - why aren't these things further scoped by the /images, /javascripts, and /stylesheets directory when referenced via url?