/dev/null

I recently managed to publish my latest project:

chaoticpattern.net

While reading the book Sync, I felt that this is the right name – an explanation can be found on the Web site. There is already some blog with the same name, but it does not seem to be very active. The last post is from May 2009. Hope I don’t cause any confusion.

Chaotic Pattern will be used as an incubator/lab. The Wiki itself is the first in a series of experiments. It fits my personal needs as a publishing platform and is more convenient than any typical word processor out there. You can not only edit documents online, like in every other Wiki, but also produce wonderful PDFs and include content from Flickr, Youtube and Google Maps. If you like it, use it for free. There are no strings attached and I will not sell your documents or your Facebook profile

Also the Wiki is there to demonstrate what you can do with JavaScriptMVC 3.0 and a little bit of PHP. Theoretically, it should scale to billions of users. But as long as I don’t get any venture capital, we will never find out. Anyhow, I want people to use it as a prototype for their own apps. Until end of this year, everything should be properly documented and you can download some of the components.

The next step is to get the Ajax push server finally running, so that you can use the real time features. Ape is installed, but seems to cause connection problems under certain conditions. Either I can fix that with my limited resources or I have to switch to another server. It’s not like I don’t have anything else to do, so please be patient^^

If you are reading this, I can safely assume you know what JavaScriptMVC (JMVC) is, what its features are, and what components it consists of, namely:

• FuncUnit (the test framework)
• DocumentJS (the documentation engine)
• jQuery (JavaScriptMVC uses a special fork with added bugfixes)
• JavaScriptMVC  (the core framework)
• Steal (the code manager / script loader; sometimes referred to as “StealJS”)
• Phui  (component library for JavaScriptMVC, not part of the core)

These partly independent components (together with Selenium) are bundled using git submodules (http://book.git-scm.com/5_submodules.html) in a repository called “framework” for your convenience:

http://github.com/jupiterjs/framework

A detailed description of the components can be found on the Jupiter JS blog:

http://jupiterjs.com/#news/javascriptmvc-features

Release 2.0

You should be aware that release 2.0 is still out there. Don’t get confused. It is hosted on Google Code at http://code.google.com/p/javascriptmvc/ and uses Subversion instead of git. Don’t use the issue tracker there for bugs you find in 3.0.

The respective project Web sites are:

• http://www.javascriptmvc.com/ for 2.0
• http://v3.javascriptmvc.com/index.html for 3.0

Ways to contribute

There is a number of ways to contribute:

1. You can fork the project repositories on github (http://github.com/jupiterjs/) into your own repository and send a “pull request”, every time you want to submit changes (pull requests replace patches and preserve your authorship)
2. After contributing for a while, you can become a member of the JavaScriptMVC team and ask for direct write access to the project repositories
3. You can just checkout/download  the latest framework version and report bugs to the developers (http://github.com/jupiterjs/javascriptmvc/issues) or answer questions on the mailing list (http://groups.google.com/group/javascriptmvc?hl=en)

A few words about git

In every case you should become more or less familiar with git (the distributed version control system) and github (the project hosting Web site).  The Git Community Book is a great reference: http://book.git-scm.com/index.html. If you are already familiar with Subversion, you might want to read the “Git – SVN Crash Course” (http://git.or.cz/course/svn.html). The main idea of git is that everyone has its own repository (that’s why it’s called distributed). It allows everyone to contribute easily to Open Source projects and was initially designed and developed by Linus Torvalds for Linux kernel development.

In contrast to “normal” version control systems, a commit is happening on your local computer only. If you want to actually send your code to the server, you have to push it back to the remote repository. If this is not the main repository of the project but your own fork, you additionally have to send a “pull request” to the project owner (jupiterjs in that case). As a previous user of Subversion I found the whole process of checking out the source, committing my changes and pushing it back to the repository tremendously complicated, even though I understand the need for distributed version control. In part this is because git sometimes comes up with error messages that look pretty scary and that don’t help to understand the actual problem.

Setup your github account

After you created your free github account at http://github.com/, you need to go to the Account Settings page, click on “SSH Public Keys” in the left hand side navigation and then enter your public key. If you don’t have one yet, you open a text terminal on the local computer and enter:

ssh-keygen -t rsa -C “youremail@address.com”

There is a more detailed howto at http://help.github.com/msysgit-key-setup/.

If you run Windows, you are probably out of luck. Try to ask Google for assistance

Next step is to fork the repositories you want to work on. This process is pretty convenient – you simply click the fork button on the main repository page. The complete list is visible on:

http://github.com/jupiterjs

Install Git and Java

If not yet done, you should also install Git and Java on the local computer by using the package management software that comes with your Linux distribution. On Ubuntu you can use the “Synaptic Package Manager” for example. At least in the past, I had bad experiences with OpenJDK and Rhino (the engine that executes JMVC’s command line scripts), so I recommend installing the “original” Sun/Oracle Java version. The Selenium server also needs Java.

Clone the repositories

I recommend cloning (that means checkout) the “framework” repository as the first step. The build script in there allows creating the downloadable packages you see on http://github.com/jupiterjs/framework/downloads. To do this, follow these 4 easy steps:

1. Create an empty local directory that should contain all your github repositories and change into it
2. git clone git://github.com/jupiterjs/framework.git
3. git submodule init
4. git submodule update

That’s it!

The procedure to clone your forked project repositories is similar. First you need the URL that you have to provide to Git.

Go to your Git home page ( http://github.com/[username]/) and then click on the forked respository, e.g. javascriptmvc. On the top you’ll see 3 different URLS:

• SSH with read and write access: This is what you want to use
• HTTP: This is slower and offers read only access
• Git Read-Only: Since we want to push (write) our changes to the server, this does not make sense

Now simply clone it into your local github directory by typing:

git clone git@github.com:[username]/[repository].git

Push changes

With Git, you can do as many commit as you want locally. Please note that the equivalent to “svn commit” is “git commit –a”. Please be nice and provide a commit message every time, for example:

git commit -a –message ‘Changed github URLs from pinhook to jupiterjs’

To see what would be commited or to see your changes since the last commit you can use “git status” and “git diff”.

After testing all your changes locally (this is probably worth another article, but I’m running out of time now), you are free to commit your changes to the remote repository:

git push origin master

You can also configure git to always push to the matching remote branch (there are other options as well, but I’m not that much into git to fully understand them, to be honest):

git config push.default matching

Next time you push something into that repository, it is enough to use “git push”.

As mentioned earlier, you have to do send a “pull request” (there is a nice button on github to do just that), if you want to send your changes from your fork to the original project repository.

Pull changes for submodules

In my naïve thinking, I assumed that Git will automatically update the submodule directories in the framework repository when you type “git submodule update”. This is not the case.

You have to manually pull the latest changes in each sub directory that contains a submodule like this:

cd funcunit/

git checkout master (only required the first time)

git pull

You will only get the changes in the submodules via “git pull” in the main framework directory if somebody with write access to framework does this “cd [submodule]; git pull” procedure locally and then pushes back to the repository on github:

http://github.com/jupiterjs/framework/commit/1affd96d632b34621e9e0d06707ea83d1d4c3b9d

Thanks

Thanks for reading this. You see, I’m not a Git expert yet and I really hope I didn’t give any bad advice concerning its usage. As I improve my knowledge and find best practices how to do things most efficiently, I will update this page or post a follow-up. Especially testing is not covered in this article, which is a shame.

I was totally enthusiastic, when I first learned about JavaScriptMVC – because it brings all the best practices you are accustomed to into the JavaScript world. That means the MVC architecture, Object-oriented development with simulated inheritance, EJS templates, easy testing, OpenAjax support, powerful event delegation and so on.

However, I often have a hard time to convince other developers about those advantages. I guess testing is the easiest one, because no developer would dare to argue against testing, even though many developers don’t do it or don’t do it right. Questions and discussions about testing seem to be very rare. Developers just discuss it with their non-technical managers, if they want to get around that “effort”.

But it already starts with OpenAjax. I rarely find a developer that knows what it is and what it is good for and why you might need an event hub in JS at all. Zend, W3C, IBM and Google are some of the more well-known members of the OpenAjax Alliance. If you hear about it the first time now, take the time to have a look.

Just yesterday I was talking to an obviously experienced JS dev (who was working for a well known mobile phone company), that was browsing through the JavaScriptMVC documentation and stopped in the chapter about EJS templates: “Yeah, I saw something similar in our own project and removed it. It is confusing, if the JS code and the HTML are in separate files, because you don’t see the code together in one file. HTML should only be edited by developers who know the JS code very well”. Not sure what to say about that. Why not give up separation of concerns at all and put everything in one big file? Then you see the big picture on your hopefully big screen. Also you always need to check the full version control system diff after each change because the file name alone doesn’t tell you what was changed and you probably have more of those exciting conflicts to resolve. And: You REALLY need to be an expert to understand what is going on. The company can never fire you, because no one else would ever understand that kind of code. Sorry for being sarcastic, but that kind of attitude can seriously damage the health of innocent other people who need to clean up the mess in 20h work days afterwards, to meet the deadline of the project. Or the project is so small that it doesn’t matter. Or the company you work for can afford to pay a lot of developers who have the time to maintain a JS app without templates.

Convincing people about the usefulness of controllers is not easy too. Either you start showing an OpenAjax subscribe action – then the developer most likely won’t understand what that is good for (see above). Or you talk about event delegation, then the developer will point to jQuery’s live() method, which does event delegation too. In fact, JavaScriptMVC’s event delegation used to be one of the strong points and I admit that I am not totally up to date with the whole discussion right now, because I was busy with a PHP project during the last 6 months. I know Justin Meyer is/was working on the controller code so that it makes more use of jQuery, he contributes code to jQuery and he is in contact with John Resig. Anyways, the whole point about controllers again is separation of concerns and not so much the technical details behind the scenes. The JavaScript world is moving very fast and implementation details change a lot, but separation of concerns is not really something that you can look at as something you probably don’t need. The JS world sometimes feels like the PHP world about 10 years ago.

Ok, after getting rid of those thoughts, I will return to work now. The next 2 weeks will be quite relaxed with very little work. So maybe, I can read through the new JavaScriptMVC code and update my knowledge. It really feels uncomfortable not the be able to answer questions in detail. Also I start confusing PHP and JS syntax *lol*

See also: http://jupiterjs.com/news/talkin-javascriptmvc-is-hard

A new stable release of the JavaScriptMVC framework was released yesterday Justin Meyer developed it, while working with us on a new Web application. Some features like the new custom events are direct results of this project. He did an excellent job!

New features include:

• Env.js/Shrinksafe based compression: A custom env to simulate the browser. As the browser encounters script tags, it adds them to a collection and then compresses them. This means instant file concat and compression from the command line with no extra work.
• Env.js Testing: Prior to 1.5, tests ran in the browser only. With Env, the same tests can be run from the command line. Great for projects where you need a quick way of checking if functionality works before check-in.
• Documentation:New JavaScript based documentation library split between JSDoc and NaturalDocs.
• Code Generators: Added code generators and made building custom ones easy too by using EJS.
• Scaffolding: Helps you develop iteratively by connecting to default Rest services and providing an easily expandable CRUD interface.
• Engines+Plugins: Added a command line plugin and dependency installer. So, if a developer wanted a jQuery plugin, he can install it from the command line, and it will also grab jQuery.
• Custom Event Delegation: Besides improving event delegation to cover all the cases that even live doesn’t do, they’ve expanded it to include custom events such as drag+drop, lasso, hoverenter, mousenter. Developers can have the benefits of event delegation with these complex events.
• Easy Update: JavaScriptMVC can update itself from the command line.

Shall I use JSON or XML as AJAX transport format?

There actually is no reason to use XML instead of JSON, if you don’t plan to use XSLT. JSON is part of JavaScript which has been around since the dark ages of the web. Also, there are very fast server-side JSON implementations and JSON is the more compact protocol, thus saving bandwidth.

Isn’t XSLT much faster than to use JSON and modify the DOM with JavaScript?

First, it is highly unlikely that there are more browsers that understand XSLT than {a: ‘b’}. In fact, with JSON you are on the safe side. For a client-side templating engine like EJS, the performance is not an issue.

Processed EJS templates simply add strings together.  They do not use eval() or have a processing step which is probably why typical templates run slow.

Also, XSLT is loaded from another file while EJS can be packaged with the JavaScript payload.  This necessitates an extra request by the client.

Is it better to render the HTML on the client or an the server side?

There is a general decision, you need to be aware of: Are you building a Rich Internet Application or do you want to beautify some HTML output with additional (JavaScript-)effects? Is the site required to be usable without JS? If you can rely on JS and if it’s a RIA, then I would suggest using a client-side MVC framework like JavaScriptMVC, which comes with client side controllers, template engine and model classes. Your server will mainly provide JSON-API functions and the basic page grid then, and not deliver complete HTML pages. Lots of logic (and therefore code) will move to the browser/JavaScript: In effect, you will need more JavaScript developers and less server-side developers for your projects. On the other hand, you will see a gain in consistency and also performance, as you have way less client/server communication, once the JS code is loaded (which can be pretty fast, thanks to compression). Many popular Web applications are using this approach.

Which JavaScript library shall I use (Dojo, jQuery,…)?

The choice of JavaScript DOM/Ajax library is IMO not so important, as long as it doesn’t leak memory, has a small code size and offers fast DOM queries. See

http://www.domassistant.com/slickspeed/

The real problem you will have, is that normal event binding causes problems because of memory leaks (circular references which can’t be handled by the garbage collection of most browsers) and also because you need to loop through all elements and refresh everything after doing changes to the DOM (for example after AJAX requests that modify the HTML). So, at the end of the day, you will want to use event delegation, which comes for free with the controllers of JavaScriptMVC.

I know there is JavaScript “integration” in my favorite Java/PHP/Python/Perl development framework. Am I ready for building a Rich Internet Application now?

Server-side generated JS, like what you will get when using any framework that offers JavaScript support (I’m not talking about JSON/REST here), can be a pain, because almost certainly, you will have a very limited feature set and maybe also problems with deployment and testing, because the common server-side frameworks can’t really test JavaScript code nor is there a smart way to optimize generated code or structure it. Most of the approaches I’ve seen just generate some inline JavaScript which is good for enhancing forms a bit with new input elements/validators or offer some nice visual effects – but that’s it. If something like this needs to be customized a lot and grows big, you might be in trouble. Again, the question is “JavaScript enhanced Web site” or “Rich Internet Application”? To generate JavaScript with a server-side language/framework  has it’s limits and is very often not consistent, if you use hand-written JavaScript code at the same time.

What features should a JavaScript framework for Rich Internet Applications offer?

I would argue that a good JavaScript framework allows an average developer to produce effective and structured code. jQuery and the like however are just browser abstraction layers (like Zend_Db is an abstraction layer for databases). Raw jQuery will almost certainly produce the same mess as raw PHP or any other server-side language that does not imply a clean application structure (MVC). You end up with reinventing the wheel and you will have different code to do similar things. Or it will just be plain slow, because without dispatching events (event delegation) you end up with looping over elements to attach events and effects, which simply gets slow for many elements. Also you will have no overview and especially inline JavaScript can’t be compressed, which means you will transfer the same source code with every request. That said, I vote for separating server-side code, JavaScript and HTML and have clearly defined interfaces. A sound architecture is a great help for all developers, as soon as there are more lines of code than fit on a screen. You don’t want to start guessing, if the code you look for is hidden in a JavaScript file, in a template, in a view helper or some other file.

Which browsers are commonly supported for Rich Internet Applications?

Compatibility is not a big issue anymore, if you are fine with supporting IE 6+, Safari 3+ (incl. iPhone), Firefox 2+ and Opera 9.5+. IE < 6 is very hard to support (I would simply refuse to do so). IE 6 is slow with JavaScript, but the real pain is it’s lack of good CSS support.

I want to use W3C standards only (W3C DOM, XML and semantic XHTML) for my Rich Internet Application. JSON, innerHTML and DIV elements are evil, right?

No. If you have a closer look, then you will notice that semantic HTML (for example, using the table element to render a table instead of DIVs) or not using innerHTML sometimes offers very bad performance for DOM manipulations, thus render the complete application useless. innerHTML very often delivers the best performance for inserting content to the DOM. So, don’t decide on certain implementation details upfront, especially if you plan to support IE6, which simply is an old browser and needs some special treatment. JavaScript forces you to be very pragmatic sometimes.

See http://www.quirksmode.org/dom/innerhtml.html

JSON is a standard because it is implied by EMCAScript. See JSON vs. XML.

Using parallel requests to get data from the server seems to be a good idea to speed up performance. Is there any downside?

Depends. A major issue with modern Web applications is that the initial loading causes many parallel requests, as you just deliver the page grid and the JavaScript client side code then renders all the other elements from client-side templates and loads the corresponding data (if needed) using the mentioned JSON-APIs. A constant overhead is produced when using comet. Parallel requests frequently cause problems, as some server-side session handler implementations lock the session (other requests have to wait then, until the session is accessible again). Also the many requests might require lots of memory and server processes.

I want to thank Justin Meyer (maintainer of JavaScriptMVC) for all the interesting discussions and his input on the JSON vs. XSLT debate.

My blog was a calm place for a while now, because I’m terribly busy with writing code (like it should be…). With this posting, I want to present the prototype of JavaScript Components, which are comparable to Swing Components in Java. They make use of the Composite Pattern. That means every Component can act as a container for other Components. Together with Models, Views and a Controller (Event Dispatcher), they can form reusable widgets. Other projects did work on similar things (for example jMaki Widgets), but from what I see, they target different audiences. The suggested Components use a client-side only approach. They are tested with JavaScriptMVC, but may be useful for other frameworks as well. jQuery is required for DOM operations.

There already is a LayersComponent, which implements a content stack. Any component can be added as a new layer and only the respective top layer is visible.

Please check out the demo application (download as ZIP).