Visual Basic, PHP, Rails. Is Node.js next?

I had a nerdy conversation on what might be the next mainstream framework for building web products, and in particular whether the node.js community would ultimately create this framework, or if node.js will just be a fad. This blog post is a bit of a deviation from my usual focus around marketing, so just ignore if you have no interest in the area.

Here’s the summary:

Programming languages/frameworks are like marketplaces – they have network effects
Rails, PHP, and Visual Basic were all successful because they made it easy to build form-based applications
Form-based apps are a popular/dominant design pattern
The web is moving to products with real-time updates, but building real-time apps hard
Node.js could become a popular framework by making it dead simple to create modern, real-time form-based apps
Node.js will be niche if it continues to emphasize Javascript purity or high-scalability

The longer argument below:

Large communities of novice/intermediate programmers are important
One of the biggest technology decisions for building a new product is the choice of development language and framework. Right now for web products, the most popular choice is Ruby on Rails – it’s used to build some of the most popular websites in the world, including Github, Scribd, Groupon, and Basecamp.

Programming languages are like marketplaces – you need a large functional community of people both demanding and contributing code, documentation, libraries, consulting dollars, and more. It’s critical that these marketplaces have scale – it needs to appeal to the large ecosystem of novices, freelancers and consultants that constitute the vast majority of programmers in the world. It turns out, just because a small # of Stanford-trained Silicon Valley expert engineers use something doesn’t guarantee success.

Before Rails, the most popular language for the web was PHP, which had a similar value proposition – it was easy to build websites really fast, and it was used by a large group of novice/intermediate programmers as well. This includes a 19-yo Mark Zuckerberg to build the initial version of Facebook. Although PHP gained the reputation of churning out spaghetti code, the ability for people to start by writing HTML and then start adding application logic all in one file made it extremely convenient for development.

And even before Rails and PHP, it was Visual Basic that engaged this same development community. It appealed to novice programmers who could quickly set up an application by dragging-and-dropping controls, write application logic with BASIC, etc.

I think there’s a unifying pattern that explains much of the success of these three frameworks.

The power of form-based applications
The biggest “killer app” for all of these languages is how easy it is to build the most common application that mainstream novice-to-intermediate programmers are paid to build: Basic form-based applications.

These kinds of apps let you do a some basic variation of:

Give the user a form for data-entry
Store this content in a database
Edit, view, and delete entries from this database

It turns out that this describes a very high % of useful applications, particularly in business contexts including addressbooks, medical records, event-management, but also consumer applications like blogs, photo-sharing, Q&A, etc. Because of the importance of products in this format, it’s no surprise one of Visual Basic’s strongest value props was a visual form building tool.

Similarly, what drove a lot of the buzz behind Rails’s initial was a screencast below:

How to build a blog engine in 15 min with Rails (presented in 2005)

Even if you haven’t done any programming, it’s worthwhile to watch the above video to get a sense for how magical it is to get a basic form-based application up and running in Rails. You can get the basics up super quickly. The biggest advantages in using Rails are the built-in data validation and how easy it is to create usable forms that create/update/delete entries in a database.

Different languages/frameworks have different advantages – but easy form-based apps are key
The point is, every new language/framework that gets buzz has some kind of advantage over others- but sometimes these advantages are esoteric and sometimes they tap into a huge market of developers who are all trying to solve the same problem. In my opinion, if a new language primarily helps solve scalability problems, but is inferior in most other respects, then it will fail to attract a mainstream audience. This is because most products don’t have to deal with scalability issues, though there’s no end to programmers who pick technologies dedicated to scale just in case! But much more often than not, it’s all just aspirational.

Contrast this to a language lets you develop on iOS and reach its huge audience – no matter how horrible it is, people will flock to it.

Thus, my big prediction is:

The next dominant web framework will be the one that allows you to build form-based apps that are better and easier than Rails

Let’s compare this idea with one of the most recent frameworks/languages that has gotten a ton of buzz is node.js. I’ve been reading a bit about it but haven’t used it much – so let me caveat everything in the second half with my post with that. Anyway, based on what I’ve seen there’s a bunch of different value props ascribed to its use:

Build server-side applications with Javascript, so you don’t need two languages in the backend and frontend
High-performance/scalability
Allows for easier event-driven applications

A lot of the demo applications that are built seem to revolve around chat, which is easy to build in node but harder to build in Rails. Ultimately though, in its current form, there’s a lot missing from what would be required for node.js to hit the same level of popularity as Rails, PHP, or Visual Basic for that. I’d argue that the first thing that the node.js community has to do is to drive towards a framework that makes modern form-based applications dead simple to build.

What would make a framework based on node.js more mainstream?
Right now, modern webapps like Quora, Asana, Google Docs, Facebook, Twitter, and others are setting the bar high for sites that can reflect changes in data across multiple users in real-time. However, building a site like this in Rails is extremely cumbersome in many ways that the node.js community may be able to solve more fundamentally.

That’s why I’d love to see a “Build a blog engine in 15 minutes with node.js” that proves that node could become the best way to build modern form-based applications in the future. In order to do this, I think you’d have to show:

Baseline functionality around scaffolding that makes it as easy as Rails
Real-time updates for comment counts, title changes, etc that automatically show across any viewers of the blog
Collaborative editing of a single blog post
Dead simple implementation of a real-time feed driving the site’s homepage

All of the above features are super annoying to implement in Rails, yet could be easy to do in node. It would be a huge improvement.

Until then, I think people will still continue to mostly build in Rails with a large contingent going to iOS – the latter not due to the superiority of the development platform, but rather because that’s what is needed to access iOS users.

UPDATE: I just saw Meteor on Hacker News which looks promising. Very cool.

Like this post?
Get new updates via newsletter..

http://twitter.com/kmooney Kevin Mooney

You said yourself that Node.js’s strong suit is highly scalable event-driven apps.

The thing that was really exciting about rails in 2005 was that it was purpose-built to solve a problem that lots of web developers had, just like VB was built to solve a problem that lots of client-side developers had.I’m not sure the real-time collaboration problem space is big enough or painful enough for Node to really shine as a scaffolded, easy-to-onboard answer.

“Build-a-blog-on-a-super-tight-deadline” was powerful because the example was not at all contrived and it solved a real-world high-value problem.

I’m not convinced that live collaboration on comments, blog posts, live comment counts is enough of a high-value feature-add to justify the buildout of a framework, and I’m not sure it would be compelling enough to excite developers.

What real-time problems are we solving over and over again? If the node.js community can put their finger on that, then they can produce a really compelling demonstration of the platform’s power.
http://twitter.com/pt Parker Thompson

Andrew, the frame of what technology will best support realtime applications is a good one. This is how we should be evaluating technology for the class of apps that require realtime interactions.

That said, my understanding is that Node isn’t actually any better (and arguably worse) for these kinds of applications from a runtime perspective. Java, for example, can hold open many more connections than the Node runtime AND is more performant AND is more mature.

Rails represented an order of magnitude improvement over what came before. We’re seeing JS and functional programmers flock to Node because it allows them to write their kind of code everywhere. We’re not yet seeing people abandon other frameworks/paradigms because the value proposition just isn’t there yet.

As well, as a minor point Node is more like JRruby than Rails. It’s a runtime and an (immature) SDK. Node still lacks the Rails-like framework that would make actual application development a breeze. Check out Express if you want to see the state of the art there.

You might also want to look at how the folks that are in this business architect their systems. Last time I checked Pusher, for example, was using Ruby for their infrastructure. And this is their whole business.
http://andrewchenblog.com Andrew Chen

My take is that real-time moves to become more and more of an aspect of websites, the same way that social is an aspect of websites, rather than real-time being a category in itself. In that way, the market for products with this real-time aspect is huge since it actually encompasses a majority of all products out there.

As a result, the idea is that the “build a blog” scenario is just as relevant as it was before, but you could use a real-time framework to address is and build a more modern product.

I think mainly I’m reacting to the fact that when I build a new Rails app these days, the gap between where you start and what it’d take to actually build something more like Quora/Asana/GDocs is such a huge gap that it’s unachievable. My take is that if node made their event-based approach significantly more easy-to-use to the level where you could apply it trivially to form-based apps, that’d be a huge win. (I don’t think you disagree with this, but just reiterating)
http://andrewchenblog.com Andrew Chen

My understanding with node is that it’s really the event-driven model of programming that really makes the real-time updating more achievable, but that you have to significantly rearchitect your application differently from the vanilla Rails approach to get there.

That said, your points are all valid in that I just care about being able to build a modern, real-time app like Quora/Asana/Twitter more conveniently than the technology currently allows. Maybe that will end up Ruby.
http://needforair.com/ Stanislas Marion

While you make some interesting points, I think you’re comparing apples to oranges here. Node stands at a much lower level than Rails. In node core you get really the most basic functionalities that would be too painful to implement well for the end users. Rails packages a ton of stuff.
So it’s really a difference in philosophies: node apps are built from the ground up in a very modular fashion, where you add in functionality when you need it, starting with a very small core.
With Rails you get everything in one big package and if you don’t have to bother adding in a lot of stuff, it’s all there already for the most part. In fact you should probably think about removing the stuff you don’t use.

So node makes you think more about your exact needs and yields to more tailormade apps whereas a lot of rails apps probably are way too fluffy for their needs. This is why hackers seem to love node, there is no magic, you have to bleed through it, but at the end of the day, you know your app inside out whereas most rails programmers probably don’t have a clue how 99% of their app work.
http://andrewchenblog.com Andrew Chen

I know that rails and node aren’t apples to apples which is why, where it made sense, I tried to use the term “node.js community” – not sure the best way to describe the community of node, express, and all the other folks working on server-side JavaScript.

In a way, you are making my point- for this set of technologies to become mainstream, there needs to be sufficient packaging of a “whole product” where novice/intermediate programmers can build form-based apps easily. Until then, it’ll be the domain of experts who can assemble a collection of fragmented libraries into a product.

I’m sure it may not be the goal of some programmers to make node mainstream, but that’s what it’ll take to get there.

Sent from iPhone
http://profile.yahoo.com/FWZGSNJKFRSYN2ZRV6EVJCREEI Joe

“Before Rails, the most popular language for the web was PHP”

Kind of an odd statement. Rails never came anywhere near PHP in actual market penetration much less overtook it. Still today, Rails is on well under 1% of all web sites while PHP is somewhere in the 30-35% range. .NET-based technologies have a similar share to PHP with better penetration in Intranet-development.
http://cnnr.me Connor Montgomery

I’d argue (although I don’t have the stats to back it up) that the PHP “dominance” has a lot to do with popular CMS’s (WordPress, Joomla, Drupal). So, while a PHP back-end may be “more popular”, I think it’d be hard to argue against Rails’ popularity today. It seems like almost every startup or new company utilizes a Rails backend.
Just my two cents…
http://profile.yahoo.com/FWZGSNJKFRSYN2ZRV6EVJCREEI Joe

I guess it depends on how you define popularity and whether the reasoning behind the marketshare numbers matter. And if you look across all startups, not just the ones the ones you might read about on HackerNews I think you will find that the Rails penetration isn’t much better there. And even the HackerNews ones I think there are more Python-based ones than Rails these days. What do you think Pinterest is written in, for example? Certainly not Rails.
http://cnnr.me Connor Montgomery

Solid points, @yahoo-FWZGSNJKFRSYN2ZRV6EVJCREEI:disqus - thanks for the reply. I do know Pinterest (at least the early versions) were a heavily modified version of Django.
Appreciate the reply!
http://petegrif.tumblr.com/ Pete Griffiths

Another good piece. I have 3 observations:
a) the history of online applications is an evolution of the form. All that has happened is that as we have moved from mainframes apps, to mini-computer apps, to client-server apps, to browser-based apps, to single page js apps the form has become embedded in a different context – notably richer media and more widely available (mobile0. I don’t care how rich a webpage is – it is sugar around a set of input fields i.e. a form. The rise of social media which requires UGC is predicated on such form based apps which allow users to input their content. So you are absolutely right that anything that can make building form based apps easier in the context of currently available hardware/software is a big win. It will always be a big win.

b) Does the realtime web change anything? IMHO not really. Of course it has implications for the underlying plumbing and the tools required to facilitate development. But it doesn’t change the central importance of forms. All RT does is change the context for such forms.

c) What of node? You make a great point about the importance of community. The pattern I observe (and I would be interested in the opinions of others on this) is that grizzled old programmers who have experience with client server computing look at node and shake their heads. It’s not new, radical or path breaking to them. It is just a spin on an old song with the only really distinguishing characteristic that there are a lot of js programmers out there and this could enable them to program back as well as front end. The question is whether this is a good way of doing it. IMHO it isn’t but that doesn’t mean it won’t succeed because of community. Google has written a lot of large js programs – the language was never designed for such programs. And Google, having had that experience, is proposing DART as a better answer to the problem. A likely scenario is that the grizzled old heads will get DART but the hordes of relatively inexperience js programmers will overwhelm them.

I am not persuaded that the realtime feed ‘advantage’ will be telling, but I think the excitement of the hordes of js programmers is a real force.
http://www.chadmoran.com/ Chad Moran

Rails isn’t a language, it’s a framework on top of Ruby.

I also doubt that an ability to create a “blog” in 15 minutes makes it worthy of “web scale” applications.

Why does it have to replace these other frameworks? Why can’t it just coexist?
http://twitter.com/pt Parker Thompson

With my less than expert knowledge (caveat required) I think that’s a misunderstanding. Because JS in the browser is event loop based—and because the language begs you to use function pointers—callbacks are commonly used.

However, this isn’t a unique property of the language or runtime, just a common idiom in the community and libraries in common use. I got intimate with callbacks doing desktop Java UI programming!

Further, IO is blocking in Node, as it is elsewhere. Node doesn’t magically solve this problem and the programmer still has to code consciously around IO.

If I were building a realtime web app today I’d be more interested in the client-aside challenges of managing thick-client UIs. That, I think, is where there are big improvements to be made and where tech will give companies a short-term advantage.

Check out Ember.js and Backbone for examples of framework innovation client-side. I think there’s a lot less hype there, but a lot more exciting technology that has the potential to reduce the cost of amazingly rich interactive app development.
nateps

(mispost)
nateps

Hey Andrew,

I recommend you check out http://derbyjs.com/. It is a framework that I have been developing on top of Node.js, and while it is not complete yet, I believe we do a good job providing a solution to a lot of the problems that you addressed. Our goal is to provide a solution very much like what you outlined.
http://twitter.com/leggettsteven Steven Leggett

PHP is extremely popular because of a combination of factors. One of the dominating ones is due to the low cost hosting providers available and ease of use for those who are non technical. One click installations of popular CMS’s like WordPress are a huge factor. Node will never take away that market share, it’s not meant to. You have to remember that specific technologies are best for specific use cases. Why would anyone bother to rewrite something like WordPress or Drupal with Node? It just doesn’t make sense. What does make sense is to rip out various functions that those applications that are still very static, like comments, etc that rely on page refreshing. I believe Node is best to augment, not completely replace other technologies we’re currently using.
Calvin_77

You find relevant statistics here: http://w3techs.com/
77% of the websites run on PHP, only 30% use any CMS. That leaves a lot of sites actually programmed in PHP. That compares to 0.6% for Ruby, the majority of that may be Rails.
http://andrewchenblog.com Andrew Chen

thanks! I’ll check it out.
http://andrewchenblog.com Andrew Chen

lol at unironic use of “web scale” :)

There’s a limited # of developers, and the choice of a product’s primary technology is mutually exclusive (you are unlikely to use both a Rails and node stack simultaneously), and so as a result, node/rails are implicitly in competition with each other.

In some ways I’m sure they will just co-exist but I’m just talking about what will become the most popular technology for novices/intermediates to create their products.
http://andrewchenblog.com Andrew Chen

Great comment, and well said.

Re: real-time, it’s completely anecdotal but I’m just seeing a gap between modern web applications versus what Rails lets you build out of the box. This starts with styling but goes all the way towards real-time validation, feeds, etc. IMHO the majority of startups in silicon valley are building their products in this style versus what Rails can generate with scaffolding right out of the box.

As a result, I see a big market for a framework that gives you all of that for free, but it’s also totally possible I see some overrepresentation based on what people are building out here.
http://andrewchenblog.com Andrew Chen

Yes, true, and I was trying to do my best to focus on custom webapps rather than the 1-click WordPress/Drupal type products. That’s what I was thinking earlier on that PHP used to be the most popular programming language for constructing new applications, but that mantle has definitely moved to Rails over the years. So the context of my article is really about what it might take to make node the successor in this line of frameworks.
http://petegrif.tumblr.com/ Pete Griffiths

The characteristics you describe are related to the biggest thing I see out there in the ‘web’ space is the advent of websites that aren’t websites – one page sites that are really javascript apps. This is a fundamental change to the web and encourages the growth of more walled gardens with hidden resources. Quite a few such sites have the kind of characteristics you describe. So a really good framework here, allied to hordes of js fanboys may well speed the restructuring of the traditional web. Whether you think that is a good or a bad thing is a different question. :)
http://coley.co Stephen Coley

I’ve yet to really dive into Node.js, but I’ve heard of a framework that makes this form creation process a lot like Rails. It’s called Geddy. It provides an auto generated mvc, and has ActiveRecord-esque modeling. I’m sure some of you have heard of it.

Great post! I’m still a php guy, but I would love to spend more time in the node.js environment.
George Sovatzis

Having a VB6/VB.NET/ASP.NET (Webforms) background, its easier for me to start thinking forms and event-driven programming when I design an application (either Web or Thick-client). Also, this is the way that I will communicate my progress with Sales, Project Managers etc etc. I now use Java/Javascript for Web development and it wasn’t easy for me to shift to the MVC pattern, where I firstly have to think models, controllers and actions and it seems that in this pattern the view (forms) comes last at the process. Lately I am checking some PHP frameworks (I’d like to shift to PHP for Web apps) and I see that the same pattern continues there.

I really want to see a Web framework, language …whatever that will come complete with its IDE and will allow the developer to start from the user experience, create a mockup of the screens by throwing controls, finalize with Business people and then add application logic either by following the MVC pattern or by using event driven programming, below every form control like in VB. If this framework comes with an easy way to integrate Realtime/AJAX on a web form I think it will be a huge win!

I strongly believe that this would make the web development process more easy to novice-intermediate programmers and more fun. I also agree that real time web apps (that look and behave like thick client apps) are the future and up to today, developing such an app is not easy or fun (I have some experience with the Dojo/JQuery frameworks).

I have briefly checked the Wavemaker platform (wavemaker.com) and I think that they have successfully applied the form driven development to web apps (but I don’t know how it behaves on the Real time/Ajax field).
http://andrewchenblog.com Andrew Chen

Yes! That’s exactly the kind of apps I’m talking about. I am definitely looking forward to seeing them get built, and I tend to prefer the user experience. It’s really fulfilling the original promise of Java applets from back in the day :)
http://petegrif.tumblr.com/ Pete Griffiths

There is a real user benefit in terms of usability. There is also a real cost in terms of transparency. For example, you can’t crawl them without the active cooperation of the site. It’s harder to link to content on such sites etc etc
http://www.facebook.com/everton.inocencio Everton Inocencio

I agree with much of what he wrote,
and although I like a lot of PHP, and have studied it since 2009 on their own, now suffer horrible to work with codesthat make my head hurt: x
I saw some of the Rails and liked it, I recently read articles and tutorials Node.js and looks very attractive too.
And the popularity of Rails, said in the local context, right?
Antoine Fraticelli

Your stats need to focus on new tech company (and not website or website made by web agency because they reused their home made CMS in many case).
http://www.facebook.com/javier.roca.a Javier Roca

JavaScript is the lingua franca of the web, Node.js is a Event-driven I/O server-side that use Google V8 engine the thing that makes chrome fast, i have 7 years developing software with JavaScript if you need a blog i recommend you the Extjs JavaScript framework http://www.sencha.com/products/extjs or the Yahoo YUI framework, extjs was build by a ex-staffer of yahoo yui project, thanks to node.js JavaScript is ubiquitous client/server side, the PHP, Rails and other server side developers create forms and html tags with the server, JavaScript developers use the client side for the UI and manipulate the DOM, the fuzz about Node.js is because the JavaScript developers pass the data with ajax request, xmlrequest, http request, web sockets socket.io, etc and use the server to process the data with php, .net , etc now with node.js we can handle events on the server side, create the server with the headers, etc side, maybe this namespace that use socket.io will help you http://nowjs.com/ everything on JavaScript is a object helloworld.initialize oreilly book is a good start http://ofps.oreilly.com/titles/9781449398583/ if you want a scalable app i recommend you read about NoSql like Mongodb http://blog.mongodb.org/post/812003773/node-js-and-mongodb ”In the last 10 years, the Internet has challenged relational databases in ways nobody could have foreseen. Having used MySQL at large and growing Internet companies during this time, I’ve seen this happen firsthand. First you have a single server with a small data set. Then you find yourself setting up replication so you can scale out reads and deal with potential failures. And, before too long, you’ve added a caching layer, tuned all the queries, and thrown even more hardware at the problem.
Eventually you arrive at the point when you need to shard the data across multiple clusters and rebuild a ton of application logic to deal with it. And soon after that you realize that you’re locked into the schema you modeled so many months before.
Why? Because there’s so much data in your clusters now that altering the schema will take a long time and involve a lot of precious DBA time. It’s easier just to work around it in code. This can keep a small team of developers busy for many months. In the end, you’ll always find yourself wondering if there’s a better way—or why more of these features are not built into the core database server.
Keeping with tradition, the Open Source community has created a plethora of “better ways” in response to the ballooning data needs of modern web applications. They span the spectrum from simple in-memory key/value stores to complicated SQL-speaking MySQL/InnoDB derivatives. But the sheer number of choices has made finding the right solution more difficult. I’ve looked at many of them.
I was drawn to MongoDB by its pragmatic approach. MongoDB doesn’t try to be ev- erything to everyone. Instead it strikes the right balance between features and com- plexity, with a clear bias toward making previously difficult tasks far easier. In other words, it has the features that really matter to the vast majority of today’s web appli- cations: indexes, replication, sharding, a rich query syntax, and a very flexible data model. All of this comes without sacrificing speed.
Like MongoDB itself, this book is very straightforward and approachable. New MongoDB users can start with Chapter 1 and be up and running in no time. Experi- enced users will appreciate this book’s breadth and authority. It’s a solid reference for advanced administrative topics such as replication, backups, and sharding, as well as popular client APIs.” ,regards
http://andrewchenblog.com Andrew Chen

spam.

@andrewchen