JavaScript Ecosystem Overview (2018)

JavaScript , one of the pillars in which the web content engineering rests, turns twenty-three this year. It’s been around for a long time now and, no matter how you look at it, its importance is undeniable.

Here I will take the ambitious task of analyzing the JavaScript ecosystem, starting by describing the history of this programming language and, later, covering the different front-end and back-end technologies that exist in it.

The Language

The JavaScript programming language was created by Brendan Eich in 1995. It was the time of the browser wars: Internet Explorer and Netscape Navigator were fighting to provide new features and there was no time to lose.

Mr. Brendan Eich, an american technologist, was working for Netscape and had to create a programming language that would allow web developers to embed small programs into webpages, making them more interactive and dynamic. The final product had to be flexible, easy to learn, have a popular appeal, and be easily implementable.

Mr. Eich only had 10 workdays to produce the first implementation - a fact that we confirmed with the man himself - but, against the odds, he did succeed to deliver.

Here are some existing programming languages that influenced the result:

  • Scheme - the functions as first class values and the dynamic typing;
  • Perl - weak-typing;
  • Self - simple-to-implement prototype-based object-system;
  • Java - mainstream syntax and naming conventions.

The language was developed under the name Mocha and, later, published together with Netscape under its current de-facto name: JavaScript. This name choice was a marketing move by Netscape, in order to showcase its new creation piggybacking on the hype of the day: Java.

Since JavaScript was a success, Microsoft reverse-engineered it in order to compete with Netscape, publishing its own version under the name JScript.

From the beginning, there were several incompatibilities between the different implementations of the language and, in order to reduce the impact of this problem, Netscape submitted JavaScript for standardization by the Ecma International standards organization in 1997.

The resulting standardized version was named ECMAScript and progressively attained universal acceptance as the language specification. Despite this, the name JavaScript remained in use as the de-facto name of the language.

The Evolution of the EcmaScript Standard

In order to add new features to the language and to become compatible with the work of other standardization organizations, the standard has been changing from time to time.

The deliverables for each version are known as "EcmaScript " and here I’ll list a brief description of some of the most relevant changes for each.

EcmaScript 1

The first version of the language standard.

EcmaScript 2

This version was produced in 1998 when the language was also standardized by ISO/IEC. It updated the Ecma standard so that it would match the document produced by ISO/IEC.

EcmaScript 3

In 1999 there were several new features being added to the language. The most conspicuous were:
Regular expression literals;
New control statements;
Exception handling.

EcmaScript 4

This version draft was abandoned in 2008 due to disagreements about its feature set. It was decided that the output was too radical to be introduced and there was an agreement to create an intermediate version before such drastic developments would take place.

EcmaScript 5

The version that supported most browsers in early 2016 was created in 2009. Its most noticeable features were:
High-order iteration functions (map, reduce, filter, forEach);
JSON support;
Getters and setters;
Better reflection and object properties;

EcmaScript 5.1

Just like EcmaScript 2, this spec was produced to align the Ecma standard with the document that was produced by ISO/IEC for EcmaScript 5.

EcmaScript 6 (AKA ES6, EcmaScript 2015, Harmony)

This version added several big changes in 2015, namely:
Promises.
Modules;
Classes;
Block-scoped variable declarations (let);
Arrow functions;
Template literal;
Spread operator;
De-structuring assignment;
Parameter default values;
Rest parameters;
Symbols;
Generator functions.

EcmaScript 2016 (AKA ES7)

This version only made minor changes to the language, such as adding an exponentiation operator and specifying the <Array.includes> method.

The browser support for this version is incomplete though there is a transpiler that converts ES6 code into older ES versions.

EcmaScript 2017 (AKA ES8)

Mainly adds support for async functions.

EcmaScript 2018

Still a very early draft.

Frontend Technologies

It was with the advent of AJAX in 1999 (standardized in 2006) that JavaScript started to grow. Before AJAX, JavaScript was mostly used to animate and provide interactivity to some webpages. Though, as this interactivity would not involve interchanging data with the server - it would only consist on accessory display-logic.

With AJAX, JavaScript started to perform key tasks in our web-applications application-logic, to the point that nowadays it’s hard to find web-applications that do not require a JavaScript-enabled browser in order to work. It was after this change that the JavaScript ecosystem started to expand as a client-side programming language.

DOM Manipulation Libraries

The main needs of the developers working with AJAX web applications are:
• Performing AJAX calls;
• Manipulating the Document Object Model (DOM) - the data structure kept by the browser representing the webpage's structure;
• React to the user's interactions.

The core JavaScript functions providing these features were not very practical and sometimes their behavior was not compatible with different browsers.

The first libraries to become popular addressed precisely these three tasks, providing a collection of functions and objects that made these operations easy and functional on every browser. Some of these libraries would also provide plenty of other general-purpose functions:

Prototype

A JavaScript framework that was released in 2005 as part of the Ruby on Rails AJAX support. It directly extended many JavaScript core objects with behavior that would be somewhat similar to the Ruby programming language, also adding a class-based system to the language.

These core extensions were regarded as troublesome by other programming communities and the provided functions as weirder than the following competing library:

jQuery

The jQuery framework was released in 2006 and - unlike Prototype - has let the language core as it were and rather focused on providing plenty functionalities related with the DOM, events, asynchronicity and effects.

One feature to highlight is that jQuery supports a plugin system that allows library programmers to extend its functionality. This library was the most successful library of this kind, being used nowadays in the majority of the websites.

Despite jQuery’s immense popularity, as well as its ability to provide many useful functionalities, it’s important to notice that modern versions of JavaScript already provide many of the functionalities that made jQuery popular. Maybe you might not need jQuery at all.

It’s increasingly popular not to use jQuery and to rather rely on either only the current core JavaScript functions or on libraries that are specialized in each specific task (e.g. fetch for AJAX calls).

MVC-like Frameworks

While using DOM-Manipulation libraries, it’s common to keep the application data in the DOM itself, thus there are DOM-Manipulation selector functions that have to be called when needing to access or modify this data.

This kind of code is often filled with selector-names and easily becomes a mess as the application's complexity evolves. There is a set of frameworks that allow us to address this problem in an elegant way by leveraging on application patterns that are similar to the famous Model-View-Controller (MVC) pattern.

On the other hand, there is a recent trend in which web-applications are fully loaded in the browser on the first page-load, performing every subsequent operations via AJAX.

This kind of applications is called Single-Page Application (SPA) and, as their code is often quite elaborate and complex, it’s recommendable to pick one of these frameworks when implementing a SPA. Let’s look into some of them.

Backbone

Backbone is an Model-View-Presenter (MVP) framework published in late 2010. it’s focused on SPAs and enforces the separation between application-logic, display-logic and templates.

The communication between these entities is managed in an event-based fashion and it also supports a Router class that allows the applications to react to URLs and to update these according to the triggered events.

Angular

Angular is a framework implemented by Google that used to follow a MVC architecture and that has evolved into an MVVC framework. Similar to Backbone, it’s focused on SPAs and enforces the separation between application-logic, display-logic and templates.

It’s interesting to notice that Angular allows the programmer to extend HTML by creating new tags that can be used in the templates and that encapsulate parts of the page.

Other distinctive feature is that it allows the establishment of two-way data-bindings in-between the templates and the JavaScript objects that are to be rendered by them.

In this way, any update to these objects will automatically trigger HTML updates, and - in a similar way - the JavaScript objects will be updated when their corresponding template input fields are changed.

Virtual-DOM based frameworks

As we have described on the previous sections, the display layer of our web applications moved from a simple computational model where the interface to display was only function of some data calculated by the server-side code into something much more dynamic and complex, where it would depend not only on the retrieved data but also on the user interactions and scripts that were processing these.

This complexity often translates itself into complex code that’s hard to share and to reason about. Thus, some people at Facebook started experimenting with a simpler computational model to code the front-end. A computational model that would allow us to think of what is displayed in the page again as result of the state of some few variables instead of the result of the interaction between a "zillion" of Model-View-Controller classes.

The Virtual DOM

In order to make such change take place, something radical had to happen: if we wanted the display to only be the result of applying a function to some state variables we would have to stop manipulating parts of the DOM directly and instead render each version of the display as a whole.

As rendering the whole DOM whenever changing something in it would be too slow, **it was decided to map the DOM onto a lightweight representation of it that would be easy to regenerate and compare. **

Thus, whenever changing one variable, we are able to generate a new version of this lightweight representation and compare it with its previous version, thus calculating which parts of the DOM need to be updated. We call Virtual DOM to this lightweight representation of the DOM.

React

React was the first framework to use a Virtual DOM. It was created by Facebook and the architecture that needs to be followed by the applications using it it’s minimal.

Regarding MVC, React does not stick to it and is rather only about the View part of the code. With it, the View is represented as a tree of entities called components that might be composed of other components themselves.

A component holds some state, can receive properties upon creation, and knows how to render itself for a given set of properties and state variables. Whenever its state is changed, a new version of its display's Virtual DOM is rendered and compared with its previous version, and the needed DOM is updated accordingly.

If some interactions performed on a given component are to change the state of parent components, then the parent component has to pass the handling code to the subcomponent in the form of a callback. This approach guarantees that components are loosely coupled between themselves, thus making them quite easy to be reused.

One rather controversial choice taken by the creators of this framework is, instead of using HTML templates like the other frameworks, an HTML-like representation of the code is written as XHTML mixed within the JavaScript code, therefore requiring the use of a compiler that knows how to transform this JS and XHTML mix (called JSX) into HTML-generating JavaScript.

Flux and Redux

The component-based approach of React is effective for smaller applications, though for structuring bigger applications Facebook proposes an architecture called Flux, in which the state of the application is stored away of the components in something called state containers.

The original implementation of the Flux architecture was rather complex but there is a simpler and more recent one called Redux that’s becoming very popular and starting to be used even to implement application back-ends.

Its architecture is functional and instead of directly mutating state, the programmer is meant to create new versions of it by applying reducer functions, thus allowing easy Undo/Redo and time-traveling debugging.

Vue

Vue started as a MVC framework similar to Angular but focusing on being simpler and requiring less boilerplate. The last version already features a component-based approach and a virtual-DOM in a similar way that React does.

The documentation describes it as a progressive framework, in the sense that it is thought to be progressively adopted and thus allowing a user to start by only using the view-part of the framework and later add more complexity if needed. There is a focus in reducing boilerplate and the number of files used when possible.

GraphQL

When developing complex Web applications people noticed that when fetching data through AJAX calls there was often under-fetching and over-fetching as well as that it was not easy to manage caching.

In order to sort out this problem Facebook developed a query language that allows describing the data each part of the interface needs. These queries are meant to be interpreted and aggregated into efficient sets of AJAX requests by a graphQL client.

There are at least two GraphQL clients avaliable:
Relay, the former implementation by Facebook, highly tangled together with React;
Apollo, which is a more recent framework-agnostic project.

Mobile Development Frameworks

While JavaScript conquered the web, it also made advances as a mobile application development technology. It’s often chosen as a way to deal with mobile in a cross-platform way.

Apache Cordova

Apache Cordova enables the development of mobile applications using JavaScript CSS and HTML. It also supports a FFI mechanism through which the programmers can include portions of native code and easily create bridges in between JavaScript and native functionality.

Ionic Framework

While Apache Cordova allows to develop mobile applications with JavaScript, it does not provide the high-level features needed to do so. Ionic is a framework that provides such functionality: widgets, screen-transitions and other functionality. It follows an MVC architecture.

React Native

React Native applies React's component based approach to the mobile development, but instead of running JavaScript and allowing it to interact with native code it rather compiles the JavaScript to native code.

The Back-end Ecosystem

Despite JavaScript being used on the server side since the mid-90's, its implementations were quite slow and it hasn't become popular for such use before 2008, when Google published the V8 JavaScript engine bundled together with the first version of Chrome web-browser.

V8

When implementing the called V8 engine, Google used optimization techniques that had both been created during the development of already forgotten programming languages compilers, namely of Self's and Strongtalk's compilers. The result was remarkably fast for a dynamically typed programming language like JavaScript, vigorously surpassing other implementations' performance.

On the other hand, both Chrome and V8 were released as open-source software (with a BSD license), thus allowing anyone to build their work on top of them and to distribute them as part of any software without having to pay royalties to Google.

Node

It did not take long until Ryan Dahl, an american software engineer, created an environment called Node.js where JavaScript could be run on V8 away of the browser. **Besides bringing V8 to the server-side, Node is also event-driven, thus allowing any blocking IO operations to be written in an non-blocking asynchronous fashion. **

This way to address IO together with the speedy V8 made Node suitable to develop server-side real-time applications that required the swiftness of non-blocking IO together with the expressivity of a high-level programming language such as JavaScript.

NPM

As JavaScript's core specification is quite small and not thought for the server side, using it for such task usually requires plenty of external libraries. There is the need to have a package manager for Node, thus allowing developers to publish packages and easily retrieve libraries that are to be included in the developer's applications.

The Node Package Manager (NPM) is such package manager, just in the style of Perl's CPAN or Ruby's RubyGems.

Module Loaders

As a browser language, JavaScript missed statements to specify dependencies between files and to tell the compiler to load these. In order to solve this problem there were two different APIs being specified and competing for this task:
• Asynchronous Module Definition (AMD);
• CommonJS Modules;
ES6 Modules.

In the ES5 world, Node.js implemented CommonJS modules while front-end programmers preferred AMD, thus creating chaos on this field. There was thus invented a way to require modules and declare them called Universal Module Definition (UMD), which is rather ugly but makes modules compatible with both module APIs.

Anyway, this mess is about to be wiped by the adoption of ES6 - that already supports a language-level constructs aimed at dealing with modules.

Module Bundlers

When using modules while developing web applications there is often the need to merge the different module files into a single file to be included by the web application. There are two libraries that allow developers to perform this task:

Browserify

Browserify is the oldest web module bundler and allows developers to use the CommonJS module syntax when programming web-applications.

Webpack

Webpack is a more modern web module bundler that does not only bundle JavaScript but also other assets such as images and CSS. It allows the different assets to be hot reloaded by the browser when there is a change in the modules code.

Task Runners

As projects become bigger and there is the need to transform their assets in order to run them, there's the need to write scripts that manage the execution of these transformations - something like make, ant or rake. There are two dependency-based task runners implemented in JavaScript:

An older one called Grunt - in which data that’s intermediate to the different transformation steps is kept in temporary files. And Gulp - which is a more recent one that rather uses memory instead of the temporary files.

Express

As JavaScript is a language born out of the browser, why couldn’t we use it to develop the server side of our web applications? Express is the framework tackling this issue. It’s aimed at being minimalist (like Sinatra), though there are plenty of big libraries developed for it (e.g. Automatic Backoffice generation).

MongoDB

As V8 was used to create Node, it was also used as interpreter by the MongoDB document-oriented database. When interacting with the database the data is shared in JSON-like structures and the query language is about invoking JavaScript functions. This makes this database very popular amongst JavaScript developers.

Yeoman

Yeoman is an interactive project skeleton generator written in JavaScript. It allows to create plenty of JavaScript project types with different configurations - though it’s not only aimed at JavaScript, it may also be used to generate skeletons of Java projects, for instance.

Opinionated Full Development Stacks

There were some attempts to bundle both the tooling and the technologies aimed at creating full-stack applications. Here I'll talk about two of these attempts.

Meteor

Meteor is a web framework that uses MongoDB as database. It uses a publish-subscribe protocol applied to resources, allowing clients to get updates when the data is updated on the server.** It can be used with any client-side framework.**

MEAN

The MEAN stack stands for a package with the following set of technologies:
• MongoDB;
• Express;
• Angular;
• Node.

It also comes with the "mean" command-line tool, that allows generating projects, files and performing other useful command-line tasks.

MERN

The MERN Stack is similar to MEAN but focuses on React rather than Angular.

Transpilers and related tools

As not everybody is satisfied either with the JavaScript language or with its support by the browsers, there have been produced several transpilers which convert other programming languages into browser-supported JavaScript versions. Follows a few examples of these:

Babel

Babel is a transpiler that converts modern JavaScript into older, browser-supported JavaScript. It’s very customizable and allows cross compiling between several JavaScript versions.

Web Assembly

Web assembly (also known as wasm) is a bytecode format now supported by all major browsers as an alternative to JavaScript. For now there are few compilers targeting Web Assembly. It should be noticed that - for now - Web Assembly does not feature a garbage collector.

Thus, when compiling to wasm one should either use a programming language that requires no garbage collector or will otherwise have to compile a runtime that enables garbage-collection together with the remaining application/module.

Typescript

This transpiler, created by Microsoft, converts a gradual typed superset of JavaScript called TypeScript into JavaScript.

Flow

Flow is a static type checker developed by Facebook. As Typescript, it is a gradual typed system, though Flow is only focused on the types and does not add anything else to the language besides the type system.

Elm

Elm is a framework and a pure typed functional programming language inspired by Haskell that compiles to JavaScript and follows the same architecture as Redux, which is heavily inspired in Elm.

Unlike other static type systems, this one guarantees 100% type coverage through type inference, thus not requiring the programmer to explicitly enter any type signature at all. The type system is so safe that it does not allow any runtime exception to occur.

ReasonML

This is Facebook's take on using a typed functional programming language to generate Javascript. ReasonML has O'Caml semantics and a Javascript-friendly curly-braces-based syntax. It can be seen as an Elm's cousin who is not so keen on functional purity and who is backed by Facebook.

Concluding (for now...)

JavaScript's ecosystem tells much about the history of the language, and it’s amazing how a language that was originally specified in 10 days gained such traction.

It also shows well how the evolution of software sometimes requires patchy solutions from which the essence is later distilled into standards and more mature tools (e.g. the modules system). Another noticeable element is how the application of old ideas has driven breakthroughs in the ecosystem (e.g.: V8).

Though, we should not be naïve to the point of believing that this success is only the product of the language's feature-set. Its standardization as "the programming language of the web browsers" had a very big impact in getting JavaScript the status of Lingua Franca that it has unprecedentedly attained.

Current technological needs regarding speed, safety and expressivity in browser applications are creating pressure both to turn JavaScript into a different language and to replace it by something more flexible than a high-level programming language.

WebAssembly is one of such projects, and, if successful, it might represent a big turning point in the history of this language.

If such a change takes place it’s hard to tell if JavaScript's status will be kept or whether this "Babel tower" will fall apart, scattering the browser-programming landscape across the diversity that has always characterized the history of programming languages.

At Imaginary Cloud, we simplify complex systems, delivering interfaces that users love. If you’ve enjoyed this article, you will certainly enjoy our newsletter, which may be subscribed below. Take this chance to also check our latest work and, if there is any project that you think we can help with, feel free to reach us. We look forward to hearing from you!