*(not to be confused with node.js)
Nodes was created and developed by Nick Nikolov and Marcin Ignac between 2017 and 2019 inside Variable. Even as the very nature of the tool is to experiment and explore new ways to play with code and data, Nodes has been used in several production-grade projects and tested in various real-world scenarios. We've used it to create realtime 3D graphics installations, explore and visualise data, experiment with AI and export results in various formats like images for print, videos for social media, 3D models for mobile AR and data files for further processing.
On this page we are going to explain our motivations, the evolution and the current state of Nodes and how it changed our approach to creative coding.
Here’s a very quick primer if you’re a programmer but you’ve never heard of visual or node-based programming. In some tools, the actual programming language is visual, instead of writing text you construct your program visually. Think Scratch, or a modern take like Luna (which technically has a dual nature – visual and textual).
Most of the popular visual programming tools today however use the visual interface as an abstraction one level above textual code. Usually, instead of files you have 2D representations of your program. Very often they are rectangles of some form, connected by wires of some form. Just like most programming languages have similar syntax (for loops and class constructs), visual tools generally tend to be rectangles wired up together representing data flows. This makes them very popular in areas like visual effects programming, real-time graphics, data-processing pipelines, procedural architecture and so on.
These days, neural networks is another area where computational graphs tend to be visualised with nodes and wires.
At Variable – our data visualisation and generative design studio – we are programmers first and foremost. But each of us has their own unique background: interaction and graphic design, real-time graphics, new media art, music production etc. What these fields have in common is that artists rely on professional-grade, usually very visual and sophisticated software-based tools. As our main medium of expression shifted to code, we've enjoyed unparalleled creative flexibility but at the same time we traded that for our ability to “see”.
We made a provisional breakdown of the modern visual and web-based programming landscape in this Visual Tools Landscape Figma Board. You can see high res version here: visual-tools-landscape.pdf.
As of June 2019 we are currently at version 0.11.0.
You use Nodes by installing the package from npm and run a local server. A cloud-based version for collaboration and client work was always one of our goals but isn’t a priority just yet. The benefit of running locally is that you maintain control over your data and don’t have to worry about assets sizes.
In Nodes you write programs by connecting “blocks” of code. Each node – as we refer to them – is a self contained piece of functionality like loading a file, rendering a 3D geometry or tracking the position of the mouse. The source code can be as big or as tiny as you like. We've seen some of ours ranging from 5 lines of code to the thousands. Conceptual/functional separation is usually more important.
You create a node by double clicking anywhere on the Graph Editor canvas and choosing a node from a list of templates. Nodes can declare public properties called ports allowing them to receive and send data via connections. There are two types of ports:
So what’s happening here?
radiusin param. We additionally connect
Time.timeout param to
Rectangle.rotationin param to animate it.
Each node is defined by its code. By declaring triggers and params and evaluating the node, at the same time the graph view of the node is updated and the inspector view is populated with widgets reflecting given port type.
The graph canvas is of unlimited size which allows you to organise your code spatially and navigate around by panning (space + drag) and zooming (mouse scroll) or search (
/ + type, see Powerful Search.
You can select a node by clicking on it or select multiple nodes by Shift+Click or drawing selection rectangle around them. Nodes can be deleted by pressing Backspace, and duplicated using copy (Ctrl/Cmd+C) and paste (Ctrl/Cmd+V). We support copying and pasting nodes between different graphs and browser windows. As the copied node is simply a piece of JSON, you can even paste piece of graph into an e-mail or Slack message.
It's also possible to Undo/Redo your modifications using familiar Cmd+Z, Cmd+Shift+Z (or Ctrl on Windows).
One common differentiation between visual programming tools is the level of abstraction they operate on. Some tools like Origami for example are meant to prototype interactions of user interfaces and not much more. Others – like Pure Data for instance – are meant to be very low-level and let users craft their own environments. Some – like UE4 Blueprints or the Unity Visual Effects Graph – are embedded and part of a larger system (e.g. a game engine environment).
Nodes is more directly comparable to Cable.gl, FlowHub or MaxMSP does come with some batteries included but there are less preconceptions of what you use it for. However, unlike most of them we adopt a more programmer-first approach where the nodes themselves are generally a lot more high-level and users are encouraged to peek inside, copy and modify, and write their own. While you could drop on a lower level and express language-level semantics like if-checks and for-loops, or math concepts like add or multiply on a node level we generally don't feel that is a productive way to work and prefer to write that logic in a textual programming language. This might feel conceptually familiar if you have used NoFlo.
Whatever your preference, with the built-in code editor we are able to fluently move between such levels of abstraction and create nodes that range from rounding up a number to a complicated particle system running on GPU. Edge cases can be handled by simply copying a node and adding more code. Reusable components can be created by splitting big nodes into few smaller ones. As long as we keep the corresponding types of input/output ports intact, this can be done live on a running graph without needing to reload the whole application. We focused on opening, modifying and seeing changes as straightforward and as instant as possible.
This brings us to the most powerful feature of the code editor: live eval. At any point in time while editing the code, you can evaluate it by pressing
enter. All ports already present in the node will have their values and connections preserved. If you make a mistake or typo you can edit code and compile again or roll back to the last saved version.
Changing code on the fly while the state and relationships are both persistent and dynamic can be very freeing. If we introduce far ranging and disruptive changes all nodes with errors will be highlighted in the graph and are usually quick to find and fix. Nodes tries to preserve the last working state as much as possible giving the programmer the chance to fix all mistakes until everything runs smoothly again.
We believe that while an unlimited 2D canvas can be very helpful to think about the “bigger picture” and reason about the architecture of your app, zooming in and jumping around the codebase should be just as fast and fluid. To search you can simply press
/ and start typing. Use up/down arrows to jump between results and automatically focus on the given node. To search for code prefix your search query with
". For example
"ctx.texture2D to find all nodes that call
ctx.texture2D function. You can then select a node using arrows and press
enter to open code editor. The editor opens at the line of code that matched your query.
To help you better understand the internal flow of the application, we fade out nodes with trigger ports that haven't been updated in the last frame. Similarly, the profiling mode visualises the CPU usage of each node. By colouring nodes by the amount of time the trigger took in the previous frame, we can identify hot paths of the graph worth optimising.
We think we are just scratching the surface here and other similar dynamic visualisations of the source code and the program logic can be very helpful in large projects. Before Nodes, we would very often created custom “debug views” alongside the deliverable of your project but now it usually happens right there in the graph.
Continuing that line of thinking, you can create a generic textual comment box using
left mouse click. We allow for some basic color and font size options. Dynamic node comments are created by adding
'my comment' and are attached below the name of the node.
We use this feature to visualise global state as serialised JSON, loading status, timers, simple graphs and many more. It allows for a basic but powerful and really fast way to “annotate” your source code and present whatever is the most important top-level information in your program.
Node comments can also have images – for example looking at raw textures in big rendering graphs or even custom “drawings” using an image buffer. See more in Visual Literate Programming.
This functionality allowed us to build tools for other designers, screen based installations and visualisations on the web or anything else that has to be tweaked on the fly.
Some of the more common modules that seem to work well in such an environment and that we keep coming back to.
The choice felt natural as both PEX and Nodes are developed at Variable. Having full control over the source code allowed us on improving features of PEX that were needed for Nodes and vice versa. One of the drawbacks of that is less examples and documentation compared to eg. Three.JS that increased the learning curve for new Nodes users.
In the last 2 years we have used Nodes for multiple types of projects and output formats. Below is a list of a few possible use cases.
We noticed two things when working on bigger graphs and writing comments for our nodes.
Firstly, spatial layout of the graph allows you to organise nodes in area or sub-graphs responsible for distinct functionality of your app (e.g. data processing, geometry generation etc) or part of user flow like home screen or an interactive scene. This layout provides an excellent way to see an overview of your application and explain it to a co-worker or client before jumping into the code.
On bigger projects it also helps coming back to an old graph and refreshing your brain on how it works (where as normally you would usually have a bunch of text files and hopefully a README file). By creating empty nodes, giving them appropriate names and writing comments in the graph (not code) you can brainstorm the architecture of your app similar to UML or data-flow diagrams.
Secondly, as we mentioned a few times already, we naturally started including richer data in the node comments like text templates with live values, images and mini graphs. We use those not only for debugging but also for navigation as different parts of the graph become recognisable even on further zoom levels. This allows you to quickly read or scan the graph and get a rough idea of what it’s doing. Our vision is to extent that functionality and have true digital canvas with your graph, snippets of code, visualisations and references all in one place. Which brings us to...
Creative coding environments like Processing popularised idea of "sketching in code" – quick and dirty prototyping allowing you to explore multiple ideas quickly. This type of programming is especially suitable for visuals problems and early explorations where focusing too much on code structure and optimisation can slow us down and limit possibilities. If we compare this way of working to pen and paper sketching, the analogy breaks down quickly as we would be limited to draw only one thing per page. Having an explicit "edit, compile, run" loop (as opposed to surgical live changes) and no easy way to maintain multiple versions of the code (as opposed to quickly copy-pasting nodes or parts of the graph) usually hold you back from easily exploring multiple branches of your ideas. Small changes in the ergonomics of your tool can amount to unlocking hidden creative powers.
Nodes on the other hand allows you to not only edit code live but also run multiple branches of graph at the same time e.g. 2D visualisation in d3.js, basic raw WebGL visualisation and full-blown 3D scenes with physically-based lighting using pex-renderer. Another example would be to start with CPU implementation of particle system and later implement GPGPU version while keeping the same input data and final rendering parts of the graph, and even run both versions at the same time.
Programming in Nodes enables easier code decoupling and in combination with copy-and-paste, we quite often "transplant" parts of the graph between projects (e.g. data parsing pipeline or lighting setup).
In July 2017, after 5 months of work (on what was back then still a side project for our studio), we could create nodes from templates, program new ones in the browser, live recompile them with Shift + Enter and see instant changes, expose and tweak their parameters and manually add npm modules to be required in their source code.
It was so much fun to use that we immediately started using it for prototyping in our commercial projects. Despite common crashes, sporadical data losses and UI quirks, the productivity gain was tremendous. See Learnings and Observations for more info on how our workflow has changed after adopting Nodes.
As our graphs grew, we experimented with breaking away from the columns and grid-based interface so common in content creation tools. Beside the appealing aesthetics, it proved to be only minor improvement from a UX point of view as you still need your code editor, inspector and preview to be visible and accessible.
Data in our graphs flows from top to bottom; inputs are at the top of the node, outputs at the bottom and text labels in the middle. This way vertical space becomes very scarce. Borrowing some ideas from Houdini, we moved labels outside of the nodes to compress the graph vertically and remove constraints on the name length of the nodes. That layout allowed us to add node comments used to describe node behaviour but also peek at its internal state. We talk more about it in [Visual literate programming] section.
These are only few major iterations out of many many more developed both in code and manually in Figma. You can see the whole journey in this Nodes Themes Figma Board or ui-evolution.pdf
We keep using nodes on a daily basis and actively work on making it better.
That said, Nodes is not yet ready for public release.
Subscribe to our newsletter to be first to know about public release.
Want to know more or schedule a demo? Drop us a line: firstname.lastname@example.org