The D3 Update Pattern: matching your data with visuals

What is the update-pattern all about?

In a nutshell, the update-pattern is a D3 methodology we can use to link pieces of information with graphical elements; and later, manipulate the appearance of such elements.

We, humans, are visual beings; therefore, it is natural trying to give a graphical representation to the data. The great power of the update-pattern is how it allows us to alter the visual properties of the elements (for instance, color, size, position, etc), based on the information linked to them. This is possible thanks to the mechanisms the update-pattern offers to manipulate the DOM.

As you may recall, the DOM is the object that keeps a record of all the active HTML elements in the browser’s window. D3 maintains a register of this data-element relationship. It is also possible to change what datum is linked to what element.

We are going to call join-process to the mechanism used to link pieces of information to visual elements. This is an important concept used throughout this post. Since D3 is a JavaScript library, the pieces of information will usually follow the JSON standard, while the visual elements will be HTML elements, most likely, SVG (Scalable Vector Graphics) elements.

If you are yet not sure what the SVG elements are, actually, it is very simple; they are just HTML elements used to draw shapes on the browser; for instance, rectangles, circles, triangles, etc. What is really important, though, is to make sure that these elements are embedded inside a <svg> element.

The elements we are going to be working with, in this little tutorial, are: <rect> (a square or rectangle) and <circle>. As an example, allow me to show you a snippet of the code we’d use to draw a green square; with 50 pixels of width and height; and located at the (0, 0) position of our canvas.

    <rect x="0" y="0" width="50" height="50" fill="green"/>

By the way, I’ll be using the term “canvas” to refer to the <svg> element employed to draw the graphic elements. Please, do not confuse this term with the <canvas> element; I won’t make use or further refer to such element.

So, how about we learn more about this versatile technique?

The selection states

To better understand how the update-pattern operates, imagine you have two sets: a data-set and an HTML elements-set. After we execute the join-process, there may be 3 possible selection states: enter-selection, update-selection, and exit-selection. In the next diagram, we can see an overview of this process:

Inside the update-selection, we can find the HTML elements that were present on the browser before the join-process took effect, and have been linked to some datum. If before the join-process any, the data-set or the elements-set were empty, this selection will also be empty.

On the other hand, if we had more pieces of information than HTML elements before the join-process, these “extra” data, that cannot be linked to any element, produce the enter-selection. On the contrary, when each datum is linked to some HTML element, this selection will be empty. To be more precise, there aren’t really any elements in this selection, but rather markers for the data; however, as we will see, we can use said markers to manipulate the DOM and create new elements.

Finally, in the cases where there exist more HTML elements than pieces of information, the “extra” elements fall inside the exit-selection. When the number of elements is minor or equal to the number of datum, we will find this selection to be empty. Usually, the elements found inside this selection are those we want to be removed from the DOM; why is that? Remember that our aim is to create a visual representation of the data; if you think about it for an instant, you’ll realize there wouldn’t really make any sense to keep graphic elements that do not represent some datum.

Next, we are going to examine several cases for you to better understand the join-process and the resultant selection states.

How I present the examples

Almost all the examples I’ll be using have two main parts: on the one hand, a diagram laying out the selection states we have already discussed; these will vary depending on how the join-process was performed.

On the other hand, I’ll also present to you the code working the magic; below it, you’ll be able to see a <svg> element with the visual result of the code’s execution. The best part is, though, that the code lies inside interactive blocks; so, you can modify it and see the result! To do this, you only need to change the content inside the text area and press the “run” button located under the code block.

By Pressing the “restart” button, you can return the interactive block to its basal state; i.e., before you ever pressed the run button. Both, our <svg> canvas, as well as the editable text area code will be restarted. It’d be convenient to press this button before you change and run your code; that way, you’ll be able to accurately assess the impact of your changes.

Going deeper on the enter-selection: Appending new elements in an empty canvas

Imagine you have a data-set as the one below:

[{x:5, y:5}, {x:90, y:30}, {x:105, y:65}, {x:190, y:55}]

As you may notice, it is an array with four objects; each object, in turn, has two properties: x and y. We can use this information to designate the position of four HTML elements. To create said elements, we are going to use the code I present you next (don’t forget to press the run button to see the result)

What is going on?

So, before we execute any code, we are defining a <svg> element whose id attribute is “enter-selection-demo-svg”. We are going to use this element as a “canvas” where our visualization will be drawn.

Now, let us explore the JavaScript snippet. At the first line of code, we are only defining the dataset to use. Afterward, we create a selection object that makes reference to the <svg> canvas.

Up until this point, we haven’t really done anything extraordinary; however, the fun is about to start! In the next line of code, where we define the update_selection, we are also joining the dataset with the selection of <circle> elements found inside the <svg> canvas; however, since the <svg> element is actually empty, the circles selection will also be empty.

Mmm, and what is an empty selection good for? One interesting property of D3 is its capability to join data even to empty selections, and that is exactly what the invocation of the method .data(dataset) is doing. As a matter of fact, this tiny method is key for the update-pattern; it performs the join-process, linking data and elements together; likewise, it creates the three selection states we’ve already talked about. The .data(dataset) method must be called from inside a selection object and it needs to receive, as an argument, the dataset we wish to link with the HTML elements. This method returns directly a update_selection object; with it, we can access the elements inside the update-selection.

In this example, after executing the join-process, we’d have the ensuing selection states:

As you may notice, both, the update-selection as well as the exit-selection, are empty and we only have elements inside the enter-selection; nevertheless, the .data(dataset) method only gave us access to the update-selection; then, how can we rather access the enter-selection? Lucky for us, the update_selection object possesses methods to access the enter-selection, as well as the exit-selection; for the first case, we need to call the .enter() method, that returns an object enter_selection. In our example the line of code that grants us access to the enter-selection is:

enter_selection = update_selection.enter();    

Now, I want you to pay close attention to the following lines of code; these are responsible for inserting the <circle> elements inside the <svg> canvas:

        .attr("cx", (d) => d.x)
        .attr("cy", (d) => d.y)
        .attr("r", (d, i) => (i + 1) * 5);

As you may remember, the enter-selection does not contain elements but markers; in our example, we have four markers. Now it is time to turn these markers into elements; the method .append("circle") does right that; for each marker in the enter-selection, it adds a <circle> element. Through the successive invocation of the .attr() method, we are setting the atributes of these four elements; in our example, these atributes are color, horizontal position, vertical position, and radius, respectively.

This time, I want to tell you a little bit more about the .attr() method. It can receive up to two arguments. The first one is simply the name of the attribute we want to modify. The second parameter represents the value we will assign to the attribute and it can take two possible forms: a constant value or a function. In the case it is a function, this must return the value for the attribute. The function can receive 3 possible arguments: the current datum (usually represented with a d letter), the index of the datum (i), and the current group of nodes.

In the example, we assign the fill attribute through a constant; to assign the value of the attributes cx and cy, we use a function that only requires the current datum as argument; finally, to set the value of the r attribute, we use a function that takes two arguments: the current datum and the index.

You may be wondering what I mean with “the current datum”; well, it is the particular piece of information being linked to each of the HTML elements during the join-process. In our example, each of the JSON objects inside the dataset array, work as the current datum at some point. As we have seen, these objects have two properties, x and y; we are using their values to position the <circle> elements.

What do you think? pretty cool, eh? The update-pattern provides an alternative to the use of loop blocks; in their place, we tell D3 what we want it does with each graphic element inside the selection.

How about we keep exploring this awesome pattern?

Going deeper on the update-selection: Linking the graphical elements to a new data-set

Now, suppose you have a <svg> canvas that already has elements inside it. Maybe you need to change the attributes of these elements; for this, the update selection would be ideal.

Check out the code block I’ve prepared next. As you’ll see, we have four gray circles inside a the <svg> canvas. Proceed to run the JavaScript code and see what happens!

What is going on this time? Actually, the code is fairly similar to the one we saw in the former case; however, in this occasion the selection of circles inside the <svg> element will not be empty. After the execution of the join-process, the selection sets would look this way:

This time the data-set and the elements-set completely overlap for we have four pieces of information linked to four elements; as a result, all the elements fall inside the update-selection; then, both, the enter-selection and the exit-selection are empty.

Now let us see the last line of the script. At this point, the code should be familiar to you. Given that our data has the color attribute, we can use this value precisely to color the circles. Pretty simple, don’t you think?

Going deeper on the exit-selection: removing elements that don’t stand for any data

What happens when the number of registers in the dataset is smaller than the number of graphical elements we are linking them to? In this scenario, we have “extra” elements that are not helping to represent any piece of information. Generally, we want to get rid off such elements; nonetheless, I must clarify, this is not mandatory and you can do whatever best suits your needs with this group of elements.

It is time to explore our sample code block. We have four circles before the execution of the join-process. Fine, run the code and see what happens.

In this occasion, the array dataset only has two pieces of information; as a matter of fact, we are not going to make any use of this data to alter the graphical elements; rather, we will use it in order to create the exit-selection. After the execution of the method .data(dataset), we’d find that each datum is linked to some HTML element; however, not every element is going to be linked to a datum. To make this situation clearer, let me show you a diagram with the resultant selection states.

At this stage, we can find elements in both, the update-selection and the exit-selection; nevertheless, for now, we only care for the latter. The method .exit() of the object update_selection permits us to access the exit selection. The final line of code is fairly simple, it is just removing all the elements found inside the exit-selection, from the browser. And, which ones are such elements? As we saw at the beginning, we only count with two pieces of information; these are being linked to the first two HTML elements found, i.e., the green circles. Since the red circles cannot be linked to any data, they fall in the exit-selection.

Linking data and graphical elements through a key

Up to this point, we have linked the pieces of information and the graphical elements in the order they appear; the first datum with the first element, the second datum with the second element, etc. Nevertheless, it’d be highly convenient to link the elements with specific pieces of information, based not in the order they appear but rather on some specific key. Lucky for us, D3 offers a mechanism to do just that. Before anything else, let me introduce you the code that would make the trick; we start with a <svg> canvas with four <circle> elements inside it.

I’ve divided the JavaScript code into two parts to better understand the process. In the first part, we are just linking the four <circle> elements with the data contained in the variable linkDataset. Can you figure out what kind of selection we are creating? think about it for a minute before proceeding… that’s right! we could only have elements in the update-selection. Actually, we are not going to perform any operation over this update-selection; we are only calling the .data(linkDataset) method to link the HTML elements with some piece of information. At the end of this first stage, the data is linked to the elements in the same order as they appear, just as we have seen up until now.

Maybe you are wondering how D3 keeps a record of what datum is linked to what element; well, do you remember how everything we see in the browser’s window is registered by the DOM? The strategy of D3 is to find all the graphical elements involved in the join-process, inside the DOM, and add a new property called __data__ in each of them. In our example, every circle object will have a __data__ property, whose value corresponds to one of the JSON objects from the array linkDataset; the object “alpha” for the first circle, the object “beta” for the second circle, and so on. One characteristic of this mechanism is that the graphic elements remain linked to the same datum until the join-process is executed anew; in the meantime, no matter how many times we select the elements, the relationship datum-element stays constant.

Let us explore the second part of the code. In this occasion, our goal is to change the data linked to the second and the fourth circle. As you can see, this time our dataset, called updateDataset, has only two components. We are going to use the “keyLink” property of this data to link them with the right HMLT elements; also, we’ll use the “color” property precisely to change the color of these elements.

Okay, but, how do we tell D3 we want it to use the “keyLink” property to link pieces of information and elements, instead of taking their position as a reference? You see, in fact, the method .data() can accept a second parameter called “key”: a function that, in turn, receives the current datum as one of its arguments. The value returned by this function will be used to assess what element must be linked to what piece of information. D3 applies this function to every new datum as well as the datum currently linked to each element in the selection. In case the function returns the same value for both cases, the new datum and the corresponding element will be linked together.

In our example, the “key” function is applied to every datum in the array updateDataset and also to every datum currently linked to the circles; in both cases, it will return the value of the property “keyLink” of these objects. Elements and data related to the same value for this property will be linked together; these elements fall inside the update-selection.

In the final line of code, we are solely updating the color of the elements found in the update-selection.

By the way, when we insert new elements through the enter-selection, D3 is automatically linking these elements with the corresponding piece of information; so, you don’t need to do it again.

Working with the three selections at the same time

With all you’ve learned, I’m sure you are an update-pattern master. But before letting you go with your graduation diploma, I’d like to present you one last strategy. So, What happens when you want to manipulate more than one selection at the same time? For instance, imagine you want to perform the same modifications on the elements in the enter-selection and the elements in the update-selection. Obviously, you could alter each selection separately and repeat the same code; however, D3 offers a better way: the .merge(other) method.

In the following example, we are going to perform an operation over each of the selection states.

In this case, the code is also separated into two stages. In the first one we insert four circles; each of them is going to be linked to one piece of information from the insertDataset array. We will use the value of the property keyVal to identify each of the objects inside the data arrays employed in this example; likewise, we can use this property to link the HTML elements to a new dataset. Each circle inserted at this stage is going to be black since we are not indicating any color.

Now, let us check the second stage out. As you can look, the elements inside the array updateDataset also have the property keyVal. Besides, notice that in this case, the data array no longer has pieces of information identified as “a” or “b”; furthermore, now we have two new datum identified with “e” and “f”. After calling the method .data(updateDataset, d => d.keyVal) (line 2.6) we are going to find elements in each of the three selections.

Before going any further, think for an instant what objects will fall in each selection. Have you already given a thought to it? In the update-selection, we can find the objects identified as “a” and “b” given that before and after the join-process we can find objects identified with these keys; inside the enter-selection, we have the objects identified as “e” and “f”, because these are “new” pieces of information that we didn’t have in the former dataset; finally, in the exit-selection we will find the objects identified with the “c” and “d” keys, since these are objects from the former dataset that don’t have a counterpart in the new one.

By using the method .remove() (line 2.7), we can eliminate from the browser all the elements in the exit-selection. Next, we use the markers from the enter-selection to add two squares and adjust their appearance (lines 2.8 to 2.10).

The next step is something we had not done up until this point; we are calling the method .merge(update_selection) from inside the enter-selection (line 2.11). This method must be called from a selection object and receives a parameter called “other”; said parameter is, in turn, a selection object. The method .merge() creates a new selection object based on the selection object that called it (the enter-selection, in our example) and complements it with the elements found in the selection “other” (the update-selection, in our case).

Watch out! the method .merge(other) does not join both selections, even though it may appear the case; what it does is to fill the “missing” elements in the first selection with those in the second one. Given the case where both selections possessed an element identified in the same fashion, only the element from the selection that called the method would be taken into account.

Once we have the new selection object, we can simultaneously modify the elements in both, the enter-selection and the update-selection; in our case, we are setting the color, border, and width (lines 2.12 to 2.14).

Wrapping up

What do you think? At this stage, you have surely realized how powerful and versatile is the update pattern. You should also feel comfortable enough to start using these tools in your next projects.

You may be thinking, what is the difference between the join-process and the update-pattern? they pretty much seem the same thing, do they not? It is actually very easy, the join-process is the mechanism used to link pieces of information with graphical elements; we perform it calling the selection object’s .data() method. In contrast, the update-pattern consists of the execution of the join-process, followed by operations over the enter-selection, the update-selection, and/or the exit-selection.

If you still have doubts about the general operation of D3, fear not! My next post will address precisely this subject; so, stay tuned!