Raspberry Pi Pico Tips and Tricks

Wednesday 20 March 2013

A simple d3.js map explained

The following post is a portion of the D3 Tips and Tricks document which is free to download. To use this post in context, consider it with the others in the blog or just download the pdf  and / or the examples from the downloads page:-)

-------------------------------------------------------


Starting with a simple map

Our starting example will demonstrate the simple display of a World map. Our final result will looks like this;

The data file for the World map is one produced by Mike Bostock's as part of his TopoJSON work.

We'll move through the explanation of the code in a similar process to the one we went through when highlighting the function of the Sankey diagram. Where there are areas that we have covered before, I will gloss over some details on the understanding that you will have already seen them explained in an earlier section (most likely the basic line graph example).

Here is the full code;
<!DOCTYPE html>
<meta charset="utf-8">
<style>
path {
  stroke: white;
  stroke-width: 0.25px;
  fill: grey;
}
</style>
<body>
<script type="text/javascript" src="d3/d3.v3.js"></script>
<script src="js/topojson.v0.min.js"></script>
<script>
var width = 960,
    height = 500;

var projection = d3.geo.mercator()
    .center([0, 5 ])
    .scale(900)
    .rotate([-180,0]);

var svg = d3.select("body").append("svg")
    .attr("width", width)
    .attr("height", height);

var path = d3.geo.path()
    .projection(projection);

var g = svg.append("g");

// load and display the World
d3.json("json/world-110m2.json", function(error, topology) {
    g.selectAll("path")
      .data(topojson.object(topology, topology.objects.countries)
          .geometries)
    .enter()
      .append("path")
      .attr("d", path)
});

</script>
</body>
</html>

 One of the first things that struck me when I first saw the code to draw a map was how small it was (the amount of code, not the World).  It's a measure of the degree of abstraction that D3 is able to provide to the process of getting data from a raw format to the scree that such a complicated task can be condensed to such an apparently small amount of code. Of course that doesn't tell the whole story. Like a duck on a lake, above the water all is serene and calm while below the water the feet are paddling like fury. In this case, our code looks serene because D3 is doing all the hard work :-).

The first block of our code is the start of the file and sets up our HTML.
<!DOCTYPE html>
<meta charset="utf-8">
<style>
 This leads into our style declarations.
path {
  stroke: white;
  stroke-width: 0.25px;
  fill: grey;
}
We only state the properties of the path components which will make up our countries. Obviously we will fill them with grey and have a thin (`0.25px`) line around each one.

The next block of code loads the JavaScript files.
</style>
<body>
<script type="text/javascript" src="d3/d3.v3.js"></script>
<script src="js/topojson.v0.min.js"></script>
<script>
In this case it's d3 and topojson. We load `topojson.v0.min.js` as a separate file because it's still fairly new. In other words it hasn't been incorporated into the main d3.js code base (that's an assumption on my part since it might exist in isolation or perhaps end up as a plug-in). Whatever the case, for the time being, it exists as a separate file.

Then we get into the JavaScript. The first thing we do is define the size of our map.
var width = 960,
    height = 500;
Then we get into one of the simple, but cool parts of making any map. Setting up the view.
var projection = d3.geo.mercator()
    .center([0, 5 ])
    .scale(900)
    .rotate([-180,0]);
The projection is the way that the geographic coordinate system is adjusted for display on our flat screen. The screen is after all a two dimensional space and we are trying to present a three dimensional object. This is a big deal to cartographers in the sense that selecting a geographic projection for a map is an exercise in compromise. You can make it look pretty, but in doing so you can grievously distort the land size / shape. On the other hand you might make it more accurate, in size / shape but people will have trouble recognising it because they're so used to the standard Mercator projection. For example, the awesome Waterman Butterfly

There are a lot of alternative projections available. Please have a browse on the wiki where you will find a huge range of options (66 at time of writing).

In our case we've gone with the conservative Mercator option.

Then we define three aspects of the projection. Center, scale and rotate.

 center

If center is specified, this sets the projection’s center to the specified location as two-element array of longitude and latitude in degrees and returns the projection. If center is not specified the default of (0°,0°)  is used.

Our example is using `[0, 5 ]` which I have selected as being in the middle (`0`) for longitude (left to right) and `5` degrees North of the equator (North is positive values of latitude, South is negative values). This was purely to make it look aesthetically pleasing. Here's the result of setting the center to `[100,30]`.
The map has been centered on 100 degrees West and 30 degrees North. Of course, it's also been pushed to the left without the right hand side of the map scrolling around. We'll get to that in a moment.

scale

If scale is specified, this sets the projection’s scale factor to the specified value. If scale is not specified, returns the current scale factor which defaults to 150. It's important to note that scale factors are not consistent across projections.

Our current map uses a scale of 900. Again, this has been set for aesthetics. Keeping our center of `[100,30]`, if we increase our scale to `2000` this is the result.

rotate

If rotation is specified, this sets the projection’s three-axis rotation to the specified angles for yaw, pitch and roll (equivalently longitude, latitude and roll) in degrees and returns the projection. If rotation is not specified, it sets the values to [0, 0, 0]. If the specified rotation has only two values, rather than three, the roll is assumed to be 0°.

In our map we have specified `[-180,0]` so we can assume a roll value of zero. Likewise we have rotated our map by -180 degrees in longitude. This has been done specifically to place the map with the center on the anti-meridian (The international date line in the middle of the Pacific ocean). If we return the value to `[0,0]`(with our original values of `scale` and `center` this is the result.
In this case the centre of the map lines up with the meridian.

The next block of code sets our svg window;
var svg = d3.select("body").append("svg")
    .attr("width", width)
    .attr("height", height);
The follow portion of code creates a new geographic path generator;
var path = d3.geo.path()
    .projection(projection);
The path generator (`d3.geo.path()`) is used to spcify a projection type (`.projection`) which was defined earlier as a Mercator projection via the variable `projection`. (I'm not entirely sure, but it is possible that I have just set some kind of record for use of the word 'projection' in a sentence.)

We then declare `g` as our appended svg.
var g = svg.append("g");
The last block of JavaScript draws our map.
d3.json("json/world-110m2.json", function(error, topology) {
    g.selectAll("path")
      .data(topojson.object(topology, topology.objects.countries)
          .geometries)
    .enter()
      .append("path")
      .attr("d", path)
});
We load the TopoJSON file with the coordinates for our World map (`world-110m2.json`). Then we declare that we are going to act on all the `path` elements in the graphic (`g.selectAll("path")`).

Then we pull the data that defines the countries from the TopoJSON file (`.data(topojson.object(topology, topology.objects.countries).geometries)`).  We add it to the data that we're going to display (`.enter()`) and then we append that data as `path` elements (`.append("path")`).

The last html block closes off our tags and we have a map!
The code and data for this example can be found as World Map Centered on the Pacific  on bl.ocks.org.

Zooming and panning a map

With our map displayed nicely we need to be able to move it about to explore it fully
.
To do this we can provide the functionality to zoom and pan it using the mouse.

Towards the end of the script, just before the close off of the script at the `</script>` tag we can add in the following code;
var zoom = d3.behavior.zoom()
    .on("zoom",function() {
        g.attr("transform","translate("+ 
            d3.event.translate.join(",")+")scale("+d3.event.scale+")");
        g.selectAll("path")  
            .attr("d", path.projection(projection)); 
});

svg.call(zoom)
This block of code introduces the `behavior`s functions. Using the `d3.behavior.zoom` function creates event listeners (which are like hidden functions standing by to look out for a specific type of activity on the computer and in this case mouse actions) to handle zooming and panning gestures on a container element (in this case our map). More information on the range of zoom options is available on the [D3 Wiki](https://github.com/mbostock/d3/wiki/Zoom-Behavior).

We begin by declaring the `zoom` function as `d3.behavior.zoom`.

Then we instruct the computer that when it 'sees' a 'zoom' event to carry out another function (`.on("zoom",function() {`).

That function firstly gathers the (correctly formatted) `translate` and `scale` attributes in...
        g.attr("transform","translate("+ 
            d3.event.translate.join(",")+")scale("+d3.event.scale+")");
 ... and then applies them to all the path elements (which are the shapes of the countries) via...
        g.selectAll("path")  
            .attr("d", path.projection(projection)); 
 Lastly we call the zoom function.
svg.call(zoom)
Then we relax and explore our map!
The code and data for this example can be found as World Map with zoom and pan on bl.ocks.org.

 Displaying points on a map

Displaying maps and exploring them is pretty entertaining, but as anyone who has participated in the improvement of our geographic understanding of our world via projects such as [Open Street Map](http://www.openstreetmap.org/) will tell you, there's a whole new level of cool to be attained by adding to a map.

With that in mind, our next task is to add some simple detail in the form of points that show the location of cities.

To do this we will load in a csv file with data that identifies our cities and includes latitude and longitude details. Our file is called `cities.csv` and looks like this;

code,city,country,lat,lon
ZNZ,ZANZIBAR,TANZANIA,-6.13,39.31
TYO,TOKYO,JAPAN,35.68,139.76
AKL,AUCKLAND,NEW ZEALAND,-36.85,174.78
BKK,BANGKOK,THAILAND,13.75,100.48
DEL,DELHI,INDIA,29.01,77.38
SIN,SINGAPORE,SINGAPOR,1.36,103.75
BSB,BRASILIA,BRAZIL,-15.67,-47.43
RIO,RIO DE JANEIRO,BRAZIL,-22.90,-43.24
YTO,TORONTO,CANADA,43.64,-79.40
IPC,EASTER ISLAND,CHILE,-27.11,-109.36
SEA,SEATTLE,USA,47.61,-122.33


While we're only going to use the latitude and longitude for our current work, the additional details could just as easily be used for labeling or tooltips.

We need to place our code carefully in this case because while you might have some flexibility in getting the right result with a locally hosted version of a map, there is a possibility that with a version hosted in the outside World (*gasp* the internet) you could strike trouble.

The code to load the cities should be placed inside the function that is loading the World map as indicated below;
d3.json("json/world-110m2.json", function(error, topology) {
    g.selectAll("path")
      .data(topojson.object(topology, topology.objects.countries)
          .geometries)
    .enter()
      .append("path")
      .attr("d", path)
                           // <== Put the new code block here
});
Here's the new code;
    d3.csv("data/cities.csv", function(error, data) {
        g.selectAll("circle")
           .data(data)
           .enter()
           .append("circle")
           .attr("cx", function(d) {
                   return projection([d.lon, d.lat])[0];
           })
           .attr("cy", function(d) {
                   return projection([d.lon, d.lat])[1];
           })
           .attr("r", 5)
           .style("fill", "red");
We'll go through the code and then explain the quirky thing about it.

First of all we load the `cities.csv` file (`d3.csv("data/cities.csv", function(error, data) {`). Then we select all the circle elements (` g.selectAll("circle")`), assign our data (`.data(data)`), enter our data (` .enter()`) and then add in circles (`.append("circle")`).

Then we set the x and y position for the circles based on the longitude (`([d.lon, d.lat])[0]`) and latitude (`([d.lon, d.lat])[1]`) information in the csv file.

Finally we assign a radius of 5 pixels and fill the circles with red.

The quirky thing about the new code block is that we have to put it inside the code block that loads the world data (`d3.json("json/world-110m2.json", function(error, topology) {`). We could place the two blocks one after the others (load / draw the world data, then load / draw the circles). And this will probably work if you run the file from your local computer. But when you host the files on the internet, it takes too long to load the world data compared to the city data and the end result is that the city data gets drawn before the world data and this is the result.
To avoid the problem we place the loading of the city data *into* the code that loads the World data. That way the city data doesn't get loaded until the World data is loaded and then the circles get drawn on top of the world instead of under it :-).
The code and data for this example can be found as World map with zoom / pan and cities on bl.ocks.org.
The topojson file is here; and the cities.csv file is here (while you can find these from the github page, it isn't particularly obvious where they are).

The above description (and heaps of other stuff) is in the D3 Tips and Tricks book that can be downloaded for free (or donate if you really want to :-)).

Mapping with d3.js overview

Mapping with d3.js


Another string to the bow of d3.js is the addition of a set of powerful routines for handling geographical information.

In the same sense that a line graph is a simple representation of data on a document, a map can be regarded as a set of points with an underlying coordinate system. When you say it like that it seems obvious that it should be applied as a document for display. However, I don't want to give the impression that this is some sort of trivial matter for either the original developers or for you the person who wants to display a map. Behind the scenes for this type of work the thought that must have gone into making the code usable and extensible must have been enormous.

Mike Bostock has lauded the work of Jason Davies in the development of the latest major version of d3.js (version 3) for his work on improving mapping capability. A visit to his home page provides a glimpse into Jason's expertise and no visit would be complete without marveling at his work with geographic projections.

Examples

I am firmly of the belief that mapping in particular has an enormous potential for adding value to data sets. The following collection of examples gives a brief taste of what has been accomplished by combining geographic information and D3 thus far. (the screen shots following have been sourced from the biovisualize gallery and as such provide attribution to the best of my ability. If I have incorrectly attributed the source or author please let me know and I will correct it promptly)



Above is an interactive visualization showing the position of the main map on a faux D3 3d globe with a Mapbox / Open Street Map main window. Source dev.geosprocket.com  by Bill Morris.



This is a breakdown of population in  Kentucky Counties from the 2010 census. Source: ccarpenterg.github.com by Cristian Carpenter.



This map visualizes air pollution in Beijing. Source: scottcheng.github.com by Scott Cheng.



This is a section of the globe that is presented on the Shuttle Radar Topography Mission tile downloading web site. This excellent site uses the interactive globe to make the selection of srtm tiles easy. Source dwtkns.com by Derek Watkins.



This is a static screen-shot of an animated tour of the Worlds countries. Source bl.ocks.org  by Mike Bostock.



This is one of the great infographics published by the New York Times. Source: www.nytimes.com by Mike Bostock, Shan Carter and Kevin Quealy.



This is an animated graphic showing a series of concentric circles emanating from glowing red dot which was styled after a news article in The Onion. Source: bl.ocks.org by Mike Bostock.



Here we see earthquakes represented on a selectable timeline where D3 generates a svg overlay and the map layer is created using Leaflet. Source: bl.ocks.org by tnightingale.



Carrying on with the earthquake theme, this is a map of all earthquakes in the past 24 hours over magnitude 2.5. Source: bl.ocks.org by benelsen.



An interactive satellite projection. Source dev.geosprocket.com by Bill Morris.

GeoJSON and TopoJSON

Projecting countries and various geographic features onto a map can be a very data hungry exercise. By that I mean that the information required to present geographic shapes can result in data files that are quite large. GeoJSON has been the default geographic data file of choice for quite some time, and as the name would suggest it encodes the data in a JSON type hierarchy. Often these GeoJSON files include a significant amount of extraneous detail or incorporate a level of accuracy that is impractical (too detailed).

Enter TopoJSON. Mike Bostock has designed TopoJSON as an extension to GeoJSON in the sense that it has a similar structure, but the geometries are not encoded discretely and where they share features, they are combined. Additionally TopoJSON encodes numeric values more efficiently and can incorporate a degree of simplification.  This simplification can result in savings of file size of 80% or more depending on the area and use of compression. Although TopoJSON has only begun to be used, the advantages of it seem clear and so I will anticipate it's future use by incorporating it in my example diagrams (not that the use of GeoJSON differs much if at all). A great description of TopoJSOn can be found on the TopoJSON wiki on github.

Monday 18 March 2013

Directional Force Layout Diagram (varying link opacity)


The following post is a portion of the D3 Tips and Tricks document which is free to download. To use this post in context, consider it with the others in the blog or just download the pdf  and / or the examples from the downloads page:-)
-------------------------------------------------------


The next variation to our force layout diagram is the addition of variation in the link to represent different values (think of the number of packets passed or the amount of money transferred). It follows on from the Directional Force Layout Diagram (Node Highlighting).

There are a few different ways to do this, but by virtue of the inherent close linkages between the arrowhead marker and the link line, altering both in synchronicity proved to be beyond my meagre talents. However, I did find a couple of suitable alternatives and I will go through one here.

In this example we will take the value associated in the loaded data with the link and we will adjust the opacity of the link line in a staged way according to the range of values.

For example, in a range of link strengths from 0 to 100, the bottom 25% will be at opacity 0.25, from 25 to 50 will be 0.25, 50 to 75 will be 0.75 and above 75 will have an opacity of 1. So the final result looks a little like this;

The changes to the code to create this effect are focused on creating an appropriate range for the values associated with the links and then applying the opacity according to that range in discrete steps.

The first change to the node highlighting code that we make is to the style section. The following elements are added;
path.link.twofive {
  opacity: 0.25;
}

path.link.fivezero {
  opacity: 0.50;
}

path.link.sevenfive {
  opacity: 0.75;
}

path.link.onezerozero {
  opacity: 1.0;
}
 This provides our four different 'classes' of opacity.

Then in a  block of code that comes just after the declaration of the `force` properties we have the following;
var v = d3.scale.linear().range([0, 100]);

v.domain([0, d3.max(links, function(d) { return d.value; })]);

links.forEach(function(link) {
    if (v(link.value) <= 25) {
        link.type = "twofive";
    } else if (v(link.value) <= 50 && v(link.value) > 25) {
        link.type = "fivezero";
    } else if (v(link.value) <= 75 && v(link.value) > 50) {
        link.type = "sevenfive";
    } else if (v(link.value) <= 100 && v(link.value) > 75) {
        link.type = "onezerozero";
    }
});
 Here we set the scale and the range for the variable `v` (`var v = d3.scale.linear().range([0, 100]);`). We then set the domain for v to go from 0 to the maximum `value` that we have in our link data.

The final block above uses a cascading set of if statements to assign a label to the `type` parameter of each link. This label is the linkage back to the styles we defined previously.

The final change is to take the line where we assigned a class of `link` to each link previously...
    .attr("class", "link")
 ...to add in our `type` parameter as well;
    .attr("class", function(d) { return "link " + d.type; })
 Obviously if we wanted a greater number of opacity levels we would add in further style blocks (with the appropriate values) and modify our cascading if statements. I'm not convinced that this solution is very elegant for what I'm trying to do (it was a much better fit for the application that Mike Bostock applied it to originally where he designated different types of law suits) but I'll take the result as a suitable way of demonstrating variation of value.

The code and data for this example can be found as Directional Force Layout Diagram with varying link opacity on bl.ocks.org.

The above description (and heaps of other stuff) is in the D3 Tips and Tricks book that can be downloaded for free (or donate if you really want to :-)).

Saturday 16 March 2013

Directional Force Layout Diagram (Node Highlighting)

The following post is a portion of the D3 Tips and Tricks document which is free to download. To use this post in context, consider it with the others in the blog or just download the pdf  and / or the examples from the downloads page:-)

-------------------------------------------------------


Following on from the Basic Force Layout Diagram, our next goal is to highlight our nodes so that we can get a better view of what ones they are (the view can get a little crowded as the nodes begin to increase in number).

To do this we are going to use a couple more of the mouse events that we first introduced in the tooltips section.

For this example we are going to use the  `click` event (Triggered by a mouse click (mousedown and then mouseup over an element)) and the `dblclick` event (Triggered by two clicks within a short time over an element).

The single click will enlarge the node and the associated text and the double click will return the node and test to its original size.

The way to implement this is to first set a hook to capture when the event occurs, which calls a function which is laid out later in the script.

The hook is going to be part of the JavaScript where we define our nodes;
var node = svg.selectAll(".node")
    .data(force.nodes())
  .enter().append("g")
    .attr("class", "node")
    .on("click", click)        // Add in this line
    .on("dblclick", dblclick)  // Add in this line too
    .call(force.drag);
The two additional lines above tell the script that when it sees a click or a double-click on the node (since it's part of the node set-up) to run either the `click` or `dblclick` functions.

The following two function blocks should be placed after the `tick` function but before the closing curly bracket and bracket as indicated;
function tick() {
    path.attr("d", function(d) {
        var dx = d.target.x – d.source.x,
            dy = d.target.y – d.source.y,
            dr = Math.sqrt(dx * dx + dy * dy);
        return "M" + 
            d.source.x + "," + 
            d.source.y + "A" + 
            dr + "," + dr + " 0 0,1 " + 
            d.target.x + "," + 
            d.target.y;
    });

    node
        .attr("transform", function(d) { 
            return "translate(" + d.x + "," + d.y + ")"; });
}

        // <= Put the functions in here!

});
 The `click` function is as follows;
function click() {
    d3.select(this).select("text").transition()
        .duration(750)
        .attr("x", 22)
        .style("fill", "steelblue")
        .style("stroke", "lightsteelblue")
        .style("stroke-width", ".5px")
        .style("font", "20px sans-serif");
    d3.select(this).select("circle").transition()
        .duration(750)
        .attr("r", 16)
        .style("fill", "lightsteelblue");
}
 The first line declares the function name (`click`). Then we select the node that we've clicked on and then the associated text before we begin the declaration for our transition (`d3.select(this).select("text").transition()`).

Then we define the new properties that will be in place after the transition. We move the text's x position (`.attr("x", 22)`), make the text fill steel blue (`.style("fill", "steelblue")`), set the stroke around the edge of the text light steel blue (`.style("stroke", "lightsteelblue")`), set that stroke to half a pixel wide (`.style("stroke-width", ".5px")`) and increase the font size to 20 pixels (`.style("font", "20px sans-serif");`).

Then we do much the same for the circle component of the node. Select it, declare the transition, increase the radius and change the fill colour.

The `dblclick` function does exactly the same as the `click` function, but reverses the action to return the text and circle to the original settings.
function dblclick() {
    d3.select(this).select("circle").transition()
        .duration(750)
        .attr("r", 6)
        .style("fill", "#ccc");
    d3.select(this).select("text").transition()
        .duration(750)
        .attr("x", 12)
        .style("stroke", "none")
        .style("fill", "black")
        .style("stroke", "none")
        .style("font", "10px sans-serif");
}
 The end result is a force layout diagram where you can click on nodes to increase their size (circle and text) and then double click to reset them if desired.

And here's a live version.

The code and data for this example can be found as Directional Force Layout Diagram with Node Highlighting on bl.ocks.org.

The above description (and heaps of other stuff) is in the D3 Tips and Tricks book that can be downloaded for free (or donate if you really want to :-)).

Thursday 14 March 2013

d3.js force directed graph example (basic)

The following post is a portion of the D3 Tips and Tricks document which is free to download. To use this post in context, consider it with the others in the blog or just download the pdf  and / or the examples from the downloads page:-)
-------------------------------------------------------

Basic force directed graph showing directionality

As explained in the previous post, the example graphs explained here are a combination of Mike Bostock's  Mobile Patent Suits graph and  Force-Directed Graph with Mouseover graph.

The data for this example graph has been altered from the data that was comprised of litigants in the mobile patent war to fictitious peoples names and associated values (to represent the strength of the links between the two). In the original examples the data was contained in the graph code. In the following example it is loaded from a csv file named force.csv. The values loaded are as follows;

source,target,value
Harry,Sally,1.2
Harry,Mario,1.3
Sarah,Alice,0.2
Eveie,Alice,0.5
Peter,Alice,1.6
Mario,Alice,0.4
James,Alice,0.6
Harry,Carol,0.7
Harry,Nicky,0.8
Bobby,Frank,0.8
Alice,Mario,0.7
Harry,Lynne,0.5
Sarah,James,1.9
Roger,James,1.1
Maddy,James,0.3
Sonny,Roger,0.5
James,Roger,1.5
Alice,Peter,1.1
Johan,Peter,1.6
Alice,Eveie,0.5
Harry,Eveie,0.1
Eveie,Harry,2.0
Henry,Mikey,0.4
Elric,Mikey,0.6
James,Sarah,1.5
Alice,Sarah,0.6
James,Maddy,0.5
Peter,Johan,0.7


The code is as follows;
<!DOCTYPE html>
<meta charset="utf-8">
<script type="text/javascript" src="d3/d3.v3.js"></script>
<style>

path.link {
  fill: none;
  stroke: #666;
  stroke-width: 1.5px;
}

circle {
  fill: #ccc;
  stroke: #fff;
  stroke-width: 1.5px;
}

text {
  fill: #000;
  font: 10px sans-serif;
  pointer-events: none;
}

</style>
<body>
<script>

// get the data
d3.csv("data/force.csv", function(error, links) {

var nodes = {};

// Compute the distinct nodes from the links.
links.forEach(function(link) {
    link.source = nodes[link.source] || 
        (nodes[link.source] = {name: link.source});
    link.target = nodes[link.target] || 
        (nodes[link.target] = {name: link.target});
    link.value = +link.value;
});

var width = 960,
    height = 500;

var force = d3.layout.force()
    .nodes(d3.values(nodes))
    .links(links)
    .size([width, height])
    .linkDistance(60)
    .charge(-300)
    .on("tick", tick)
    .start();

var svg = d3.select("body").append("svg")
    .attr("width", width)
    .attr("height", height);

// build the arrow.
svg.append("svg:defs").selectAll("marker")
    .data(["end"])
  .enter().append("svg:marker")
    .attr("id", String)
    .attr("viewBox", "0 -5 10 10")
    .attr("refX", 15)
    .attr("refY", -1.5)
    .attr("markerWidth", 6)
    .attr("markerHeight", 6)
    .attr("orient", "auto")
  .append("svg:path")
    .attr("d", "M0,-5L10,0L0,5");

// add the links and the arrows
var path = svg.append("svg:g").selectAll("path")
    .data(force.links())
  .enter().append("svg:path")
    .attr("class", "link")
    .attr("marker-end", "url(#end)");

// define the nodes
var node = svg.selectAll(".node")
    .data(force.nodes())
  .enter().append("g")
    .attr("class", "node")
    .call(force.drag);

// add the nodes
node.append("circle")
    .attr("r", 5);

// add the text 
node.append("text")
    .attr("x", 12)
    .attr("dy", ".35em")
    .text(function(d) { return d.name; });

// add the curvy lines
function tick() {
    path.attr("d", function(d) {
        var dx = d.target.x - d.source.x,
            dy = d.target.y - d.source.y,
            dr = Math.sqrt(dx * dx + dy * dy);
        return "M" + 
            d.source.x + "," + 
            d.source.y + "A" + 
            dr + "," + dr + " 0 0,1 " + 
            d.target.x + "," + 
            d.target.y;
    });

    node
        .attr("transform", function(d) { 
            return "translate(" + d.x + "," + d.y + ")"; });
}

});

</script>
</body>
</html>
 In a similar process to the one we went through when highlighting the function of the Sankey diagram, where there are areas that we have covered before, I will gloss over some details on the understanding that you will have already seen them explained in an earlier section (most likely the basic line graph example).

The first block we come across is the initial html section;
<!DOCTYPE html>
<meta charset="utf-8">
<script type="text/javascript" src="d3/d3.v3.js"></script>
<style>
The only thing slightly different with this example is that we load the d3.v3.js script earlier. This has no effect on running the code.

The next section loads the Cascading Style Sheets;
path.link {
  fill: none;
  stroke: #666;
  stroke-width: 1.5px;
}

circle {
  fill: #ccc;
  stroke: #fff;
  stroke-width: 1.5px;
}

text {
  fill: #000;
  font: 10px sans-serif;
  pointer-events: none;
}
 We set styles for three elements and all the settings laid out are familiar to us from previous work.

Then we move into the JavaScript. Our first line loads our csv data file (`force.csv`) from our `data` directory.
d3.csv("data/force.csv", function(error, links) {
Then we declare an empty object (I still tend to think of these as arrays even though they're strictly not).
var nodes = {};
 This will contain our data for our nodes. We don't have any separate node information in our data file, it's just link information, so we will be populating this in the next section...
links.forEach(function(link) {
    link.source = nodes[link.source] || 
        (nodes[link.source] = {name: link.source});
    link.target = nodes[link.target] || 
        (nodes[link.target] = {name: link.target});
    link.value = +link.value;
});
 This block of code looks through all of out data from our csv file and for each link adds it as a node if it hasn't seen it before. It's quite clever how it works as it employs a neat JavaScript shorthand method using the double pipe (`||`) identifier.

So the line (expanded)...
link.source=nodes[link.source] || (nodes[link.source]={name: link.source});
 ... can be thought of as saying “*If* `link.source` *does not equal any of the `nodes` values then create a new element in the `nodes` object with the name of the `link.source` value being considered.*”. It could conceivably be written as follows (this is untested);
if (link.source != nodes[link.source]) {
    nodes[link.source] = {name: link.source}
};
 Then the block of code goes on to test the `link.target` value in the same way. Then the `value` variable is converted to a number from a string if necessary (`link.value = +link.value;`).

The next block sets the size of our svg area that we'll be using;
var width = 960,
    height = 500;
 The next section introduces the `force` function.
var force = d3.layout.force()
    .nodes(d3.values(nodes))
    .links(links)
    .size([width, height])
    .linkDistance(60)
    .charge(-300)
    .on("tick", tick)
    .start();
 Full details for this function are found on the D3 Wiki, but the following is a rough description of the individual settings.

`var force = d3.layout.force()` makes sure we're using the `force` function.

`.nodes(d3.values(nodes))` sets our layout to the array of `nodes` as returned by the function `d3.values` (https://github.com/mbostock/d3/wiki/Arrays#wiki-d3_values). Put simply, it sets the nodes to the `nodes` we have previously set in our object.

`.links(links)` does for links what `.nodes` did for nodes.

`.size([width, height])`  sets the available layout size to our predefined values. If we were using `gravity` as a force in the graph this would also set the gravitational centre. It also sets the initial random position for the elements of our graph.

`.linkDistance(60)` sets the target distance between linked nodes.  As the graph begins and moves towards a steady state, the distance between each pair of linked nodes is computed and compared to the target distance; the links are then moved towards or away from each other, so as to converge on the set distance.

Setting this value (and other force values) can be something of a balancing act. For instance, here is the result of setting the `.linkDistance` to 160.


Here the charged nodes are trying to arrange themselves at an appropriate distance, but the length of the links means that their arrangement is not very pretty. Likewise if we change the value to 30 we get the following;


Here the link distance allows for a symmetrical layout, but the distance is too short to be practical.

`.charge(-300)` sets the force between nodes. Negative values of `charge` results in node repulsion, while a positive value results in node attraction. In our example, if we vary the value to 150 we get this result;


It's not exactly easy to spot, but the graph feels a little 'lazy'. The nodes don't find their equilibrium easily or at all. Setting the value higher than 300 (for our example) keeps all the nodes nice and spread out, but where there are other separate discrete linked nodes (as there are in our example) they tend to get forced away from the center of the defined area.

`.on("tick", tick)` runs the animation of the force layout one 'step'. It's these progression of steps that give the force layout diagram it's fluid movement.

`.start();` Starts the simulation; this method must be called when the layout is first created.

The next block of our code is the standard section that sets up our svg container.
var svg = d3.select("body").append("svg")
    .attr("width", width)
    .attr("height", height);
 The next block of our code is used to create our arrowhead marker. I will be the first to admit that it has entered a realm of svg expertise that I do not have and the amount of extra memory power I would need to accumulate to understand it sufficiently to explain won't be occurring in the near future. Please accept my apologies and as a small token of my regret, accept the following links as an invitation to learn more: http://www.w3.org/TR/SVG/coords.html#ViewBoxAttribute and http://www.w3schools.com/svg/svg_reference.asp?.
What is useful to note here is that we define the label for our marker as `end`. We will use this in the next section to reference the marker as an object. This particular section of the code caused me some small amount of angst. The problem being when I attempted to adjust the width of the link lines in conjunction with the `value` set in the data for the link, it would also adjust the stroke-width of the arrowhead marker. Then when I attempted to adjust for the positioning of the arrow on the path, I could never get the maths right. Eventually I decided to stop struggling against it and the encode the value as line in a couple of different ways. One as opacity using discrete boundaries and the other using variable line width, but with the arrowheads a common size. We will cover both those solutions in the coming sections.
svg.append("svg:defs").selectAll("marker")
    .data(["end"])
  .enter().append("svg:marker")    
    .attr("id", String)
    .attr("viewBox", "0 -5 10 10")
    .attr("refX", 15)
    .attr("refY", -1.5)
    .attr("markerWidth", 6)
    .attr("markerHeight", 6)
    .attr("orient", "auto")
  .append("svg:path")
    .attr("d", "M0,-5L10,0L0,5");
 The `.data(["end"]) ` line sets our tag for a future part of the script to find this block and draw the marker.

`.attr("refX", 15)` sets the offset of the arrow from the centre of the circle. While it is designated as the `X` offset, because the object is rotating, it doesn't correspond to the x (left and right) axis of the screen. The same is true of the `.attr("refY", -1.5)` line.

The `.attr("markerWidth", 6)` and `.attr("markerHeight", 6)` lines set the bounding box for the marker. So varying these will vary the space available for the marker.

The next block of code adds in our links as paths and uses the `#end` marker to draw the arrowhead on the end of it.
var path = svg.append("svg:g").selectAll("path")
    .data(force.links())
  .enter().append("svg:path")
    .attr("class", "link")
    .attr("marker-end", "url(#end)");
 Then we define what our nodes are going to be.
var node = svg.selectAll(".node")
    .data(force.nodes())
  .enter().append("g")
    .attr("class", "node")
    .call(force.drag);
 This uses the `nodes` data and adds the `.call(force.drag);` function which allows the node to be dragged by the mouse.

The next block adds the nodes as an svg circle.
node.append("circle")
    .attr("r", 5);
And then we add the name of the node with a suitable offset.
node.append("text")
    .attr("x", 12)
    .attr("dy", ".35em")
    .text(function(d) { return d.name; });
 The last block of JavaScript is the `ticks` function. This block is responsible for updating the graph and most interestingly drawing the curvy lines between nodes.
function tick() {
    path.attr("d", function(d) {
        var dx = d.target.x – d.source.x,
            dy = d.target.y – d.source.y,
            dr = Math.sqrt(dx * dx + dy * dy);
        return "M" + 
            d.source.x + "," + 
            d.source.y + "A" + 
            dr + "," + dr + " 0 0,1 " + 
            d.target.x + "," + 
            d.target.y;
    });

    node
        .attr("transform", function(d) { 
            return "translate(" + d.x + "," + d.y + ")"; });
}
 This is another example where there are some easily recognizable parts of the code that set the x and y points for the ends of each link (`d.source.x`, `d.source.y` for the start of the curve and `d.target.x`, `d.target.y` for the end of the curve) and a transformation for the node points, but the cleverness is in the combination of the math for the radius of the curve (`dr = Math.sqrt(dx * dx + dy * dy);`) and the formatting of the svg associated with it. This is sadly beyond the scope of what I can comfortable explain, so we will have to be content with “the magic happens here”.

The end result should be a tidy graph that demonstrates nodes and directional links between them.


The code and data for this example can be found as Basic Directional Force Layout Diagram on bl.ocks.org.

The above description (and heaps of other stuff) is in the D3 Tips and Tricks book that can be accessed from the downloads page of d3noob.org (Hey! IYou can get it for free. Why not?)