View Page
This week completely focused on the view page, using D3 to build the time series graph and annotations. Since this was my first time using D3 the start of the week was much slower compared to previous weeks. The graph visualization panning and zooming, as well as annotations display and the ability to edit them, are fully complete, luckily no unknown unknowns have popped up.
D3
D3 stands for Data-Driven Documents, allowing developers to present data using HTML SVG and CSS. Unlike other data visualization libraries D3 works by giving you a set list of components to build any type of visualization wanted, many other visualization libraries lockdown to very specific visualizations with specific properties. This can be extremely problematic, the developer relies heavily on a set feature list but with D3 the core components provided can be used to build any specific feature such as panning on one axis.
D3 works by manipulating the Document Object Model (DOM) according to the given data, using a combination of select and modify methods. The select method is D3’s most basic but most useful, allowing the selection of the DOM either by tag, class name or id name. Once an element has been selected a child element can be appended or an attribute can be set or get. Using only the select method a developer can iterate through an array appending an h1 tag to the body as well as styling each tag independently. The example below demonstrates how to select all div tags and colour each div background red.
<html> <body> <div> <h1> Hi </h1> </div> <div> <h2> Hi there </h2> <h2> How are you </h2> </div> </body> </html>
d3.select('body') .selectAll('div') .style('background-color','red')
First the select statement searches the DOM and selects the first occurrence of the body tag, second, the selectAll statement selects all div elements within the body and finally, the style adds a style attribute to each div setting the background color to red.
One thing to note about D3 is a selection method typically returns the current selection or a new selection allowing methods to be easily chained. By convention, if a selection method returns the current selection use four spaces to indent if a new selection is returned use two spaces of indentation.
D3 is a powerful tool but with great power comes great responsibility. D3 takes considerably long time to build a visualization but when used correctly responsive, intuitive and beautiful visualizations can be built. The best example of D3 is The New York Times 2013 Obama’s Budget Proposal visualization, inspecting the element you can view the core elements used.
D3 Graph
To build the time series graph D3 was used, this whole week had me working closely with D3 slowly building each component. The first element that needed to be built was the graph itself, akin to past weeks I used a combination of tutorials, D3 documentation and my past knowledge of javascript. Building a line graph within D3 is relatively simple, the graph consists of two main components axis and the line itself.
To build an axis within D3 you first need to specify the scale. A D3 scale maps data to a visual representation (x and y coordinates in pixels), D3 provides multiple methods both for continuous and discrete data. The example below specifies two linear scales and sets their ranges.
//set scales and ranges var x=d3.scaleLinear().range([0,width]); var y=d3.scaleLinear().range([0,width]);
After the scale ranges have been set, the domains need to be specified. The domain is the complete set of values when specifying the domain you provide an array of two number specifying the minimum and maximum data points.
The example below uses the d3.min and d3.max methods returning the minimum data point and maximum data point from an array respectively. d3.extent may be used instead, which returns the minimum and maximum values.
x.domain([ d3.min(data.Xdata), d3.max(data.Xdata) ]); //or x.domain(d3.extent(data.Xdata));
The final step is to render the actual trend line using the d3.line method which generates a line from an array of data. Using both the x and y scale specified earlier the line method can generate an svg path on screen.
var line=d3.line() .x(function (d){ return x(d.Xdata); }) .y(function (d){ return y(d.Ydata); });
Voila, a line graph is built, well almost. You might notice that the trend line is rendered on screen but no axis is this is because we haven’t told d3 to render any axis. To render an axis simply use the d3.axis[position] method specifying the position and scale.
var xAxis=d3.axisBottom(x); var yAxis=d3.axisLeft(y);
The example below shows the fully how to implement a line graph using D3.
//select svg and specify width and height var graph=d3.select('svg') .attr('width',800) .attr('height',500) //graph data var data=[data] //specify the ranges of the x and y scales var x=d3.scaleLinear().range([0,800]); var y=d3.scaleLinear().range([0,800]); //specify to d3 to render x and y scales var xAxis=d3.axisBottom(x); var yAxis=d3.axisLeft(y); //specify to d3 how to render the line var line=d3.line() .x(function (d){ return x(d.Xdata); }) .y(function (d){ return y(d.Ydata); }); //specify the domains of the x and y scales x.domian(d3.extent(data.Xdata)); y.domain(d3.extent(data.Ydata)); //create a group tag and append the x axis graph.append('g') .attr('class','axis axis--x') .attr('transform','translate(0,'+500+')') .call(xAxis); //create a group tag and append th y axis graph.append("g") .attr("class", "axis axis--y") .call(yAxis); //create a group then a path tag and calculate the path definition to be drawn. graph.append('g').append('path') .attr('class','line') .attr('d',function (d){returnline(data)})
Zooming and Panning
Initially, I predicted that zooming and panning would take a considerable amount of development time instead I completed both the zooming and panning just under half the predicted time mostly due to the in-built d3 zooming capability. D3.zoom method provides a transition vector when the user either zooms or pans, the vector contains three parts a scale, x and y. The transition vector can be used to rescale both the x and y scales appropriately.
function zoomed() { var t=d3.event.transform; var xt=t.rescaleX(x); var yt=t.rescaleY(y); }
Using both xt and yt variables any component can be re-rendered.
The problem with using D3.zoom is it doesn’t distinguish between zooming and panning, for our graph multiple trends line can be viewed, panning should offset only one trend line whilst zoom should zoom all trend lines. To distinguish between zooming and panning, I compared previous transition vectors to the current one, if the scale component has changed I know the user has zoomed if not I know the user had panned.
//last transition vector var lastTransitionVector; function zoomed(){ //current transition vector var t=d3.event.transform; //k is the scale component, if not equal to the //last transition vector user has zoomed var isZooming=endZoomvector.k!=t.k if(isZooming){ console.log('user is zooming'); }else{ console.log('user is not zooming'); } //update the lastTransitionVector lastTransitionVector=t; }
Annotations D3
Annotations can be built directly using D3 but this can be a long and laborious process, instead I used the d3-annotation library. D3-annotation is a library, built by Susie Lu, used to build responsive annotations.
Using the library drastically sped up the development time clawing back a few hours lost from previous weeks. Building basic annotation is very quick process, each annotation needs an x and y pixel or data co-ordinate. If a data co-ordinate is provided the accessors method can be used to it into an x and y pixel co-ordinate, d3-annotation will take care of rendering process. The example below demonstrates how to create a simple annotation.
const type=d3.annotationLabel //annotations const annotations=[{ note: { label: 'Description' bgPadding: 20, title: 'Annotation' }, data: { xPoint: '100', yPoint: '20' }, className: 'show-bg' }] //set up range and domain (not shown here) const makeAnnotations=d3.annotation() .notePadding(15) .type(type) //convert data points to x and y pixel co-ordinates .accessors({ x: d=>x(data.xPoint), y: d=>y(data.yPoint) }) .annotations(annotations) d3.select('svg') .append('g') .attr('class','annotation-group') .call(makeAnnotations)
Once a annotation can be rendered on screen, I had to make sure each annotation maintains its correct position when the user pans and zooms the graph. Akin to the time series graph, I used the rescale method to rescale the x scale to calculate each annotation position. The example below demonstrates how to re-render each annotation, the function expects an array of annotations and a transition vector. Its important to remember the d3-annotation library clears all annotations then renders them back therefor if one annotation moves but a second doesn’t both need to be re-rendered.
function annotationRescale(annotations,t){ const type=d3.annotationLabel var xt=t.rescale(x); var yt=t.rescale(y); const makeAnnotations=d3.annotation() .notePadding(15) .type(type) .accessors({ x: d=>xt(data.xPoint), y: d=>yt(data.yPoint) }) .annotations(annotations) }
One of the main features of annotations is the ability to drag each one to a new position. Luckily the d3-annotation provides a drag event calling a function when an annotation is dragged. The function simply translates the annotations x -coordinate to the cursors x-coordinate.
Building the mechanics of the annotations was a relatively smooth process compared to the annotations UI design. In the first few weeks of design I completely fleshed out the UI for the whole system, both the search and import pages followed the UI wireframes, on the other hand, the annotations UI did not. I quickly realized the original design was extremely clunky and would not work well on a touch device, not wasting too much time, Laurie and I designed a better more intuitive system that would scale nicely on all devices.
Conclusion
Overall this week has been very satisfying,taking raw data from a database and displaying it on screen. Due to last weeks, extra features and bugs, development still isn’t finished, all that remains is the tags and columns panel. By next week all features should be complete leaving the rest of the time for debugging and testing.