We find we’re living in an increasingly data-rich world; our data lakes feed the data we harvest and fill our data silos and data warehouses with; it dissipates into the cloud and pours back down to grow anew.
Accessing this data is becoming harder and harder, though; even with GDPR, many systems require extensive manual work or navigating arcane, computer-science based API interfaces. Even instruments and sensors insist on network or hardware interface controls to access their goods.
Once the data is got, it’s another ordeal to do something with it; to publish it, convert it into something a familiar tool can use, or even let you know when a long duration activity has run it’s course or a certain number of samples have been collected.
While we’ve spent great effort teaching computational thinking and using Microbits and Raspberry Pis, we think we’ve missed a trick; we’ve sweated the details, but ignored the bigger picture – how do we get data from that to this and make a record of it, without spending and involving a huge team of specialists?
We think we have you covered.
Data, Not Programmes
Node-Red is a system for working with flows of data, rather than writing programmes, as in traditional computational applications. Systems are built by dragging nodes onto a page and connecting their inputs and outputs, rather than by writing reams of syntactically structured code. Changes can be made and tested easily, and data can be used in multiple different ways without significantly impacting existing systems.
Node-Red is lightweight and can even run on a £10 Raspberry Pi. With a huge number of pre-built expansion modules available, the basics for most activities are already in place, whether that’s gathering data from FitBits or environmental sensors, to publishing data to Amazon or physical actuators.
Data-driven systems and their applications
Researchers may already have workflows that could benefit from some automation; others may be trying to find ways to realise systems that they can just about see but are beyond their funding and technology reach. Here we’re going to look at a few applications where Node-Red can provide an easy solution to realise previously challenging, expensive, or resource-demanding applications.
Social Data analysis
We have access to some extraordinary social and publishing tools. Twitter provides a real-time interface between writing and global publishing, which has been a remarkable asset for a number of unexpected applications.
- Public health monitoring, disease outbreaks, and disaster prediction
- Disinformation campaign analysis
Collating specific datasets or samples is a challenging activity with social media; the data is sent in realtime, then disappears; storage costs become large quickly without whittling down the inbound messages. Node-Red makes all these activities easy.
As machine-readable and open data becomes prevalent, journalism is finding new ways to make use of this tool to understand events and tell stories. Tasks such as plotting data on maps, over time, can still be extraordinarily challenging, however, as our tools have not caught up to the kinds of time-based, geolocated, and multivariate data we have to work with, still steeped in two-dimensional spreadsheets.
Internet of Things, Smart Cities, and Smart Homes
Node-Red excels at working as the intelligence layer in systems automation. As our emphasis shifts to maximising power usage, our devices are outsourcing their intelligence to ‘the cloud’, where online systems perform the analysis and storage work that would have taken place on-device a few years ago. Node-Red natively provides access to messaging and real-time data access, making it an ideal platform to provide not only the backbone to smart systems, but providing the hooks to integrate them with larger systems and big data and machine intelligence applications.
- The Things Network uses Node-Red for consuming and pushing information to smart sensors and devices over LoRaWAN
- Smart Smoke alarms have been piloted to reduce the cost and false-positives for GMFRS
Instrumentation and data-gathering
Traditional tools for interfacing with hardware and sensors are expensive and limiting. They require specialist knowledge and expertise, as well as on-going licensing costs. Node-Red, Raspberrys Pi, and cheap microcontrollers work incredibly well together, reading data not only from network interfaces but also serial interfaces and GPIO pins.
Embrace the flow!
We’re really excited about the potential of this tool to transform how research and design can approach computational tasks. It’s cheap, forgiving, and immediately responsive.
At Digital Labs, we’re planning a number of workshops to support your activities with Node-Red, and develop custom functionality that may fit your specific demands. Please talk to us or get in touch through our contact page to discuss ways we can help.