D3 Tips and Tricks v6: Interactive Data Visualization Online Course

Wednesday, 22 September 2021

What do we mean when we describe d3.js as an API?


An application programming interface (API) is a connection between computers or applications that allows specified interactions to expose services to be provided.

D3.js (where 'D3' is short for 'Data Driven Documents') is a JavaScript library which is designed to produce data visualisations. It does this by binding data to the Domain Object Model (DOM) which can be presented in the browser.

D3 provides the specification for a data driven approach to DOM manipulation and as such the d3.js library provides the API for the interaction between the data and the DOM.

API structure

The D3.js API contains several hundred functions, and they can be grouped into following logical units:

  • Arrays: Array manipulation, ordering, searching, summarizing, etc.
  • Axes: Human-readable reference marks for scales.
  • Brushes: Select a one- or two-dimensional region using the mouse or touch.
  • Chords: Visualize relationships or network flow with a circular layout
  • Colors: Color manipulation and color space conversion.
  • Color Schemes: Color ramps and palettes for quantitative, ordinal and categorical scales.
  • Contours: Compute contour polygons using marching squares.
  • Voronoi Diagrams: Compute the Voronoi diagram of a set of two-dimensional points.
  • Dispatches: Separate concerns using named callbacks.
  • Dragging: Drag and drop SVG, HTML or Canvas using mouse or touch input.
  • Delimiter-Separated Values: Parse and format delimiter-separated values, most commonly CSV and TSV.
  • Easings: Easing functions for smooth animation.
  • Fetches: Convenience methods on top of the Fetch API.
  • Forces: Force-directed graph layout using velocity Verlet integration.
  • Number Formats: Format numbers for human consumption.
  • Geographies: Geographic projections, shapes and math.
  • Hierarchies: Layout algorithms for visualizing hierarchical data.
  • Interpolators: Interpolate numbers, colors, strings, arrays, objects, whatever!
  • Paths: Serialize Canvas path commands to SVG.
  • Polygons: Geometric operations for two-dimensional polygons.
  • Quadtrees: Two-dimensional recursive spatial subdivision.
  • Random Numbers: Generate random numbers from various distributions.
  • Scales: Encodings that map abstract data to visual representation.
  • Selections: Transform the DOM by selecting elements and joining to data.
  • Shapes: Graphical primitives for visualization.
  • Time Formats: Parse and format times, inspired by strptime and strftime.
  • Time Intervals: A calculator for humanity’s peculiar conventions of time.
  • Timers: An efficient queue for managing thousands of concurrent animations.
  • Transitions: Animated transitions for selections.
  • Zooming: Pan and zoom SVG, HTML or Canvas using mouse or touch input.

Each of these categories provides access to standard specified interactions that allow data bound to the DOM elements to be manipulated and presented to the user.


For example, the manipulation of data in an array is a common function to allow analyzing and visualizing data.

To access the d3-array function in an html document we can either load the d3.js library in full or just the specific module. If we were doing this from a traditional CDN service that might look something like the following when loading the full library;

<script src="https://cdn.jsdelivr.net/npm/d3@7"></script>

... or like the following when loading just the d3-array module ...

<script src="https://cdn.jsdelivr.net/npm/d3-array@3"></script>

To test the function to find the mean value of an array of numbers, we can use the d3.mean function like so;


var a = [10, 53, 8, 99, 1, 6];
new_size = d3.mean(a);


The output to the console will be the value 29.5 which is the mean value of the numbers 10, 53, 8, 99, 1 and 6.

To extend the example slightly the following adds two paragraph lines to an html page, computes the mean value of an array and then selects the first of our paragraphs and changes the size of the text to the mean value;


<!-- load the d3.js library -->        
<script src="https://cdn.jsdelivr.net/npm/d3@7"></script>

<p>This is the first line</p>
<p>This is the second line</p>


var a = [10, 53, 8, 99, 1, 6];
new_size = d3.mean(a);

d3.select("p").style("font-size", new_size + "px");



And what we see should look like the following in the browser....

This is the first line

This is the second line

Wednesday, 1 September 2021

Raspberry Pi GPIO Pins Overview - The Movie!

 I made a thing!

It's that common lockdown story. There I was with some time on my hands and thinking of something new to learn. What about that new fangled thing that all the cool kids are talking about called Youtube? What'ts that all about?

I googled it, and apparantly it might catch on. So I thought how do I learn how to Youtube?

Like everything, try it and see....

So behold! my first ever attempt at doing some content for consumption via video.

An overview of Raspberry Pi GPIO Pins. 

So be gentle with me since this is my first time. It's very much a way to learn a new skill and perhaps someone will find it useful.

Maybe I might get excited and do more!

If so, you can expect them to appear here on my 'Channel' (Whatever that is).

Saturday, 12 June 2021

D3 Tips and Tricks v6: Online Training Now Available from Educative

D3.js is an extraordinary piece of software, but more importantly it has been a gateway to data visualisation that has enabled users to be able to turn information into storytelling.

The journey for d3 began over 10 years ago and in that time the central tenants of using pre-built functions to bind arbitrary data to a Document Object Model (DOM), and then apply data-driven transformations to the document have been well established.

However, I can testify as someone that does not have a formal coding background that the barrier to entry can be daunting for some. That's why I started down the journey of writing down the things that I was learning as I started to understand d3 and why ultimately that ended up becoming the book D3 Tips and Tricks (of which there have now been 4 editions!).

That first edition was not just a journey in learning how to wrangle d3, it was also a learning experience in how to gather information into a book form and to make it available to others. My default start point was to make the information available for free to whomever wanted to read it and as it turns out, this wasn't so easy. However, cometh the hour, cometh the platform. 

Leanpub provided an excellent ecosystem for supporting the development, publishing and front-end for making the book available to others and in the process I found that I enjoyed the act of writing. I won't claim to have a particularly 'cultured' delivery style. I am frequently irreverent and prefer describing what I am learning in a fashion that makes sense to someone who might be somewhat naive about the background technical niceties.  

I'll be honest and say that I was definitely encouraged by the feedback and uptake of the book. People kindly offered advice and suggestions for improvements. Even going so far as to do proof reading and cover design work!

The end result (although this is by no means the end!) is that people have downloaded my books over 92,000 times. That is staggering and quite frankly, a pretty daunting prospect to consider that so many people might be taking my advice on something, let alone a JavaScript library that I wanted to learn about because it looked like fun :-).

Doing this has been extremely time consuming and while I don’t begrudge the effort that it entails, it isn’t my day job and I have to be careful about where I place my focus so that I can try and balance the enjoyment that I get out of writing and making sure that the end users are getting the information they need.

So the prospect of putting the quite considerable content that I have amassed over the years into a form of online learning programme has not been something that I have seriously considered because of the amount of time that it would have entailed.

But again... Cometh the hour, cometh the platform. In this case it came in the form of the team from Educative. The contacted me out of the blue to ask if I was interested in partnering with them to make the contents of the book available as an online course. I explained that I wasn’t going to be able to dedicate a considerable amount of time to support them, but they had a scheme for such partnerships that allowed me to make some alterations to the book (in particular to improve the resolution of the images) and for them to be able to deliver the content in an appropriate way.

The end result has been the publishing of D3 Tips and Tricks: Interactive Data Visualization as an online course.

It includes 144 lessons, nine separate quizzes, 18 challenge problems, 295 illustrations and over 60 code samples! The estimation for completion time is 102 hours. That's a lot of learning.

The first time that the Educative team provided me with an overview of the content I was staggered by the presentation and volume of information. It really is a great collection and I’m very glad that they have been able to make it available to a wider audience.

Will this be something that will be a different direction for me and perhaps for some of my other books? Really good question. That will definitely depend on what sort of reaction to the course there is. I am intending to be as active as possible on the Educative forum to provide assistance to students. Perhaps this will be another rewarding outlet that I will want to pursue.

Thursday, 16 July 2020

Controlling the Activity LED on a Raspberry Pi

I updated the ebook 'Just Enough Raspberry Pi' the other day (get it for free from Leanpub) with the method for turning the activity light on and off from the command line.


Turn the activity light on or off

The main board on the Raspberry Pi has power and activity LEDs to indicate when power has been applied (red) and when the on-board SD card is being accessed (green). These are situated on the opposite end of the board to the Ethernet connector on the B models. They can however be on different sides of the display ribbon connector depending on which B model.

LED Positions
LED Positions

Embarrassingly, I have found that when running multiple Raspberry Pi’s I have forgotten which ones are running which software or operating system (This is what I get for writing books on monitoringGhostownCloud etc). This is exacerbated by mounting the Pis in an open stack configuration similar to the following (Imagine it as a slightly higher stack).

Stack o Pi
Stack o Pi

What to do then when faced with a stack o’ Pi and difficulty in telling which is which?

The good news is that we can log into each and force the activity LED to illuminate and hence identify each device.

Cut to the chase and just do it

The first thing we need to do is to set the trigger for the activity LED to GPIO mode;

Then we can turn the LED on by writing a ‘1’ to the ‘led0’ brightness file with the following command;

If we want to turn it off we write a ‘0’ like so;

And to return it to the state where it indicates activity on the SD card we use mmc0 which is shorthand for multi media card 0 (or the SD card);

The explanation of how it works

The /sys directory exists as an interface between the kernel-space and the user-space. As such it is an implementation of the system file system (sysfs). The /sys/class subdirectory is exported by the kernel at runtime and presents devices on the system as a ‘class’ in the sense that it abstracts out the detailed implementation that might otherwise be exposed (the example used in the ‘makelinux’ description of classes is that a driver might see a SCSI or ATA disk, but as a class they are all just ‘disks’).

The following is a highly abridged hierarchy of the /sys/class directory where we can see the range of classes and their respective links.

pi@raspberrypi /sys/class $ tree
├── bcm2708_vcio
│   └── vcio -> ../../devices/virtual/bcm2708_vcio/vcio
├── gpio
│   ├── export
│   ├── gpiochip0 -> ../../devices/soc/3f200000.gpio/gpio/gpiochip0
│   └── unexport
├── graphics
│   ├── fb0 -> ../../devices/virtual/graphics/fb0
│   └── fbcon -> ../../devices/virtual/graphics/fbcon
├── i2c-adapter
├── input
│   └── mice -> ../../devices/virtual/input/mice
├── leds
│   ├── led0 -> ../../devices/soc/soc:leds/leds/led0
│   └── led1 -> ../../devices/soc/soc:leds/leds/led1
├── mem
│   ├── full -> ../../devices/virtual/mem/full
│   ├── mem -> ../../devices/virtual/mem/mem
│   ├── null -> ../../devices/virtual/mem/null
│   ├── random -> ../../devices/virtual/mem/random
│   ├── urandom -> ../../devices/virtual/mem/urandom
│   └── zero -> ../../devices/virtual/mem/zero
├── misc
│   ├── autofs -> ../../devices/virtual/misc/autofs
│   ├── cachefiles -> ../../devices/virtual/misc/cachefiles
│   ├── cpu_dma_latency -> ../../devices/virtual/misc/cpu_dma_latency
│   ├── memory_bandwidth -> ../../devices/virtual/misc/memory_bandwidth
│   ├── network_latency -> ../../devices/virtual/misc/network_latency
│   └── network_throughput -> ../../devices/virtual/misc/network_throughput
├── mmc_host
│   └── mmc0 -> ../../devices/platform/mmc-bcm2835.0/mmc_host/mmc0
├── net
│   ├── eth0 -> ../../devices/platform/bcm2708_usb/usb1/1-1/1-1.1:1.0/net/
│   └── lo -> ../../devices/virtual/net/lo
├── power_supply
├── scsi_device
├── scsi_disk
├── scsi_host
├── sound
│   ├── card0 -> ../../devices/virtual/sound/card0
│   └── timer -> ../../devices/virtual/sound/timer
└── vtconsole
    └── vtcon1 -> ../../devices/virtual/vtconsole/vtcon1

The leds class contains directories for ‘led0’ and ‘led1’.

Inside this directory are the trigger file which determines which kernel modules activity will flash the led and the brightness file that will determine the brightness (duh!) of the led.

If we cat the trigger file we can see that there is a range of different things that can be used as the trigger to illuminate the led.

pi@raspberrypi /sys/class/leds/led0 $ cat trigger
none [mmc0] timer oneshot heartbeat backlight gpio cpu0 default-on input

The multimedia card (mmc0) is set as the default.

The led can only have two levels of brightness; ‘on’ or ‘off’. This corresponds to a ‘0’ or a ‘1’ respectively. To illuminate our led all we have to do therefore is to signal the brightness file that it has the value ‘1’ (per the example above).

To revert to control of the brightness we echo the device responsible for controlling the led to the trigger file. In this case for the activity led it is the ‘mmc0’ device.

Thursday, 21 May 2020

Upgrading Prometheus after installing as a binary

The following are steps that I work through when upgrading Prometheus on my Raspberry Pi based system. It was installed as a standalone binary, so there is a little more to the process. The information is an excerpt from the book 'Raspberry Pi Computing: Monitoring with Prometheus and Grafana'. You can download the full book for free or read online if you wish. There are no catches :-).
Upgrading Prometheus is something that we should do as new versions with new features become available. Have installed a system by downloading and running it as a standalone binary, the simple method such as using the apt-get track won’t work for us.
However, that doesn’t mean that it’s a difficult task. In fact, it’s blissfully easy.
We can make the process fairly straight forward and painless by installing the new version alongside our older version and then just copying over the configuration and database.
What we will do is;
  • Download and decompress our new version
  • Copy the configuration and data from our old version to our new version.
  • Stop Prometheus and Grafana
  • Run our new version of Prometheus manually (not as a service) and test it.
  • Stop the newer version of Prometheus
  • Change the directory name of our old and new versions
  • Start the Prometheus and Grafana services.


In much the same way that we installed Prometheus the first time, the first thing we need to do is to find the right version to download. To do this browse to the download page here - https://prometheus.io/download/. Select the architecture as ‘armv7’ (assuming that we are installing on a Pi 2,3 or 4)
Note the name or copy the URL for the Prometheus file that is presented. On the 10th of May 2020, the version that was available was 2.18.1. The full URL was something like;


Note that we can see that ‘armv7’ is in the name. That’s a great way to confirm that we’re on the right track.
Just for reference, the previous version that we are upgrading from is 2.17.1
On our Pi we will start the process in the pi users home directory (/home/pi/). We will initiate the download process with the wget command as follows;

The file that is downloaded is compressed so once the download is finished we will want to expand our file;

Remove the original compressed file with the rm (remove) command;

We now have a directory called prometheus-2.18.1.linux-armv7. During a new installation we would have renamed this with the mv (move) command to change the directory name to just ‘prometheus’. However, in this case we will work with our new version in this default folder till we’ve tested that it works correctly. This way we can back out of the upgrade if (for whatever reason) it doesn’t go smoothly.

Stop the services

Stopping the Prometheus service means that we need to think a bit about implications. While we have the program stopped, there won’t be any data available for the Grafana service. This means that anything that will be affected by an absence of data will get triggered. For example, if we had an alert set up in Grafana to notify when a metric was absent, that alert will get triggered. If we have other users that are relying on the service to be operating we will need to ensure that we discuss the plans with them ahead of time. To remove the possibility of Grafana thinking that something horribly wrong has happened, we will stop the Grafana service as well.
Stopping both of the services is nice and easy;

Copy the configuration and data

The two things that we will want to copy over from our old installation are our configuration file prometheus.yml and our collected data.
Since we haven’t started our new version of Prometheus yet, it won’t have a data folder, so we can just copy that straight into the appropriate place.

Then copy in our configuration file from our current Prometheus instance

Run the new version manually and test

We can now run Prometheus manually. We will need to ;

To test that it is working correctly, we can open our browser to the Prometheus web page and check that all the things that are in there seem good. Once we are happy we can move on.

Stop the newer version

In your terminal use ‘ctrl-c’ to stop the running manual instance of prometheus.
We should also change back to the home directory.

Change the directory names

Now that we’re happy that everything is in order we can change the name of our older Prometheus instance to reflect its version number (in other words, we won’t get rid of it just yet, because it’s always prudent to keep things about just in case.

And now we can rename the directory of our new version of Prometheus as simply prometheus.

Start the services.

With everything in it’s proper place we can restart the services again and they will automatically start our new version of Prometheus.

Just as a final check, we should go back to our browser and go over the system (both Prometheus and Grafana) to confirm that everything is good.
If you have any users of the system you can advise them that everything is operating well again.

Sunday, 26 April 2020

Current detection, measurement, monitoring and alerting.


The aim of the project is to provide visibility of when the pump from the outflow chamber of a septic system is operating. Then if it is running for too long we want to be sent an alert to notify us as this is a potential indicator of a problem (blockage leading to overflow).


The method that we will use to achieve this will be to measure the current flowing to the pump and to make that data available to a Prometheus / Grafana instance that will then be used for visibility / alerting.



  1. Current Sensor: An ACS712 based module. We'll use a 5A unit since the pump is not expected to exceed 500W.
  2. Analog to Digital Converter: The ADS1015 will be more than sufficient for our needs.
  3. Enclosures: The current sensor must be protected to prevent contact with the live terminals. The A2D converter should be protected where possible.
  4. Voltage divider: We will use three 1k Ohm resistors.
  5. Wiring: Hook up wires with female Dupont connectors and appropriate wiring for the load connection (I used 2.5mm cabling)
  6. Compute: Raspberry Pi Zero W with case, micro SD Card, case and headers.


This project is one that has been born out of necessity. Our septic system is a good unit that operates as required however, there have been a few occasions where we have had the outflow from the final chamber blocked for one reason or another (and there have been a variation here). The end result of this has been the increase in water level in the final chamber. In two cases this has not been caught in spite of the inbuilt alarm (that uses a float switch and an annunciator in the garage) resulting in the septic system flooding (not good).
To add some redundancy to the system and to provide remote alerting capability I decided to add my own alerting system.
I considered a couple of different options including;
  • Looking at the outflow from the septic tank and measuring the volume of water pumped. While possible, this option involves moving parts in a portion of the system that is space constrained and the method for alerting would involve determining the absence of a thing over a period of time that would be indeterminate.
  • An additional (but different) float switch in the final chamber. This was practical and do-able, but the environment is fairly harsh (water, humidity, 'stuff') and it would have been an alert that occurs later in the process of failure (similar to the default installed alarm which once had its float get stuck and failed to alert properly)
  • Current sensing for the pump operation. This has the advantage of providing very quick feedback if there is a problem and keeps any measurement in a 'cleaner' space. The down side is that this does mean connecting a sensor to a high voltage part of an electrical circuit and therefore requires doing the work within local regulations and taking the appropriate safety precautions.
I opted for the current sensing option since it was the best of the three options in terms of getting a good result (early detection). I have the advantage of being a registered electrical service technician and therefore am able to do the work safely and appropriately.


The current sensor we will use is based on the ACS712 Module. I sourced mine from Banggood, but they are widely available for low cost. This specific one used here is the ACS712ELC-05B which can measure plus or minus 5 Amps corresponding to an analog output of 185 mV/A. The unit is powered from a 5V supply, so we will be connecting it to a 5V output from the Pi and using a voltage divider to reduce the voltage output appropriately for input to the analog to digital converter
If you have access to a 3d printer, you can print an enclosure from the design here.
To enable the current sensing option we will also need to include a digital to analog converter. This should be a fairly simple task similar to the method used here for measuring levels from a gas sensor.
If you have access to a 3d printer, you can print an enclosure for the D2A converter from the design here.
The connection layout is as below (again very similar to the method here);

Operating System Setup

The installation below was carried out using the Raspbian 'Buster' OS. This is available from the Raspberry Pi website.
Install the OS onto an SD card and power up the Pi.
The first thing we should do is to configure the Pi appropriately for use. Some of the steps may or may not be needed depending on your circumstances (i.e. will you be using a model Pi with a physical Ethernet connection or Wifi). Where in doubt check out one of the Raspberry Pi Computing books which will walk through the options.
We can start by running the command;
sudo raspi-config
This will give us access to the options for localisation (WiFi country, time-zone, locale) but most importantly (in this case) for enabling the I2C protocol.
Since the ADS1015 uses the I2C protocol to communicate, we need to load the appropriate kernel support modules onto the Raspberry Pi to allow this to happen.
Since we are using the Raspbian distribution there is a simple method to start the process of configuring the Pi to use the I2C protocol.
On the first page select the Interfacing Options with the arrow keys and then tab to select
Then we select the I2C option for automatic loading of the kernel module;
Would we like the ARM I2C interface to be enabled? Yes we would;
Press 'OK' to acknowledge that the interface is enabled.
Press tab to select 'Finish'.
Once this is done we should do what we should always do when embarking on a fresh install on a Raspberry Pi and that's to update and upgrade the software.
sudo apt-get update
sudo apt-get upgrade
From here, you should look to configuring a static IP address or a WiFi connection if desired. Again, where in doubt check out one of the Raspberry Pi Computing books for options.

Software Setup

There's still some work to do to get things sorted, especially for the I2C functionality. We need to check the /etc/modules file using:
sudo nano /etc/modules
Where we need to ensure that the following line is at the end of the file:
Under some circumstances (depending on the kernel version we are using) we would also need to update the /boot/config.txt file. We can do this using;
sudo nano /boot/config.txt
Make sure that the following line is uncommented (the '#' is removed from in front of the line) in the file;
Then we should load tools for working with I2C devices using the following command;
sudo apt-get install i2c-tools
... and now we should reboot to load the 'config.txt' file if we changed it earlier
sudo reboot
We can now check to see if our sensor is working using;
sudo i2cdetect -y 1
If we were using an older B model of Raspberry Pi with 256MB of RAM, we would need to use sudo i2cdetect -y 0.
The output should look something like;
pi@raspberrypi ~ $ sudo i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- 48 -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
This shows us that we have detected our ADS1015 on address '48' (The ADS1015 can support four different addresses as shown on page 17 of the data sheet).
Now we want to install the Python libraries designed to read the values from the ADS1015. The library we are going to use was designed specifically to work with the Adafruit ADS1015/ADS1115 ADCs. In carrying out this library development, Adafruit have invested a considerable amount of time and resources. In return please consider supporting Adafruit and open-source hardware by purchasing products from Adafruit!
We need to change our default version of Python running on the Pi to Python 3
We can check what version is running by executing the following command;
python --version
If that indicates python 2.x, then we need to change that.
To find out what version of Python 3 is available, run the following
ls /usr/bin/python*
Hopefully you will see a 3.x version.
To change the default python version system-wide we can use the update-alternatives command. First list all available python alternatives;
update-alternatives --list python
There is a good chance that the output will be something like;
update-alternatives: error: no alternatives for python
The above error message means that no python alternatives have been recognised by the update-alternatives command. For this reason we need to update our alternatives table and include both python 2 and 3. Just check that the numbers for the versions in the commands below match the versions that the previous command indicated were installed.
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.7 2
The last number on each of the previous lines is the priority, with the higher number being the
highest priority
We can check again by running;
python --version
Now the default version should be Python 3
Install pip3 (for Python 3)
sudo apt-get install python3-pip
Then install setuptools via pip
sudo pip3 install --upgrade setuptools
Install python libraries
pip3 install RPI.GPIO adafruit-blinka
Install the ADC libraries (Note: I occasionally get errors when carrying out the following command and need to re-run it a few times);
sudo pip3 install adafruit-circuitpython-ads1x15
We should now be able to run a simple program to test our sensor.
Create a file called current-test.py (using nano) with the following contents in the home directory (this code can be downloaded from GitHub here);

import time
import board
import busio
import adafruit_ads1x15.ads1015 as ADS
from adafruit_ads1x15.analog_in import AnalogIn

# Create the I2C bus
i2c = busio.I2C(board.SCL, board.SDA)

# Create the ADC object using the I2C bus
ads = ADS.ADS1015(i2c)
ads.gain = 1

# Create single-ended input on channel 0
chan = AnalogIn(ads, ADS.P0)

# Create differential input between channel 0 and 1
#chan = AnalogIn(ads, ADS.P0, ADS.P1)

print("{:>5}\t{:>5}".format('raw', 'v'))

while True:

  readValue = 0         
  maxValue = 0      

  for i in range(0,300):
    readValue = chan.value

    if (readValue > maxValue):         
      maxValue = readValue

  # write the current integer value to a file
  f = open("current.txt", "w")

  print("{:>5}\t{:>5.5f}".format(maxValue-6680, chan.voltage))
The file above is essentially the simpletest.py file that Adafruit publish with some amendments.
One of these is that we are only looking at an instantaneous measurement of current. For AC current that could be anywhere along the sine wave in a positive or negative direction.
The solution (which is in the code) is to put in a small loop that determines the maximum value while the loop is running. This indicates the peak value in one direction.
As well as this we can reduce the gain and put in an offset that allows our returned value to read close to zero with no current flowing.
It also includes a small section that writes our current value to a text file. We will use this to contain our value to pass onto Prometheus / Grafana for monitoring later.
Now we can run it using
python current-test.py
We should see a print out of the values represented by the value of the ADC as an integer and the voltage from the ADC as a floating point value (this is without any current flowing in our sensor).
  raw       v
   -8   0.83203
   56   0.82403
   40   0.82603
   56   0.82803
   56   0.82403
   56   0.82403
And we should also have a file (called 'current.txt') created and updated with the most recent reading.

Create a Service to Run Our Python File

I configured the current measurement as a service as while testing the installation I noticed that the program would occasionally fail at random times. This will allow it to restart if necessary and to start on boot.
The first step in this process is to create a service file which we will call current.service. We will have this in the /etc/systemd/system/ directory.
sudo nano /etc/systemd/system/current.service
Paste the following text into the file and save and exit (the file is also available from GitHub).
Description=Python Current Measurement

ExecStart=/usr/bin/python /home/pi/current.py

The service file can contain a wide range of configuration information and the case above there is 'Restart' which will restart (duh) the service if it fails and 'ExecStart' which describe where to find python and the current.py executable.
The other important part allows the service to restart if it fails.
Before starting our new service we will need to reload the systemd manager configuration again.
sudo systemctl daemon-reload
Now we can start the current service.
sudo systemctl start current
You shouldn't see any indication at the terminal that things have gone well (or otherwise), so it's a good idea to check current's status as follows;
sudo systemctl status current
We should see a report back that indicates (amongst other things) that 'current' is active and running.
Now we will enable it to start on boot.
sudo systemctl enable current

Integrate with Prometheus / Grafana

If you haven't used Prometheus and Grafana for monitoring before, let me take a moment to enthusiastically recommend it to you. Combined, they make a very powerful platform for checking up on your various pieces of IT and they are tools that are widely used in the outside world (in other words, the skills you build in using it are transferable). Better yet, they will run really well on a Raspberry Pi!
I'm not going to describe how to install both here, but I would recommend working your way through this book which you can read online or download (for free) to step through the process.
Assuming you have an instance of Prometheus / Grafana available, we can add a custom exporter for monitoring our current as follows;
We are going to add this instrumentation via the officially supported libraries for Python.
When implemented it will gather our information and expose it via a HTTP endpoint (the same way that Prometheus's node_exporter does with http://<IP Address>:9100/metrics.
Our metric name will reflect the type of thing that is being measured. You can see a range of good examples in the metrics exposed from node_exporter. For example 'cpu_frequency' and 'boot_time'. For our measurements we will have 'current'.
The last part of the name should describe the units being used for the measurement (in plural form). In our case the current will be in 'milliamps'.
The full name of our metrics will therefore be;
  • septic_current_milliamps
To use the custom exporter on our target machine (in this case the IP address of the Pi on the septic tank is, we’ll need to install the Prometheus client library for Python and pip as follows;
sudo apt-get install python-pip
pip install prometheus_client
To collect information, we need to make sure that we can gather it in a way that will suit our situation.
Our 'current' value is read by a Python script that is running in the background and writing the value to a file called 'current.txt'.
So, our python exporter script (which in this case is named septic-exporter.py) looks like the following (this file is also available from GitHub);
import prometheus_client
import time


current = prometheus_client.Gauge('septic_current_milliamps',
            'Value relating to current flow with zero being effectivly nil.')

if __name__ == '__main__':

while True:
  with open('/home/pi/current.txt', 'r') as f:
    current_value = f.readline()
    except: pass

The main actions in our exporter can be summarized by the following entries:
  • Import the Prometheus client Python library.
  • Declare gauge metric with the metric name that we decided on earlier.
  • Instantiate an HTTP server to expose metrics on port 9999.
  • Start a measurement loop that will read our metric value every 15 seconds
  • Gather our metric value from our text file.
In the same way that we made sure that our current measurement program starts up simply at boot, we will configure our Python exporter script as a service and have it start at boot.
The first step in this process is to create a service file which we will call septic_exporter.service. We will have this in the /etc/systemd/system/ directory.
sudo nano /etc/systemd/system/septic_exporter.service
Paste the following text into the file and save and exit (this file is also available at GitHub).
Description=Septic Exporter

ExecStart=/usr/bin/python /home/pi/septic-exporter.py

The service file can contain a wide range of configuration information and in our case there are only a few details. The most interesting being the 'ExecStart' details which describe where to find python and the septic-exporter.py executable.
Before starting our new service we will need to reload the systemd manager configuration again.
sudo systemctl daemon-reload
Now we can start the septic_exporter service.
sudo systemctl start septic_exporter
You shouldn't see any indication at the terminal that things have gone well, so it's a good idea to check septic_exporter's status as follows;
sudo systemctl status septic_exporter
We should see a report back that indicates (amongst other things) that septic_exporter is active and running.
Now we will enable it to start on boot.
sudo systemctl enable septic_exporter
The exporter is now working and listening on the port:9999
To test the proper functioning of this service, use a browser with the url of our current measuring Pi:
This should return a lot lot statistics. Some of them will look a little like this;
# HELP septic_current_milliamps Value relating to current flow with zero being effectivly nil.
# TYPE septic_current_milliamps gauge
septic_current_milliamps -520.0
There are a lot more metrics than just our septic info, but you can see our metrics there.
Now that we have a computer exporting metrics, we will want it to be gathered by Prometheus

Adding adding our custom exporter to Prometheus

We need to add the IP address of our new metrics source to the Prometheus prometheus.yml file. To do this we can simply add the IP address of a node that is running the septic_exporter as a new target and we are good to go.
On our Prometheus server;
nano /home/pi/prometheus/prometheus.yml
At the end of the file add the IP address of our new node - targets: [''];
  # The job name is added as a label `job=<job_name>`
  # to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    - targets: ['localhost:9090']
    - targets: ['']
    - targets: ['']
Then we restart Prometheus to load our new configuration;
sudo systemctl restart prometheus
Now if we return to our Prometheus GUI to check which targets we are scraping we can see our new node at

Make a graph in Grafana

Making sure that we can see what is happening with our sensor seems like a really good idea. Check out the book for a good overview of the process, and for those who would like a very simple graph in json format for importing, there is one available in GitHub here.

Add an alert

Part of the point for this entire project was that I wanted to know when the pump for the system was running for too long. Grafana and its alerting feature are the end game for that problem.
Alerts in Grafana allow us to set limits on our graphs such that when they are exceeded, we can send out notifications (for instance have one that indicated a drop in temperature that sent an alert to let you know to cover up delicate seedlings or an increase in used disk space that indicated a problem on a computer).
Looking at a graph of data that represents the operation of our pump (via the measurement of the current that it's drawing), we can set an alert that will trigger if our current draw goes above 1000 for more than 5 minutes (given that our pump will drain the final chamber in just under 2 minutes, this is a fair time).
Check out the book to see an explanation for configuring Grafana to enable sending email alerts and for adding an alert to our graph. There is a Grafana panel saved in GitHub that includes an alerting component.