D3 Tips and Tricks v4

Wednesday, 18 February 2015

Raspberry Pi Smackdown: Email vs Google+ vs reddit

If you build it they will come right?

Well, they're only going to come if they know where to go.

Writing books is honestly a pretty fun thing to do, and I won't lie to you when I say that it's kind of neat to have people actually read the books as well. So, when you write a book and make it available for free as I have done with D3 Tips and Tricks, Leaflet Tips and Tricks and Raspberry Pi: Measure, Record, Explore, you can be fooled into thinking that there is no barrier to people who are interested in the topic downloading it right?

Brace yourself.... The internet doesn't care.

I know... I know... I doesn't seem to make sense. People will care on an individual basis, but as they scale up in a network, the capacity for caring becomes diluted and the Internet is a really big network. There are just sooooo many other things to care about and caring is a finite resource.

The extreme in this example is when there is something that could genuinely help them out for no investment on their part, but they don't get or even know that the opportunity exists.

So how do people get to know that there's something 'out there' that they might care about?

If you've read this far hoping that there is going to be a magic revelation, I'm afraid that [SPOILER ALERT] the following will not provide one. However, it may provide (as it did to me) some measure of wonder at the way that Internet allows communities to form and people to interact in social structures that mimic islands of interest and activity.

So why the revelation?

When I published my latest book (Raspberry Pi: Measure, Record, Explore) I did so on with the on-line service Leanpub. They do a great job and have an awesome service that I would heartily recommend to anyone, but the act of publishing a book does not mean that suddenly your weighty tome becomes the top return in any given Google search. So I did what I thought would be a good idea and I emailed the readers of one of my previous books (D3 Tips and Tricks,). This is a facility available via Leanpub and I am really reluctant to do it since it means disturbing close to 17,000 people. There is a strictly relevant reason for doing so since Raspberry Pi: Measure, Record, Explore includes a fair bit of d3.js goodness as well, so I think my conscience is clear.

So what was the result of this?


As you can see, although I published it on Christmas day, there were quite a few people who were interested enough to visit the site and at least get the option to decide if the book was interesting enough to download. We can notice that the interest spikes at around 500 new visitors to the site and then trails off slightly as (I assume) a diminishing number of people read their email and opt to act or not. This is fairly interesting in its own right, but that would be a digression.

Once it trailed off we were left with a relatively constant 10-15 new users visiting the page every day. Towards the end of January I published some new material in  D3 Tips and Tricks, and took the opportunity to re-send the link to the Raspberry Pi book through again.


This time (as should be expected), the response was more muted with an initial peak of around 75 new visits. In other words, pretty much anyone in that community who had an interest had visited. Fair enough.

At this point I don't really recall why, but I decided that for the first time I would use Google+ to look for information on the Raspberry Pi. It appeared that there was a good community there, so I posted the latest update to the book there and had the following response.


This topped out at about 100 new users and that was pretty neat I thought for a group that I had never considered.

And then.....



One evening I'm minding my own business and I see a notification that a reader has donated when downloading the book. Always a nice moment and I have a little smile.

5 minutes later... Another email... Another reader has donated. Wow! that's really unusual to have two people donate in quick succession, perhaps one person talks to another person.. "Check this out" kind of thing. Sure... That explains it.

10 minutes later... Another email... Hold on.. What's going on here....

A quick trip to Google analytics tells me who the culprit is...

Now I know that it's small potatoes in the scheme of things, but the level of interest had just gone up considerably for this page.

So a trip to reddit and after a bit of a hunt there it was...


So at this point I start to have a look around at reddit. Wow, there are quite a few folks here who are interested in the Raspberry Pi! OK, time to become a member of reddit.

Now I'm all joined up and have even been brave enough to comment a few times on posts.

It's come time however, to take the next step. I'll have to post something of my own. I'll need a post. Oh, hang on here's one :-).

Lastly, here's the graph with the results of the comparison;


As a small experiment I'll try the following....

  • First I'll just publish this post and see how many page visits it gets.
  • Then I'll tweet to my hoard of ardent followers (you're all awesome!) and measure the response
  • Then I'll push a post to Google+ and check the numbers
  • Lastly I'll see if putting it onto reddit has an equivalent response.

This will probably be the most un-scientific thing ever, but hey, welcome to the internet!

If any readers are considering posting something about this to skew the results, go ahead! How bad could it get :-)

For a noob like myself this is a fascinating experience. Now of course I'm wondering just what other communities are out there.....

Thursday, 12 February 2015

Raspberry Pi System Information Measurement: 3.b Explore

The following post is a section of the book 'Raspberry Pi: Measure, Record, Explore'.  The entire book can be downloaded in pdf format for free from Leanpub or you can read it online here.
Since this post is a snapshot in time. I recommend that you download a copy of the book which is updated frequently to improve and expand the content.
---------------------------------------
This is the second part of the third post working through a project looking at Measuring Recording and Exploring system information from and with the Raspberry Pi.

PHP
As mentioned earlier, our html code requires JSON formatted data to be gathered and we have named the file that will do this job for us sys_info.php. This file is per the code below and it can be downloaded as sys_info.php with the downloads available with the book.
We will want to put this code in the same directory as the sys_info.py and bullet.js files, so they can find each other easily.
$hostname = 'localhost';
$username = 'pi_select';
$password = 'xxxxxxxxxx';

try {
    $dbh = new PDO("mysql:host=$hostname;dbname=measurements",
                               $username, $password);

    /*** The SQL SELECT statement ***/
    $sth = $dbh->prepare("
       SELECT *
       FROM `system_info` 
       ORDER BY `dtg` DESC
       LIMIT 0,1
    ");
    $sth->execute();

    /* Fetch all of the remaining rows in the result set */
    $result = $sth->fetchAll(PDO::FETCH_ASSOC);

    /*** close the database connection ***/
    $dbh = null;
    
}
catch(PDOException $e)
    {
        echo $e->getMessage();
    }

$bullet_json[0]['title'] = "CPU Load";
$bullet_json[0]['subtitle'] = "System load";
$bullet_json[0]['ranges'] = array(0.75,1.0,2);
$bullet_json[0]['measures'] = array($result[0]['load']);
$bullet_json[0]['markers'] = array(1.1);

$bullet_json[1]['title'] = "Memory Used";
$bullet_json[1]['subtitle'] = "%";
$bullet_json[1]['ranges'] = array(85,95,100);
$bullet_json[1]['measures'] = array($result[0]['ram']);
$bullet_json[1]['markers'] = array(75);

$bullet_json[2]['title'] = "Disk Used";
$bullet_json[2]['subtitle'] = "%";
$bullet_json[2]['ranges'] = array(85,95,100);
$bullet_json[2]['measures'] = array($result[0]['disk']);
$bullet_json[2]['markers'] = array(50);

$bullet_json[3]['title'] = "CPU Temperature";
$bullet_json[3]['subtitle'] = "Degrees C";
$bullet_json[3]['ranges'] = array(40,60,80);
$bullet_json[3]['measures'] = array($result[0]['temperature']);
$bullet_json[3]['markers'] = array(50);

echo json_encode($bullet_json); 
The PHP block at the start of the code is mostly the same as our example code for our single temperature measurement project. The difference however is in the select statement.
       SELECT *
       FROM `system_info` 
       ORDER BY `dtg` DESC
       LIMIT 0,1
It is selecting all the columns in our table, but by ordering the rows by date/time with the most recent at the top and then limiting the returned rows to only one, we get the single, latest row of data returned.
Returned system data
Returned system data
Most of the remainder of the script assigns the appropriate values to our array of data bullet_json.
If we consider the required format of our JSON data…
[
  {
    "title":"CPU Load",
    "subtitle":"System Load",
    "ranges":[0.75,1,2],
    "measures":[0.17],
    "markers":[1.1]
  }
]
…we can see that in our code, we are adding in our title
$bullet_json[0]['title'] = "CPU Load";
… our subtitle
$bullet_json[0]['subtitle'] = "System load";
… our ranges
$bullet_json[0]['ranges'] = array(0.75,1.0,2);
… our system load value as returned from the MySQL query…
$bullet_json[0]['measures'] = array($result[0]['load']);
… and our marker(s).
$bullet_json[0]['markers'] = array(1.1);
This data is added for each chart that we will have displayed as a separate element in the bullet_json array.
Finally, we echo our JSON encoded data so that when sys_info.php is called, all d3.js ‘sees’ is correctly formatted data.
echo json_encode($bullet_json); 
Now every 60 seconds, the d3.js code in the sys_info.html script calls the sys_info.php script that queries the MySQL database and gathers our latest system information. That information is collated and formatted and converted into a visual display that will update in a smooth ballet of motion.
As a way of testing that this portion of the code is working correctly, you can use your browser from an external computer and navigate to the sys_info.php file. This will print out the JSON values directly in your browser window.

The post above (and heaps of other stuff) is in the book 'Raspberry Pi: Measure, Record, Explore' that can be downloaded for free (or donate if you really want to :-)).