The first section follows the pipeline prominently marked on Topo50 just south of Otira township. There’s a corresponding parking area on the east side of the road. The map and other trip reports are a bit misleading in terms of the first section of the route. The narrow pipeline reaches a large metal box then widens and proceeds to the creek intake. If you’re doing this trip, ensure you pick up the ribbon-marked track which turns off at this metal box to steeply climb the spur. We overshot and reached the creek before backtracking. At least it provided an opportunity to fill water supplies for the dry day ahead!

From the tops, we sidled around in the vague direction of Mt Barron summit. There’s no need to climb point 1629 or reach the ridge early.

Scree below the summit may be suitable for descent, but for our upwards route we chose to head up the more stable rock to the right. This reaches a narrow saddle.

We sidled behind a rock formation to reach the main saddle (visible at the top of the main scree chute in the earlier image). This side yielded a fresh view of Kellys Range. From the saddle it was a matter of carefully climbing between rocky outcrops to reach the low peak of Mt Barron.

The aim of the descent route was to follow the ridge over point 1569 then down the scree below point 1265. We made a mistake by picking up the ridge too soon—this would have been sensible in the opposite direction but really just lost us time. Instead the goal should be to get to the bottom of the basin south of the summit before picking up the ridge. This is where we ended up regardless, but there was a much more direct scree we could have ridden there from the low peak!

A benefit of our little detour was the chance to see and hear some elusive rock wrens nearby.

We found our way to the ridge and enjoyed some easier travel as the tops step down towards our scree exit point.

Our exit scree chute was the one below point 1265—we specifically took the northern branch visible on Topo50. We found the rock very loose (especially near the top), so care, patience and liberal spacing were needed to get all of us down without any concussions.

I suspect the track is a more practical descent route, especially with a large group and/or tired knees. That said, the scree is certainly a more interesting option. At the base of the chute, a surprisingly straightforward downhill bush bash took us to the (incredibly solid and level) unsealed service road for the Otira rail tunnel, a short walk from the carpark where it joins SH73.

Before long, the group made it to the (very busy) summit for a snack while looking out over the surrounding mountains.

From here the goal became to make it to the ridge leading towards the dramatic Mount Rolleston. To get there, we dropped off the summit to the south down some nice scree before looping back to the saddle just west of the summit.

We followed the ridge towards the highlight of the trip: the lengthy scree descent into Crow Valley. The drop was further along the ridge than I expected but is quite obvious, with a sturdy pole at the top and (on this bluebird day) a clear line of sight right to the base of the valley.

We followed our noses down the valley, picking up track markers just above Crow Hut. This would make a great overnight stopover but we pressed on, exiting to Klondyke Corner on the same day.

My favourite part of the track is the approach to the tops, with spectacular shrubs and buttercup clinging to the hillside as the tree cover lessens and the view open out.

Shortly past this point, the tops open up and Carroll Hut becomes visible. There were many visitors during the holiday season, with a full hut and families camping on the flat below the hut. We filled our bottles from the stream before continuing to our destination: the tops above the hut.

There is good camping on the tops, but it took us a while to find a level spot among the rolling terrain.

We awoke to the grating sound of my favourite alpine alarm clock: nearby kea investigating our tents at 5:30am.

Instead of going back to sleep, I climbed up Kellys Hill itself, to check out the radio station at one of the peaks and enjoy the views.

There is a poled route up to the radio station from the the main Kellys track. It branches of the track near the tarns.

The track was better marked than I thought, with poles present for the entire loop. This is despite the northern part of the route being entirely missing from Topo50. We completed the loop in clockwise direction which is better for getting to the summit quickly. A counter-clockwise attempt would be worth considering to ensure the steepest section (the northern section of the track, directly above the saddle) is an ascent.

There are two ways up the hill from the Courthouse Flat trailhead: up the spur or via Blue Creek. I entered up the spur and exited by Blue Creek. The creek is muddier, more secluded and steeper than the spur but has some lovely bush. It also passes by interesting mining artifacts on the flat lower section, including the stamping battery marked on Topo50.

It’s a steady uphill slog to the sidle around point 1335. The section ominously marked “Staircase” on the map is indeed steep but thankfully required no scrambling or climbing.

After this point it’s a straight shot to the hut. The last section of the approach is along a streambed.

The hut is stunningly positioned in the open neck of Granity Pass with clear views up to the surrounding rock formations and tops.

With two nights at the hut, I spent the next day exploring the open Mt Owen tops. This area is a bit of a playground for climbers and cavers at certain times of year. The scenery is astounding, with gnarly rock faces and disappearing streams. Sentinel Hill is just a short stroll above the hut and has excellent lines of sight to surrounding peaks. The path is not marked above the hut but is fairly well trodden.

I attempted Mt Owen summit during the day, but abandoned the effort due to the unsettling combination of ravines and late-season ice still settled around the place. I had little confidence that there was solid ground under any ice I stepped on!

As consolation, a pair of heli-tourists who had parked up at the base of the rocky peak handed me half a bottle of wine on my way back down, which I enjoyed at the hut with a book and some snacks. The afternoon was sunny and calm. I didn’t see a soul until the following day.

My home office lacks natural light, and this can lead to a fairly grainy and poorly-lit webcam feed. I’ve been considering upgrading the lighting setup with a key light but realised that, in theory, I could make my own camera light with parts and tools I already had:

• Decent-quality LED strip (the same 24V strip I once used for a ceiling light)
• WiFi LED driver (the awesome ElectroDragon ESP32-C3 one I used in that project; it’s currently driving a 24V LED strip behind my desk as a bias light)
• Some silicone LED strip conduit I purchased a while back but never ended up using
• 3D printer

After some tinkering, I now have a VESA-mounted camera light which automatically turns on whenever an application starts the webcam. Here’s how I did it!

Mounting the light

The silicone conduit is flexible and from the get-go I imagined the light as a curve of the conduit which would “pop up” from behind the monitor. I also knew that I wanted it to attach to the monitor’s VESA mount so that I didn’t need to rely on adhesive (which so often leads to disappointment!).

I measured the conduit then whipped up a basic design in OpenSCAD, which I printed then iterated. The part is essentially an “adapter” which connects to half a VESA mount at one end and the conduit at the other. By printing two mirrored parts, I can mount both ends of the conduit to a VESA mount.

Powering the light

I’m using an ESP32-C3 driver board for the bias lighting behind my monitor. The new light will be in a similar place—it’s just pointing in the opposite direction (towards my face instead of the wall). I decided to drive the new LED strip from the same board. This works well as the board has four PWM output channels. They’re labelled “RGBW”, but that doesn’t mean they can only be used for coloured LED strips. Each temperature adjustable strip uses two channels (one for cold white, one for warm white), therefore I can wire the new strip to the currently unused “blue” and “white” pins. The common 24V anode on the strip will be soldered to the same “V” pin as the other strip, as both strips are the same voltage.

I also double-checked to ensure my power supply’s 24V rail would be able to drive the extra strip at max brightness. These strips max out at 8W/m, so the 40cm camera light will draw no more than 4W or so at max brightness on both channels. On top of the 10-15W already used by the bias light, this should be easily managed by the 24W supply.

Now, I updated the ESPHome config to make it aware of the second light on the “blue” and “white” pins:

esphome:
  name: 'esphome-desk-light'
  platformio_options:
    board_build.flash_mode: dio

esp32:
  board: esp32-c3-devkitm-1
  framework:
    type: arduino
    version: latest

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: 'Desk Light Fallback Hotspot'
    password: !secret wifi_ap_password

captive_portal:

web_server:
  port: 80

logger:

api:
  # Redacted

ota:
  platform: esphome

light:
  - platform: cwww
    name: 'Desk bias light'
    cold_white: redpin
    warm_white: greenpin
    cold_white_color_temperature: 6000 K
    warm_white_color_temperature: 2700 K
    constant_brightness: true
  - platform: cwww
    name: 'Desk camera light'
    cold_white: bluepin
    warm_white: whitepin
    cold_white_color_temperature: 6000 K
    warm_white_color_temperature: 2700 K
    constant_brightness: true

output:
  - platform: ledc
    pin: GPIO5
    id: redpin
    frequency: 19531Hz
  - platform: ledc
    pin: GPIO8
    id: greenpin
    frequency: 19531Hz
  - platform: ledc
    pin: GPIO4
    id: bluepin
    frequency: 19531Hz
  - platform: ledc
    pin: GPIO3
    id: whitepin
    frequency: 19531Hz

And everything magically appears in Home Assistant!

Turning the light on with the Mac webcam

At this point, I have a sweet camera light that I can turn on using Home Assistant, just like my other lights. But a smart home’s only smart if it’s automated, and in this case the automation is clear: when my laptop’s camera is in use, I want the light to turn on; when the camera is not in use, the light should turn off.

In MacOS, there are a few commands that can help achieve this. The following ones work on MacOS Sonoma 14.5, but tweaks may well be needed for different versions.

To detect when the camera is off on on, I can use the long-running command:

exec log stream --predicate 'process == "appleh13camerad" && (eventMessage CONTAINS "PowerOnCamera" || eventMessage CONTAINS "PowerOffCamera")' --style compact --color none

I should be able to hook into the events I need now, but it’s important that I only toggle the light when I’m actually at my desk! To work this out, I can check to see whether the laptop dock is connected:

system_profiler SPThunderboltDataType | grep -q "UID: 0x014AFA4DCC5947BE"

This will get the Thunderbolt devices connected to the laptop and fail if my dock’s specific UID is not present.

Finally, I need to actually toggle the light from the command line. Because I enabled web_server in ESPHome, I can just use cURL to POST the relevant endpoint on the light:

curl -X POST 'http://esphome-desk-bias-light.local/light/desk_camera_light/turn_on'
curl -X POST 'http://esphome-desk-bias-light.local/light/desk_camera_light/turn_off'

Putting it all together

Much of this is inspired by an AskDifferent question which used a similar solution for a “do not disturb” light outside a room.

Using the commands above, the following script waits for relevant events, checks whether the dock is attached, then requests the relevant state from the light.

#!/bin/bash

exec log stream --predicate 'process == "appleh13camerad" && (eventMessage CONTAINS "PowerOnCamera" || eventMessage CONTAINS "PowerOffCamera")' --style compact --color none |
  /usr/bin/grep -vE --line-buffered '^Filter|^Timestamp' | # Remove the useless "header" row in output
  tee /dev/stdout | # Display matching events
  /usr/bin/sed -Eu 's/.*(PowerOnCamera|PowerOffCamera).*/\1/' | # Extract the relevant word from the event
  while read -r event; do # Store that word in the $event variable
    echo "Camera $event"

    system_profiler SPThunderboltDataType | grep -q "UID: 0x014AFA4DCC5947BE" # Check to see whether Thunderbolt Dock attached
    OFFICE_DOCK_ATTACHED=$?  # Capture the exit status of the previous command (0 if success)

    if [ "$OFFICE_DOCK_ATTACHED" -eq 0 ]; then
      if [ "$event" = "PowerOnCamera" ]; then
        echo "Lamp on"
        curl -X POST 'http://esphome-desk-bias-light.local/light/desk_camera_light/turn_on' &
      else
        echo "Lamp off"
        curl -X POST 'http://esphome-desk-bias-light.local/light/desk_camera_light/turn_off' &
      fi
    fi
  done

I haven’t tried this yet for Windows or Linux, but I’m sure there’s an approach which works similarly on each of those operating systems. Let me know if you have any ideas as to how it can be done!

The result

The light works just as I imagined: when any application starts using the camera, a request is fired off to enable the light. Here’s the behaviour switching between Google Meet and Photo Booth:

Further improvements

While I’m happy that the automation is possible, there are a few things that could be better about this setup:

• There’s some delay in turning the light on. The HTTP web server on ESPHome has fairly poor performance. I’m not sure whether this is the fault of ESPHome or just an ESP32-C3 hardware limitation. ESPHome’s native API—the one used by Home Assistant to talk to ESPHome devices—is a much more lightweight setup based on protocol buffers and encryption. In theory I could work out the exact command to send to the device over TCP, either by replaying the payload or building it up using the official definitions. I opted for HTTP as it’s easier and lets me use cURL to remove the need to install specific dependencies on a work laptop.
• The light could be brighter. This amount of lighting helps to fill out some shadows but it’s not sufficient to greatly reduce camera noise. There are many ways that this could be improved, including using more lights or more transparent conduit. The most tempting solution is to completely swap out this conduit-based light with an old screen which has been converted to diffuse a set of bright LEDs. I have a broken monitor lying around which might be perfect for this purpose. This may well be an upcoming project.

I’ve been recently reading about tools like htmx and Hotwire Turbo. These frameworks rely on sending HTML, not JSON, over the wire, with a focus on graceful degradation and reduced custom client-side JavaScript code. The simplicity is tantalising!

To learn more about this tech, I re-wrote my Obfuscator project using htmx. You can find the latest incarnation here, with source code on GitHub. The functionality of Obfuscator remains the same as ever: it translates some user-entered text through multiple languages in sequence, returning the result in the original language.

Then and now

The first version of Obfuscator was written in PHP and used the Bing Translate API, but I had no server to host it on an ongoing basis. The second version was a React/Redux setup which ran on AWS Lambda plus AWS Machine Translation. It had 174 runtime dependencies for what basically amounts to a simple form with five inputs and a button.

The latest verion is hosted on CloudFlare Pages, my preferred static host these days. There’s no framework or additional third-party dependencies to be seen here—just htmx at runtime, plus TypeScript and CloudFlare’s CLI for development (thankfully, the CLI wraps all the build shenanigans for backend code). I enjoyed writing a fairly stripped-back frontend in vanilla HTML. As for the backend: it’s amazing how painless shipping serverless backend code has become with the likes of Netlify and CloudFlare Pages Functions, compared to when I first wrote Obfuscator in 2016. I was even able use CloudFlare for translation using the m2m100 model which is surfaced as part of CloudFlare Workers AI. It’s certainly slower and less accurate than Amazon’s offering, but is much cheaper with a perpetual free tier. Besides, accurate translation was never really the point!

Diving into htmx

One of the cool things about the new Obfuscator is that there is no custom frontend JavaScript. The page is basically just a form with some data-hx- attributes added to indicate desired form submission behaviour to htmx. Here’s a compacted version of the page:

<!doctype html>
<html lang="en">
	<head>
		<title>Obfuscator</title>
		<!-- Other head fields (omitted) -->
		<meta name="htmx-config" content='{"responseHandling": [{"code":".*", "swap": true}]}' />
	</head>
	<body>
		<form
			action="/obfuscation"
			method="get"
			data-hx-get="/obfuscation"
			data-hx-indicator="#indicator"
			data-hx-disabled-elt="#form-fields"
			data-hx-select="#result"
			data-hx-target="#result"
			data-hx-swap="outerHTML"
			class="form"
		>
			<fieldset id="form-fields">
				<!-- Fields in here (omitted) -->
				<div class="block-submit">
					<button type="submit" class="button">
						<div id="indicator" aria-hidden="true">
							<svg viewBox="-10 -10 20 20" class="loading-spinner"></svg>
						</div>
						<span class="button-content">Obfuscate</span>
					</button>
				</div>
			</fieldset>
		</form>

		<div id="result"></div>
	</body>
</html>

The non-htmx form attributes are correct here: the form submits to the /obfuscation endpoint using GET, which will return a full HTML page which looks a bit like:

<!doctype html>
<html lang="en">
	<head>
		<title>Obfuscator</title>
		<!-- Other head fields (omitted) -->
	</head>
	<body>
		<div id="result">
			<div class="result-content">
				<span class="result-label">The result:</span>
				<details>
					<summary>The entrance</summary>
					<ol>
						<li><span class="result-lang">English</span> input</li>
						<li><span class="result-lang">Chinese</span> 入口</li>
						<li><span class="result-lang">Russian</span> Вход</li>
						<li><span class="result-lang">Hebrew</span> הכניסה</li>
						<li>
							<b><span class="result-lang">English</span> The entrance</b>
						</li>
					</ol>
				</details>
			</div>
		</div>
		<a href="/">Do another Obfuscation</a>
	</body>
</html>

That’s a good baseline, and means that the page gracefully degrades in the absence of JavaScript. When htmx is available, however, it does its magic of replacing a full browser form submission with an HTTP call which happens in the background, making the page feel smoother. Once the call is completed, it “swaps” the source page with the response.

This “swapping” can occur in many ways. It can replace the source page entirely, just a part of it, or even disparate elements across it! The behaviour is configured by the various attributes on the submitting element, in this case the form:

<form
	action="/obfuscation"
	method="get"
	data-hx-get="/obfuscation"
	data-hx-indicator="#indicator"
	data-hx-disabled-elt="#form-fields"
	data-hx-select="#result"
	data-hx-target="#result"
	data-hx-swap="outerHTML"
	class="form"
></form>

• data-hx-get=“/obfuscation” re-iterates the action and method attributes, as in this case htmx should still GET the /obfuscation endpoint. Unlike the browser, htmx will do this request in the background when the form is submitted.
• data-hx-indicator=“#indicator” tells htmx that the element with ID indicator is the loading indicator here; htmx will add the .htmx-request class to this element while the request is in flight. Styling possibilities ensue.
• data-hx-disabled-elt=“#form-fields”: htmx will mark this element as disabled while the request is in flight. Putting all the fields in a single fieldset is a clean way to disable them all using one parent element.
• data-hx-select=“#result” means htmx will only look at the #result element in the response when swapping out the target.
• data-hx-target=“#result” tells htmx that the #result element on this page should swapped with the selected element in the response.
• data-hx-swap=“outerHTML” ensures the entire #result element is replaced with that in the response.

In all these cases, the data- prefix is optional and I could use hx- directly, but I prefer data- for being able to quickly spot custom attributes in HTML.

I like this approach for its graceful fallback. It reminds me a bit of Turbo Frames, which encourages sending full pages which are surgically selected and swapped.

The last piece of the puzzle is the meta element at the top:

<meta name="htmx-config" content='{"responseHandling": [{"code":".*", "swap": true}]}' />

This configures htmx to perform its swapping even if the response is an error—I want errors to be visible to the user! If the input starts with “!!!error” it will trigger an “artificial” 400 error to demonstrate the behaviour.

What’s next

Obfuscator probably won’t need much maintenance for the next few years, especially now that there are so few dependencies to update. I’d love to internationalise it someday, or play around with a template engine that isn’t just strings. As for htmx: I’m impressed with its simplicity for basic applications like this and I’m keen to try it for something (a little) more complicated.

The trail is well-maintained and relatively straightforward for a West Coast track—but it’s still a West Coast track, with a couple of short rock-hopping sections but thankfully not much steep climbing. Experienced hikers or trail runners will take less than the DOC time of seven hours, but inexperienced hikers will find things more challenging; this can lead to a long day on the legs!

We were subjected to an absolute deluge while attempting to reach the hut, and side streams became uncrossable. Architect Creek Hut provided a great place to rest up for a couple of hours until the rain and waterways subsided. If you’re attempting this, make sure to allow extra time for such inclement weather, which is fairly common in the area. DOC is quite good at communicating with those who have booked the hut and on this weekend were offering booking refunds, even though the track stayed open.

Our day in to the hut saw us walking through lush and very damp bush. It was particularly impressive to look up to the steep rocky hills on either side of the valley, upon which waterfalls materialised in the heavy rain.

The hut is generously-sized, with four sleeping rooms upstairs plus the Sierra Room downstairs—a four-bunk room which can only be booked with sole occupancy. The downstairs fire didn’t seem particularly effective but with enough people, the space warmed up a little. The most impressive feature of the hut is the flush toilets which are available all year round.

I was particularly excited to check out the hot pools, which I had heard much about. The pools are just a short walk from the hut along a 50-metre track. A small shelter near the pools provides a place to change and hang dry clothes.

The pools are absolutely stunning, and I proceeded to spend about seven hours in them the following day. The hot water, dramatic view and cold air truly is a winning combination. In this single trip, Welcome Flat became one of my favourite places in the country and I’m already planning a return.

Isthmus Peak track is accessible from State Highway 6 just a few minutes’ drive north of Hāwea. There are multiple car parks opposite the trailhead. On the blissfully calm autumn day we visited, the track seemed only moderately busy. I imagine it must be heaving in the summer months, though presumably not to the level of Roys Peak.

Apart from a short bush-clad start, the route is almost entirely 4WD track through private farmland. We visited during the roar season and got to see a couple of stags fairly close to the track. They’re much tamer than deer I’ve experienced in the backcountry as they’re farmed for tourists who hunt them on site. This is a good reason to stay on the track!

After many switchbacks, the track straightens and levels somewhat for a beautiful tops approach to the summit.

The view from the top is truly panoramic, with great sight-lines over both Lake Hāwea and Lake Wānaka. Pride of place to the north is Mt Albert which sits above Albert Burn. We got a glimpse of the snowy peaks in Mt Aspiring National Park, too.

After having a lovely picnic in the shade of the summit shelter, we decided to go back the way we came. There is the option of forming a loop with South Ridge, but this route looked more exposed and rocky.

I was recently inspired by some impressive 3D-printed geometric vase designs and considered how I could replicate a similar effect for some decorative pot covers. It struck me that many of these designs could be described by a mathematical function. I love the simplicity of “printing” a 3D function, but I was unsure as to how I could achieve this in practice.

There are a few solutions I have seen proposed for tackling this kind of thing:

1. Use an off-the-shelf graphical CAD program supporting functions. There are tools that support defining surfaces as functions. SideFX Houdini and Full Control Designer are mentioned. The disadvantage here is that I would need to learn these specific workflows and be at the mercy of their limitations.
2. Work out how to do it in OpenSCAD. The polyhedron feature, combined with a lot of iteration, could be used. While I love OpenSCAD for basic parts, I suspect this approach would be very slow and cumbersome to output a complicated shape with the resolution I desire.
3. Code it from scratch. I chose this option as it would give full flexibility, maximum speed and let me learn a bit more about STL files along the way.

How to make a pot with code

Enter my solution: vasestl. I opted to use Rust for its blazing-fast performance, and because I have recently been learning the basics of Rust through the Advent of Code challenges. On reflection, I doubt the performance advantages were necessary because the program can build tens of thousands of triangles in a matter of milliseconds.

Compute points

The first step is to come up with an equation describing the shape. I decided to use polar co-ordinates for this process, which makes it easier to define periodic shapes that will wrap around the pot seamlessly. At each increment of height ($z$) and angle ($\theta$), the distance from the origin ($r$) can be defined—which yields a point in 3D space. One of the simplest options is a cylinder, which would have a mathematical definition like this:

With four iterations of z and 14 iterations of θ, the corresponding point cloud for this equation would look something like:

Adding a cool swept sine pattern to this shape is as simple as incorporating the angle and offsetting it with changing z-height:

In the Rust code, we could consider the base cylindrical shape and the decorative “ripple” separately, representing it as something like:

fn r(t: f32, z: f32) -> f32 {
    const CYLINDER_RADIUS: f32 = 30.0;

    // "Base": Just the cylinder, in this case
    let base = CYLINDER_RADIUS;

    // "Decoration": A sine wave to add a rippled texture
    let decoration = 1.0 + f32::sin(12.0*t + z);

    // Return the sum of these components
    base + decoration
}

Where this can get a bit painful is for non-circular shapes, such as squares and rectangles. These need to use a polar representation of the shape. For example, a square pot shape of side length $s$ would be defined as:

Awkward, right? In code it’s at least a tad clearer to build these shapes up piece by piece. For example, here is the definition of a pot which sweeps from a circular bottom to a square top. Once this “base shape” is defined, it is easy to add any decorative elements just like before:

fn r(t: f32, z: f32) -> f32 {
    // Pot is circle ø88 at bottom; square 104×104 at top

    const SQUARE_SIZE: f32 = 52.0; // Half the width (minimum "radius") of square
    let square = if (PI * 0.25..PI * 0.75).contains(&t) || (PI * 1.25..PI * 1.75).contains(&t) {
        f32::abs(SQUARE_SIZE / f32::sin(t))
    } else {
        f32::abs(SQUARE_SIZE / f32::cos(t))
    };

    const CIRCLE: f32 = 44.0; // Radius of circle

    // Ease sinusoidally between circle base and square top
    let z_ease = -(f32::cos(PI * z) - 1.0) / 2.0;
    let base = CIRCLE * (1.0 - z_ease) + square * z_ease;

    // "Decoration": A sine wave to add a rippled texture
    let decoration = f32::sin((t + 0.5 * z) * 80.0) * 1.0;

    // Combine and return
    base + decoration
}

This yields the following shape:

It is possible to also build rectangles using a similar strategy. See the examples in the repository.

Convert the points to triangles

After computing the points at every increment of $z$ and $\theta$, the code has a point cloud where every point lies on the surface of the pot shape. This needs to be converted to a series of triangles which will define the surface which can later be printed.

Because the points were computed at discrete increments of $z$, the array of points is effectively a series of vertically-stacked “rings” of points, where each “ring” has the same number of points (determined by the increment of $\theta$). Building triangles is a matter of iterating through the rings and connecting adjacent points to form squares, which are then stored as two triangles. Additionally, the normal vector of each triangle is determined, which can be computed using a cross product. This normal is later embedded in the STL file to indicate which side is the “outside” of the surface.

Build an STL file

The STL file itself was surprisingly simple to build. I opted to use the binary variant of the file for its more efficient structure. After a mostly-empty header, the data for each triangle is simply stuffed into the file bytes. Each triangle is defined by 12 floating point numbers (x, y, z coordinates of each corner, plus a 3D normal vector) and Rust’s to_le_bytes() function made the data dumping trivial, if verbose.

Printing the pots

In part, this whole project started an excuse to play around with vase mode. Vase mode is offered by many 3D slicers and enabled the printing of a single-walled part with a continuous spiralling layer, as opposed to the more typical discrete layers used in conventional prints. This process leads to a fairly flimsy part, but saves on filament and is perfect for aesthetic pot and vase covers.

In Cura, enabling this mode is as simple as checking the “Spiralize outer contours” box. I’ve had great success using this feature with PLA and PETG.

Access is easy—park at Deer Valley campsite then go straight up Foley’s Track to the bushline. After this, it’s a matter of following the ridge to the summit. There are a couple of particularly steep sections, but cairns and the occasional pole help to navigate. We made it to the summit in just over two hours. If you’re doing this trip, be wary of the steep dropoffs to the south of the final ridge climb.

The peak itself has a few nice nooks to sit down and enjoy some snacks while enjoying the stunning view. It’s amazing how open the tops are around here. Traversing the Libretto Range—or at least spending a night camping up here—could be a fun challenge.

If you’re tackling this trip, make sure to bring plenty of water and keep an ear out for kea.

The two tracks meet just shy of the peak. This final section is a rocky ridge with stunning views all around.

As we left the ridge and commenced our descent, I was distracted by a roving gang of young kea that landed nearby. Now I think of it, maybe they were the ones being distracted by me. It’s always hard to tell with these intelligent creatures.

Day one

With our sights set on Butler Junction Hut, we set off from the carpark adjacent to the Whataroa River. The track proved easy-going until the main swingbridge at the Perth River Track junction. Beyond that, the track turns to follow the Whataroa River valley. There are a couple of large undulations, then things become increasingly slow-going and technical as the hut approaches. Thankfully, overgrown vegetation had been recently cleared on some of the track sections near the hut. Still, it took us over nine hours to reach the hut—longer than DOC’s eight-hour estimate.

Butler Junction Hut is perched above a bend in the Whataroa River and comes into sight with a decent amount of walking remaining. With the hut intentions book only going back to about March 2023, it was hard to get a good idea of visitation rates. It looked like there had been about one visit per month since the start of the book, but I imagine that the rate would likely increase in summer.

Day two

The mission was simple: get to Ice Lake. In line with DOC times, the Butler River Track took us about four hours to the lake, with Top Butler Hut at the halfway mark. This hut is in a dramatic clearing, surrounded by imposing mountain peaks. There is a notable river crossing of the Butler River South Branch near Top Butler Hut. The area hadn’t received any rain for about a week, so this crossing was easily manageable.

Like the later parts of day one, the Butler River track was undulating and slow-going, with a fair amount of windfall. There were some fascinating rock faces and cave-like areas along the way, as well as a spicy section of ongaonga overgrowth.

The track emerges from the bush for the last 30 minutes of the approach to the lake. Eventually, we crested a ridge to see Ice Lake in all its glory.

The colour of the water is striking, with a light cloudy colour imparted by the rock dust delivered by the glaciers beyond the head of the lake. We sat near the lake for over an hour, soaking in the view and making some drinks with the nearby ice (it’s called Ice Lake so the ice must be worth a try, right?).

Another four hours of walking brought us back to Butler Junction Hut, where I was delighted to take off my boots for the day. Thankfully we had several hours of daylight left—plenty of time for a dip in the river and a sit in the sun.

Day three

Up bright and early, day three was simply a matter of retracing our steps to get back to the cars and then to Christchurch.

The track to the hut is easy-going and could be safely walked at night by flashlight. Note that the hut is popular and had scarce water in the tank when we visited. Pack in plenty of water, and either arrive early to claim a bunk or bring a tent as backup.

I have been using the impressive Hyprland compositor on my Linux laptop for the last month or so, and it occurred to me recently that I had all the ingredients for a better setup: a setup where the laptop’s brightness keys would just change the brightness of whichever monitor I was using in that moment.

The ingredients

To change the brightness of the internal display, I use brillo because it supports smoothed brightness adjustments. This is the command I previously had bound to the brightness up key in Hyprland’s config:

# Increase the brightness 8% over 150ms
brillo -u 150000 -A 8

To increase the brightness of the external display, ddcutil is perfect as it uses DDC/CI to change the display’s paramters over I²C (magic!):

# Increase the brightness of the first external monitor by 8%
ddcutil --display=1 setvcp 10 + 8

What I really wanted to do was run the relevant command for the currently focused monitor (ie, the one where the cursor is). Here’s where Hyprland comes in:

# Get a list of monitors including focused state and ID, in JSON format
hyprctl monitors -j

The (truncated) output of this command is:

[
	{
		"id": 0,
		"name": "eDP-1", // The internal monitor
		"focused": false,
		...
	},
	{
		"id": 1,
		"name": "DP-2", // My one external monitor
		"focused": true,
        ...
	}
]

Using a bit of jq, I plucked out the ID and name of the focused monitor:

focused_name=$(hyprctl monitors -j | jq -r '.[] | select(.focused == true) | .name')
focused_id=$(hyprctl monitors -j | jq -r '.[] | select(.focused == true) | .id')

Putting it together

Now it’s just a matter of getting the current monitor and running the correct command based on its name:

#!/usr/bin/env sh

# Accept an arg '+' or '-'
direction=$1

# Get monitor info
monitor_data=$(hyprctl monitors -j)
focused_name=$(echo $monitor_data | jq -r '.[] | select(.focused == true) | .name')

if [ "$focused_name" == "eDP-1" ]; then
    # Internal display is focused -> use brillo
    if [ "$direction" == "-" ]; then
        brillo -u 150000 -U 8
    else
        brillo -u 150000 -A 8
    fi
else
    # External display is focused -> use ddcutil
    # But *which* external display?
    focused_id=$(echo $monitor_data | jq -r '.[] | select(.focused == true) | .id')
    ddcutil --display=$focused_id setvcp 10 $direction 8
fi

I named this script hypr_brightness.sh and mapped it to my brightness keys in my hyprland.conf a bit like the following:

binde = , XF86MonBrightnessDown, exec, /path/to/hypr_brightness.sh -
binde = , XF86MonBrightnessUp, exec, /path/to/hypr_brightness.sh +

Since implementing this change on my personal device, I’ve found myself confused on other laptops when the external monitor brightness does not change when I use the brightness keys. That’s just how intuitive it feels.

You can find the script at albertnis/hypr-brightness on GitHub. The version there is a bit more refined, with argument validation and some optimisations.

The track to Buckland Peaks Hut starts at the carpark situated at the end of Bucklands Peak Road*. The first 5.5 kilometres of hiking follows the private section of the road, crossing farmland to reach the base of the hills. Note that this section is accessible on foot only, as the landowner has disallowed bicycle and vehicle travel. The track is well-marked through intersecting farm tracks, and with some tactical jogging along the way I reached the hills in a bit under an hour. At this point the track crosses a fenceline then follows a spur steadily up to the bushline.

Just above the bushline is a short stretch of tops travel to a saddle, at which point the poled route drops down to the hut which is about a 10 minute walk down rocky terrain. Reaching the hut from the carpark is noted by DOC as taking five hours, but I found it doable in just over 3.5 hours. Your mileage may vary, but getting up to the hut and back down would be quite possible as a daytrip.

After refilling water bottles at the hut, I ascended back up to the saddle with a cooker to eat soup while watching the sunset. The view out to the coast was glorious on this pristine winter’s evening. With barely a breeze in the air, I enjoyed the peaceful moment.

The hut is weathertight and double-glazed, and it retained its heat well into the evening. That said, it lacks a fireplace and modern insulation, leading to a chilly morning after being woken by a mouse in the ceiling.

Judging by the intentions book, it is common to walk up to Buckland Peaks themselves on the second day, before descending in the afternoon. I walked along the ridge to get a better view of the peaks but decided not to venture any further, to allow myself plenty of time to drive back to Christchurch. It’s epic country up here, and somewhere I’d love to come back to one day for more exploration.

​* The naming I experienced along this track was weirdly inconsistent. The hut is called “Buckland Peaks Hut”. The peaks are called “Buckland Peaks”. But, somehow, the road is called “Bucklands Peak Road” and there are plenty of signs talking about “Buckland Peak Hut” and other permutations. It seems like nobody could remember the correct name—or that there are definitely multiple peaks.

Spoiler alert: here’s the end result! By simply adding a GPX file to the routes key in the frontmatter of a Markdown blog post, I can see an interactive map and elevation profile. Here is the example for the Top Hope Hut adventure.

title: Top Hope Hut
date: 2022-09-24T19:42:00+1200
description: Meandering river valley track up Hope Valley near Lewis Pass
tags: [adventures]
routes:
  - ./Top_Hope_Hope_Kiwi_.gpx

Let’s jump in and see how this is achieved, or check out the code in GitHub.

Desired features

The first step for tackling this problem was to identify the features I wanted from this functionality. Here are some features I want to replicate from Strava:

• View the route on an interactive map
• View an elevation profile
• See metadata:
  • Elapsed time
  • Total distance
  • Gross elevation gain

And here are some features I desire beyond Strava’s functionality:

• Ability for anybody to download original GPX file
• View overnight stops along the route (most of my blogged adventures are multi-day hikes and the stops usually correspond to notable huts or campsites)

Setting up the Vite plugin

Before tackling most of these features, a Vite plugin needs to be set up.

Create a minimal plugin

A Vite plugin is simple in that it’s a function that is directly referenced by config. That is great for type-safety and ease of getting started. To create one, I simply created the following stub in a src/plugins/vite-plugin-gpx/index.ts file:

import type { Plugin } from 'vite'

export function gpxPlugin(): Plugin {
	return {
		name: 'vite-plugin-gpx'
	}
}

Loading this stub plugin is as easy as adding a reference into vite.config.ts or equivalent:

import type { UserConfig } from 'vite'
import { gpxPlugin } from './src/plugins/vite-plugin-gpx'

const config: UserConfig = {
	// ...
	plugins: [
		// ...
		gpxPlugin()
	]
}

export default config

I find it awesome that Vite supports type-safety all the way from the config file. If gpxPlugin didn’t properly implement Plugin, we would see errors in the IDE well before we built any output. This code “works” great but doesn’t achieve anything!

Enabling importing of GPX files

Let’s go over what this Vite plugin will let us achieve. Specifically, we want to be able to import the GPX file directly. The return value of the import will be an object with the data we’re interested in.

import routeData from './my-route.gpx'

console.log(routeData)

// Object { fileName: "my-route.gpx", ... }

Initially, we want the output object to just have a fileName property, just to prove that we can get some output from the plugin. We’ll make the output more useful in subsequent steps.

Let’s add a load function to our gpxPlugin.

import { basename } from 'node:path'
import type { Plugin } from 'vite'

export interface ViteGpxPluginOutput {
	fileName: string
}

export function gpxPlugin(): Plugin {
	return {
		name: 'vite-plugin-gpx',
		load(id) {
			if (!/\.gpx$/.test(id)) return null

			const srcURL = new URL(id, 'file://')
			const pluginOutput: ViteGpxPluginOutput = {
				fileName: basename(srcURL.pathname)
			}

			return {
				code: `export default ${JSON.stringify(pluginOutput)}`
			}
		}
	}
}

The load function is called whenever import is invoked, with the absolute path of the imported file present in the id parameter. In this example, we do a few things:

1. If the file does not end with .gpx, ignore it!
2. Get the name of the imported file
3. Wrap the name with an object matching the output interface defined as ViteGpxPluginOutput
4. Return a plugin output object. The code key is set to a string containing JavaScript code which exports the output object, in string form.

The last part is particularly wild to me. We are effectively transforming our GPX file into a JavaScript file! Importing a GPX file will “trick” the compiler into thinking it has imported whatever JavaScript file contents this function returns.

Enabling type-safe imports

Running the import example above should work well now. But there’s a problem:

The imports fail with the error Cannot find module ‘./my-route.gpx’ or its corresponding type declarations. even though the file does exist. What’s possibly even worse is that the return type of the import is any, as demonstrated by asyncRouteData having an any type.

This can be easily fixed with a TypeScript module declaration. In my case, I put it in src/plugins/vite-plugin-gpx/ambient.d.ts. The main thing is to ensure the file matches an include rule from tsconfig.json.

declare module '*.gpx' {
	const output: import('./index').ViteGpxPluginOutput
	export default output
}

With this file in place, import errors are gone and we see beautiful, beautiful autocomplete on the imported variable! Even import paths will autocomplete .gpx files now.

Now the plugin is fully set up and type-safe. By adding to our pluginOutput, we can return more data about the file. Time to incorporate the first feature.

Enabling downloading of the original file

The first and most trivial feature for the plugin is to enable the download of the original GPX file. Frustratingly, this feature is withheld by Strava; I personally love the idea of letting people download GPX files as it can help with navigation for their own travels.

The main aim here is to instruct Vite to either render or serve the file then obtain the hosted path for the file. I was inspired by Jonas Kruckenberg’s work on the imagetools library and implemented a simplified version of imagetools’ Vite plugin definition. I’ll place the code below and then walk through it.

import { basename } from 'node:path'
import { readFile } from 'node:fs/promises'
import type { Plugin } from 'vite'

export interface ViteGpxPluginOutput {
	filePath: string
}

export function gpxPlugin(): Plugin {
	let basePath: string
	const gpxPaths = new Map()
	return {
		name: 'vite-plugin-gpx',
		configResolved(cfg) {
			const viteConfig = cfg
			basePath = (viteConfig.base?.replace(/\/$/, '') || '') + '/@gpx/'
		},
		async load(id) {
			if (!/\.gpx$/.test(id)) return null

			const srcURL = new URL(id, 'file://')
			const fileContents = await readFile(decodeURIComponent(srcURL.pathname))

			gpxPaths.set(basename(srcURL.pathname), id)

			let src: string
			if (!this.meta.watchMode) {
				const handle = this.emitFile({
					name: basename(srcURL.pathname),
					source: fileContents,
					type: 'asset'
				})

				src = `__VITE_ASSET__${handle}__`
			} else {
				src = basePath + basename(srcURL.pathname)
			}

			const pluginOutput: ViteGpxPluginOutput = {
				filePath: src
			}

			return {
				code: `export default ${JSON.stringify(pluginOutput)}`
			}
		},
		configureServer(server) {
			server.middlewares.use(async (req, res, next) => {
				if (req.url?.startsWith(basePath)) {
					const [, id] = req.url.split(basePath)

					const gpxPath = gpxPaths.get(id)

					if (!gpxPath)
						throw new Error(
							`gpx cannot find GPX file with id "${id}" this is likely an internal error. Files are ${JSON.stringify(
								gpxPaths
							)}`
						)

					res.setHeader('Content-Type', 'application/gpx+xml')
					res.setHeader('Cache-Control', 'max-age=360000')
					const buffer = await readFile(gpxPath)
					const contents = buffer.toString()
					return res.end(contents)
				}

				next()
			})
		}
	}
}

There’s a lot going on here. But at its core you should still able to see the return statement within the load function. Here’s what is happening:

1. When Vite is started and “sees” the plugin, configResolved is called. The cfg argument contains a bunch of configuration and context about Vite. In this case, we use it to get base path information and store it for later use.
2. Whenever we import a file, the load function is called, just like before. It has a bit more functionality now, though. First, it reads the file using Node’s readFile. It then needs to ensure this GPX data is hosted somewhere before returning the path at which it is hosted. This is achieved quite differently depending on whether Vite is running with a dev server or as a build command:
   • In build mode, the file contents are registered with Vite using the this.emitFile function. The output path looks like __VITE_ASSET_{uniqueid}__. Behind the scenes, Vite will replace this magic path with the path at which it has stored the emitted file. (I can’t find much documentation on this process—reach out if you know more).
   • In dev server mode, a path is generated for each loaded gpx file with the form /@gpx/{filename}. The plugin also registers each loaded gpx file in the gpxPaths map. To ensure the files actually exist in this location, the configureServer function is used to register a sever middleware which intercepts requests to /@gpx/* paths. If the request is for a loaded GPX file, the file contents are read and returned.

Now, when a GPX file is imported, there is easy access to the file path—perfect for a download link or button.

Extracting data from the GPX file

It’s time to delve into the file contents and extract some data. It might be time to refactor the output into its own function:

export function gpxPlugin(): Plugin {
	// ...
	return {
		// ...
		async load(id) {
			// ...
			const pluginOutput = gpxDataToOutput(fileContents.toString(), src)
			// ...
		}
	}
	// ...
}

const gpxDataToOutput = (gpxData: string, path: string): ViteGpxPluginOutput => {
	return {
		filePath: path
	}
}

The rest of the code in this section will be within the scope of this gpxDataToOutput function, unless specified otherwise.

Understanding GPX

A GPX file is an XML file with a schema. The most useful thing to do with it in this plugin is to parse it into an GeoJSON format. This makes it easier to work with in JavaScript. Besides, most mapping libraries will accept GeoJSON as input. The @xmldom/xmldom and @mapbox/togeojson libraries work well here.

import geojson from '@mapbox/togeojson'
import { DOMParser } from '@xmldom/xmldom'

// ...

const gpxXmlDocument = new DOMParser().parseFromString(gpxData)
const gj = geojson.gpx(gpxXmlDocument)

It’s worth noting that the @mapbox/togeojson library, while reliable, is quite old and has no type definitions. Thankfully, the GeoJSON output from the library is externally standardised so we can simply use a separate library for typing and apply a module declaration. I achieved this by creating a src/@mapbox/ambient.d.ts file with the following declaration which uses the geojson library to provide typing:

declare module '@mapbox/togeojson' {
	import type { GeoJSON } from 'geojson'

	interface MapboxToGeoJson {
		gpx: (data: Document) => GeoJSON
	}

	const mapboxToGeoJson: MapboxToGeoJson

	export default mapboxToGeoJson
}

Once we have converted to a GeoJSON value stored in gj, we can evaluate the contents:

{
	"type": "FeatureCollection",
	"features": [
		{
			"type": "Feature",
			"properties": {},
			"geometry": {
				"type": "LineString",
				"coordinates": [
					[171.96891, -41.855557, 653],
					[171.968706, -41.855599, 653],
					[171.968748, -41.855728, 653],
					[171.968787, -41.855936, 654], // ...
				]
			}
		}
	]
}

That’s a GeoJSON! Each co-ordinate is displayed as [latitude, longitude, elevation].

Data compression

It might be tempting to surface our path and elevation data at this point by simply returning:

return {
	filePath: path,
	geoJson: gj
}

Before doing that, consider file size! Everything that we return here will have to travel across the wire at some point, and GPX files can be huge—often tens of megabytes for long trips. That includes position, heart rate and timestamp data taken every second or two. Without intervention, most of this data will end up in our GeoJSON file as metadata. Compressed server responses will help to some extent, but we have a prime candidate for optimisation here. Let’s see how this can be achieved for path and elevation data.

Path data

First, the plugin calculates the target number of datapoints for the route. The input count is raised to the power of 0.7 (a number I settled on after much experimentation) to obtain the desired number of coordinates in the output data. This means that longer trips are compressed more aggressively. A short route with 1,000 points will be downsampled to about 125 points, while a 30,000-point adventure will result in around 1,361 points in the output.

const coordinatesDataCount = feature.geometry.coordinates.length

const targetPathDataCount = Math.pow(coordinatesDataCount, 0.7)
const pathSamplingPeriod = Math.floor(coordinatesDataCount / targetPathDataCount)
const downSampledCoordinates = downSampleArray(feature.geometry.coordinates, pathSamplingPeriod)

Downsampling occurs by only taking values spaced pathSamplingPeriod apart:

const downSampleArray = <T>(input: T[], period: number): T[] => {
	if (period < 1 || period % 1 != 0) {
		throw new TypeError('Period must be an integer greater than or equal to 1')
	}

	if (period === 1) {
		// Return a copy of input
		return [...input]
	}

	const output: T[] = []

	for (let i = 0; i < input.length; i += period) {
		output.push(input[i])
	}

	return output
}

After downsampling the coordinates, we can exclude elevation data from the GeoJSON as this will be handled separately. Removing this data will decrease the filesize further by a third.

import type { Geometry } from 'geojson'

// ...

const downSampledGeometry: Geometry = {
	...feature.geometry,
	coordinates: downSampledCoordinates.map((c) => [c[0], c[1]])
}

Elevation data

I used a similar method for elevation data, but with a different approach to obtaining a count target. Here I want the target to be at least 900, as this is approximately the number of pixels my elevation graph takes up at its maximum width.

const targetElevationDataCount = 900
const elevationSamplingPeriod =
	coordinatesDataCount < targetElevationDataCount
		? 1 // Use all the datapoints if there are fewer
		: Math.floor(coordinatesDataCount / targetElevationDataCount) // Downsample
const downSampledGeometryForElevation = downSampleArray(
	feature.geometry.coordinates,
	elevationSamplingPeriod
)

The final step is to extract just a list of elevations from the GeoJSON:

const downSampledElevations = downSampledGeometryForElevation.map((g) => g[2])

We could downsample elevation at the same rate as position data but generally speaking there are fewer elevation points required, so this saves space overall.

Bringing it together

With some changes to the ouput interface, we can set the ouput of gpxDataToOutput function to look a bit like the following:

return {
	elevationData: {
		downSampledElevations,
		samplingPeriod: elevationSamplingPeriod
	},
	pathData: {
		geoJson: buildGeoJSONFromGeometry(downSampledGeometry),
		samplingPeriod: pathSamplingPeriod
	},
	metadata: {
		gpxFilePath: path
	}
}

buildGeoJSONFromGeometry simply reconstructs a minimal GeoJSON to wrap geometry. In this case it’s used to strip out any of the extra bloat that may have been in the original GeoJSON.

It’s coming together! This is enough information to show path data and basic elevation data.

Computing metadata

Distance

There are two purposes for computing the distance of the path:

1. Display the total distance of the route
2. Display an accurate elevation graph

The first item is probably obvious, but the interaction between distance and elevation graph is particularly interesting. You see, an elevation graph typically has distance as the x-axis, not data point index. Currently we only have a list of elevations and no distance data for correlation.

To provide this distance data, a list of cumulative distances is required. This enables us to map an index to a distance.

/**
 * Compute cumulative distances along a path
 * @param input List of coordinate arrays representing a path
 * @returns Array of distances. Each distance is the distance along the path from the start of the path to that point.
 */
const computeCumulutiveDistanceMetres = (input: Position[]): number[] => {
	let distanceMetres = 0
	const distancesMetres = [0]

	for (let i = 1; i < input.length; i++) {
		distanceMetres += haversineDistanceMetres(
			input[i] as [number, number],
			input[i - 1] as [number, number]
		)
		distancesMetres.push(distanceMetres)
	}

	return distancesMetres
}

The distance between each successive pair of co-ordinates is computed using the Haversine formula then added to the previous distance and appended to the array.

The total distance is simple the last item in the cumulative distance array.

Elevation gain

Gross elevation gain is a useful metric for assessing how hilly a route was. Conceptually it’s the total cumulative upwards travel. In practice, it can be a bit nuanced to compute given that elevation data can be imprecise and noisy at times. I implemented a fairly simple version which includes some downsampling and a threshold for elevation gain. I don’t think it’s perfect, but it gives a fairly good estimate.

const computeCumulativeElevationGainMetres = (input: Position[]): number => {
	let vertGain = 0
	const gainThreshold = 2
	let currentBaseline = input[0][2]

	for (let i = 0; i < input.length; i += 3) {
		const alt = input[i][2]
		const climb = alt - currentBaseline

		if (climb >= gainThreshold) {
			// We have gone up an appreciable amount
			currentBaseline = alt
			vertGain += climb
		} else if (climb <= -gainThreshold) {
			// We have gone down an appreciable amount
			currentBaseline = alt
		}
	}

	return vertGain
}

Time

There are two key times to capture from an activity: duration and start time.

Start time is easy. Most of the time the GeoJSON feature will contain a “time” which can be used. Just in case, I fall back to the first coordinate time.

Times in GeoJSON are represented in ISO format, so liberal use of parseISO from date-fns is appropriate here.

import { parseISO } from 'date-fns'

/**
 * Calculate the start time from a feature
 * @param input Feature to use for calculating start time
 * @returns Date object representing the feature's `time` property, or the first `coordTime` if there is no such property
 */
const computeStartTime = (input: Feature): Date | null => {
	const startTime = input.properties?.time

	if (typeof startTime === 'string') {
		return parseISO(startTime)
	}

	const times = input.properties?.coordTimes

	if (Array.isArray(times)) {
		return parseISO(times[0])
	}

	return null
}

Elapsed duration can be calculated by taking the difference between the first and last coordinate time. The date-fns library has some very helpful methods and types to assist with this. The timestamps appear in the properties key of the GeoJSON. I’m unsure whether this is standard or arbitrary behaviour in @mapbox/togeojson.

import { intervalToDuration, parseISO } from 'date-fns'

/**
 * Calculate the duration from the `coordTime`s stored in a feature's property
 * @param input Feature to use for calculating duration
 * @returns Duration between first and last `coordTime` in the feature
 */
const computeDuration = (input: Feature): Duration | null => {
	const times = input.properties?.coordTimes
	if (times == null || !Array.isArray(times)) {
		return null
	}

	const start = parseISO(times[0])
	const end = parseISO(times[times.length - 1])

	return intervalToDuration({ start, end })
}

Breaks

Many of the trips I track are multi-day hikes (or “tramps” as they’re lovingly known by New Zealanders). It is nice to know where the huts and campsites were along the route. Manually entering the locations of these spots is probably the most foolproof way of doing this, but I opted for an alternative, lazy, method. I simply extract the locations where there are no GPX entries for over four hours. This tends to be a good indicator of where I paused my tracking device overnight—and a great proxy for hut and campsite locations.

import { differenceInHours, parseISO } from 'date-fns'

/**
 * Compute the locations of breaks exceeding four hours within a feature
 * @param input Feature with coordTimes data used to evaluate breaks
 * @returns Array of indexes where each index corresponds to a coordinate after which there were no coordinates recorded for at least four hours
 */
const computeBreakIndices = (input: Feature): number[] => {
	// Self-contained sampling rate to accelerate calcs
	// Can ususally set this high unless there are lots of stops
	const samplingRate = 30

	const times = input.properties?.coordTimes

	if (!Array.isArray(times)) {
		return []
	}

	// Get the date objects for a subsampled set of timestamps
	const parsedTimes = times.filter((_, i) => i % samplingRate === 0).map((t) => parseISO(t))

	const indices = []
	for (let i = 1; i < parsedTimes.length; i++) {
		if (differenceInHours(parsedTimes[i], parsedTimes[i - 1]) > 4) {
			indices.push((i - 1) * samplingRate)
		}
	}
	return indices
}

Displaying data

Now that all the data and metadata has been computed, it’s time to put this to use! Let’s import a gpx file like before and see what we’re dealing with:

import geoData from '../top-hope-hut/Top_Hope_Hope_Kiwi_.gpx'

Inspecting or logging geoData shows that has the following value:

{
	"elevationData": {
		"downSampledElevations": [
			602,
			602,
			602,
			603 // ... [907 items total]
		],
		"elevationGainMetres": 1073,
		"samplingPeriod": 54
	},
	"metadata": {
		"gpxFilePath": "/@gpx/Top_Hope_Hope_Kiwi_.gpx",
		"breakIndices": [23940, 34560],
		"duration": {
			"years": 0,
			"months": 0,
			"days": 2,
			"hours": 3,
			"minutes": 21,
			"seconds": 45
		},
		"startTime": "2022-09-23T21:34:06.000Z",
		"distanceMetres": 57605.837060918944
	},
	"pathData": {
		"geoJson": {
			"type": "FeatureCollection",
			"features": [
				{
					"type": "Feature",
					"properties": {},
					"geometry": {
						"type": "LineString",
						"coordinates": [
							[172.382772, -42.584167],
							[172.38245, -42.58434],
							[172.382203, -42.584338],
							[172.381928, -42.584328],
							[172.381573, -42.584365] // ... [1958 items total]
						]
					}
				}
			]
		},
		"cumulativeDistancesMetres": [
			0,
			36.508624548783956,
			61.228052816957145,
			88.76666712805523,
			124.48251406266579 // ... [1958 items total]
		],
		"samplingPeriod": 25
	}
}

We can see all the geo information is stored there as we expect! Elevation data has a sparser sampling rate than point data and both sampling rates are present. Plus, the object gives us a gpxFilePath which can be used to download the original file. Here the file starts with /@gpx/ because the example output is from a local server.

Displaying the file is now a matter of writing some code that takes this data and renders something with it. It could be plain old JavaScript, a React component, or just about anything. For my Sveltekit blog, I created a <MapGroup> Svelte component that renders an elevation graph using SVG and a map using MapBox. The details are a bit beyond this article, but suffice it to say that I can use the component with the following code snippet:

<script>
	import MapGroup from '$lib/components/MapGroup.svelte'
	import geoData from '../top-hope-hut/Top_Hope_Hope_Kiwi_.gpx'
</script>

<figure>
	<MapGroup geo={geoData} />
	<figcaption>Example of GPX-derived data being loaded via map component</figcaption>
</figure>

For my blog, I check the blog post for the routes key, then import every GPX file reference and insert a component similar to the above. See how it works on GitHub.

Graceful degradation in the absence of JavaScript

An early version of my site’s GPX integration used lazy loading for the entire map group, consisting of elevation graph, metadata display and map. But it’s always worth considering how much can be rendered without JavaScript present in the browser at all. In my case, I could easily pre-render or server-side-render the SVG elevation graph, as well as the metadata block.

<script lang="ts">
	import type { ViteGpxPluginOutput } from '../../plugins/vite-plugin-gpx/types'
	import type { ComponentType, SvelteComponentTyped } from 'svelte'
	import { onMount } from 'svelte'
	import ElevationGraph from '$lib/components/ElevationGraph.svelte'
	import MapMetadata from '$lib/components/MapMetadata.svelte'
	import MapLoading from '$lib/components/MapLoading.svelte'

	export let geo: ViteGpxPluginOutput
	let hoveredIndex: number | undefined = undefined
	let mapComponent: ComponentType<SvelteComponentTyped> | undefined

	onMount(async () => {
		mapComponent = (await import('$lib/components/Map.svelte')).default
	})
</script>

<svelte:head>
	<noscript>
		<style>
			.maploading {
				display: none;
			}
		</style>
	</noscript>
</svelte:head>

<ElevationGraph {...elevationGraphPropsGoHere} />

<MapMetadata {...metadataPropsGoHere} />

{#if mapComponent == null}
	<div class="maploading">
		<MapLoading />
	</div>
{:else}
	<svelte:component this={mapComponent} {...otherMapPropsGoHere} />
{/if}

Here, the map component itself is lazy-loaded (and will be hidden if there is no JavaScript on the client) but elevation graph and metadata will be present regardless of whether the code is running on server or client. The next level would be to fall back to Mapbox’s image API to render an image instead of an interactive map in the absence of JavaScript!

Conclusion

In building a Vite GPX plugin, I set out to move a small slice of Strava to my personal site. A Vite plugin achieves this in a fairly elegant way, abstracting the messy work away from application code which simply imports it. With creative downsampling it is possible to minimise file sizes while retaining the full GPX file for download.

The next step might be to make some improvements to the performance of the loader, given that it can take 2-3 minutes to build the entire site. There are also plenty of options for enabling configuration which I will explore if I ever publish vite-plugin-gpx as a standalone library.

I hope you have enjoyed this walkthrough of building a Vite GPX plugin! You can find the full source code for the plugin on GitHub.

The LD2410 uses a serial bus for configuration and for detailed output from the sensor. The configuration in particular is quite powerful here: with the right tweaking of sensitivity values, the sensor can be set to only detect motion within certain distance ranges. For example, you could configure it to trigger when something is between 3 and 4.5 metres of the sensor.

So how can this configuration be achieved? HiLink provides a first-party tool but it only runs on Windows and is closed source. That’s why I have recently developed and released an alternative!

LD2410 Configurator is my first foray into the WebSerial API. It was very interesting to be able to dabble with serial functionality right within the web browser. Using SvelteKit and Cloudflare Pages, it was quite easy to build and deploy this simple app. The most interesting learning was how to use custom Svelte stores for basic reactive features; I used one to implement an event-based serial data store that other components could subscribe to.

Check out the LD2410 Configurator at ld2410.albert.nz or view the source code!

Flora and fauna

The birdlife is plentiful in this part of the country. Hours of walking were set to the sounds of clusters of bellbirds calling in unison, kakariki chattering (though rarely seen) and the cheeps of tomtits and riflemen. Kiwi were around, too; I saw no fewer than seven during the trip. They’re audible early and late in the day making horrendously shrill cries to each other, but we saw a few alongside the track in daylight, too. They’re faster, bigger, and more oblivious than I ever expected, often just about colliding with a walker before turning around and zooming off in the other direction.

Robins were numerous in the sheltered low-lying stretch between Mason Bay and Freshwater River.

There’s a varied range of plants around the place. Unfortunately the huge deer population is successful at continually suppressing new broadleaf growth, so there are large stretches of forest consisting of ferns, mature broadleaf trees, and nothing in between. I suppose hunters enjoy the good sight lines afforded by this distorted ecosystem.

The forest is incredibly damp from the large amount of precipitation received on Stewart Island. Windfall and dead trees rot quickly and give rise to new plant life.

We saw some bright orange and blue fungi, too.

Landscape

The vast and pristine beaches did a great job of making me feel very small. It’s a humbling wilderness.

Possibly my favourite terrain feature was the incredibly tall sand dune adjacent to Big Hellfire Hut. I was not prepared for a hut at 200m elevation to have a beach on its doorstep.

What to expect if you’re doing the North West Circuit

Here’s a bit more information if you’d like to learn more about doing this tramp.

North West Circuit itinerary

Counter-clockwise is the standard direction for the North West Circuit but you can totally do it the other way, too.

The standard track description includes all 10 huts on the circuit, which would take 11 days. If you’re staying at all huts, the amount of walking per day will vary from 3 hours to 7 hours. Note that the first and last (Port William and North Arm huts) are Great Walk huts on the Rakiura Track, meaning they book out early and cost more money than the $5 standard huts on the rest of the track. It’s common to skip these huts to avoid the bustle and make it to the cheaper huts more quickly. If you do this, be aware that you will need to walk from Freshwater Hut to Oban in one day, which has a DOC-provided time of 10-11 hours. I’d recommend getting a water taxi for this leg! See the transport section below. Some experienced hikers we encountered also skipped Yankee River Hut, Long Harry Hut or Mason Bay Hut to reduce the amount of days.

We took 10 days and skipped the Great Walk huts only. This meant we had an additional rest day, which we spent at Mason Bay Hut.

Transport to and on the island

Transport to Rakiura / Stewart Island is via boat or plane. The boat is operated by realnz and costs $99 each way from Bluff. I’d agree with their website that it is quite an experience; the Foveaux Strait lives up to its name and even on a calm day those prone to seasickness (like myself) will be reaching for the sick bag during the hour-long crossing. Get on the boat mentally prepared and with an empty stomach—or catch the plane. The flight is $240 return from Invercargill. It takes less than 30 minutes but is limited to 15kg baggage allowance.

Water taxi services are available within the island. For $70 per person, they will shuttle you between Bungaree Hut and Oban, or Freshwater Hut and Oban. We used a taxi to skip our last day from Freshwater Hut to Oban—the taxi arrives at the landing adjacent to the hut at high tide then slaloms down Freshwater River at impressive speed, reaching Oban in about 45 minutes. You need at least three people to charter a boat (or get lucky and book onto an already-chartered boat) and you’ll need to book ahead. Freshwater Hut has no cellphone reception and you’re not guaranteed any reception from Rocky Mountain, either. There are two operators who sail to Freshwater Hut: Rakiura Adventure and Rakiura Charters.

Huts

Apart from the Great Walk huts, all huts are standard. You can use $5 hut tickets or a hut pass. There is alternatively a North West Circuit Pass which I am told is cheaper than individual tickets. There’s little information online about this pass but you should enquire at the DOC visitor centre.

The standard huts are quite generously sized. Most have 10-12 bunk beds with the exceptions being Mason Bay Hut (20 bunks) and Freshwater Hut (16 bunks). They all have at least two sleeping areas, with the exception of Long Harry Hut and Big Hellfire Hut which are single rooms with a “marae-style” sleeping area.

The northern huts (Bungaree, Christmas Village, Yankee River and Long Harry) have cell reception in or near to the hut.

Long Harry hut is dramatically positioned above the beach with a stunning view to the north. It would have been my favourite if it wasn’t for the sandfly leakage—the window mesh is too coarse and sandflies just crawl through.

Many of the huts have great rivers nearby to bathe in. Being somewhat clean makes a big difference on such a long hike.

People

The circuit was far busier than I expected. We walked it in March as a group of three and didn’t have a single hut to ourselves. We heard from some circuit hikers a day behind us that they experienced full huts and were forced to hut skip as a result. If you’re doing this trip in summer or autumn and are looking forward to 10 days without seeing another soul, adjust your expectations!

A few of the circuit walkers we encountered had just finished Te Araroa southbound. Early autumn may be a common time for such walkers as they would have started TA in late spring and taken about four months to complete it.

Solo and duo circuit hikers are common because it’s hard to organise a group for a 10-day tramp on an island. That said, the hut books did have reports of groups up to six people in size. You might find that you form an ad-hoc group with people you encounter along the way, if they’re pleasant company!

Some of the huts (at least Bungaree, Yankee River and East Ruggedy) can be used by hunters as the corresponding hunting blocks do not have dedicated hunters’ huts. They’re only allowed to use half the bunks in these huts but in some cases will unpack their stuff throughout the hut, leading to some tension with trampers. The friendlier hunters will offer hot water, good conversation and maybe even meat.

Mason Bay Hut is commonly used by wealthy groups who charter a helicopter in, stay a few nights in the $5 hut, then walk the three hours to Freshwater to catch a water taxi out. Needless to say, this can cause tension with circuit walkers who arrive after seven days of walking and dehydrated meals, looking for a bunk and perhaps some solidarity. It was certainly jarring for me to walk into a party of six drinking wine and gorging on delicious-looking hamburgers.

Tidal considerations

There are several parts of the trip that are difficult in high tide. There may be other parts we missed out of sheer luck, but here are the two we noticed:

1. The section east of East Ruggedy Beach. There’s no high tide track here. This one nearly caught us out but the tide wasn’t quite high enough to cut us off.
2. The rock in the Mason Bay section. It’s described on the DOC webpage. There is a high tide route but it looks difficult—DOC says it adds an hour of walking.

For Mason Bay tides, the guidance from the DOC visitor centre was to subtract two hours from the official LINZ Oban tide predictions. For Mason Bay, aim to avoid this time by two hours either side.

Track condition

The track was well marked—much better than I expected. When we did the track there were many sections between Long Harry Hut and Mason Beach that were overgrown and had significant windfall. DOC told us there hasn’t been much maintenance in the last couple of years due to COVID-19 but that they were expecting to get maintenance teams in there over winter 2023.

Side trips

Mt Anglem / Hananui

We skipped the side trip to Mt Anglem / Hananui from Christmas Village Hut. According to our DOC informant, it’s a stunning view on a clear day but generally you can’t tell whether it’s a nice day until you’re 80% of the way up the mountain. It wasn’t worth the gamble for us.

East Ruggedy Beach lookout

There’s a lookout signposted just before the descent to East Ruggedy Beach. It’s only 50m from the main track and well worth it.

Hellfire tops

A local staying at Big Hellfire Hut informed us there are some great views to be had along the ridge south of the hut. He suggested departing the track by a few metres to reach the various false peaks along the ridge. We found the views from the track were good enough!

Big Sandhill

Definitely worth the 90-minute return trip from Mason Bay Hut. There are nice views from up there and it’s easy to squeeze into a day even if you’re walking on to Freshwater Hut. Playing on the dunes is fun, too.

Rocky Mountain

2-3 hours return from Freshwater Hut, the top of Rocky Mountain has a grand view back towards Mason Bay, and even towards Oban—or so we were told. As we climbed the hill, the weather closed in and it was cold, wet and windy at the top. Probably only worth doing in good weather. Some say you can get cell reception at the top (theoretically great for booking or rescheduling a water taxi from the hut) but on this stormy day we had neither the patience nor the line of sight to find it.

Weather

The weather was all over the place. Most days would go from cloud to rain to sun and back to rain again—in the span of an hour. We had warm sunny times and cold blustery southerlies. Forecasts were good enough to predict storm systems passing through, but don’t rely on them for day-to-day weather. Pack for all conditions!

What follows is hopefully a more complete description of getting to and from Mt Curtis Hut based on our experience. It is up-to-date as at time of writing.

How to get to Mt Curtis Hut

Access is from the bridge which crosses Dee Creek, just East of Inangahua. There’s a decent-sized layby on the West end of the bridge for parking. Cross to the Eastern side of the bridge and descend to the river on the true right. You’ll have to jump a fence to get down to the river from the road. The next step is to follow Dee Creek for a couple of kilometres—there are no markings in this section and travel is through farmland along a paper road which follows the river.

Eventually the river plunges into forest. Continue following the river until you reach point 1515277E 5364463N on the TL bank.

At this point there exists a marked trail but you need to head directly away from the river to find it. Look for orange and pink ribbons visible from the parallel side creek a few metres to the south of the main river. If you struggle to find it, consider continuing 100-300m further up the main river then bush-bashing south to intercept the track while it is close to the river.

The track runs vaguely parallel to the river at first, tracking the top of the bluffs marked on Topo50. It then moves to follow the spur past Pt 489 and all the way up to the hut, becoming increasingly well-marked along the way. We encountered a wasp-infested section of beech about halfway along the trail—watch out for that if you’re doing this tramp in the summer months.

How we got to Mt Curtis Hut

This, of course, was not the way we went up. Only after coming back down do I have the privilege of describing the actual track to you.

With such concealed marking and little information, we completely missed the turnoff for the track, heading up the river for another few hours. Dee Creek develops gorge sections higher up, leading to beautiful sights, slow travel and steep bush-bashing to bypass unnavigable parts.

Eventually the river rose out of a gorge to meet the bank on the TL. We took this opportunity to depart the river to head in the direction of the hut. Fully prepared to bash the remaining two kilometres to the hut, we hedged our bets by detouring to the spur—a decision inspired by recalling a vague report of a marked track up one of the spurs in the area. After pushing through scrub for a while, we reached the spur and found an orange track marker! The going got steep shortly afterwards and it was amazing to experience the shift from mentally-exhausting bush-bashing to physically-exhausting climbing.

The hut itself

Even before we could collapse on the hut deck, we were greeted by a weka scurrying through the adjacent flax plantation. (An entire family of curious weka meandered past while I was on the toilet in the morning, much to my surprise.)

Mt Curtis Hut is a modern-but-basic four-bunker with a spare fifth mattress. The hut book indicates about a dozen visits per year. There’s a fairly well-stocked pantry here with a few pots and pans. It’s double-glazed and we certainly had no need for the fire on this clear summer night. The hut sits just below the bushline and the view is largely obscured by trees. Just five minutes’ walk up the hill (quite doable in sandals) is a small knoll which has unobstructed views to the West. Mt Curtis summit itself is also easily accessible but might be a more appropriate side trip for a two-night stay.

The return

Descending the marked track was straightforward enough experience (despite a wasp sting) and one we had been looking forward to since the previous day. We expected to have missed an obvious turnoff but once we got to the bottom of the track the markers thinned then stopped well out of sight of Dee Creek. In the end, then, we didn’t have any reason to beat ourselves up for missing the track on the way up.

Our misguided trip in to the hut took 6½ hours. Out, it had taken just 3½.

Paying for Mt Curtis Hut

This is not a DOC hut so hut passes and tickets do not apply. Payments towards the hut can instead be made to the following account which was displayed on a sign in the hut.

Mount Curtis Community Hut
38-9012-0653480-00

Also listed in the hut are details to contact some of the people involved in its maintenance. Contact me if you would like those.

Day 1

After an early start and a disappointingly closed Sheffield pie shop, we got walking from Greyneys Shelter. It was an easy going three-and-a-bit hours up to Edwards Hut, where we stopped for a sit-down and some lunch.

Beyond the hut the trail continues all the way to Taruahuna Pass, with the only addition being overgrown scrub and multiple small river crossings. The increasingly open surroundings really show off the surrounding mountains.

At the pass, travel converts to rock-hopping then sliding down loose banks towards Otehake River.

The track markers peel off to Tarn Col; to get down the valley, we picked a route that brought us out on the true right of the river as the terrain levels out at the bottom of the rockfall. From this point we could see a track marker leading us to the left. The marked track crosses all the way to the left of the valley, sidling through forest above the bluffs before descending at the main river junction. This is the route we took. Apparently it is also possible to simply follow the river between the bluffs when levels are low.

The track on the West Coast side of Taruahuna Pass is starkly different to the Canterbury side. The forest is more dense and markers harder to spot—there is a mix between orange DOC markers and white Permolat markers. Despite the slower travel we found our way across the river and on to Otehake Hut, where we prepared a pleasant cheeseboard before dinner.

Day 2

Allowing 10 hours for our exit day through Hawdon Valley, we got up nice and early. Backtracking to Taruahuna saddle was slow and steady; I found the steep climb up to the saddle easier than the descent. At the saddle we were greeted by a delightful sight: a pair of rock wren hopping between the rocks and calling to each other.

We turned our attention to the Tarn Col ascent. The face is an imposing wall of rock and in the clouded visibility we largely missed the poles marking the ascent route through vegetation, instead deciding to climb a scree face. As a scree novice, this was definitely out of my comfort zone—though possibly good practice for times when such an ascent would be necessary. For clarity, here’s a summary of the various routes which I devised after the trip:

During breaks in the cloud, we glimpsed sights of the otherwordly tops up here. The rocky tussock atmosphere reminded me a lot of Routeburn, a track I also completed in damp grey weather.

The route is poled all the way down to the intersection with Otehake River East Branch. Follow that river upstream until the marked turn off towards Walker Pass.

With surprisingly fast travel through Walker Pass and out Hawdon Valley, we made it out safely after a 9-hour day.

Day 1

With packrafts inflated, we paddled across the northern tip of the lake. It was an absolute bluebird day and a lot of fun on the water. We made it across in about 30 minutes.

One of the benefits of this trip is that the paddling leg is adjacent to the carpark. This means no need to carry rafts! Kayaks would work even better here. We stashed the raft amidst a manuka forest just north of the Albert Burn inlet.

Walking up the track was a fairly straightforward affair. We zigzagged across the water following the river outlet’s twists and turns, eventually picking up the marked track on the true left of the river just downstream of the narrow gorge section. The track climbs up the bank a wee bit before dropping down for a crossing in the middle of the gorge. On this clear day the river was waist deep here, so we locked arms and made it across safely. This is definitely not a crossing to be attempted at rainy times.

Continuing along the track on the other side of the river, it became increasingly clear that this track is not well-frequented. Pieces of fallen beech debris were scattered everywhere and we shoved many branches out of the way as we walked. After about a kilometre, the track opens up to a beautiful wide river valley. We spotted numerous waterfalls coming down from the steep rocky hills lining the valley as we meandered up the river.

The hut sits above a large clearing, a bit further up a hill than I thought. It’s a surprisingly nice hut for one which receives about a dozen parties each year: built in 1995 and having two levels of four bunks each, with lovely evening sun and only minor sandfly leakage.

The valley is really wide at this point and we took the opportunity to check out the dramatic waterfall on the opposite side of the valley. It made for a sublime spot for some swimming, frisbee throwing, and refilling of water bottles. This was absolutely a moment to remember.

Day 2

With some showers overnight, I was worried that the river would be swollen. It was reassuring to see many of the waterfalls, presumably fed by snow melt, were actually lower than the previous evening. We crossed the river without incident.

What became somewhat more troubling was the ever-increasing wind as we approach the lake shore. A stiff northerly—stronger than forecast—was whipping down from the Makarora Valley. With minimal delay, we loaded up with sugar and carbs, inflated the rafts, then set out back to the campsite. It took more than an hour to get back in the inefficient and highly blowable little packrafts—over twice the time it took in the other direction. With tired arms, but thankful to have made it back safely, we regrouped at the carpark and prepared for the trip home.

About halfway between St Jacob’s Hut and Top Hope Hut, the track peters out on the true right of the river. It looks like undermining from the river is eating away at the track. We wasted a fair amount of time trying to find the track higher up on the bank, as marked on Topo50. But it turned out to be easier to just hop into the river and walk the final 30 minutes of the approach with wet feet.

Colour strips are great for decoration, but there are better options for illuminating a room with high-quality light. I stumbled upon some nice temperature-adjustable white LED strips on AliExpress recently. Unlike individually-addressable strips where control is done via a signal line, these strips are what I like to call “direct drive”, where the input voltage is applied directly to the LED strip and brightness is changed via current control or PWM.

ESP8266 boards aren’t ideally suited to driving these strips directly because their PWM implementation is in software, not hardware. This can cause flickering. But I was recently overjoyed to notice ElectroDragon is now selling an ESP32-based version of the driver board. The ESP32 has the notable benefit of hardware PWM—perfect for LED strips that require direct voltage!

Armed with this knowledge and a bunch of components, I set out to install my new temperature-adjustable ceiling light.

Hardware

The LED strip

I’m using a 24V two-channel white COB LED strip. One channel is warm white, and one channel is cold white. I went with this strip for a few reasons:

• This is for ceiling lighting; I don’t need colour and I don’t need individually-addressable LEDs.
• I enjoy the idea of having controllable colour temperature so I can change it during the day or connect it to Flux for automatic temperature changing over the course of the day.
• The CRI of these strips is excellent at Ra>90. For comparison: the white channel of my colour SK6812 strips is generally rated at Ra>70. This all means the COB strip gives off a much fuller-feeling light that is more suitable to illuminate a room.
• 24V is a high enough voltage that I can run over two metres of the strip without having to wire up extra power lines to intermediate points along the strip (something I had to do with my 5V addressable strips).
• The COB (chip on board) design of this particular strip is incredibly bright at over 700 lumens per metre.
• With an insane 640 LEDs per metre, the strip basically has all the diffusion it needs built right in. This is less of a concern given I will be bouncing the light off the ceiling.

You’ll need a 24V power supply to use with these. I bought a cheap supply from BTF Lighting, the same seller as the LED strip. I would recommend a more reputable 24V supply, perhaps one of those more traditional “plastic brick” style ones.

The driver

I have loved working with ElectroDragon’s ESP8266 boards in previous projects. It’s a basic LED strip controller with built-in MOSFETs and voltage regulation—configurable to drive addressable LED strips like the SK6812 or “direct” LED strips by sending PWM signals to the MOSFET pins.

A big downside of the ESP8266 is its lack of hardware PWM. This means a direct drive LED strip (like the one I’m using in this project) would be more likely to flicker as the microcontroller juggles PWM signalling, WiFi connection, and other tasks with competing priorities. That’s simply not an issue with the new ESP32 boards. PWM is managed using what are called LEDC channels: just specify a PWM frequency and it will be taken care of in hardware. At 7USD a pop for the upgraded version, more people should know about these delightful boards.

To configure the board’s hardware in this project, I set both power jumpers to the “5-27V” setting. This means that I can use the 24V power supply required for the strip as input and power will be passed straight through. At the output side, I set the jumpers to R (as opposed to GND) and G (as opposed to IO2). This is the configuration for driving those “direct” LED strips via the MOSFETs.

The board comes with screw terminals for the power connections. I was happy to see that unlike previous revisions, the power input terminals are labelled on the silkscreen. Like on my other projects with the board, once I was happy with my configuration, I soldered the input wires, output wires and jumper settings in place.

Software

There are two options here: the flexible ESPHome, or the more LED-focused WLED, which now supports two-channel white strips. I went with ESPHome, mostly because WLED doesn’t have support for ESP32-C3 (read on for more about that) but also because ESPHome has supported this kind of strip for much longer.

The end result I want is a temperature-controllable light entity in Home Assistant. Fortunately, ESPHome has the cwww light platform which is exactly for this application. The ledc output component lets me use that glorious hardware PWM, too.

The configuration I went with looks something like this:

esphome:
  name: ceiling_light
  platformio_options:
    board_build.flash_mode: dio

esp32:
  board: esp32-c3-devkitm-1
  framework:
    type: arduino
    version: latest

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: 'Ceiling Light Fallback Hotspot'
    password: !secret wifi_ap_password

logger:

api:

ota:

light:
  - platform: cwww
    name: 'Ceiling light'
    cold_white: greenpin
    warm_white: redpin
    cold_white_color_temperature: 6000 K
    warm_white_color_temperature: 2700 K # From the strip's datasheet but pretty sure it's more like 3000K in reality
    constant_brightness: true

output:
  - platform: ledc
    pin: GPIO5
    id: greenpin
    frequency: 19531Hz
  - platform: ledc
    pin: GPIO8
    id: redpin
    frequency: 19531Hz
  # - platform: ledc
  #   pin: GPIO4
  #   id: bluepin
  #   frequency: 19531Hz
  # - platform: ledc
  #   pin: GPIO3
  #   id: whitepin
  #   frequency: 19531Hz

There are some important bits in here:

• board: esp32-c3-devkitm-1 signifies that this isn’t an ESP32 but rather the related-by-name ESP32-C3. The ESP32-C3 is less powerful but contains many of the ESP32’s key features. It’s also much newer than the ESP32 and runs on the completely different RISC-V architecture. The small package size of the ESP32-C3 means it can be used by manufacturers as a drop-in replacement for the ESP8266 on boards like the ElectroDragon one, and I can only see the popularity of this chip skyrocketing.
• board_build.flash_mode: dio is absolutely required for flashing to work successfully. Without this line, the board will default to QIO mode which resulted in boot loops on my ESP32-C3 board.
• frequency: 19531Hz is taken from the table of recommended frequencies published in the ESPHome documentation. Initially I tried 1220Hz as this results in the most steps available for buttery transitions. Unfortunately, my power supply produced a loud whining sound and some flickering at that frequency. So I switched to the lowest recommended frequency above my hearing limit of about 18kHz; 19.5kHz it was! A worthwhile change for the eliminated flickering and noise—and I don’t notice a difference in transition smoothness. A lower recommended frequency may work well for you if you have a better quality supply.

A note on pins

I’ve included the other pins on the board, commented out, for reference purposes. This took some sleuthing as ElectroDragon originally didn’t publish the pin numbers for the ESP32 variant of this board. Here’s a table of what I found:

Flashing the board

I used the ESPHome dashboard interface to enter and export the config. From the dashboard, find the device you’re flashing and select Install -> Manual download -> Modern format. It should compile and download a bin file which can be flashed to the ESP.

The documentation for “Modern format” is entirely absent but after much pain and learning about bootloaders I can confirm this is the option you want. I believe “modern” generates an image including the bootloader, which makes it really simple to flash to the 0x0 offset. “Legacy” exports an image for tools which provide their own bootloader.

Connect the ESP board to a USB FTDI adapter. My FTDI adapter is based on the CP2102 and includes a 3.3V output. Connect GND->GND, RX->TX, TX->RX and 3V3->3V3. The ElectroDragon documentation recommends using 5V->5V instead of 3V3->3V3 (I think this is what I have done on previous projects) but either should work.

I flashed the board on Linux using esptool.

Before each esptool command, hold down the IO0 button on the ESP board while plugging the FTDI adapter into the computer. This signals the ESP to boot in flash mode.

First, erase the contents of the flash:

python -m esptool -p /dev/ttyUSB0 -b 460800 --before default_reset --after hard_reset --chip esp32c3 erase_flash

Now replug the board and flash the firmware:

python -m esptool -p /dev/ttyUSB0 -b 460800 --before default_reset --after hard_reset --chip esp32c3 write_flash 0x0 firmware-factory.bin

Easy as! The device should eventually appear as online in the ESPHome dashboard. Over-the-air flashing will be possible now.

For troubleshooting, I found picocom a useful tool to monitor serial output over USB:

picocom -b 115200 /dev/ttyUSB0

The result

It’s simple and it works. Introducing my new ceiling light! I’m really impressed by the quality of the light. While it’s fun being able to generate rainbows from lights like the SK6812, there’s something really beautiful about a natural, bright, high-CRI white. I’ll be sticking to this kind of strip whenever I just want illumination, not decoration, from a lighting project.

There is a small network of tracks around the Mataketake Range. Confusingly, they’re not all on topomaps and the cattle track is closed south of Maori Saddle. We decided to go to and from the hut via the Mica Mine tops route to maximise our chance of getting great views. Going up via Maori Saddle Hut would have the benefit of forming a longer loop and opening up a side trip to Lake Law.

With a sketchy-but-passable weather forecast for ANZAC weekend, we committed to the trip and set off!

Day one

After the long drive, it was under an hour’s walk to Blue River Hut. What a great feeling to stretch the legs before arriving at this historic, cosy, building for the night! We even got a clear sky overnight. A noisy group of three kea zoomed overhead while we took some photos.

By night the top-bunkers among us discovered a downside of this hut: the low ceilings made it comically easy to bump limbs and heads during the night.

Day two

With the skies still clear against the odds, we were keen to get up the hill! The track was generally straightforward, fast travel the whole way to the bushline, becoming more of a grind at the top. A highlight of this section was seeing many kererū flying through the bush and hearing the calls of kākā and kea mix across the forested valley.

Before we knew it, we were out into openness with the sky still clear. Beautiful views of the surrounding ranges emerged, only to be swallowed up by low cloud as we approached Mataketake Hut.

Day three

As a personal first, I had planned for an entire hut day on this trip. This was based on the nice things I had heard about the hut. Hopefully it would increase our chance of getting good weather and having the building to ourselves. Turns out neither of those things occurred during the first night at the hut. Sunday proved to be an excellent day to be indoors, with a storm bring plenty of rain and wind. Plus, over the course of a cosy day in the hut filled with board-gaming, tarn-swimming, and book-reading, we realised we would have a clear evening and sole occupancy in the second night.

As sunset approached and the cloud layer lifted, it became clear that this hut’s situation is truly special.

Day four

We awoke at dawn. This seemed fitting for ANZAC day and meant we would get back to Christchurch earlier. With a crisp breeze behind us and the day dawning in front of us, we packed up and headed out before sunrise. The views were distractingly stunning.

We made it out from the hut in 4.5 hours, a fair bit faster than the 8 hours estimated by DOC. Before we knew it we were back in Christchurch and dreaming about getting back to the hills.

Access to the Mt Pfeifer tops is via a stream up to Waharoa Saddle. If you’re going up there, make sure to stick to the left branch and you should have no trouble reaching the saddle.

The saddle was rewarding with views of the Pfeifer tops and provided a great place to top up on snacks before the steepest part of the climb.

Beyond the bushline, the track fades. There are a couple of large basins to cross before reaching the small hut that is Pfeifer Biv. We had originally intended to proceed beyond the biv but quickly realised we were (a) running out of daylight hours, and (b) quite looking forward to a rest after walking up a mountain. This cosy biv sleeps two but was perfect for all six of us to sit in to cook up some dinner before dispersing to tents. Thankfully, water from the nearby tarns didn’t seem to be particularly poisonous.

A cool, rainy morning broke and saw us heading down the mountain towards Lake Kaurapataka. Will expertly navigated us to our exit route: a scree chute marked as a stream on Topo250, a few hundred metres northeast of the biv beginning around the 1120m contour. This descent provided me with my first experiences in both bush-bashing and scree-skiing as we lost over 500m of altitude in a kilometre.

After reaching the bottom of the not-stream, we followed Pfeifer creek out to Lake Kaurapataka track. This track is very fast travel until near the end of the lake when it becomes surprisingly undulating, a situation which worsens until the track eventually joins the Otehake River further south. The distance and elevation changes are definitely concealed by the contours on the map.

Eventually, though, we were out of the forest and walking along the river to the hot pools. These are situated on a riverbed between two campsites. There are some excellent swimming holes nearby, too.

The hot pools are delightful and most of us agreed these were the best natural pools we had experienced. All six of us could fit in them; I even found a spot sheltered from the rain by a protruding log. With a perfect temperature, neck-high water, and few hotspots, I can certainly recommend these pools to anybody willing to walk the 5-6 hours to get there from the road.

With the delayed itinerary, we decided to split the party: Will and Hayley would speed onwards to complete the traverse, while the four others would head back to Morrisons Footbridge (the easy way—that is, not going over a sizeable mountain). While it was a shame to miss the views of Tarn Col in brilliant weather, I was happy to return home before nightfall and in one piece, with plenty of Arthur’s Pass secrets remaining for me to explore another day.

The track consists of three parts: a hilly section over Casey Saddle, a flat bike track along the Poulter riverbed between Casey Hut and Poulter Hut, and finally a gentle uphill approach to the lake.

Day one

A mere 200 metres in to the track, I tripped and almost fell into a bush of Matagouri. Not the most elegant start.

The following 15 or so kilometres to Casey hut has a couple of surprisingly steep ups and downs: don’t underestimate the change in altitude between 550m and almost 900m! On the downhill approaching Casey hut I felt my IT band starting to play up. With 12km of walking remaining, this was not a great sign.

Before I knew it, I was back on the flat in the Poulter Valley, and the pain vanished. It’s quick travel up the track all the way to Poulter Hut.

Nearing the hut, I heard a vaguely familiar “kekekekekek” sound and looked up to see a kārearea (NZ falcon) chasing a kāhu (harrier hawk). The falcon wasn’t flying much faster than the hawk and would take a while to catch up to it, performing a dive bomb then repeating the whole process. They carried on like this for a good few minutes; it was majestic to watch.

Poulter Hut looked very tidy. I stopped for a snack and filled out the intentions book before resuming up to the lake. It would be quite nice to come back here another day.

It didn’t take long to cover the ground to Lake Minchin. The track meets the Lake at its southern point and continues around to the north. This northern shore is where I set up camp on some nice spongy ground.

I should pause here to make a couple of points that I wish I had known before camping here:

• The sandflies can be vicious. They sounded like rain on my tent. If you’re going here, bring DEET and/or your favourite long clothing. I’m personally quite sensitive to insects and I found their presence made it a fair bit harder to relax and enjoy the view!
• More importantly, with rivers running low I struggled to find a clean water source near the lake. There was a creek I had topped up at halfway between Poulter Hut and the lake, but I really should have taken that opportunity to fill up my bottles.

Nevertheless, this spot was truly beautiful and ticked the boxes I had set out to tick. I felt truly physically isolated from anyone else and completely in the middle of nature. Most people don’t get to feel that feeling very often!

Day two

Slightly thirsty and plagued by the gradually-awakening sandflies, my plan in the morning was simple: wake up, pack up, and hurry up to Poulter Hut for breakfast. It became quickly apparent I had aggravated injuries to both my IT band and ankle; walking was pretty uncomfortable. But there’s not much that can be done in these situations. I scoffed some painkillers and focused on moving. This would be a day to get over and done with.

I couldn’t help but stop just a half hour later when I spotted a kea chilling on a trunk off the side of the track. It seemed very interested in me as I watched it. A second kea appeared behind me on the track and started hopping closer while screeching. Next thing I knew, both of them were dive-bombing me as I escaped towards the hut. I’ve never seen such odd behaviour from these birds!

After filling up on porridge and some much-needed water, I was back out on the riverbed. My two kea friends (frenemies?) emerged from the forest and circled me for a surreal few minutes, screeching the whole time.

A day of backtracking proceeded. I saw a human for the first time in 23 hours somewhere just beyond the Casey Hut turnoff. Eventually I reached Andrews Shelter, clutching a sore leg but thankful for the weekend adventure that had been.

Update: I considered revisiting this track in 2024, but unfortunately a rockfall near the bridge has closed the track we took. The hut is still accessible from the east, from Gannons Road via Montgomerie Hut.

The hut is fairly modern and warm, but we had a hard time getting the coal burner started on a surprisingly cold New Year’s Eve.

On our outwards journey, we attempted to locate “Fiery Cross Gold Mine”, as present on the Topo50 map. We found neither mine nor track, despite our attempts to ascend Caples Creek.

I hear the views from the ridgeline parts are expansive and some of the best in New Zealand. Unfortunately we received low cloud and only received brief glimpses of the surroundings.

During the climb through forest, increasing glimpses of Lake Kaniere are offered. By the time we reached the bushline we were surrounded by dense cloud which, thankfully, started clearing soon after.

Upon reaching the hut, we instantly recognised it as the building pictured on the information sign in just about every DOC hut in the country. Ironically, the hut is not actually a DOC hut, being managed by the Kokatahi Tramping Club.

Mt Brown Hut is a cosy hut with only four bunks and a fireplace. Its small size belies its high popularity so I would recommend bringing a tent if you are heading up here, especially on summer weekends. The hut sits directly adjacent to a lip which drops off steeply to reveal gorgeous views out to the coast. Views are even more stunning from the hill just to the south of the hut. It was exciting to watch these views improve as the cloud cleared into the evening.

Back in St Arnaud

We were only in town for a couple of nights but enjoyed heading down to Lake Rotoiti which was a short walk from our accomodation. I already want to come back to this region, possibly for some fast-packing in summer - there are so many interesting longer trails to explore.

Day 1

To maximise our hot pool time on day 2, we arrived late on Friday and walked to Dillons Hut in the dark. The first 2.5km is 4WD track which we tackled without too much difficulty in a Prado. We got out and started walking when it got too muddy to proceed in the vehicle, arriving at the hut around 9:30pm.

Day 2

We set off the next day, now able to see the valley views all around us as we proceeded to Mid Taipo Hut and beyond. There is a three-wire bridge incorrectly labelled as a cableway on Topo50. Following the markers when approaching the bridge from downstream will lead you onto the flood track. If possible I’d recommend staying on the riverbank all the way to the bridge.

Shortly after arriving at Julia Hut, it was time to find the hot pools. Follow the track beyond the hut down to the stream and then head about 400m futher downstream. We found some shallow pools directly adjacent to the river on the true right. With the assistance of some spades and pots left by past groups, we dug some shallow pools big enough for six of us to sit half-submerged. The combination of dark skies, hot water and cold air made for a surreal stargazing experience.

Day 3

The next day we got up early and retraced our steps 23km back to the car.

Day 1

We started at the Forest Road trailhead, oppositing the Te Ahumata Track on a saddle near Whagaparapara. We were planning to finish the tramp 2km down the road at the Kaitoke Hot Springs Track. So I dropped the others off at the saddle, parked at the Hot Springs carpark, then ran back along the road.

We had lunch at a picnic table at the junction between Forest Road, Kiwiriki Track, and Maungapiko Track. The Maungopiko Track proved to be a delightful excursion. Just a few hundred metres and some rock scrambling leads to a beautiful view out from the centre of the island all the way to the western coast and Little Barrier Island beyond.

A couple of kilometres before reaching Kaiaraara Hut, there is a lovely short nature walk to some incredibly large kauri trees - probably the biggest I have seen in my life.

There was light left in the day so a few of us took an excursion to Bush’s Beach. It’s about 1.5km each way and well worth it. I’m sure this would be a lovely spot to relax in summer.

Day 2

There are two ways from Kaiaraara Hut to Mt Heale Hut: the Kaiaraara Track (which is the one DOC marks on the Aotea Track map) or the South Fork Track. We were leaning towards the South Fork Track as it has less vertical gain. This decision was confirmed by chatting to a ranger who arrived at the hut in the morning - he said it is one of his favourite tracks on the island!

The track follows close to Kaiaraara Stream and is poorly marked for this distance. This isn’t a big deal, though; just make sure to find the track at the stream fork about 600m after the track first hits the stream. A couple of us simply walked along the stream for this stretch. Some members of our party were less enthusiastic about wet feet.

The remainder of the track is a straightforward and increasingly beautiful ascent up to Mt Heale Hut. See if you can spot the hut on the other side of the valley as the track climbs. Thankfully, the views keep on getting better all the way to the hut. From the balcony, the vista looks down the bush-clad valley out to the sea, with Little Barrier Island perfectly framed.

We arrived at the hut early, around 2pm, just in time for a late lunch and reading on the deck until the sun went down and the winter cold forced us inside. Sunsets don’t get much better than this.

Day 3

We woke up at 6am to climb Mt Hobson / Hirakimata. Unlike the Kaiaraara Track, the South Fork Track hadn’t taken us up to the summit and we were really keen to watch sunrise from the tallest peak on the entire island! So we set off in the dawn light.

The climb up is short and steep - I was grateful for the very nice stairs all the way to the top.

Exiting was straightforward; it’s a gentle downhill most of the way from the hut to the hot springs carpark. The hot pools themselves were delightful. There is one large pool, then more along a stream which stretches up the hill beyond. On advice from a local, we waded up the stream to a deep pool 50 metres along. The water was lukewarm with hot mud lining the streambed. While far from a spa, I really enjoyed this first natural hot pools experience and I would certainly recommend it for the last day of the trip.

A big benefit of this section of trail is that the three major river crossings are bridged. It’s clear the waters were higher than usual.

A highlight of the hut were the new memory foam mattresses—a clear upgrade from the old ones—which made for a great night’s sleep.

Appendix: Fungi

Here are some neat fungi which we spotted beside the trail.

The track starts with a river crossing right near the carpark. We did the walk in autumn and the crossing was trivial. As usual, though, check forecasts—especially if walking with young ones.

It’s also worth noting that the hut gets busy! I’d highly recommend our route—leave early, drop bags at the hut, then do the 12km loop in the hills with daypacks. It makes the excursion nicely challenging and means you can claim a bunk early.

The run was harder than expected but I managed to beat my four hour goal. Highlight: making a fair few overtakes in the 5km of technical flat trail between Flays Road and the ridgeline. Lowlight: the cramps, which begun 5km before the finish line and didn’t relent for hours!

Back in Coromandel Town

The weather was far from perfect but there’s no denying it’s a walk with beautiful surroundings.

Check the weather! It seemed pretty doable on the sunny Saturday we attempted it, but the very next day the weather packed in; crampons were necessary on the route and we wouldn’t have been able to do it in such conditions.

Other things

We ventured up the Hooker Valley Track the following day.

There was enough clear sky at night to get some nightscape photos.

I’d like to come back here in better weather and potentially create a longer loop with Gable End Ridge.

If running the track, the counterclockwise direction would probably make more sense. This way the steepest ascent is done early and gradual downhill is maximised.

Day 1

Most of the day is a gradual climb up Cass River. The scenery gets dramatic at Cass Saddle.

Hamilton Hut is halfway along the track. It’s nicely located on a bank set back from Hamilton Creek. We were the first group at the hut, arriving around 3pm. This way we were able to make the most of the sunshine hours and great weather by reading in the sun and throwing a frisbee around.

Day 2

We set off up Harper River as the sun rose above the hills.

The second day was a bit harder than the first and more interesting too. We made our way up the river sections with help from cairns. It’s lovely scenery, with dense beech forest and river junctions all over the place.

Beyond A-Frame Hut, the vegetation falls away to reveal what must be some of the best views in the area.

I was not mentally prepared for the last few kilometres of the trail. The climb to Duke Knob near the carpark looks innocent enough on the map but really took it out of me. It was a luxurious feeling to relax on a camping chair once I got back to the car, despite the feasting sandflies. My legs were cramping all the way home.

We were worried that the cloud would mean no views. But as we climbed higher along the steep (and surprisingly lush) Te Ara Pātaka Walkway, it was revealed that the cloud was simply a low layer appearing to sit across the entire mid Canterbury region.

To form a bit of a loop, we returned to Packhorse Hut along Faulkner Track on the other side of Mt Bradley.

I would definitely recommend this for a day walk or run in the region. For a faster, fuller loop with some road running, consider continuing past Mt Herbert summit and decsending Monument Track and Kaituna Valley Road—that’s what I would try next time.

• I had less than a week until my move to Christchurch
• The weather was perfect
• I was fit enough to run a marathon, maybe
• It was Wellington Anniversary Day
• I had told a bunch of people I would do this before leaving Wellington

I had more or less experienced each individual section of the WUU2K route over the years so nothing would be unfamiliar. I opted for the marathon route as it’s a bit more sensible than the ultra (a more sensible route and also more sensible for my body).

Without overthinking it, then, the night before the run I asked my flatmate to give me a ride to the start line. He kindly obliged and before I knew it I was on the trail!

I mentally broke down the trail into four sections.

Section 1: Skyline Walkway (Runway?)

My favourite stretch of the run! After a quick walk to Mt Kaukau it’s lovely running along the undulating Skyline Walkway. In worse weather this track gets some awful wind, so be prepared for that.

Section 2: Makara Peak to Wrights Hill

Now for the best bush trail on the entire route. A solid downhill along a 4WD track then on to Lazy Fern downhill bike track. This was the dodgiest part of the marathon; during the event I’m guessing they close this track but in my case I had to remain vigilant for any bikes approaching from behind. Thankfully it’s blissful smooth trail and pleasant to get done quickly.

Then it’s up Salvation Track, a gentle uphill which is quite runnable. Cross the road near the top of the hill to continue along the trail.

Section 3: Zealandia fenceline

A simple but steep 4WD section to the turbine. The legs were really starting to feel it now and I began to fade a bit.

Section 3: Tip Track and Island Bay

Backtracking from the turbine but now on the other side of the road, the trail is good going all the way to Hawkins Hill. It was still slow travel on weary legs.

The tip track heads down to Happy Valley from here and is easy cardio-wise—but like a jackhammer on the knees.

Just before the Berhampore Golf Course there’s a compact but confusing mess of trails known as Tawatawa Reserve. I became lost here despite knowing I wanted to head to the golf course, despite having visted this area the week before and despite colour-coded trail maps throughout the park.

The official route from The Parade up to Mt Albert Road is also unclear. It appears the route heads directly up the hill. I think I found the intended track but it was so overgrown as to be unnavigable. I opted for the longer left track instead.

Section 4: Southern Walkway

I was familiar with this one. Mostly just follow the signs, but pay attention to the shimmy past the zoo: there’s no need to go up to Truby King park so just get onto the hill road as soon as possible. Further along, take the detour along the left of the velodrome, then it’s straight up to the summit. This entire section I wasn’t doing much running. Better nutrition probably would have helped.

I struggled to walk for the rest of the day, but the sense of achievement more than made up for it. How cool is it that this marathon route is almost entirely on good-quality trail close to town and only crosses three major roads? I’ll miss you, Wellington!

This was my second overnight tramp and the first I’ve organised myself.

Day 1

Mostly a slog up a staircase to Powell Hut. The hut sits just above the bushline. We glimpsed it from Rocky Lookout before the clouds rolled in. We had heard that the hut had a grand view out over the Wairarapa so it was a bit disappointing to arrive at the hut with only a few metres of visibility.

Day 2

Prayers be answered! I was woken up at 6am by footsteps on the balcony and vivid red sunlight streaming in through the windows. Venturing outside, I was greeted by sunrise between two layers of cloud. The streaks of shadow created by the hills and clouds below was a hauntingly gorgeous sight and one I won’t forget easily. It may well be the most beautiful sight I’ve ever seen.

We set off up the hill and then along the ridge towards Jumbo Hut. We had calm weather which allowed us to enjoy the views. We heard from some seasoned local trampers that such days don’t happen often up here.

We arrived at Jumbo Hut a bit after midday for lunch. It’s a significantly more crowded hut than Powell with only a single room. We knew the hut was booked out so relaxed around the hut for a couple of hours, waiting see if many people would turn up. Sure enough, more hut dwellers arrived as well as some campers. With plenty of daylight hours remaining, we decided to cut the tramp short and exit that same afternoon.

Immediately below Jumbo Hut is Rainguage Spur Track. It’s a steep, brutal mess of root steps, descending over 600m in less than 2km. Running down here would suck; if you’re running this loop as a half-marathon endeavour to do it counterclockwise so you get to run down the Powell Hut stairs.

Beyond this, we passed the secluded Atiwhakatu Hut and left along the trail of the same name.

I did the track on New Year’s Day to ring in 2021 with plenty of sweat and vertical gain. The weather was exceptional: sunny, calm and just the right temperature.

The track to the summit consists of three tracks: Orongorongo Track, Big Bend Track, and Mt Matthews Track.

Orongorongo and Big Bend Tracks

The first 8km is along these tracks. They’re great non-technical tracks so I decided to run them in both directions. This reduced the 9-10 hours return DOC time to around 6 hours. Follow Big Bend Track past Haurangi Hut to the junction with Whakanui Track. Then cross onto the riverbed and start looking for Matthews Stream to the true left of the river.

Mt Matthews Track

This is the meaty bit. With well over 750m of vertical gain in 5km, I decided to walk this section. It’s a straightforward track with tons of roots for steps. It levels off near the top. I didn’t see another soul in either direction.

If you’re short, the summit may be underwhelming; at only 941m altitude, the peak is forested and I needed to step on a log to see above the bush. Peeking above the trees, the view spans in three directions: Palliser Bay, Wellington, and Remutaka Range.

The return

I exited in similar fashion: walking down to the river, then running Big Bend and Orongorongo Tracks back to the car. I was exhausted and probably should have taken more food.

Climbing up Mt Matthews was a special experience—more special than I thought it would be. The mountain is visible from almost everywhere in Wellington and while descending the track, I pondered that fact: in a way, the mountain had been in the backdrop of my entire Wellington upbringing and clambering up its slopes had made me appreciate that fact. It somehow seems corny to write it, but here goes: I don’t think I have ever felt so connected to the land. This walk has left a lasting impression. If you’ve got a day to adventure in Wellington—and certainly if you live there—I hope you consider Mt Matthews Track and perhaps have a similar experience to mine.

Headphones are listed in order of when I acquired them.

Previous headphones

2010-2014: Sennheiser HD212 Pro

My first over-ear headphones, purchased second-hand as an upgrade from some old Panasonic on-ears. While I can’t remember how much I bought them for, it must have been a hell of a deal as I wore these things until they starting disintegrating. I remember being impressed by by the isolation and comfort from the soft oval pads. They had a very bassy sound signature but I can’t recall the details. I do recall enjoying listening to music through these, however.

2014-2015: GRADO SR125e

Perhaps it was my Sennheisers falling apart, or online research during a largely idle summer, but I found myself wanting to try something different. Open headphones seemed popular and promised a more impressive sound, so I picked up some discounted Grados and an Audioquest DragonFly DAC. More or less instantly I was disappointed by these cans. They were sublime for acoustic tracks (The Summer You Never Meant by The Exponents)… and nothing else. This was due to an incredibly airy upper-end which brought so much realism to strings and vocals. Lacking bass made them fall flat for electronic music. Build quality, cable and comfort were appalling also. I’ll never realise how people rave about these lower-end SR Grados. If you listen exclusively to soft rock at home and your ears are either tiny or numb I’d highly recommend these.

A clumsy entry into the world of nice headphones. I sold them a few weeks after purchase.

2015-2019: Audio-Technica ATH-M50xBL

I had actually tried these before the Grados and avoided them - but the Grado experience made me reconsider what I thought I knew. The M50x sound signature had no real surprises but was a clear upgrade from my old Sennheisers in terms of clarity and balance. Moreover, I suddenly appreciated the practicality in the M50x. Removable cables, foldable design, carrying bag - perfect! Plus these things paired so well with the DragonFly DAC, had very little sound leakage, and great isolation. I remember jamming to iconic EDM tracks (Afterglow by Wilkinson) through these things during study sessions throughout university. I bought the 2014 Limited Edition blue and tan model which looked sublime. Oddly, the tan-coloured pads seemed less comfortable than the black on the M50xs owned by my friends. A good number of those friends had purchased the M50x on my recommendation.

Tried and true, but I eventually became either sick or bored of their one-trick-pony sound signature which only seemed to sound great for electronic music.

2016-2019: Superlux HD668B

I heard some ravings online about this semi-open model. 55NZD from Massdrop was so tempting I just bought them. My reasoning at the time was something like: Maybe the “semi” in “semi-open” meant they’d be less bad than the fully-open Grados. They were certainly less bad but the comfort was horrendous. The sticky vinyl pads were large but shallow and my ears rubbed on the driver. I bought some 15NZD velour pads from ebay and they instantly became some of the most comfortable headphones I’ve ever owned. The cable design is ingenious, too: The earcups have a stubby male connector and the provided cables are simply extension cords!

The HD668B, helped by the plush new earpads, had an awesomely spacious sound and were my go-to gaming headphones. This soundstage helped with music rendition also, but unfortunately the high frequencies were just too aggressive to derive much enjoyment from the experience, even on the Topping D30/A30 stack I purchased around this time. Even for a dedicated gaming headphone though, these are quite the deal! I sold them for the same price nearly four years later but threw in the bonus pads.

2016-2020: Sennheiser/Massdrop HD 6XX

There was so much hype around the release of the HD 6XX. This is basically a full-fat HD650 at half the price, after all. A friend and I bought into the hype, each buying multiple pairs with the intention of keeping a pair and flipping the rest on the New Zealand market.

Frankly, the hype was well-deserved and remains to this day. I can’t say enough good things about the HD 6XX. I used it almost daily for five years and it didn’t skip a beat. The mid-heavy sound of these things is lush. Vocals take on this blissfully full and natural tone. High frequencies sparkle but don’t dominate. Even after getting used to the sound signature, I would reliably get goosebumps listening to certain tracks on the HD 6XX (Amandine Insensible by Sevdaliza, Going Home by Ásgeir). This is not to mention that the oval pads and light weight of this headphones provided the most comfortable listening experience I’ve ever had.

Bass was the main weakness of the HD 6XX in my eyes (ears?). It was there, but it was not sufficient to drive electronic music. I found it took on something of a one-note tone and never had much texture or impact to it.

Still, the HD 6XX provides a true high-end headphone experience. I’d recommend this or the HD58X for really taking a dive into awesome-tier headphones. Eventually I sold these to a friend. I can only hope this person derives some of the same joy from these bad boys. Hell, they’ll probably last at least another five years.

2017-2020: Sony WH-1000XM2

The problem with hi-fi headphones like the HD 6XX is that they’re really only good for home listening. There’s minimal convenience there. My theory was that a closed-back, wireless, noise-cancelling headphone would provide all the convenience I needed for when I wasn’t listening to my HD 6XX - plus it would add some more bass for electronic listening. The Sonys were pleasant. I found the sound quality nice enough: sometimes muddy but usually even more appealing than the ATH-M50x. The comfort was a mixed bag - they were hot and sweaty, especially on my walking commute. They also had a fair bit of heft and undersized earpads. These problems were mostly fixed in the XM3 and XM4, I believe. Maybe I should have waited. Luckily the Sonys are always in-demand and a co-worker bought them off me.

2019: Meze/Massdrop 99 Noir

I had been looking for ATH-M50x replacements in pursuit of a better home-listening experience for electronic music. The identical 99 Classics were selling for almost twice the price domestically. People online left very positive reviews praising a bassy sound signature that retained excellent detail and musicality.

Those people were wrong.

In a similar situation to the Grado SR125i, I found myself wondering how the hell these get such good press. The bass was overwhelming and bloated. The comfort was borderline. They were built attractively but seemed fragile. I bought replacement pads thinking I was going crazy - they didn’t help. Thank goodness the Mezes sold quickly second-hand, on TradeMe. I refused to sell them to anybody I actually knew.

2020: HiFiMan HE4XX

I felt ready to continue on my open-backed journey. I had heard about planar magnetic headphones and how they could provide some of the better open-backed bass. What if the HE4XX was the open-backed electronic-music headphone I longed for to live alongside my HD 6XX? I bit the bullet and purchased from Drop.

The HE4XX was less comfortable than the HD 6XX. The pads were plush but smaller, and the whole device was heavy. Unlike the Sennheiser it was hard to forget about wearing the HE4XX. The sound was surprisingly different to the HD 6XX. It was as if some of the mid-focused prowess of the HD 6XX had been traded for more textured bass and much clearer high-frequencies. I really enjoyed the HE4XX but I returned the headphone due to a notable driver imbalance.

Current headphones

2019-: Beyerdynamic DT770 Pro 80 Ohm

After the Mezes I was still on the hunt for a bassy-enough closed-back headphone to complement the HD 6XX. The reviews of the DT770 are all over the place, largely due to the three impedance versions which have their subtle differences. I settled on the 80 Ohm ones as they ostensibly have the most bass and least treble. I bought them from a UK supplier for a good price - well under what I paid for the M50x.

I really didn’t know what to expect from these, but overall I’m pleased. Pleased enough that I sold my M50x for these.

Upon unboxing I was struck with the indestructible build quality and awesome comfort. The bass on these headphones is gorgeous and just what I was after. With the DT770, it’s all about sub-bass. It’s cavernous, as if there’s no note deep enough to miss (I Found by Amber Run). The best part is the rest of the bass isn’t too bloated. Otherwise, the sound signature is more controversial. Vocals are consistently recessed and highs can be quite sharp - though the good soundstage does add immersion. The DT770 is most at home for electronic instrumental tracks, but is a surprisingly good all-rounder once you get used to the sound. Hell, it’s not bad for gaming, either.

2020-: HiFiMan Sundara

The HE4XX experience got me thinking. I really liked the tradeoff compared to the HD 6XX.

1. The top-notch detail afforded by the more present high frequencies was undeniable. I enjoy treble that’s crispy but not overwhelming and HE4XX had nailed that.
2. If my open-backed pair could provide enough bass for electronic listening, maybe I could get sell the DT770. I enjoy downsizing after upgrading.
3. The path to improve the HD 6XX involves purchasing expensive subjective audio gear like tube amps and the like, to “improve” the sound by adding imperfections to the audio chain. Would something like the HE4XX be the best choice on my more analytical chain?

I decided to go up a level, purchasing the higher-end Sundara, the bigger brother of the HE4XX/HE-400i. Spending 580NZD on a headphone was a big purchase. To help justify it, I set the expectation that this would replace my HD 6XX, not live alongside it. A high bar, but I wasn’t disappointed.

The Sundara solved many of the issues with HE4XX. A better headband, a smaller and lighter design, and slightly comfier earpads were highlights. And the sound is astonishingly enjoyable. Somehow it manages to be neutral but fun. High frequencies are emphasised somewhat, leading to tons of beautiful detail. Guitar strums and vocal consonants are always clear (Should Have Known Better by Sufjan Stevens). Mids are less rich than the HD 6XX but plenty present: vocals seem to surround your head as opposed to caressing your eardrums (Watch Me Read You by Odette). The bass is noticeably nicer than the HD 6XX. It has texture and impact - nothing dominating (I’d still love a touch more) but enough to make electronic music pleasant to listen to (Disarm by Alltta).

On poorly-produced tracks, the Sundaras won’t help: they’ll sound flat and boring. Conversely, well-produced tracks will be presented in all their glory (Good News by Mac Miller, Year of the Cat by Al Stewart).

The Sundaras dethroned my HD 6XX. They’re not better in all ways but for me are definitely better overall. I’ve barely touched my DT770 since using the Sundara; I think I know which headphone to sell next.

2020-: QCY T5

True story: These 40NZD true wireless earbuds replaced my 420NZD Sony WH-1000XM2. I realised my Sonys mostly delivered the convenience I wanted but at the expense of large size and sweaty earpads. I wondered: what would happen if I maximised convenience even at the expense of sound quality? The QCY T5 sounds alright, with super nice controlled bass even if the highs can be a tinny at times and there’s a slight buzz. But the convenience is extraordinary. They fit in a pocket, I can choose to use only one earbud, and the noise isolation is decent. Absurdly good for the price.

Next steps

The Sundara is well in the zone of diminishing returns; I don’t see myself entering into the zone of super-high-end headphones any time soon. Rather, I’ll probably sell my DT770 and upgrade the QCY T5 to higher-end true wireless earbuds.

Reflections

It’s not so much headphones that I find enjoyable, it’s finding different ways to experience the music I listen to. I enjoy owning a headphone for a while then selling it as I upgrade. I don’t want to become a hoarder (refer Zeos’ wall 2020). I appreciate that I’ve been able to experience music through all of these headphones over the years!

Switching to WLED over-the-air (OTA)

Migrating to WLED is super easy using the OTA features of ESPHome combined with the pre-compiled binaries available for WLED.

1. Enable the web_server for the ESPHome device (if it isn’t already). Do this by adding the following to the device config and uploading:

   web_server:
     port: 80 # Or your preferred port
   

2. Download the relevant binary from the WLED releases page. All ESP32 and many ESP8266 devices are supported - you’ll probably want the _ESP32.bin or _ESP8266.bin files respectively. My Electrodragon ESP LED boards use an ESP-12F module which is supported.

3. Navigate to the webserver of the ESPHome device using either IP address or mDNS name. Under the “OTA Update” section a “Choose file” button should be present.

   

4. Click “Choose file”, select the WLED binary, then click the “Update” button.

5. Now you’re ready for WLED setup, starting at step 3 in the official quick start guide.

Reflections on the migration

I was looking for an alternative to ESPHome due to some major bugs which were introduced with later versions. Most notably:

• Transition time was ignored so all transitions were the default duration.
• One of my strips would sometimes glitch out and flash random, strobing colours. This happened once or twice a week, especially when effects were running.

Some really compelling features attracted me to WLED.

• Home Assistant integration
• Hyperion/real-time support (that’s one for a separate post)
• Customisable white-channel behaviour
• Gamma correction for colour but not for brightness
• Effects are more intuitive (they respect the colour and power state of the strip)

I’ve been pleasantly surprised by WLED so far. The only real loss from ESPHome is the file-based configuration. What impresses me the most is how scalable the software is: It has everything you need for basic control via the app or web interface - even timers and macros are baked in. But it also has great support for integration in larger systems with Home Assistant, MQTT, and Hyperion control. Because of this, WLED is great for anyone from LED strip first-timers to experienced home automators.

At this point, my recommendation for almost any LED setup is simple: Buy a SK6812 LED strip and the correct power supply. Control it with a Electrodragon ESP board running WLED.

A couple of weeks ago I was given a Kogan smart plug. This controls a wall outlet over WiFi and measures power draw. It’s a feature set similar to the Sonoff Pow R2 which I’m currently using to control my PC. It has a couple of notable benefits over the Pow R2:

• It is sold officially in New Zealand
• It requires no mains wiring

The Kogan smart plug is manufactured by Tuya, a company making IoT devices on behalf of many brand names around the world. Thanks to a security vulnerability in Tuya devices, they’re known for being easy to reflash over-the-air with a tool called tuya-convert. If you’ve read many of my previous posts, you may be aware that I like ESPHome and that I don’t like brands telling me what to do.

With that in mind, I set out to install ESPHome on my Kogan smart plug. Here’s how I did it using tuya-convert.

This could totally brick your device. Follow these steps at your own risk and be aware that this is not a substitute for the official guides linked below!

Unsure if your device is made by Tuya? Check out the tuya-convert wiki to ensure your device is supported.

Flashing the device

Getting ESPHome flashed was surprisingly easy and just a matter of following instructions on the relevant guides.

1. Build an ESPHome binary

Before flashing, we need a flashable binary file for the device. For ESPHome, this means creating an entity and compiling a binary. To do this I mostly just followed the official guide so read that for more details. But the process for me boiled down to this:

1. Go to ESPHome dashboard
2. Create a really basic config with WiFi and OTA enabled. (This is so that there’s enough information to build a firmware file and enable updating. The exact device configuration will be fleshed out later.)
3. Compile
4. Download binary

2. Flash it

I used tuya-convert in Docker on a Linux laptop for this process. For this you’ll want to follow the tuya-convert Docker guide. Here’s a summary:

1. Clone the tuya-convert repo

2. Copy your ESPHome binary to the files directory. (You’ll need to do this before the Docker build as for some reason the binaries are copied to the image rather than mounted in the container by default.)

3. Build the image

   docker build -t tuya .
   

4. Copy the docker-compose template

   cp docker/docker-compose-sample.yml docker-compose.yml
   

5. Open the Docker Compose file and ensure the WLAN environment variable is the same as the one on your PC. (ip link will show all interfaces; I had to change mine from wlan0 to wlp3s0).

6. At this stage I needed to manually disconnect my laptop from my home WiFi network

7. Follow the instructions to run tuya-convert

   docker-compose up -d
   docker-compose exec tuya start
   

8. The remaining instructions will be displayed in the command line

Getting it working with the new firmware

At this stage the Kogan plug has been flashed with a minimal ESPHome install. It should show up as “online” in the ESPHome dashboard. The good news is that it’s free from the manufacturer’s firmware! The bad news is that the minimal firmware has no real functionality so the device won’t actually do anything.

Make a configuration for the device

You’ll need to build up the ESPHome configuration with components to get it working like it did before. That requires you to know which pins on the ESP8266 do what inside the device. There are a few ways to obtain this information.

Look on the ESPHome Device Configuration Repository

This is the easiest option. The repository is available at esphome-configs.io. If your device has a known config, it should be present in the relevant section. I managed to find the Kogan smart plug here. Be sure to carefully copy the relevant parts of the config from the website to your ESPHome configuration. Then upload it over-the-air to the device.

Look on the tuya-convert wiki

If there are no known ESPHome configs for your device, you’ll need to make your own from scratch. This isn’t too hard if you know the pin numbers for the device. The tuya-convert wiki contains a list of all supported devices. Many have GPIO pins specified. You won’t be able to copy and paste a config to ESPHome, but the site will at least show how pins are configured for that device. That makes it much easier to build a configuration yourself.

Look on the Tasmota Devices Template Repository

At templates.blakadder.com there is a repository of devices for the popular Tasmota firmware. Look up the device and it will show which pins are used for what.

Work it out by trial and error

If you don’t want to open the device, I’m sure it wouldn’t hurt to try some pins. Look up similar devices using the sources above and try the pins for that device by writing a config which uses your best-guess of pin numbers. That could provide a good starting point. As always, Googling and poking around forums is probably helpful also.

Look inside the device

Luckily I didn’t get this far. If you still don’t know the pins for your device, the surest way to determine them is probably to disassemble the device. It should be possible to work out which pins are which by tracing outputs and inputs to the ESP-8266 board inside.

Get it integrated into the home automation system

ESPHome is flexible and supports tons of interaction methods. I’m using the Home Assistant integration but there are many other options. Getting it all hooked up is beyond the scope of this post so check out the ESPHome docs for more information.

Reflections

The whole process was easier than I expected and works marvellously on my Kogan switch. So far it seems like a good way to get my desired firmware running on locally-purchasable hardware without any signs of tampering. Now I want to buy the cheapest Tuya lightbulbs I can find and repeat the process with those.

1. Build an artifact using a full-fat build image
2. Move it to a minimal runtime image where it can be executed

Here’s a simple example. I have a TypeScript file called helloworld.ts. It greets people. I want to compile this file to JavaScript and run it with node. Here’s a typical two-stage Dockerfile that will achieve that:

FROM node:14-buster as build
WORKDIR /project
COPY package.json .
COPY package-lock.json .
RUN npm i

COPY helloworld.ts .
# Build to dist/helloworld.js
RUN ["npm", "run-script", "build"]

FROM node:14-buster-slim as runtime
COPY --from=build /project/dist/helloworld.js .
ENTRYPOINT ["node", "helloworld.js"]

I can use this Dockerfile to build an image and run it as a container quite easily:

docker build -t helloworld .
docker run --rm helloworld Albert
# Hello Albert!

Neato! So what’s the problem?

The problem

I don’t always want to run the thing I’m building! Often (but not always) I just want to obtain the build output. This can be useful for uploading the artifact - such as for a GitHub release - and is achieved by adding an extra step to the process:

1. (As before) Build an artifact using a full-fat build image
2. Move it to a scratch image where it can be exported
3. (As before) Move it to a minimal runtime image where it can be executed

But how to add this second step to the Dockerfile?

FROM node:14-buster as build
WORKDIR /project
COPY package.json .
COPY package-lock.json .
RUN npm i

COPY helloworld.ts .
# Build to dist/helloworld.js
RUN ["npm", "run-script", "build"]

# highlight-start
FROM scratch as output
COPY --from=build /project/dist/helloworld.js .
# highlight-end

FROM node:14-buster-slim as runtime
COPY --from=build /project/dist/helloworld.js .
ENTRYPOINT ["node", "helloworld.js"]

As before, I can build and run the script easily enough:

docker build -t helloworld .
docker run --rm helloworld Albert
# Hello Albert!

But now I’ve got a new trick up my sleeve for extricating the build output from the container:

DOCKER_BUILDKIT=1 docker build -t helloworld --target output -o dist .
ls dist
# helloworld.js

The key here is using a build target in combination with the -o flag. This flag is enabled by activating BuildKit using the environment variable. With -o, Docker will copy all the files from an image’s filesystem into a specified folder once the build is complete. Using a scratch image for the output stage means that we’re left with only the single artifact from the build stage.

The best part of this process is that I know that the artifact I’m building is exactly the same as what would end up in the final running Docker image. It minimises the number of miscellaneous build scripts or Dockerfiles I would otherwise need and keeps all the build steps contained in a single source of truth.

This approach is compatible with Docker-based CI/CD solutions such as GitHub actions; Docker-out-of-Docker can be used to run these build commands within a docker Docker image. It’s all a bit ridiculous, but works well.

Where have I used this?

I took advantage of a three-stage Dockerfile in mqcontrol where it’s being used for cross-compilation binary output. That project also has a special requirement in that there are multiple runtime base images depending on what commands mqcontrol will run. That makes it a perfect candidate for a separate build output stage.

Additionally, I’ve made a minimal repository for the examples above. Find it at albertnis/demo-three-stage-dockerfile. Check out the actions which automate releases and packages all using the one core Dockerfile for building.

The challenge is: how do I make the computer respond to the message and instigate a shutdown?

This is not a new challenge, and there are personal-project utilities all over the place which achieve the goal. Previously I was doing this with software called MqttClient, a .NET GUI which ran on Windows. Now that I’m splitting my time between multiple operating systems, I wanted a small, cross-platform utility which could run quietly in the background and subscribe to shutdown requests.

Introducing: mqcontrol

mqcontrol is yet another MQTT remote control utility. It’s small, cross-platform, CLI-only and availble as a Docker image. The source is available on GitHub.

How I’m using mqcontrol

Remote hibernate as part of “safe off” script

I’m using mqcontrol as part of a computer shutdown script. When I activate the script, Home Assistant does the following:

1. Send the command to hibernate the computer
2. Wait for the computer power draw to drop below 20W
3. Switch off the computer at the wall after a delay

Here’s a basic version of what that script looks like in Home Assistant YAML configuration:

computer_off_safe:
  alias: Turn computer off safely
  sequence:
    - service: mqtt.publish
      data:
        topic: "flat/abedroom/pc/hibernate"
        payload: ""
    - wait_template: "{{ states('sensor.computer_power')|int < 20 }}"
      timeout: "00:02:00"
      continue_on_timeout: "false"
    - delay:
        seconds: 2
  - service: switch.turn_off
    data:
      entity_id: switch.computer_plug

mqcontrol runs on the computer and listens for the hibernate command.

On Linux I use systemd to run the following command at boot:

mqcontrol -c "systemctl hibernate" -t flat/abedroom/pc/hibernate

On Windows I use task scheduler to run an analogous command on login:

mqcontrol.exe -c "shutdown /h /t 0" -t flat/abedroom/pc/hibernate

This way it doesn’t matter which OS I’m booted into; the command will always cause a hibernate.

Learnings

For a small one-weekend project, I managed to learn a ton while developing mqcontrol. I’ll be writing more about these for sure!

• Basics of Golang and its tooling.
• Docker BuildKit and three-stage Dockerfile to use build stage for both artifact generation and runtime image creation.
• Docker Buildx to build and push cross-platform Docker images - but using Go cross-compilation on the build platform to eliminate the need for expensive emulation.
• GitHub Actions and Docker-out-of-Docker for releases based on Git tags and automated CI/CD.

In this post I want to look at two ends of the effort spectrum when it comes to building a smart home system: “off-the-shelf” and “do-it-yourself”. I want to show why the low-effort option has some drawbacks, and why building a more customised smart home system yourself can be rewarding if you put in the effort.

Architecture

Here are what the systems look like. As an example, the smart home has the following devices in both cases:

• A Xiaomi Aqara hub plus two bulbs, temperature sensor and a button
• A Philips Hue gateway with two bulbs, and LED strip and a dimmer switch
• Two Wemo smart plugs and a Wemo light switch

The “off-the-shelf” smart home

You go to an electronics store and buy the components. After setting things up according the manuals you’ll end up with the following:

This is the typical “default” smart home setup. What you’ll notice here is that access to the devices is separated by brand (represented by blue rectangles). For device setup and control you’ll have to use the specific app or website for the device in question.

The only place where all the devices can be controlled together is in Google Assistant or Alexa. This means that to automate devices across brands, you’re forced to use the automation functionality inside Google Assistant or Alexa. For example, you might want the Wemo light switch to control the Hue LED strip. This automation functionality is dictated by Google or Amazon; it may or may not be sufficient for your needs.

For devices which do not operate on WiFi, you’ll need separate, brand-specific gateways. This can be seen for the Xiaomi and Philips systems. Both ecosystems use Zigbee for communication, but will require different hubs to communicate with the respective servers over the internet.

The “do-it-yourself” smart home

With a home server and some learning, the system can be built more like the following:

Crucially, unlike the off-the-shelf system, access is controlled by domain (represented by purple rectangles). This means that the app you open is not determined by who made the device; it’s determined by what you want to do. These domain-specific services each run as a web server, shown by the blue boxes. In my case, these are Docker containers all running on a single server and defined in a single docker-compose file.

• Want to setup, monitor or change the state of devices? Open Home Assistant. States are all managed there - with its myriad integrations, it works with almost all major smart home brands.

• Want to integrate custom WiFi devices which don’t work out of the box? Use MQTT with mosquitto. Zigbee devices like the Aqara and Hue devices earlier can share a single USB gateway which talks to mosquitto via a service called zigbee2mqtt.

• Want to configure powerful automations within Home Assistant and beyond? Node-RED is great for that.

• Want to save time-series data, graph, and query it? Prometheus and Grafana are an awesome duo enabling just that!

• Want to use voice to control devices? Google Assistant or Alexa are perfect.

Comparing the two systems

Advantages of the off-the-shelf system:

• Easy setup
• No need for own server

Advantages of the DIY system:

• Vendor lock-in is eliminated
• Apps and websites for interacting with the system are separated based on use case (“domain”), not brand
• Individual domains can be built out as desired, not limited by vendor support for various domains
• Support for open source or custom software running on IoT devices
• Data is stored in fewer locations
• Data sent to the internet can be eliminated or controlled
• Can usually be set up to retain access through vendor apps and services if desired

Why I think the DIY system is worth it

The default smart home setup is exactly what manufacturers want you to have, and it’s a mess. Vendor lock-in is my main concern with the smart home industry. If you have some Hue lights and want to buy a switch, you’ll avoid a cheap Aqara switch because that would require a second hub - even though both systems use Zigbee and are physically interoperable. You’re locked in to the Hue ecosystem and will most likely opt for a Philips device.

In my opinion, the bottom line is this: as a user, you shouldn’t have to care who made your smart home device. You should be able to pick the best device for your needs and budget and integrate it into the rest of your system. Software like Home Assistant has made this dream realisable with a low barrier to entry.

What you need to get started

All you need is a home server and some smart home devices! For a server, I’d recommend starting small. An SBC such as Raspberry Pi or Odroid-XU4 is a great choice and lets you get started for under $100. I’m currently running a second-hand Intel NUC, which is more powerful - also recommended.

In terms of software, my suggestion is Home Assistant. For an SBC you can download and install Home Assistant as a SD card image. If you want a more flexible and modular solution, I would highly recommend using Docker and docker-compose.

Search for the Home Assistant integrations for your devices and get them set up. That’s the basic work done! The next steps are up to you - pick a domain you’re interested in, learn about it, and build it yourself.

The boardwalk/bridge across upper Lake Kohangapiripiri closed for maintenance shortly after I did this walk. Please double check that it has reopened before attempting.

The device in question is a button which lives beside my bed. I use it for turning off my lights at night and for toggling my PC at the wall. It’s WiFi-connected and uses the same brilliant ElectroDragon board as my LED strips. I made some firmware for it a while ago which was based on lightt, my firmware for LED strip control. It kind of sucked. Crucially, it lacked any debouncing - so single clicks on the cheap button would register as double clicks about 30% of the time.

I started implementing debouncing in the codebase, but quickly realised I should just move to ESPHome. The problem? ESPHome doesn’t have an all-in-one multi-click button component! Luckily the platform is flexible enough for me to implement one myself.

Desired behaviour

Here I was inspired by zigbee2mqtt and its implementation of multi-click behaviour for a Xiaomi Zigbee button which I migrated recently. Upon a series of clicks, the Home Assistant device emits an event with a payload corresponding to the number of clicks (e.g. “single”, “double”) immediately followed by an empty state "". I like this representation of an instantaneous state, so decided to follow this behaviour for consistency’s sake.

Implementation

The implementation of the button in ESPHome consists of two key components: a text sensor and a binary sensor.

Text sensor

This is responsible for emitting the correct state text, and is the part visible to the “frontend” - Home Assistant, in my case. It’s basically a default sensor but with one twist: when the state is set to a non-empty string, the sensor will set itself back to a default empty state as per the desired behaviour.

text_sensor:
  - platform: template
    name: 'Bedside button'
    id: bedside_button
    icon: 'mdi:toggle-switch'
    on_value: # When a state is set
      then:
        - if:
            condition:
              text_sensor.state:
                id: bedside_button
                state: ''
            else: # If non-empty
              - delay: 20ms
              - text_sensor.template.publish:
                  id: bedside_button
                  state: !lambda 'return "";' # Reset to empty

Binary sensor

The binary sensor is the brains of the operation. It reads the actual state of the pin across which the switch is connected. The switch contains some filters: invert makes the logic more intuitive; delayed_on and delayed_off provide basic debouncing. The real logic lives in the on_multi_click section. In each case, the text sensor is invoked to set an externally visible state. Crucially, this sensor is internal because its only purpose is to manipulate the text sensor.

Here’s a config which has a single click and a hold:

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO5
      mode: INPUT_PULLUP
    id: raw_switch_state
    internal: true
    filters:
      - invert:
      - delayed_on: 30ms
      - delayed_off: 30ms
    on_multi_click:
      - timing:
          - ON for 40ms to 400ms
          - OFF for at least 50ms
        then:
          - text_sensor.template.publish:
              id: bedside_button
              state: !lambda 'return "single";'
      - timing:
          - ON for at least 2s
        then:
          - text_sensor.template.publish:
              id: bedside_button
              state: !lambda 'return "hold";'

I’ve put a more complete config up on Gist. It contains definitions for double- and triple-click, as well as components for the switch’s built-in LED.

Testing

Once getting the config uploaded and added in Home Assistant (see my LED strip migration for more details) I can see the externally-visible components of my switch in Home Assistant via the Devices screen. Great!

To test the states, I’ll load up Node-RED with a classic debugging flow:

When I press the button once, the following state changes are logged:

That’s perfect! Look at the payload attribute. The state is set to “single” before being cleared almost instantly.

Automation

Now it’s very easy to add the desired automations in Node-RED. I just create state event nodes, restricting to the bedside button entity and the state string desired:

Come the morning, we can… saturate our indoor environments with powerful blue light that shuts off any lingering melatonin. This will help us wake up faster, more alert, and with a brighter mood, morning after morning.

Sounds desirable, right? I’ve been using my bedroom LED strips as an wake-up alarm light for years now. After reading the paragraph above, I was inspired to make a couple of improvements to the system.

Blue light alarm

This is the easy bit! Previously my wake up light had a few phases. The idea was to simulate a sunrise - the lights would shift from off to orange to white. I simplified it such that now they just turn on to melatonin-busting blue. Here’s an example of my new wake-up script in Home Assistant. It’s not rocket science and simply wraps a light.turn_on service call.

script:
  wake_up:
    alias: Wake up
    sequence:
      - alias: Bedroom lights to blue
        service: light.turn_on
        data:
          entity_id: light.bedroom_lights # Group light which targets all lights in the room
          brightness: 255
          rgb_color: [0, 20, 255] # Blue, more or less
          white_value: 80 # Add some white channel to make it more natural
          transition: 300 # Fade on over 5 minutes

Calendar-based scheduling

I used to hardcode the time for triggering my light alarm. A simple automation would trigger the script at 7:15am on weekdays and 7:45am on weekends. That worked find 95% of the time, but didn’t properly account for mornings where I needed to wake up earlier or wasn’t at home.

Setting up the integration

I decided to leverage the Google Calendar integration in Home Assistant. The aim is to have the wake-up light activate when the first event of the day starts. The full setup is easy enough to follow (see the Home Assistant docs) and boils down to a couple of lines in the configuration file:

google:
  client_id: !secret google_client_id
  client_secret: !secret google_client_secret

Event strategy

By default, the Google Calendar entity is a sensor that is either “on” (event in progress) or “off” (event not in progress). The docs have some interesting options for tracking events such as sensor offsets and filter keywords, but I kept mine basic because I want my calendar to remain human-readable and simple.

Next, I created a repeating daily event for a morning routine. This establishes a “default” wake-up time. The best part is that now I can change up my scheduling when I have something early - or delete the alarm altogether if I don’t want the lights to turn on. Here’s what my calendar mornings look like:

Triggering the alarm

So far, the wake-up script is set up and so is the Google Calendar integration. Time to stitch them together with some automation goodness! As always, my go-to- for automation is Node-RED. I’m using the time range switch and Home Assistant websocket integrations.

Here is a simple subflow which does the trick. The wake up light will be triggered by the first calendar event which begins between 4am and 10am. Awesome!

Bonus round: Using Google Assistant to change the alarm time

Now that the system is hooked into the Google ecosystem, it’s really easy to add a calendar event using Google Assistant which will trigger the lights early using. It’s a one-liner:

Hey Google, add a calendar event: “Wake up early for my flight” tomorrow at 5:30am.

That’s almost as easy as setting a traditional alarm with Google Assistant! As for moving or deleting existing events, I haven’t worked that one out yet.

How can such a traditional speaker setup be automated to deliver some more smarts?

With an IR blaster and some setup, that’s how! Here’s what I’m trying to achieve:

• Amp is on when needed and off when not needed
• Amp is always on the correct input

If I can achieve these things then I’ll never have to reach for the remote again!

Hooking the system into Home Assistant

Power state

It’s important to be able to turn the amplifier on and off. Enable power control and state with a Broadlink switch within Home Assistant. Find the IR packet for your amplifier’s on and off commands. Then register it with something like the following yaml:

switches:
  - platform: broadlink
    host: 192.168.1.100
    friendly_name: Bedroom Amplifier
    mac: '4A:E8:E6:9D:51:A6'
    switches:
      amp:
        friendly_name: Amplifier
        command_on: JgBIAAABKpEVEBU1FRAVEBU1FRAVEBUQFRAVERQ1FRAVNRU1FRAVEBU1FRAVEBUQFRAVEBUQFRAVEBU1FTUVNRQ2FDUVNRU1FQANBQ==
        command_off: JgBIAAABKpEVEBU1FRAVEBU1FRAVEBUQFRAVERQ1FRAVNRU1FRAVEBU1FRAVEBUQFRAVEBUQFRAVEBU1FTUVNRQ2FDUVNRU1FQANBQ

This should appear in Home Assistant as below:

Pressing on and off should send the correct commands and toggle the amplifier!

Input state

Here’s where things get a bit harder. How can the input be changed? In my case, I want to switch between computer, Chromecast and Bluetooth inputs. Because there’s no state feedback from the amplifier, we need to keep state in Home Assistant. My amplifier has a single button to cycle through the states. Here’s how I tackled this problem.

I have two input_select entities in Home Assistant. They’re basically dropdown menus with a list of options. One is the full list of five inputs of the amplifier and will reflect the current input of the amplifier. The second is the target list - a subset of the options from the full list containing only the three desired input options.

input_select:
  amp_input_current:
    name: Input source (actual)
    options:
      - Bluetooth
      - AUX
      - USB
      - OPTI1
      - Chromecast
      - COAX
    initial: USB
    icon: mdi:video-input-component
  amp_input_target:
    name: Input source
    options:
      - Bluetooth
      - Chromecast
      - USB
    initial: USB
    icon: mdi:video-input-component

Now add a script to select the next input. The script will turn on the amp, broadcast a “next input” command then update the current input state accordingly.

amp_next_input:
  alias: Next amp input
  sequence:
    - condition: state
      entity_id: switch.amp
      state: 'on'
    - service: broadlink.send
      data:
        host: 192.168.1.100
        packet:
          - JgBYAAABKZIVEBU1FRAVEBU1FRAVEBUQFRAVEBU1FRAVNRU1FBEVEBU1FDUVNRURFBEUERQRFBEUERQRFBEVNRQ1FTUVNRU1FQAFjwABKkgUAAxZAAEpSBUADQU=
    - service: input_select.select_next
      data:
        entity_id: input_select.amp_input_current

While we’re at it, add a simple script to send the command to the amplifier without updating the state. This is useful if the amplifier becomes out of sync with Home Assistant.

amp_next_input_naive:
  alias: Correct input
  sequence:
    - service: broadlink.send
      data:
        host: 192.168.1.100
        packet:
          - JgBYAAABKZIVEBU1FRAVEBU1FRAVEBUQFRAVEBU1FRAVNRU1FBEVEBU1FDUVNRURFBEUERQRFBEUERQRFBEVNRQ1FTUVNRU1FQAFjwABKkgUAAxZAAEpSBUADQU=

Add the power switch, target input and correction script to the UI and it’ll look a bit like this:

But nothing happens when the input is changed. Now it’s time to automate the link between the input selector and the next input script.

Automating input selection with Node-RED

When the target input source is changed, I want to keep calling the “next input” script until the actual state matches the target. Here is the flow I’m using in Node-RED:

Upon a target change, the target is stored in the flow for future comparison. We check to see if the amplifier is on - no use trying to change the input if it’s off! Then we get the current input from Home Assistant. If it doesn’t match the target, we switch to the next input, wait some time for the amplifier to catch up, then repeat the process. Go ahead and try changing the input from the Home Assistant UI, then watch the amp play catch-up.

Automations

Here are some automations which have worked well for me over the last few months. I’ve been using the amp in conjunction with my PC, Chromecast Audio and Google Home.

Go to target on amplifier power-up

When the amp turns on, wait for the amp’s power-up routine, then get the target state and trigger the input selection automation.

Turn on with the computer

When the PC is turned on at the wall plug, wait 25 seconds for the boot. If the amplifier isn’t on, turn it on and set it to USB. This ensures that if the speakers are already in use then there will be no interruption or input switching. When the PC turns off, power off the amplifier if it’s switched to USB input.

Turn on when casting starts

If the Chromecast starts playing, turn on the amp and set it to Chromecast input. When the Chromecast becomes stopped or idle (ie not playing or paused), wait a few seconds in case it’s just a change of track or playlist. Then turn off the amplifier.

Instant smart speakers right there! Ask Google Assistant to play a song through the Chromecast and the speakers will magically turn on and switch to the correct input.

Reflection

In this project, I learnt that - with a bit of automation - even old and/or “dumb” equipment can be made to be more convenient. I use these automations every day - it’s nice to be able to ask Google to play music on the speakers or boot the computer without having to think about what the amplifier was last being used for. I haven’t touched the remote for a long time. Best of all, I barely need to open Home Assistant to correct the input manually. That’s what’s automation’s all about: removing repetitive actions and genuinely making things faster and easier.

1. Use a small 1-inch OLED screen with a microcontroller.

2. Buy a tablet, nail it to a wall and load up the Home Assistant frontend.

Personally, I love the simplicity of the first option. I just want a screen to read out the weather in the morning and show me what’s playing on the Chromecast. Nothing fancy. I also like the OLED aspect - this way, I can leave the screen “on” but black by default and will appear to be off. But why settle for a cheap 1-inch display when I have an old Nexus 6 lying around? To solve my problem, I built mq-display.

Mq-display is a React frontend which allows any web browser to behave as an IoT connected display. The site subscribes to a specified MQTT broker and listens for messages. Incoming messages can have a title, brightness, display duration and markdown content. If you’re interested in setting it up for yourself, check out the GitHub repo.

Integrations

I’m hosting mq-display and mosquitto on a home server and accessing the frontend from an old Nexus 6 with the screen turned to maximum brightness and auto-locking disabled. Because of the OLED panel, the phone emits no light when it’s showing a blank black page. As a browser, I’m using WallPanel. This is a full-screen Android browser recommended by some members of the Home Assistant community.

Now playing

It’s great being able to ask Google Home to play music on my Chromecast Audio. I use the feature daily. Incredibly, the system has no great way of telling you what song is playing (in the streaming world, maybe people don’t care). Usually when I want to find out the song I open up home assistant on my phone to see. That ends with mq-display.

Whenever my Chromecast Audio changes state, I check to see if the device has starting playing or has changed track. If that’s the case, I want the display to show track information. So I format it with a template and broadcast it the to the virtual/screen/nowplaying topic. If the Chromecast stops playing, I broadcast an empty message on the same topic, which causes mq-display to remove that screen. The Node-RED automation looks something like this:

And just like that, gone are the days of scrambling to find what song that is.

Weather report

Every morning, I use my phone’s browser to check what the weather’s going to be like for that day. That’s important for answering several Important Morning Questions:

1. What should I wear?
2. Can I fly drones today?
3. How wild is ultimate frisbee going to be?

Suffice it to say my weather report checking is very predictable. Therefore I can automate it.

Every morning when I get out of bed (that automation is one for another post), I set Node-RED to get the latest weather information from Dark Sky and my outdoor temperature sensor, all via Home Assistant. It aggregates a weather summary, forecasts, wind and rain information, and puts it all in a nice little summary to show on my display. The Node-RED automation is a bit more complicated for this one, but it’s repetitive. Basically I grab various bits of data from Home Assistant then append them to the end of a message. I set the message to disappear after a few minutes.

Reflections

I enjoyed mq-display as a project because it fills a niche which I can really benefit from. Hopefully other people can also see the use-case for such a project too. For me, home automation is all about being able to make repetitive behaviours effortless. I’m already finding the system useful for reducing the number of times I pick up my phone to get weather or track information; I’m sure I will find even more opportunities for automation in the future.

Next steps

As a frontend-only application, I see no reason why I can’t host mq-display in one place on the internet to be used by all. I even set up a build on Netlify. The problem: Netlify (and, well, most of the modern internet) serves traffic over HTTPS and therefore unencrypted websockets will not work properly in most browsers. Fixing this would be great, though I’m not sure if many home users have set up encrypted websockets on their brokers.

Some more fields could be quite nice for the payloads accepted by mq-display. I’m already imagining image support (cover art or camera feeds could be awesome) and maybe features for icons and key-value pairs.

I’ve been working on a React project called Obfuscator. It’s a rewrite of an older project where you can enter a phrase to translate through multiple languages in sequence. Essentially it’s a fun way to make a computer translator play Chinese whispers with itself.

With the product well-defined and my React knowledge workable at this stage, there was something else I focused on with this project: infrastructure. I wanted the entire thing to be serverless, cheap and scalable-to-zero. Single-page applications (SPAs) with serverless APIs and data stores are nothing new. But I realised nobody was running React one their Lambdas. Surely that’s a missed opportunity! So what did I do?

I ran React on a Lambda.

Code for Obfuscator can be found on GitHub

API Gateway

The infrastructure of Obfuscator looks like this:

It basically boils down to an API Gateway REST API with three resources:

• S3 Proxy: An S3 bucket contains static resources, namely the Javascript and CSS bundles. Normally with SPAs an HTML file would live in here also. But not in this case!
• /translate: Points to a Lambda which runs my translate API. This is a simple lambda which chains calls to Amazon Translate, all bundled in validation logic.
• /: The root resource is what serves the HTML when visiting obfuscator.albertnis.com/. Normally you’d want this to point to a static HTML file à la index.html. But in this case it points to a Lambda which serves up HTML based on the results on a React render.

The Application Code

The React root component here is called <App />. The fun part is how we include this component. Backend and frontend have separate ways to “reach” the App component - beyond that it’s shared code all the way down. That’s the magic of server-side rendering (SSR).

Backend

The code is pretty typical of an SSR app: it renders the app as a string and puts that string in a larger HTML template. In this case the logic gets wrapped into the Lambda entrypoint function:

// server.js
import view from '../../backend/view'
import ssr from '../../backend/ServerClient'

// Lambda entrypoint
export const server = (event, context, callback) => {
	// Get string representing app HTML
	const { content, state } = ssr()

	// Inject app HTML into full HTML template
	const page = view('Obfuscator', content, state)

	callback(null, {
		statusCode: 200,
		headers: {
			'Content-Type': 'text/html'
		},
		body: page
	})
}

The server client serves as the base React element wrapping <App />.

// ServerClient.tsx
// [...]
const ssr = (): { content: string; state: AppState } => {
	let content = renderToString(
		<Provider store={store}>
			<App />
		</Provider>
	)
	let state = store.getState()
	return { content, state }
}

Here’s the result of the Lambda run from the the console. The rendered app is clearly present:

Frontend

The client has a simpler structure with a simple client wrapper being used:

// Client.tsx
// [...]
hydrate(
	<Provider store={store}>
		<App />
	</Provider>,
	document.getElementById('app')
)

Note that hydrate is used rather than render. This indicates to React that it should expect to see an existing render from the server rather than populating a previously empty element.

Deploying the Application

Code is bundled via webpack with multiple entrypoints:

• Client.tsx: The frontend client ends up as the main JavaScript bundle which is sent to S3 during deployment and served statically.
• server.js: The backend bundle used for rendering on Lambda. This includes the HTML template which is populated by the render.
• translate.ts: The translate API entrypoint used on the translate Lambda.

But Why?

The main reason I see to use SSR is that behaviour improves in the time it takes to download the bundle to the browser. Instead of a blank page, users see a perfectly rendered - just unresponsive - application.

How Feasible is it on Lambda?

It was really quite doable to get SSR up and running on Lambda. But that doesn’t mean it’s a good idea. Lambda rendering a React application is going to be slower than statically serving a pre-rendered HTML file from S3 or especially a CDN. In the case of Obfuscator, pre-rendering is clearly a superior option as the page doesn’t change between loads or for different users.

This app is quite a simple SPA in that it’s a single route. Architecture may get more complicated for an app with routing. I’m guessing a Lambda proxy would be useful here but I’m not sure if that would play nice with the existing S3 proxy.

Then there’s the issue of cold starts. While barely noticeable, it is worth considering that performing SSR on Lambda will mean slower page loads after no requests have been made for a period of time. Caching at the API Gateway level could be a winner here.

Reflections

It’s been nice to revisit Obfuscator and give it some TLC. It’s even nicer that it now stands in a production environment with all infrastructure defined in code. Plus the serverless architecture means I don’t pay anything if it’s not used. There are definitely improvements to be made to the frontend and more features to add (perhaps saving results to DynamoDB and making them sharable).

Highlights of this coding journey included TypeScript and webpack. TypeScript never ceases to amaze me by making JavaScript not suck. I made the webpack configs from scratch for this one and for the first time I’m starting to really understand how to configure it.

As for using Lambdas for rendering - SSR for Obfuscator is totally unnecessary. One day I might look into ways of doing pre-rendering instead. But if your application is truly a good candidate for SSR just know that it is possible with Lambda.

So what is ESPHome? It’s a home automation device manager for ESP8266 and ESP32 devices. It takes a yaml configuration file and uses it to compile a flashable firmware binary for an ESP device. ESPHome also includes a dashboard for managing devices and reconfiguring them - all over-the-air and within the dashboard! There are two components to ESPHome: the software that generates binaries, and the binaries themselves which run on devices. I’m going to call the software “ESPHome” and the binaries “ESPHome firmware” to avoid confusion.

There are really four steps for setting up an ESPHome system:

• Install ESPHome on a PC
• Configure the device through the ESPHome dashboard
• Upload a binary over serial (first-time use)
• Upload a binary over-the-air

Install ESPHome on a PC

First, get ESPHome running on a PC. Here’s a docker-compose v1 entry I’m using on an Odroid XU4 (ARMv7 like a Raspberry Pi). This will also work on an x86 PC if you remove the -armv7. Remember that ESPHome is just a configuration and build tool for the devices. It’s not required to be running all the time, just when you want to configure devices. The devices will maintain connectivity without ESPHome running on a server.

esphome:
  container_name: esphome
  net: 'host'
  restart: always
  user: 0:0
  environment:
    - HOME = "/"
  image: esphome/esphome-armv7:1.13.2
  ports:
    - 0.0.0.0:6052:6052
    - 0.0.0.0:6123:6123
  volumes:
    - /mnt/dietpi_userdata/esphome-docker:/config:rw
  command: config/ dashboard

Run the docker-compose file with docker-compose -f docker-compose.yml up -d or the equivalent docker run command.

Now go to the server at port 6052 and you should be greeted with the following dashboard. At this stage there will be no devices present.

Configure the Device Through the ESPHome Dashboard

Press the pink “Add” button and follow the prompts before hitting “Submit”. You should see your new device listed on the dashboard. Click “Edit” next the new device and a yaml editor will appear. In the yaml configuration file you can specify all the features relevant to the device. This could include sensors, switches, lights, or any combination of things listed on the official website. In this case, we’re just using lights. My configuration file looks like this (secrets redacted):

esphome:
  name: desktop_light
  platform: ESP8266
  board: esp01_1m

wifi:
  ssid: 'ssid_here'
  password: 'wifi_password_here'

# Enable logging
logger:

# Enable Home Assistant API
api:
  password: 'api_access_password_here'

ota:
  password: 'ota_access_password_here'

light:
  - platform: neopixelbus
    name: 'Desktop light'
    id: desktop_light
    restore_mode: RESTORE_DEFAULT_OFF
    type: GRBW
    pin: GPIO2
    num_leds: 144
    variant: SK6812
    method: ESP8266_UART1
    default_transition_length: 300ms
    effects:
      - addressable_rainbow:
          width: 200
          speed: 1

Take some time to craft your configuration yaml, then click “Save” to save the changes before closing the editor to return to the dashboard.

Upload a Binary Over Serial

OTA firmware flashing relies on ESPHome firmware being installed in the first place! This means that for first-time setup on each ESP device, ESPHome firmware will need to be flashed to the ESP device manually over serial. Luckily ESPHome makes this quite easy. Click the menu dropdown next to the newly-created device.

Click the “Compile” button. ESPHome will do its magic, generating C++ code based on the yaml configuration before compiling it for the ESP device. It uses PlatformIO behind the scenes.

Now click “Download binary” to save the ESPHome firmware. Plug in the ESP device to your computer over serial (using an FTDI adapter if needed). To flash the chip, I recommend using esptool.py with the write_flash command.

Upload a Binary Over-the-Air

The hard work is over! Now that the ESP devices have the ESPHome firmware flashed, they should appear as “Online” in the dashboard. Future configuration changes can now be pushed directly to the devices OTA. Simply find the “Upload” button either next to the dashboard entry for the device or below the yaml editor. Pressing this button will compile the binary as before, then poll the device until it comes back online with the new configuration.

It’s all working nicely. Remember that each ESP device exposes its own API, so ESPHome itself is only really needed for configuration changes and compilation.

Adding the Device in Home Assistant

Home Assistant has a solid ESPHome integration. Simply navigate to the Configuration -> Integrations screen and discovered devices can be added easily. If you’re doing a migration, I’d recommend renaming old devices before adding the ESPHome ones to avoid IDs being suffixed by Home Assistant. If set, enter the API password used earlier. Once added, you can view the entities associated with the device.

Reflections

The migration process was quite easy once I worked out how to get the Docker image up and running (user: 0:0 and - HOME = ”/” were key). In fact, the official guide says setup is only possible on an RPi using Hass.io or pip, which is bogus. These days, if I can’t Dockerise a self-contained service then I’m quite hesitant to install it on a server.

I’m enjoying the simplicity of modifying configurations and also the gamma-correction features that are baked into to ESPHome’s light component. One point of disappointment is how the firmware manages the white channel on my RGBW LED strip. I would prefer the white channel not be directly accessible and instead be used to add CRI to the existing RGB channels (in other words, use white LEDs for brightness and add colour cast using RGB only as needed). That’s the way Xiaomi Yeelights work, and I modeled Lightt on that behaviour.

ESPHome has some quite powerful integrations and components which I’ll definitely consider in future projects as an alternative to (poorly) coding C++ myself. It’s also a good alternative to Tasmota which I’m currently running on a Sonoff Pow R2 switch. Perhaps a migration for that switch is the next project!

Automate

From the get-go, I had an idea of how I was going to implement this process. A virtual heat pump entity will be present and state-managed in Home Assistant. Changes to the state of this fake heat pump will get picked up by a Node-RED automation which will construct a packet and sent it to the heat pump via the Home Assistant Broadlink blaster component.

Home Assistant

I needed a simple virtual heat pump entity in Home Assistant. The climate.mqtt entity is perfect for this. It operates optimistically, which means it assumes any commands are applied successfully. As a result the virtual state is updated instantly upon sending a command. Optimistic state management is generally less desirable than proper state feedback, but it’s the best we can do in this case - that’s how the original remote operates, anyway.

A suitable climate entity looks like this:

climate:
  - platform: mqtt
    name: fujitsu
    qos: 2
    availability_topic: virtual/nodered/availability
    min_temp: 16
    max_temp: 30
    temp_step: 1
    modes:
      - 'off'
      - 'auto'
      - 'cool'
      - 'dry'
      - 'fan_only'
      - 'heat'
    fan_modes:
      - 'Auto'
      - 'Quiet'
      - 'Low'
      - 'Med'
      - 'High'
    swing_modes:
      - 'Off'
      - 'Vertical'
      - 'Horizontal'
      - 'Both'
    power_command_topic: 'flat/lounge/heatpump/power/set'
    mode_command_topic: 'flat/lounge/heatpump/mode/set'
    temperature_command_topic: 'flat/lounge/heatpump/temperature/set'
    fan_mode_command_topic: 'flat/lounge/heatpump/fan/set'
    swing_mode_command_topic: 'flat/lounge/heatpump/swing/set'
    current_temperature_topic: 'flat/lounge/temperature/current'

We actually don’t need the MQTT command functionality that this climate entity provides as we’ll just be directly monitoring the state from Node-RED. But I defined topics anyway as it won’t do any harm. The heat pump looks like this on the Home Assistant dashboard:

Perfect! You might not have a current temperature readout at this stage, but I’ll address that with a Node-RED automation later in the article. We can play around with the virtual heat pump at this stage. Obviously, the heat pump will not respond as we haven’t developed an automation yet. The state on the virtual heatpump will update, however.

Docker

Before we move on to automating the heat pump, we need to find somewhere for the packet-generating code to live. As a sadistic web developer, I couldn’t pass up the opportunity to wrap the code in a simple web API and put it in a Docker container.

You can totally do this without Docker. The best option would be to organise the code into script (“function”) nodes in Node-RED. JavaScript code is available at the repo. The rest of this article will assume you have the code running in some kind of web API, however.

We just need to build an image and run the container with something like the following:

git clone https://github.com/albertnis/fujitsu-ar-ry13-ir-codes
cd fujitsu-ar-ry13-ir-codes
docker build -t ir-sandbox .
docker run -p 8080:8080 -d ir-sandbox --name="ir-sandbox" --restart="always"

I’ll be adding this to my home automation docker-compose file someday, but for now the manual build and run does the trick! At this stage IR codes can be constructed via the API. For example:

$ curl "http://localhost:8080/broadlink?tempC=18&mode=heat&fanSpeed=Quiet&swing=Off&powerOn=0"
JgAGAWg0DQ0NDQ0mDQ0NJg0NDQ0NDQ0mDSYNDQ0NDQ0NJg0mDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NJg0NDQ0NDQ0NDQ0NDQ0NDSYNDQ0NDQ0NDQ0mDSYNJg0mDSYNJg0mDSYNDQ0NDSYNDQ0NDQ0NDQ0NDQ0NDQ0NDSYNJg0NDQ0NDQ0NDQ0NDQ0NDSYNDQ0NDQ0NDQ0mDQ0NDQ0NDQ0NDQ0NDQ0NJg0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDSYNDQ0NDQ0NDQ0NDSYNDQ0NDQ0NJg3/DQUAAA==

Node-RED

Time for the fun part! I made an automation flow which will run every time the virtual heat pump’s state changes. It looks like this:

I’ll run through the steps briefly:

1. Heatpump state changed runs whenever something about the heat pump is altered. This includes mere changes to the ambient temperature.

2. Current temp change halts the flow if only the current temperature is changed. We don’t need to send IR commands if the ambient temperature changes in the room. It causes the heat pump to beep all the time which is annoying. We’re only interested in actual command changes.

3. Is it off now? If the heat pump is newly off, a Home Assistant script is run which just broadcasts the stateless off code.

4. Newly on? Sets the relevant parameter depending on whether the heat pump is just turning on or has already been powered up. This parameter matters for construction of the payload for some reason.

5. Map state to remote query just reorganises new state information into a neater structure within the flow message under the query key. This step isn’t strictly needed.

6. Construct query string is a template node which changes the query object into a string to be passed to our API.

   ?tempC={{query.tempC}}&mode={{query.mode}}&fanSpeed={{query.fanSpeed}}&swing={{query.swing}}&powerOn={{query.powerOn}}
   

7. Get remote code calls the API.

8. Set packet moves the response string into the msg.payload key of the flow message.

9. Send IR code is where we close the loop. It invokes the switch.broadlink_send_packet service in Home Assistant to send our code to a Broadlink IR blaster entity. The data template looks like this:

   {
     "packet": "{{payload}}"
   }
   

There’s just one more automation we need to set up, to get current temperature showing up in the heatpump entity. I don’t have access to the heat pump’s thermostat so I circumvented this by putting a Aqara temperature sensor in the same room. The following automation flow maps any temperature changes of that sensor to the MQTT topic which was set as the current_temperature_topic of our virtual heatpump:

Wrap-up

Time to see this beast in action!

Without a doubt, working on this heat pump has been one of the most satisfying home automation projects I have worked on for a while. From learning about how IR communication really works, to cracking the code of the remote, to wrapping the whole thing in an API - it’s been a great learning experience.

Recently, I learnt about some of the basics of IR communication. These fundamentals are interesting, but I wanted to apply them to a real-world situation: automating my heat pump. The aim was to be able to set the heat pump to any setting using Home Automation software and an IR blaster, without the heat pump’s own remote control. Here’s how I did it - hopefully the steps are helpful for anybody else looking to get into heat pump automation.

There are various encodings of IR payloads below. It gets a bit technical and I’ll be switching between pronto and Broadlink notation without much explanation. If you’re new to this I highly recommend checking out the article on the basics before proceeding.

Research

Before diving blindly into the project, I did some digging. First of all, I found the model of my remote: in this case, a Fujitsu AR-RY13. Like most heat pump or A/C remotes, the state of the heat pump is managed on the remote. That means that when you, say, change the temperature on the remote, an IR code will be emitted which includes the remote’s entire state, including things like swing mode, fan speed, and operation mode. There are some codes - like the power off code - which are stateless and don’t change based on heat pump state.

In researching similar remotes online, I came across a useful document put together by David Abrams on Remote Central. It contains details of the IR protocol used in Fujitsu’s AR-RY16 remote. The model name sounded similar to my AR-RY13 so I used this document as a starting point, assuming the two protocols would be identical or similar.

Reverse-engineer protocol

There are two things that need to be reverse engineered here:

• Timing of the remote codes: How bits of data are converted into IR pulses.
• Construction of the data packets: How heat pump states are converted in bits of data.

I tackled timing of the remote codes first because it will let us try out some basic stateless codes without worrying about constructing state packets.

Timing of remote codes

The off code is an example of a stateless code which can be sent to the heat pump. Using the Broadlink RM Mini 3’s learning capability, I captured the off packet of my remote control. It looks like this:

JgB2AG4zDwsQCxAmDwwPJhALEAsPDA8mECYQCw8MDwsQJhAmDwsQCxALEAsPDA8MDwsQCxALEAsPDA8LEAsQJg8MDwsQCxALEAsPDA8LECYQCxALDwwPCxAmEAsPDA8LEAsQCxALDyYQCxAmECYPJhAmDyYQJhAADQUAAA==

That’s the base64 representation which is used by Broadlink devices. Decoding it reveals the following byte array:

[26, 00, 76, 00, 6e, 33, 0f, 0b, 10, 0b, 10, 26, 0f, 0c, 0f, 26, 10, 0b, 10, 0b, 0f, 0c, 0f, 26, 10, 26, 10, 0b, 0f, 0c, 0f, 0b, 10, 26, 10, 26, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 0f, 0c, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 0f, 0b, 10, 0b, 10, 26, 0f, 0c, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 0f, 0b, 10, 26, 10, 0b, 10, 0b, 0f, 0c, 0f, 0b, 10, 26, 10, 0b, 0f, 0c, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 26, 10, 0b, 10, 26, 10, 26, 0f, 26, 10, 26, 0f, 26, 10, 26, 10, 00, 0d, 05, 00, 00]

In Broadlink protocol, most of these numbers represent timings in multiples of 2^-15s (the first few and last few are a bit different). I know that this code, when sent to the heat pump, will turn it off. The trick now is to try to derive this same code from the information in David Abrams’ document. Then I will know that I have worked out the timing information for the heat pump and can send any bits to the heat pump. The document says this is the code:

00101000110001100000000000001000000010000100000010111111

That’s a series of bits which need to be converted to a code. The document says that “one” bits become 0010 0010 and “zero” bits become 0010 002e. Those are pronto codes which means each hex value is a number of cycles. I added the leader and trailer specified in the document then converted this to a Broadlink code by using the frequency 39kHz, close to the 38kHz specified by Abrams. I chose this because the resulting code had the “26” values which I saw in the real packet. This is a good indicator that the frequency is close. Here’s what this generated value looks like, with the real bytes (from before) alongside:

Real:      [26, 00, 76, 00, 6e, 33, 0f, 0b, 10, 0b, 10, 26, 0f, 0c, 0f, 26, 10, 0b, 10, 0b, 0f, 0c, 0f, 26, 10, 26, 10, 0b, 0f, 0c, 0f, 0b, 10, 26, 10, 26, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 0f, 0c, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 0f, 0b, 10, 0b, 10, 26, 0f, 0c, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 0f, 0b, 10, 26, 10, 0b, 10, 0b, 0f, 0c, 0f, 0b, 10, 26, 10, 0b, 0f, 0c, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 26, 10, 0b, 10, 26, 10, 26, 0f, 26, 10, 26, 0f, 26, 10, 26, 10, 00, 0d, 05, 00, 00]
Generated: [26, 00, 76, 00, 68, 34, 0d, 26, 0d, 26, 0d, 0d, 0d, 26, 0d, 0d, 0d, 26, 0d, 26, 0d, 26, 0d, 0d, 0d, 0d, 0d, 26, 0d, 26, 0d, 26, 0d, 0d, 0d, 0d, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 0d, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 0d, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 0d, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 0d, 0d, 26, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, ff, 0d, 05, 00, 00]

The good news is that the length is perfect! We’re clearly on to something here. In our generated code the “one” bits have become burst pairs of [0d, 0d] and the “zero” bits are [0d, 26]. But notice how they don’t line up very well: our first [0d, 26] lines up with a [0f, 0b] in the real code and our first [0d, 0d] lines up with a [10, 26]. It appears our bit mappings are reversed! Looks like this remote is a bit different to the AR-RY16 that Abrams documented.

I performed the same generation as before, but this time “one” bits become 0010 002e and “zero” bits become 0010 0010 - opposite to before. I left the trailer and leader the same.

Real:             [26, 00, 76, 00, 6e, 33, 0f, 0b, 10, 0b, 10, 26, 0f, 0c, 0f, 26, 10, 0b, 10, 0b, 0f, 0c, 0f, 26, 10, 26, 10, 0b, 0f, 0c, 0f, 0b, 10, 26, 10, 26, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 0f, 0c, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 0f, 0b, 10, 0b, 10, 26, 0f, 0c, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 0f, 0b, 10, 26, 10, 0b, 10, 0b, 0f, 0c, 0f, 0b, 10, 26, 10, 0b, 0f, 0c, 0f, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 26, 10, 0b, 10, 26, 10, 26, 0f, 26, 10, 26, 0f, 26, 10, 26, 10, 00, 0d, 05, 00, 00]
Generated + flip: [26, 00, 76, 00, 68, 34, 0d, 0d, 0d, 0d, 0d, 26, 0d, 0d, 0d, 26, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 26, 0d, 26, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 26, 0d, 26, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 26, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 26, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 26, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 0d, 26, 0d, 0d, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, 26, 0d, ff, 0d, 05, 00, 00]

Looks much better! Note how the first [0d, 0d] lines up with a [0f, 0b]. The real code has a bit of jitter so this is a match as far as I’m concerned. One thing I’m not too sure about is the ff which appears in the generated code near the end. Luckily, it doesn’t seem to affect things. This code, when sent to the blaster, is recognised by the heat pump as an off code.

I leant on Abrams’ work to validate this step of the reverse-engineering process. If you don’t have such a document, this is still totally doable! You’ll just have to start with the intial captured packet and derive bits directly from that packet. Play around with frequency ranges and bit mappings to get a feel for what will or won’t work on your heat pump.

Construction of the data packets

Having worked out timing information, we can move on to look at data. Abrams’ document says the heat pump broadcasts its entire state in 16 bytes of information. Let’s capture a code and take a look. This is a code for 30 degrees Celsius, heat mode, no fan, no swing, with the heat pump already on:

JgAGAXAyEAsQCxAlEAsQJhAKEQoQCxAmECUQCxALEAsQJRElEAsPDBAKEAsRChALEAsQChEKEQoQCxALEAsPJhALEAsPDBAKEQoQCxALECYQChEKEAsQCxAlESUQJhAlECYQJhAlECYQCxAKESUQCxALEAoQCxALDwwQCxAKESUQJhALEAoQCxALEAsQCw8MECUQJhAlEQoQCxAmEAoRChALDwwQCxAKEAwPJhALDwwQChEKEAsQCw8MEAoRChALEAsQCxAKEQoQCxALEAsQCxAKEQoQCxALEAsQChEKEAsQCxALEAoRChEKEAsQCxALECUQCw8MDwwQChALECYQCxAKESUQJhAADQUAAA==

Decoded into timing bytes, we get this:

[26, 00, 06, 01, 70, 32, 10, 0b, 10, 0b, 10, 25, 10, 0b, 10, 26, 10, 0a, 11, 0a, 10, 0b, 10, 26, 10, 25, 10, 0b, 10, 0b, 10, 0b, 10, 25, 11, 25, 10, 0b, 0f, 0c, 10, 0a, 10, 0b, 11, 0a, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 0f, 26, 10, 0b, 10, 0b, 0f, 0c, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 26, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 25, 11, 25, 10, 26, 10, 25, 10, 26, 10, 26, 10, 25, 10, 26, 10, 0b, 10, 0a, 11, 25, 10, 0b, 10, 0b, 10, 0a, 10, 0b, 10, 0b, 0f, 0c, 10, 0b, 10, 0a, 11, 25, 10, 26, 10, 0b, 10, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 10, 25, 10, 26, 10, 25, 11, 0a, 10, 0b, 10, 26, 10, 0a, 11, 0a, 10, 0b, 0f, 0c, 10, 0b, 10, 0a, 10, 0c, 0f, 26, 10, 0b, 0f, 0c, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 0f, 0c, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 25, 10, 0b, 0f, 0c, 0f, 0c, 10, 0a, 10, 0b, 10, 26, 10, 0b, 10, 0a, 11, 25, 10, 26, 10, 00, 0d, 05, 00, 00]

We can remove the Broadlink preamble and postamble, as well as the leader and trailer pair to give us this:

[10, 0b, 10, 0b, 10, 25, 10, 0b, 10, 26, 10, 0a, 11, 0a, 10, 0b, 10, 26, 10, 25, 10, 0b, 10, 0b, 10, 0b, 10, 25, 11, 25, 10, 0b, 0f, 0c, 10, 0a, 10, 0b, 11, 0a, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 0f, 26, 10, 0b, 10, 0b, 0f, 0c, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 26, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 25, 11, 25, 10, 26, 10, 25, 10, 26, 10, 26, 10, 25, 10, 26, 10, 0b, 10, 0a, 11, 25, 10, 0b, 10, 0b, 10, 0a, 10, 0b, 10, 0b, 0f, 0c, 10, 0b, 10, 0a, 11, 25, 10, 26, 10, 0b, 10, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0b, 0f, 0c, 10, 25, 10, 26, 10, 25, 11, 0a, 10, 0b, 10, 26, 10, 0a, 11, 0a, 10, 0b, 0f, 0c, 10, 0b, 10, 0a, 10, 0c, 0f, 26, 10, 0b, 0f, 0c, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 0f, 0c, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 0a, 11, 0a, 11, 0a, 10, 0b, 10, 0b, 10, 0b, 10, 25, 10, 0b, 0f, 0c, 0f, 0c, 10, 0a, 10, 0b, 10, 26, 10, 0b, 10, 0a, 11, 25, 10, 26]

That’s 256 timing entries in the array, or 128 burst pairs. Each pair is a bit - meaning we have 16 bytes right here, as predicted by Abrams’ document. Maybe this won’t be so hard after all. Using the timing knowledge gleaned earlier, we can go through and replace each pair with a bit. We can’t just do a find and replace. This is due to the jitter I mentioned earlier: “One” bits aren’t always [0d, 26] - sometimes they are [10, 25] or [0f, 26]. We can look at proportions instead. Here’s the kind of substitution rule which works: Each pair where the second item is at least 1.5 times the first item encodes a “one” bit. Otherwise we take the pair as encoding a “zero” bit. Applying this rule for each pair of timing values, the code becomes:

00101000110001100000000000001000000010000111111110010000000011000000011100100000001000000000000000000000000000000000010000010011

Or, in hex:

[28, c6, 00, 08, 08, 7f, 90, 0c, 07, 20, 20, 00, 00, 00, 04, 13]

Boom! We have ourselves a state payload. Are all these hex codes getting confusing? Remember that we are now in the data domain. These bytes represent encoded state data, not timing. We just made an algorithm to convert a captured IR packet to a data payload. Let’s apply the algorithm to an IR code for 18 degrees celsius instead of 30 with all other settings equal. Can you spot the difference?

30deg: [28, c6, 00, 08, 08, 7f, 90, 0c, 07, 20, 20, 00, 00, 00, 04, 13]
18deg: [28, c6, 00, 08, 08, 7f, 90, 0c, 04, 20, 20, 00, 00, 00, 04, 11]
                                         ^                           ^

Now we know that the second nibble of the ninth byte determines temperature, with 7 meaning 30°C and 4 meaning 18°C.

We are now in a position to reverse engineer the remainder of the state payload. I did this by changing one thing on the remote at a time and running a script to extract the state payload. By seeing what changes in the payload, I was able to map out what each byte means in the payload. I put the results on Github. Interestingly, the bytes are generally different to Abrams’ document but have the same position.

Checksum

The final byte of the payload is the checksum. If it’s not correct then the heat pump will ignore the entire packet, presumably because it assumes interference. Abrams gives a simple way to calculate the checksum but it didn’t work for the AR-RY13. I found a more complex algorithm online and played around with different ranges of bytes to checksum until the generated code matched real-world examples. The final algorithm is something like this:

1. Reverse each of bytes 9 - 15. e.g. 11100001 -> 10000111 (0xe1 -> 0x87) for a single byte
2. Sum those reversed bytes
3. Take result of (208 - sum) % 256
4. Reverse bytes of result

Determining checksums may take a wee bit of trial and error. You’ll most likely need to generate some packets to test, as below.

Generating packets

We have fully reverse engineered the AR-RY13 protocol at this stage. We know how the remote encodes bits as timings, plus how to get a 16 byte code from the heat pump state. At this stage we can write code to generate IR codes given a heat pump state, in the following steps:

1. Receive a desired heat pump state from a user.
2. Generate the corresponding 16-bit state payload, including checksum. Or just use a stateless “off code” if the heat pump is turning off.
3. Convert to timing information using bit mapping and frequency. This includes appending trailer and prepending leader.
4. Convert to desired format (e.g. Broadlink).
5. Send to IR blaster.
6. Profit!

What now?

In the next article I’ll go into how we can wrap this workflow in an automation pipeline using Node-RED for intuitive control through Home Assistant.

The humble Wi-Fi IR blaster is a powerful tool in home automation. Suddenly any device with a traditional remote control can be commanded over the air. With my Broadlink RM Mini 3 and its Home Assistant Component, all I need to do is tell Home Assistant to send a packet to the blaster and it will do the job! I want to answer a couple of questions here:

• What is a packet, anyway?
• What packet do I send?

These are questions I only recently answered for myself by reverse-engineering a heatpump remote. In this post I’ll use simpler examples to explain the basics.

What is a packet, anyway?

To understand packets, we need to understand how IR remotes work. When you press a button on a remote, a message is broadcast as precisely-timed pulses of infrared light which are emitted from the LED and picked up by the receiver. There can be hundreds of pulses in a single IR message - easy to take for granted when you just want to turn your TV on!

A packet is simply data sent to a WiFi blaster which contains all this timing information so that it can broadcast the IR message just as a remote would. It’s basically an encoded form of the IR message. What the packet looks like can depend on the IR blaster being used.

Time domain: raw pulses

Let’s look more closely at how IR pulses work.

The pulses adhere to a carrier frequency where the LED will turn on or off for a certain number of carrier frequency cycles in succession. The frequency is typically around 40kHz, but just like most things in the world of IR, the carrier frequency varies between manufacturers and devices. There’s really no standardisation here!

Here is an imaginary sequence of six remote pulses at a carrier frequency of 20kHz. We can think of the pulses as a series of 12 alternating on/off durations (each pulse consists of on time, then off time). Read the table left-to-right to get an idea of different ways of looking at the time-domain information. Notice how the pulse durations change between pulses.

This is the kind of timing information that can be encoded into a packet for sending to an IR blaster. Note that in the real world, messages are considerably longer than this.

Data domain: beyond the pulses

The pulses aren’t just random, of course. They encode a data payload which can be picked up by the receiving device. For simple remotes this data is typically some kind of key code, which can be parsed at the receiving end. On more complex remotes such as heatpump remotes, they can consist of state data and even have checksums!

Let consider the example from the before. Another way of writing the cycles is with [n-on, n-off] notation, like this:

[70, 130], [16, 16], [30, 16], [16, 16], [16, 16], [70, 110]

These pairs are called burst pairs. Grouping pulses into burst pairs makes outliers clearer. The long pulses and the beginning and end of the message stand out here. They are probably the leader and trailer, respectively. Such pulses are used to serve as a kind of “marker” for the receiver. But they typically don’t encode data. As we’re trying to extract data, let’s remove them, leaving the following:

[16, 16], [30, 16], [16, 16], [16, 16]

When we get to a stage of having two types of burst pair, we’re in a good spot. This means we can extract binary from the pairs. Typically the pairs with a proportionally longer “on” duration mean a 1 and the other pairs mean 0. In our case some pairs have a 30-cycle “on” duration and some have a 16-cycle “on” duration. Let’s replace the 30-cycle pairs with 1 and the others with 0:

0100

So our remote data is 0b0100. Or, in other words, the number 4. If that seems like quite a lot of pulse information for one nibble of data, that’s because it is! But we need to retain all this timing information in order to faithfully rebroadcast the signal from the IR blaster and have it received properly.

Encoding pulses as packets

Say we want to send this timing data to an IR blaster as packet. What do we send? Pulses are truly just a series of durations, but we can’t simply use the table or array notations I’ve used for demonstration. We need a more useful, standardised format. Two encodings I’m familiar with are Pronto and Broadlink.

Pronto codes

Pronto are used by Pronto-branded IR blasters and are also a generally accepted IR code format. Do a web search for “pronto codes” and you’ll find a ton for your device. Pronto codes are hexadecimal and consist of a “preamble” which declares carrier frequency and message length, followed by the burst pairs defined as cycle counts. Here’s what our previous example looks like as a Pronto code packet, with some extra annotation added beneath.

0000 00cf 0006 0000 0046 0082 0010 0010 001e 0010 0010 0010 0010 0010 0046 006e
     freq  len      leader    bit 1     bit 2     bit 3     bit 4     trailer
preamble --->       burst pairs --->

There’s much more information about this popular format at Remote Central.

Broadlink codes

Broadlink devices like my IR blasters use their own code format. Broadlink codes are basically burst-pair information converted from cycles to multiples of a certain time period. This is surrounded by a Broadlink preamble and postamble, then converted to base64 encoding before sending to the device (which I’m doing via Home Assistant). Here’s our (rather ugly) Broadlink code packet.

JgAOAHLVGhoxGhoaGhpytA0FAAAAAAAAAAAAAA==

Check out my pronto2broadlink script for conversion from Pronto to Broadlink format. A good write-up of the Broadlink format is here.

What packet do I send?

The previous example is all well and good, but it assumes we know what timing data we need to send to the blaster. Unless we already have a Pronto code or similar, it’s unlikely we know what to send. There are three main ways I have successfully obtained codes for devices.

Learning mode

Most, if not all, IR blasters come with a learn mode. In this mode, the blaster operates “in reverse” and receives signals from remotes. For example, you could enter learn mode, press the power button on your TV remote, and then receive the resulting packet back from the blaster. Sending this packet to the blaster in the future will broadcast that message to toggle the TV power. The best part? Since learning happens on your blaster, you know that it will give you the correct packet format for use in the future - so there’s no need for conversion or inspection of the packet.

Find it online

Searching for IR codes online can be surprisingly fruitful. Some manufacturers publish entire spreadsheets of Pronto codes. These documents often contain “secret” codes not found on the remote. Discrete IR codes are a great example; these codes are perfect for home automation as they remove some state-management burden. IR code databases are also present. As with Broadlink blasters, you may need to convert the Pronto codes to vendor-specific packet formats.

Reverse-engineering

For complex remotes, reverse-engineering can work. This is the wild west of IR codes. The approach basically consists of analysing a remote, extracting transmitted data for various keypresses and then attempting to work out how data payloads are structured and what they mean. The data can then be changed and reconstructed into artificially-created packets. I recently did this successfully with a heatpump (one for a future post). Here’s an example of a less successful effort, with my SMSL AD18 amplifier. In this case, each key transmitted a data payload of one byte:

I noticed that all payload values were multiples of 15, and that 9 multiples of 15 were missing! Did that mean 9 “secret” commands were present which could include discrete power commands? In this case, no. I tried broadcasting the other multiple of 15 and nothing happened. You win some, you lose some.

Summary

The humble IR remote is a powerful tool and, thanks to IR blasters, is one that can be integrated into a home automation system effectively. It’s been fascinating to learn about how the IR messaging works and the encodings people use to store and share codes. I have found that reverse-engineering remote controls can be a surprisingly engaging challenge - a challenge which is extra rewarding when it works. At $25 for one of these little blasters, it’s hard not to recommend getting one!

• Last Will and Testament (LWT) for availability updates
• Retained messages for persistent state

Here’s an explanation of what I did to get these features working for me, with some examples from Home Assistant and mosquitto. If you want to try out the examples, install mosquitto.

Last Will and Testament (LWT)

Binary devices have two basic states, on or off:

There is a third state, even for binary devices, which is very useful: unavailable:

An unavailable state allows home automation software to prevent commands being sent to a disconnected device. It is also useful for monitoring the health of devices, or as a failsafe. For instance, if an alarm system is powered down, other devices can be alerted to this fact. Google Assistant will also mention if devices are unavailable.

But how can a device tell other devices when it is disconnected? This is where the power of the broker and LWT comes in. When a device connects to a broker, it can specify an LWT topic and payload. The broker receives this information and will publish it when that device is disconnected. Essentially, the device says “I’d like to connect. When I eventually disconnect, publish this message”. The broker takes on the responsibility and publishes the LWT payload on the device’s behalf when it becomes disconnected.

Let’s give this a go in Arduino C++ using PubSubClient:

// Connect to the broker and tell it to publish "offline" to the availability topic when disconnected in the future
char availabilityTopic[] = "light/desktop/availability";
bool mqttConnected = client.connect(
    mqttClientname,
    mqttUser,
    mqttPassword,
    availabilityTopic,
    0,
    1,
    "offline");

if (mqttConnected) {
    client.subscribe(commandTopic, 1);

    // Publish "online" to the availability topic now to signify connection
    client.publish(availabilityTopic, "online", true);

    Serial.println("connected");
  }

You can play around with this by using mosquitto_sub to investigate messages. Run mosquitto_sub then turn the device on and off. You should see something like this:

$ mosquitto_sub -t "light/desktop/availability" -v -h localhost
light/desktop/availability online
light/desktop/availability offline

The “offline” message is received like magic after the device loses power! The broker has delivered the last will and testament of the client.

Home Assistant can be set up to use this topic. By default, it expects “online” and “offline” as payloads on the availability topic. Here is an example mqtt_template light entry in configuration.yaml:

light:
  - platform: mqtt_template
    name: "Desktop light"
    state_topic: "light/desktop/state"
    command_topic: "light/desktop/command"
    availability_topic: "light/desktop/availability"
    ...

Retained Messages

When a device powers up, what state should it be in? Let’s take a light for example. An easy option would be to have the light turn on to the same colour each time. Or maybe have the light remain off (“black”) to make restarts inconspicuous. The best option is to have the light retain its previous state. There are two main ways to do that:

1. The light remembers its own last state, perhaps by writing every change to persistent storage and reading this storage on boot.
2. The light gets its previous command from the broker and executes that command to restore state.

Both approaches have their merits, but I like the second option because it ensures all parts of our setup are working from the ground truth provided by the broker.

MQTT has a feature just for this - retained messages. Any message can be sent as a retained message. This message will be sent to future subscribers to the topic as soon as they subscribe. Only the latest retained message for that topic will be sent. Let’s see it in action using mosquitto_sub and mosquitto_pub.

$ mosquitto_sub -t "light/test" -v -h localhost

Nothing to see here! No messages are being published to this channel so the output is empty. Kill this command and we can do some publishing:

$ mosquitto_pub -t "light/test" -h localhost -m "retained!" -r
$ mosquitto_pub -t "light/test" -h localhost -m "not retained"

We sent two messages to the test topic, with a key difference: the -r flag in the first command means this message is retained and will be sent to new subscribers. We can check this by subscribing again:

$ mosquitto_sub -t "light/test" -v -h localhost
light/test retained!

This is excellent and can work both ways in home automation:

• If the device publishes state updates as retained, the home automation software will receive state on connection to the broker.
• If the home automation software publishes commands as retained, the device will receive commands on connection, hopefully restoring state.

On the device in Arduino C++, this is simple and looks like this:

// The "true" means retain in PubSubClient
client.publish(stateTopic, output, true);

In Home Assistant, it is also simple:

light:
  - platform: mqtt_template
    name: "Desktop light"
    command_topic: "light/desktop/command"
    retain: true
    ...

Summary

Leveraging the LWT and retain features of MQTT has made me realise just how useful MQTT can be. These features facilitate convenient use cases in home automation software. Previously I thought MQTT’s broker seemed like a redundant middleman. Now it is clear just how powerful this architecture is for transparently managing subscribers and their payloads.

This post is a bit out of date. I loved exploring LED control by writing custom software like Lightt, but nowadays I use much better software written by others. I recommend you check out my posts on WLED (great for individually-addressable colour strips like those described in this article) and ESPHome.

Over the last three years, LED strip lighting has become a staple of my desk setup. I like it for a few reasons:

• Strips behind a desk add bias lighting, making it more comfortable to look at a screen at night.
• Strips stuck along the bottom of a monitor provide a work and keyboard light.
• It adds colour to a room, without painting or doing anything permanent.

But there have been some drawbacks this whole time:

• I haven’t settled on a seriously useful control mechanism. I’ve tried an IR remote, but that’s most appropriate for a single zone. I settled on rheostats, which are simple but require physical access.
• Running from a breadbroad means there is always a tangle of wires in the corner of my desk.
• Power supply woes. I’d have an Arduino run from my computer’s USB, then power the strips with separate higher-power 12V and 5V power supplies. The 5V power supply was a computer PSU, which was loud and impractical.

I had been thinking of implementing a WiFi control system for a while, but never really saw how it would fix those last two issues. Then I found this downright amazing board from ElectroDragon, which gave me a lot of hope. I bought three and prepared to level up my Arduino skills for the ESP8266.

The Board

The ElectroDragon ESP LED Strip Board essentially consists of an ESP8266 microcontroller with input terminals for power and output terminals for and LED strip. The coolest part is that it’s designed to be powered with a single supply. Just plug in a power supply to suit your strip, select the correct pin jumper settings and all voltage regulation will be taken care of. The board is a small package and even comes with an injection-moulded enclosure and leads for power input and output. Here’s how I set it up for a ‘dumb’ 12V RGB strip and a ‘smart’ (SK6812) 5V RGBW strip.

Programming an ESP8266

Actually getting code on the ESP8266 is harder than a regular Arduino. It’s surprising how much digging I had to do to find some answers. Here’s the low-down on what I learnt about flashing this chip:

• You’ll need a USB to TTL adapter to program this board and most other ESP8266, as there is not onboard USB port.
• Despite the ESP8266 being a 3.3V board, use the 5V rail to power the ESP LED strip board when the USB adapter is connected. I don’t know why this is the case, but it works a treat.
• The Arduino IDE is convenient for programming this board. You’ll need to install the ESP8266 Arduino core. Once that’s done, I recommend switching to a more powerful editor like VSCode, which can program the board via the Arduino plugin. Check out Lightt’s GitHub page for my board settings which work well for uploading to the ESP LED strip board.
• This one’s important: Unlike most Arduinos, the ESP8266 has to be manually put into programming mode before an upload can happen. To do this, the GPIO0 pin must be connected to ground at powerup. On the ESP LED strip board, do this by holding down the ‘IO0’ button while you plug the USB adapter into the computer.

Here’s what my programming setup looks like with a CP2102 USB to TTL adapter:

All About Lightt

Feel free to clone and flash Lightt (fork and PR too, if you’re keen!). I learnt a ton while making Lightt and it has been a really rewarding undertaking. Here are some key learnings.

MQTT

MQTT is a network protocol which is perfect for IoT. Basically a ‘broker’ runs the MQTT server on your network. I’m using a Raspberry Pi 2 with Mosquitto to achieve this. MQTT works on a publish/subscribe basis. Messages can be published by devices connected to the broker, and each message has content (‘payload’) and a subject (‘topic’). Devices can also subscribe by listening for messages which have a certain topic.

In the case of Lightt, each board has two topics: command and state. The board subscribes to the command topic in order to receive commands from the home automation software. It publishes messages on its state topic to confirm a command has been executed and to regularly let the home automation server know what its current state is.

Message Formats

The board needs to know a command from the server looks like, and also how to format state messages. String processing is a bit of a pain on microcontrollers, so my aim here was to let the home automation server do the processing and keep the messages themselves very structured.

Robustness Never Hurt Anyone

I ran into issues with runtime errors and odd network disconnections which would cause resetting. It turns out the ESP8266 can reset if its loop times become too slow. I fixed the issue by removing delays in the code and instead implementing more robust checkers for MQTT and WiFi connections in the main loop.

Crossfades Are Still Cool

I’m really happy with the crossfade functionality in Lightt. There’s a default 150ms crossfade duration, but this can actually be overridden if a transition is defined in the MQTT command. This allows for some awesome behaviour like 30-minute crossfades for sunrise alarms. More on that in another blog post…

Dimming Curves Are a Thing

Human perception of brightness isn’t linear at all - it’s more logarithmic, like most senses. I implemented a dimming curve for the first time and included a Python helper script. It would awesome to have more than 255 steps to play with, but this is a limitation of the 8-bit colour signal expected by smart strips.

More Experiments to Come

I love being able to program a WiFi-controlled microcontroller. It allows for so many connected, low-power, and out-of-sight projects - and that’s exactly what has made the ESP8266 so popular for IoT projects. The ESP LED strip board in particular has made it possible to run homebrew Arduino-like software without the nest of wires and improvised controls. It looks like the ESP8266 will be driving my desk lighting for the forseeable future, in a project which finally feels finished.

Beautifully generic.

Never fear: Hex Solutions isn’t an actual company. Rather, I imagined the overly vague company in a recent project which I used to teach myself all about full stack JavaScript web development. While the project was more or less completely contrived, it does address a problem summarised below:

What if an individual or small company wants a basic, maintainable, and elegant landing page without the fuss of an entire platform such as Wordpress?

From this precept, Level was born. The idea of Level is to make content editing by the user trivially simple even for those with little technological prowess. The user can login securely without any page reloads or navigation and start editing the page straight away.

The User Workflow

First of all, the user scrolls to the bottom of the page to find the login link:

Clicking this link shows the login form, where credentials can be entered:

If the credentials are correct, the page enters editing mode. This is all done with no page reloads or redirects.

Dynamic fields are now editable throughout the page. As the user edits a field, the save status is shown in the toolbar. Changes are sent to the server, debounced by several seconds from user input to prevent request spamming.

The user is notified when the server has received and stored changes:

The process allows for simple editing. The user can see what field they are editing, can toggle spellcheck, and can logout at any time.

The Technical Details

The app is full-stack and entirely written in JavaScript using the MERN stack. Here’s a quick summary of how the stack works in Level:

• MongoDB: Uses two collections: one for users and one for fields.
• Express: Handles the app routing. Level is more or less single-page, so there’s nothing too fancy here, but there are routes set up for actions such as logins, logouts and field edits.
• React: The app is built on React.js. Interface files are used for both client-side and server-side rendering.
• Node.js: The runtime used for Level.

There are other key libraries which I used:

• Redux: Used for state management and action dispatching. In other words, a lot of heavy lifting!
• redux-observable: Brings RxJS to redux. In Level it takes care of the AJAX workload; namely, sending login attempts and edits. It also has neat functionality for debouncing those edits before sending.
• Passport: Server-side authentication management. Used with a few session libraries, makes user management quite convenient. Obviously, changes will only be pushed to the database if an authenticated session is detected.

Reflections

What an adventure! Five years ago, I avoided JavaScript where possible. So much of what I saw JavaScript used for could be easily accomplished by other means (often CSS animations). Diving into this project opened my eyes to some interesting things.

• ES2016 makes JavaScript simpler. Level was developed using babel through Webpack to compile JavaScript files. While this can seem like an unnecessary evil, it means ES2016 can be used. This flavour of JavaScript has some neat features, one of which is no semicolons. A friend commented that the ES2016 code looked more ‘Pythonic’ due to the lack of semicolons and presence of import/export statements.
• Server side rendering is awesome. I mean, it’s nothing new, really. PHP, Python, and just about any backend language do the exact same thing when used to generate websites. What’s cool about the full-stack JavaScript approach is that the server code can be pointed at the exact same code used for frontend rendering. Obviously, Webpack bundling means that server and client are using different JavaScript files, but server-side rendering enables React to pick up directly where the server left off and start handling the page right after initial load. This was a huge discovery for me: in the past JavaScript seemed to me like something which had to painstakingly hook into various pre-rendered components to attach listeners and the like. This way, your app is ‘delivered’ to the client in a ready-to-go form.
• MongoDB is surprisingly simple. It’s just saving JSON files. And because the whole app is in JavaScript, JSON is ridiculously convenient. Mongoose makes interactions with the database in code straightforward. At least for smaller projects, it seems more convenient than SQL.
• Redux took a while to learn. There are reducers, actions, and connections to manage, not to mention the state itself. I found Redux to be one of those ‘practice makes perfect’ things. I’m nowhere near perfect but I can get actions whipping through the app pretty quickly now, and linking them to components seems like less of a chore than it did initially.
• I recommend source maps. I’ll admit it: I barely realised what a source map was, let alone how it could help me. That was, until halfway through the project. Webpack can easily generate source maps and it makes debugging many times easier than probing code with ‘console.log’.

While Level is most certainly a solution looking for a problem, the app has taught me a great deal about full-stack development. Relatively simple projects like this are great for getting to grips with new ways of doing things, especially in a world with so many JavaScript frameworks floating around. I don’t have access to a Node.js host for Level, but if I ever needed a simple single page for something, I might just dust off Level someday.

It was easy to explore between the large boulders which make up the landscape.

The others explored along the top of a rock cluster.

I decided to head up Castle Hill to seek out some views.

From the top of the hill, there were fantastic views of the grassland below, as well as the mountains beyond.

My only previous experience with computer vision merged my interest in programming with my interest in music. This previous project, named “VMIDI”, used an RGB camera to track markers attached to the action of a partially-dismantled piano. The code utilised Python and OpenCV to track keypresses and their velocities. While a good learning opportunity, VMIDI was slow, intrusive and never made it very far.

When I discovered students could specify our own projects in the computer vision course, making the sequel to VMIDI seemed like a natural evolution of my previous line of thought. So I decided to give VMIDI2 a go, this time armed with one year more coding experience and departmental backing.

There was a third difference between VMIDI and VMIDI2: the C++ language. Using C++ for VMIDI2 ended up being a bit of a revelation. While Python computer vision can be fast if native OpenCV calls are maximised, C++ makes speed seem effortless. The source code for VMIDI2 is available on GitHub.

The rest of this article is a compressed version of the research paper I wrote on this topic.

Method

The Kinect camera is positioned above the keyboard, as shown by this IR frame:

The keyboard is extracted from the depth frame (not pictured) using perspective warp on manually-selected points. After applying a bilateral filter to remove some noise, the following frame is obtained:

Keys are manually overlaid over this image. These rectangles will be used as key masks for key depth tracking in a future step.

This unpressed state is used to create a difference image for successive frames. Each pixel in this difference image has a value of millimetres below its initial, unpressed state. When unpressed, it looks like this:

It’s basically black, because the keyboard is essentially still in its initial state. Each key’s rectangle is used as a mask within which the mean pixel value is calculated (shown in red for the A3 key). Each mean value is assigned to its key. Additionally assigned is an n=5 rolling average of this value (shown in blue).

When pressed, the depth reflects the change:

We’ll get to the velocity value (shown in green) later, but first there is an important consideration to make when tracking keypresses. Often hands cover unpressed keys while playing. It is important to exclude from the difference image any pixels belonging to hands. To achieve this, I made a simple hand detection procedure. Pixels a certain distance above the keyboard are thresholded:

These pixels include hands and a smattering of noise. To remove the noise, an image opening step was applied. A further dilation made the hand area more conservative, in an attempt to approximately cover parts of fingers near the key level.

Finally, this mask was subtracted from the key mask before finding average depths. Here’s an example of such a combined mask. It can be seen the subtraction of the hand frame has taken a ‘nibble’ out of the otherwise rectangular key mask.

Each key depth mean obtained from these masks is tracked and used to generate a rolling average. When this average exceeds the press depth required for the key to sound (measured at about 4.5mm on my piano), the reverse difference method is used to work out how quickly the key was going when it reached sounding point. The following graph shows how this occurs. A gradient is calculated using the points straddling the point of sound line.

Results

Various keys were pressed multiple times with varying volumes. Using a microphone recording and Adobe Audition, the peak amplitude of each keypress was extracted from an audio recording. These amplitude values were used as a baseline to which the calculated press velocities were compared. Plotting calculated keypress velocity against measured audio levels yielded the following graph:

Which is great! Lines were matched with R² values ranging from 0.81 to 0.92.

Future Research and Improvements

• Some full-on performance analysis. VMIDI2’s results were based on repeated keypresses. This makes sense for testing a proof of concept, however it made comparison with previous research difficult. The prior research which I could turn up generally neglected velocity sensitivity but had quite thorough result gathering.
• Integration with an AMT workflow. A notable piece of prior research by Akbari presented a system for automatic music transcription (AMT). Essentially an RGB camera is pointed at a keyboard, and when a performance occurs, sheet music is generated. This has three phases: keyboard registration, keypress detection and transcription. VMIDI2 as it stands is ‘weak’ in some ways, as it requires manual keyboard registration and does not contain transcription features. It would be great to try to use it as the keypress detection stage in a larger process.
• Investigation into keypress detection robustness. A wide range of keys should be tested and compared to audio amplitudes. Do, say, lower keys make less sound for the same press velocity? Does each keyboard benefit from an individual calibration ‘profile’ of some sort?
• Investigation into camera positioning. Distance is a big one here. One application of VMIDI2 may well be hanging a Kinect sensor over a publicly accessible keyboard. Keeping the sensor well out of reach would be ideal, and it would be worth testing the effect of distance on accuracy. Angle may likewise be useful to test.

Early in 2016 I attempted to combat one of my few gripes with MusicBee: the mediocre built-in themes. While the latest 3.0 version of MusicBee has helped, the only decent theme in previous versions was the rather fantastic DarkRED Flat by forum user Endeavour1934. Using these theme files as a starting point, I developed a slightly more colourful variation. The idea was to have a single accent colour which could be easily changed to any desired colour

During the process, I identified a couple of things about the skinning process which I thought could be improved:

1. The titlebar colour and border colour are actually a tiled PNG image, defined in base64 encoding within the theme file.
2. Themes cannot expose any settings to the user through the MusicBee UI.

My skin contains one accent colour, but I realised that if the user wanted to change this colour they would have to not only change the colour variable within the theme XML file, but also generate matching PNGs for the border and title bar shades before exporting to base64 and pasting into the file. Not ideal!

From this conundrum, DarkMOD was born. DarkMOD is a web app, which does the following:

1. Receives a colour via GET request, sent from a webpage frontend with a pretty preview window which I spent far too long Photoshopping.
2. Calculates required shades and tints for use in the theme.
3. Generates PNG images in base64 to represent borders, title bar and checkboxes.
4. Runs through a template of the skin, replacing placeholders with the correct colours and resources.
5. Returns the completed skin file as XML.

Don’t have or want MusicBee? Don’t have or want Windows? Head over to the project anyway; the preview window should give you a good idea of what DarkMOD is about!

The official forum thread for DarkMOD can be found on the MusicBee forum. The source code for DarkMOD is available on GitHub.

I have always found online translators so interesting. With the continuous shrinking of the world led by the continuous growth of the internet, it is hardly surprising that we have been algorithmically converting between languages for years now, with online services like Google Translate and Bing Translate establishing themselves as vital when travelling, learning and communicating. These algorithms are good. Darn good, in fact — with computer vision adding to the mix for an even more functional experience.

If my years of learning French only taught me two things about language, they would have to be the following:

1. Language is complicated.
2. When it comes to translation, darn good is not equal to perfect. Indeed, while online translators are nearly always good enough, they are not perfect by any stretch of the imagination.

I set out to explore and exaggerate the shortcomings of online translators by creating Obfuscator (I’m no longer hosting this version, but the latest version which uses a similar concept can be found here). For those who only speak one or two languages, it can be hard to gauge the inaccuracy of translation software from a single pass. Using the Bing translate API, Obfuscator takes a phrase and translates it through three successive languages before returning it to the origin language where it can be revealed to the viewer in all its glory. Depending on the languages and phrase, the translation software may completely nail it. Other times, it may not:

The source code for the original version of Obfuscator is available on GitHub. Or read more about the new version!

On the way home, we had time for a brief stop at Lake Tutira, in the hills north of Napier.

As it turns out, with three MOSFETs, an IR receiver and a bit of code, it was surprisingly easy to replicate the functionality of the bundled LED strip controller with an Arduino as the brains. With a bit more code, I was able to implement buttery-smooth colour-crossfading and coloured strobing with different speeds.

This was all very interesting, however I knew I could do more. What if I could get the strip to dance to music? A quick search showed a popular method of achieving this was the fast Fourier transform. Fourier transforms are pretty cool, however after a bit more research I found that a higher resolution and faster response could be attained with a hardware-based approach using an MSGEQ7 spectral analyser. So I bought a couple on eBay.

The MSGEQ7 is a neat little IC. Essentially, it combines a multiplexer and seven bandpass filters. When wired up properly to an audio source, the MSG receives regular signals from a microcontroller to send back amplitude data from each of the seven bands. This means only three pin connections are required between the chip and the controller - perfect for my Arduino Nano with many pins already used!

The code used for the MSGEQ7 is somewhat particular due to timing requirements, however the code I put together boils down to a nested loop which fetches spectrum data as quickly as possible, corrects for line noise, and maps three of the seven band amplitudes to the colour channels on the LED strip. I messed around with many different mapping configurations, but settled on red as band 1/7 (low bass, 63Hz), blue as band 2/7 (high bass, 160Hz) and green as band 6/7 (treble, 6.25kHz). I ended up avoiding mapping mid-tones because it didn’t look quite as impactful as emphasising bass and treble frequencies. I suppose that often these midtones are dominated by vocals, keyboard and the like, which are typically less rhythmic than bass or treble.

After settling on a colour scheme, I had a great deal of fun paging through my music library identifying the best- and worst-looking songs for the setup. One of the best was Red Dress by Submotion Orchestra. Here’s the result:

This project has been a great one so far, but there were frustrating times. The main annoyance was the capacitor used on the audio stage before input to the MSGEQ7. At relatively high volumes, the capacitor blew, resulting in a sad, flickering green output to the LED strip with which I became all too familiar. While I simply replaced blown capacitors, the problems could probably be fixed with higher quality caps, or additional circuit protection.

So what’s next for the LED strip? I am keen to continue experimenting with the MSGEQ7 project, as I think there are some refinements to make. Some sort of denoising or time averaging could reduce the amount of flickering. There may also be other algorithms worth exploring to increase the smoothness and “realism” of the light response with introducing too much lag. It may also be interesting to compare the MSGEQ7’s output with FFT methods to see what the difference really is.

After that, I’m looking forward to continuing to experiment with other hardware which can interface with the Arduino. These future experiments sure look bright.

It didn’t start off too fantastically though. Just a week after I received my second-hand Logitech G500 I started noticing frustrating double-click issues, where the mouse would erratically click and release when the left button was held down. These second-hand qualms reached the point where I switched back to my trusty Microsoft WMO1.1A while I researched causes and potential fixes. First up, I followed through a popular video on YouTube, which involved shielding the top of the switch to prevent static buildup. This only worked partially for me however and the mouse was still barely usable. Eventually I bit the bullet and opened up the mouse switch to clean the contact point. The tutorial I used is here. I opted not to remove the spring from the switch and simply scraped the contact point with a flat head screwdriver. While I couldn’t see any buildup before scraping, the trick worked marvellously; once reassembled, the mouse was as good as new!

I’ve had my G500 working for a couple of months now and it’s safe to say that I will have trouble ever going back to three-button mice. I don’t love the G500 because it’s a gaming mouse — I haven’t even been doing much gaming — I love it because it’s a productive mouse, and a comfortable one at that. The software Logitech provides for the G500 manages to get out of the way, quitely sorting out mappings and program detection in the background. And those mappings can be quite powerful. For instance, I’ve replaced the horizontal scrolling action with switching between dekstops. The first thumb button switches to the previous window in a desktop. Just these two buttons have saved me plenty of time while multitasking. By replacing the DPI down button with Windows Task View, I can get an even more detailed overview of my windows when the thumb button isn’t enough. This is perfect when many documents are open.

But it doesn’t stop there. The software can detect which program is focused and load up sets of program- or game-specific mappings. For me this is massively helpful in Adobe Lightroom, where I can now flag and develop photos one-handed without having to move my cursor up to the toolbar.

This all sounds very much like a marketing pitch for Logitech, but in reality I am simply amazed by how flexible multi-button mice are. Initially I was fearful that the additional buttons would be flashy and useless, but once I took the time to decide on useful mappings, I realised the potential these mice have to improve the entire window management experience as well as certain applications. Are there other brands who have even better mice and software? Quite possibly. All I know is that I’ve found a new personal standard for a mouse. I just wish I had discovered it sooner.

I’ve been using a Logitech K220 for over two years now. While I like having a portable keyboard which barely takes up any desk space, the K220 is truly unnatural to type on and, at least in my experience, invariably elicits negative comments from those who use it for the first time. It was this that triggered the start of my research into mechanical keyboards. Having tried a friend’s Das keyboard with Cherry MX Blue switches, I had experienced first-hand what mechanical typing felt (and sounded) like, and I liked it. My enthusiasm was short-lived. It turns out that even second-hand mechanical keyboards sell for over $100, not to mention the cost of high-end boards with individual backlighting and USB hubs. Part of me just wasn’t ready to spend so much on a keyboard. Which is why I compromised by buying a second-hand Steelseries Apex keyboard.

The Apex is a membrane keyboard, and it turns out that, in comparison to a mechanical keyboard, the Apex was a pretty big compromise indeed. Don’t get me wrong: with five backlight colour zones, macro keys and a USB hub, the Apex has some decent features in a good-looking package, but after using the keyboard for a few hours I realised that I had lost sight of my original keyboard goal of increased comfort, instead giving in to the allure of features. The allure of needless gimmicks. The truth is that the Apex isn’t particularly comfortable to type on, and is particularly cumbersome in size. And who really needs 22 macro keys? Not me.

Nevertheless, in a situation not unlike the time I went open, using the Apex has ended up being a useful learning experience in that I have a better understanding of what I want and what I don’t want in a product. I’d like a mechanical keyboard. I’d like a backlight, but I don’t need a colour-changing one. I’d like a relatively compact keyboard. I’d like a comfortable keyboard. I don’t want gimmicks.

Knowing this, I jumped onto Massdrop and investigated their fantastic mechanical keybaord section. Sure enough, the Magicforce 68-Key Mini Mechanical Keyboard seemed to fit the bill, with full-size keys, a backlight and a solid design. I got one with Gateron brown switches for their general-purpose tactility and look forward to receiving it in April. I’m really hoping that it will quench my thirst for a better typing experience — without satiating my hunger for gimmicks.

The only thing is, I can’t quite put my finger on why I think it’s sad. I’ve tried streaming services before, and they have many great features — shared public playlists, a vast array of music, a solid user base — but I’ve never been able to stick with a streaming service, always preferring to manage my own personal collection of music files. It’s a far cry from the attic of a meticulous record collector. There’s no familiar scent of papery dust. There’s no musty, almost tangible nostalgia. But there’s a sense of collection. A collection I can catalogue and tag far too pedantically. A collection I control. A collection I can play whenever and wherever I want.

That last point has always been a bit of a struggle for me however. My phone has limited storage and any music I want to play offline has to be downloaded from Google Play. It’s more convenient than Spotify, but not by much. Which is why the inheritance of a humble 5th generation iPod Nano has been a surprisingly pleasant experience for me. While adding music to the device from MusicBee wasn’t exactly seamless (thanks Apple), it is possible via a workaround, and it didn’t take long to set it up with 14GB of my favourite tunes. It also didn’t take long to remember just how great iPods were and still are.

First of all, this thing is pretty much indestructible. It’s built light and it’s built solid. While it was already a bit bruised when I received it, the Nano is quite robust to say the least, yet thin at the same time. Battery life is great, with hours and hours of playback potential, and that’s to be expected; there’s no battery-sapping WiFi and Bluetooth here, just music. Plus the click wheel remains an asset, one which myself and many others will miss in future Apple products.

The Nano isn’t perfect, however. The menus seem clumsy at times with poor sorting. Compilations are treated weirdly. I don’t know why there’s a camera. I could really do with a few more gigabytes. But features aside, the iPod’s been great so far. In a world where it would be so easy to simply listen to the same Spotify summer playlist on repeat, this device is letting me rediscover some of the most enjoyable nooks of my own collection.

In fact, it still hasn’t officially arrived to a large proportion of Moto G owners, myself included. Motorola’s rollout of Android 5.0.2 Lollipop has been frustratingly slow, having taken nearly three months and counting, releasing to individual carriers in individual countries. I’m sure that I’m missing something but the whole ordeal is a shambles given that the software on the Moto G is so very nearly vanilla. What’s more, 4.4.4 KitKat has been plagued with memory leak issues. The Facebook app started taking 30 seconds to open. Every time I returned to the home screen the launcher would have to restart after having run out of memory and music would stop playing after a few seconds if I opened another app. Clearing the cache didn’t help.

My phone is the ‘Global GSM’ version of the US Moto G XT1032. After waiting months with no update, I became fed up with my nearly-crippled phone and turned to Reddit and XDA Forums where I found many users in the same situation. I read daunting guides about manually flashing system images and custom ROMs. Yesterday I bit the bullet and somehow succeeded in manually installing Android Lollipop 5.1 on my phone (technical details below).

My first impressions of 5.1 compared to 4.4.4 are fantastic. Not only are those slick material design elements abundantly present, but memory management has come a long way. Apps open promptly, and the launcher only crashes if I do some serious multitasking. I can listen to music while browsing Facebook and using Facebook Messenger with no issues whatsoever. Plus there’s a whole host of new settings to explore, and a mysterious camera flash bug has been fixed. It’s early days for Lollipop on my phone though; I’m just hoping that performance won’t deteriorate over time. But so far, so good. In fact, 5.1 is so smooth and bug-free that this whole experience has made me even more skeptical of Motorla’s customer support and rollout strategy.

There were many steps to installing Android 5.1 Lollipop, and I’ll summarise them here:

1. Unlock the bootloader. This step has officical instructions provided by Motorla itself. This voids the phone’s warranty and erases all data on the device. Unlocking the bootloader allows the installation of images and ROMs not authorised by Motorola. I had some trouble rebooting afterwards, but it worked after retrying a couple of times.
2. Flash Android 4.4.4 GPe. pfufle’s excellent tutorial at XDA Forums is what I followed for the rest of the process. After downloading and setting everything up on my computer, I followed step 2.2 to install the Google Play Edition of 4.4.4 KitKat on my phone. This essentially changes the phone to the Google Play Edition of the Moto G which, like a Nexus Device, receives updates early and directly from Google.
3. Flash the 5.0.2 OTA update. Still following pfufle’s tutorial, 5.0.2 now needs to be flashed (manually, for some reason). This is done through booting into recovery and using ADB. After doing this step I ended up being stuck at the boot animation. After trying a suggested fix in vain, I realised I had actually missed some lines of code in step 2.2. The key things to learn here are (a) use the batch files provided, (b) if anything gets stuck while booting, holding down the power button for a few seconds will force the phone to turn off, and (c) the process can always be restarted from step 2.1.
4. Install Android 5.1. This is the easy part. After successfully booting into 5.0.2, connect to the internet and wait a few minutes if needed. Just like an unmodified phone, a notification to install 5.1 will appear. Simply download and install it.
5. Flash modem firmware (if needed). The tutorial mentions flashing modem firmware. For whatever reason, my network performed just fine, 3G and all, without additional modification. Perhaps this means the hardware in the Global GSM Moto G is identical to that in the GPe device.
6. Consider rooting (optional). It doesn’t look hard, but I decided to pass on this step. Rooting the phone does have its perks, however I’m happy with 5.1 as it stands.

When I unboxed the Dragonfly, I was surprised by how well-built this little machine is. It feels like a heavily over-engineered flash drive: heavy for its size and beautifully surrounded by a soft-touch finish. A cap and pleather case is included. Just when I thought it couldn’t look any cooler, I plugged it into my computer. The Dragonfly emblem lights up with one of four colours, depending on the sample rate at which it is being used. On Windows, sample rate is set globally, but applications can override it if needed. MusicBee can use WASAPI to essentially commandeer the Dragonfly to change its sample rate depending on the track playing.

So far, so good. However, the most important attribute of a DAC is undoubtedly sound quality. Flicking through my test track playlist with my ATH-M50XBL on, the Dragonfly performs nicely compared to my PC’s built-in soundcard. I heard the most notable improvement in the high frequencies. Crackles, pops and cymbals become far clearer, more spacious and more realistic. This effect comes off as eerie in several tracks by Baths, with the sibilance in Lovely Bloodflow and Miasma Sky suddenly seeming to breathe down my neck. The Dragonfly also shines when it comes to bass. Luckily bass isn’t boosted here, but becomes tighter. This is not always apparent in electronic music but is clear in more acoustic music: The drum kicks in Aaron Jerome’s very well-mastered version of Dancing Girl became less boomy, enabling me to hear the actual kick of the bass pedal more clearly. In terms of mids, I haven’t been hearing much difference. Vocals, guitar and piano do seem more spacious and maybe a touch more forwards, however a different pair of headphones would probably show more improvement.

So what’s the verdict? Has the humble Dragonfly revolutionised my music experience? The answer is both yes and no. For better or for worse, the Dragonfly and the ATH-M50XBL enable me to identify well-mastered albums. Tracks on these albums sound clean, detailed and generally sublime. But not everything is well-mastered (Amy Winehouse and Imagine Dragons come to mind). It is on such albums that the Dragonfly effect is wasted.

1. Rock can sound amazing. The airiness and natural sound the SR125i delivered was unlike anything else I’ve ever heard for rock and jazz music. There’s something about how guitar, piano and strings sound on these headphones that brings life to such genres. The best example I heard of this was probably The Summer You Never Meant by The Exponents. From the first strum the song absolutely brimmed with energy while sounding completely effortless.
2. A natural sound isn’t everything. That lovely acousitc sound is awe-inspiring for a lot of music, rock and jazz especially, and was one of the main reasons I settled on the SR125i in the first place. But I quickly realised that for someone who mostly listens to electronic music, a natural sound actually isn’t desirable a lot of the time, and this mostly has to do with the nature of open-backed bass. The bass was there, sure, but unlike closed cans, this bass serves only to supplement the strum of a bass guitar or to add oomph to the kick of a drum. It is simply not capable of provide a driving force to bass-heavy music - the very driving force electro and hip-hop music requires.
3. Versatility never hurt anyone. I’m pretty easy when it comes to headphone comfort, and I was never really bothered by the simple, poorly-padded design of the SR125i. What did irk me, though, was the presence of a 6.35mm audio jack and the absence of a removable cable. I mean, really, who would prefer to have a 6.35mm jack over a 3.5mm one? I guess it didn’t help that my two dollar adapter slowly began to give up the ghost shortly after I obtained the SR125i.
4. Audio-Technica knows a thing or two. I honestly don’t know where to start: style, comfort, sound, versatility; the ATH-M50X really does have it all. I managed to pop back to The Listening Post while the blue version was still available, and what a pair they are. They’re like the good-looking, more spacious, more detailed, older cousin of my old HD-212 Pro. On this note, maybe the SR125i was the old, wise (but often frustrating and irrationally inflexible) grandparent.

The SR125i is dead, long live the ATH-M50X!

The Dragonfly-SR125i combo has been a revelation for me in many ways. Rock music has come alive with a new level of detail. In more simple tracks like R.E.M.’s Nightswimming, the action of the piano is clear and vocals are reasonably forward. Complex tracks such as Dirty Work by Steely Dan sound layered. I’ll admit it though, while the SR125i’s bass does have decent extension (Clean Bandit’s Nightingale is a great subbass test) and presence, it lacks the punch of my trusty HD-212 Pros. I’ll definitely be keeping my previous headphones for their extremely fun (if far-from-neutral) sound signature, which is well suited to electronic music, as well as for their isolation.