Home Uniquely NZ Travel Howto Articles Pauline Small Firms Settings Search

Home

Uniquely NZ

Travel

Howto

Pauline

Small Firms

Advanced Search Search of pcurtis.com by Freefind

Upgrading Corinna's Electrical System with a Raspberry Pi with additional Node-RED software.

The initial document on the Upgrade to Corinna's Electrical System got far too long so I decided to cover the addition of a Gateway to allow remote monitoring using a Raspberry Pi 3B+ on new pages. There are currently two new parts: the straightforward addition of The Raspberry Pi using the Venus OS and The Extension to Venus OS Large with Node-RED to allow the addition of a dedicated Dashboard and additional smart management.

Much of the initial work using versions 2.70 and 2.80 has been overtaken by a new Venus OS versions 2.90 (and now beta versions of 3.0) with major changes including full support for the Pi 4B as well as the Pi 3B and inclusion of the Large version, which supports Node-RED, as part of the standard build. The initial work, including running on a standard Linux machine, has been preserved and is available here whilst this page only supports the latest versions 2.90 and higher

Parts are still work in progress as it serves as a working diary/log book for the development with several important sections broken out into Appendices.

Introduction to Venus OS Large and Node-RED

My current installation of Victron Smart devices I am using on Corinna and the addition of a Raspberry Pi to provide a gateway to allow remote monitoring has been covered on the previous page The Raspberry Pi using the Venus OS There is an extended build of Venus OS under development called Venus OS Large which adds Node-RED and a Signal K server. To quote Victron Energy:

Node-RED is a tool for connecting hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows. With it, one can for example program a functionality such as activating a relay based on a temperature measurement. Or make far more complex algorithms, tying relays, measurements, or other data available from Venus OS or elsewhere together. All without having to write real source code, as Node-RED calls low-code programming for event-driven applications.

Node-RED features a fully customisable dashboard, viewable in a web browser - both locally but also remotely, via the VRM Servers on the Victron Portal.

The Signal K server is aimed for yachts, and multiplexes data from NMEA0183, NMEA 2000, Signal K and other sensor inputs.

Venus OS Large runs well with Node-Red on my Raspberry Pi 3B+ when installed using the procedure in Venus OS Large. It is ideal for those with some programming knowledge who wish to tinker or customise their systems and have a sufficiently powerful processor in their Victron controller or a Raspberry Pi. Images are available for both. The idea of an improved dashboard and extra control from Node-Red was very attractive to me and I tried it out once I had got the basic system Raspberry Pi up and running. The image I installed and used for much of my early development and subsequent use on my narrowboat Corinna for the 2022 season was Venus OS Large v2.80. In parallel I was running a Raspberry Pi 4B for development using the Candidate (Beta) series of the Victron Venus OS version 2.90 which now fully supports the Pi 4B. Once 2.90 was formally Released the 'Operational' Pi 3B+ on Corinna was updated. There are considerable improvements in version 2.90 not least in fully supporting the latest version of the Pi, the 4B rev 1.5. To avoid any confusion this document is only intended to support the Venus OS version 2.90 and higher with some references to the transition. The initial work, including running on a standard Linux machine, has been preserved and is available here.

To cut the story short for this introduction: The Raspberry Pi 3B+ has adequate power for what I have tried so far and the following sections describe my journey of exploration, currently much is in a diary form. In general the use and testing of the Node-RED extension has not affected the basic operation of the Venus OS, data gathering by the Pi and transfer to the Victron Portal and I was able to mostly work remotely through the VRM Portal rather than on a very cold Corinna over the initial winter.

Exploration of Node-Red on a Desktop running Linux Mint (Not essential)

For the first few weeks I had Node-RED installed on a computer running Linux Mint, it was a useful learning exercise but not essential. I used it to understand how Node-RED works and explore Node-RED capabilities without risking my system which was working well on Corinna. There are instructions on loading the basic Node-RED software under Debian/Ubuntu and hence Mint at https://nodered.org/docs/getting-started/local. There were no Victron Nodes but it was enough to try out some of their examples and investigate how it worked, progress rapidly and enabled me to write the next section.

The addition of a Raspberry Pi 4B as a backup and development machine

I have more recently bought a Pi 4B and that has been running the most recent Venus OS Candidate (Beta versions), now version 3.00~4 . Version 2.90 has many changes which improve functionality and security and work reliably on the Raspberry Pi 4B unlike earlier versions. The Pi 3B+ is still manufactured and arguably the best matched machine but is on extended delivery and I feel much more secure having a backup available.

Using Node-RED - a simple explanation

Node-RED works on the basis of a pictorial workspace where you can drag a selection of nodes from a palette on the left of the workspace and connect them together by dragging from a nodes output to another's input. What this is doing is connecting a series of messages between outputs and inputs each of which has a payload. In essence these streams of messages are asynchronous and everything is event driven. To the right of the screen is a multi purpose sidebar (panel) which can display debug information, help and several other panels useful for configuration and diagnostics and importantly it allows one to configure a separate Dashboard - a User Interface (ui) to display and control the system.

Nodes can generate messages, change messages as they pass through (a function node) and display messages on the dashboard amongst many other things. The display nodes are very powerful and it is as easy to display on a dial or graph as a number on a dashboard.

The Display Nodes (ui) are not installed by default and have to be added - at the top right is a menu (three horizontal bars) and you need Manage Palette and search for node-red-dashboard on the Install tab and Install - you will then find a big suite of powerful display nodes in the left hand panel. One will also find a large number of other useful nodes that can be added the same way.

Victron devices such as a Victron Solar Charger are available as Nodes by default and you end up with multiple instances as each input and output from a device is added as a separate node - most nodes only have a single input and/or output so, for example, the the output representing solar power is a series of messages every 5 seconds. You add another copy of the Victron Solar node and set it up internally to, say, give the state which will be another series of messages every 5 seconds.

When one comes to use Node-RED one has to understand one fundamental difference from most programming one has done before. Node-Red is about flows. It is about messages with payloads being passed so in many ways it is transitory. An instrument produces a series of messages at regular intervals rather than a set of variables you work on. A good example is a panel switch node on your dashboard. When you turn it from off to on it sends a single message of true and a single message false when you turn it off. It can be used to carry out an action immediately by a device which responds to a single message such as 'turn inverter on'. But if you want to use that message to change anything in the future the true or false has to be captured. This takes a quite a bit of getting used to the first time.

All recent versions of node red have a feature called 'context' which provides the ability to save a value in the equivalent of the more usual variable and access them again with various scopes such as the same node, the whole flow or globally and even save them on a regular basis to a file to preserve them across restarts. Note saving and restoring from 'context' may require put() and get() functions in some cases, you need to read it up carefully. Being flow oriented is good in many ways and makes embedded device programming easier but carrying out more convention logical actions can seem overly complex initially.

Accessing Node-RED: You access Node-Red's workspace and display areas (User Interface or ui) in almost the same way as you use for the Venus OS via a web browser. The basic OS is accessed by pointing your browser at the IP Address of the Raspberry Pi on the local network or remotely via the Victron portal. To get to Node-Red locally on the same network you just access a specific port so if the Raspberry Pi is on a local port 192.168.1.127 then Node-Red is at 192.168.1.127:1881 and the Dashboard (ui) is at 192.168.1.127:1881/ui. On most local area networks you can just enter venus.local:1881 and venus.local:1881/ui . Note: This differs from the standard Node-RED where it is an http connection on port :1880. In the case of the portal an extra menu item appears to take you to the Node-Red workspace to set all up or the Dashboard to display it.

Installation on the Raspberry Pi 3B+

Since Venus OS version 2.90 Large images which include Node-RED are available for both the Pi 3B+ and Pi 4B. I used a new MicroSD card and bought several identical ones so I could make exact clones for backup. In theory the cards only need to be the same size but in practice it has been reported that anomalies can occur due to detailed differences in the internal implementation of the cards but I have not had problems and I have already covered cloning, including to different size cards, in an Appendix - Simple cloning of the microSD card as a backup - in the previous document.

There are periodically new versions which have enough differences to make one consider an update but first I need to keep track of exactly what changes I have made and how to reproduce them. I have a list of changes/settings - many (in brackets) can be made in Remote Console once you are connected.

1. Pair Bluetooth with Pi from within VictronConnect App (pin changes survive updates)
2. Provide Password for Wifi (Settings -> Wifi) , Survives update
   1. Find and note ip address - the same
3. Set access level to Superuser (Settings General, survives update))
4. Create a root password. Needs to be re-created after an update to re-enable SSH and SFTP login.
5. Enable Remote Support to enable sshd and reverse tunnel on VRM and LAN in Remote Console (Survived update)
6. Turn on Two way Communications in VRM to allow use of VictronConnect over Internet (Survived update)
7. Turn Off auto-update (Settings -> Firmware, survives update)
8. Turn on Node-RED (Venus OS Large Features, Survived update)
9. Make changes to provide persistence in Node-RED in settings-user.js from v2.90 (Survives update)
10. Extra nodes in Node-RED (added to palette by Palette manager) (Survives update)
    • node-red-contrib-victron (pre-loaded)
    • node-red-dashboard (essential)
    • node-red-contrib-cpu
    • node-red-contrib-stoptimer
    • node-red-contrib-timeswitch
    • node-red-contrib-os
    • node-red-contrib-simpletime
    • node-red-contrib-ui-artless-gauge (More flexible with linear and 270 deg gauge options)

Using Node-RED with Victron Devices

So lets actually go a bit further and look at what we need to do an actual useful task. For example, I want to be able to have a nice display with a 'dial' displaying the state of charge of my batteries and another showing the voltage from the Victron Smart Shunt and make it available on my phone locally and remotely. Once you have the Victron Large OS which is the version of the Venus OS updated to have Node-Red installed along with all the Victron Device Nodes we will find this becomes almost trivial using only half a dozen nodes total for the two displays. So lets have a look at how it all works. You access Node-Red's workspace and display areas (User Interface or ui) in almost the same way as you use for the Venus OS via a web browser as described above.

After you have made any changes in the workspace, locally or remotely, you have to Deploy them before they can be utilised - Deploy is the big red button at the top right on the black title bar and there is also a 'hamburger' menu to the right of it which has many useful items including an Import and Export to save your work and Palette to search for and install extra nodes.

Things I have learnt, mostly the hard way, which were not obvious when I started.

Node-RED runs as as a service (a daemontools service in Venus or a systemd service under Linux Mint) This is already set up for the Venus OS Large.

The dashboard layout is in a tab in the side-panel and it is possible to work top down and set up extra tabs and groups and drag nodes to reorder and drag between them regardless of the flows they come from. Groups are actually useful to improve responsive design. Deleting is done by edit -> delete in opened panel for Tabs and Groups etc. Double click the name in a flows tab to edit it.

Switches are difficult to use as one is dealing with flows of messages and on/off switches buttons and drop down menus only provide single messages which are best held using a context variables. Even then the variables do not persist through restarts in the default system but that can be fixed by a simple addition of a configuration file.

Context may need get() and put() to work so the instructions need to be read well. See https://nodered.org/docs/user-guide/context

You can join many flows (streams) into a ui display node input such as four core usages of a processor and likewise into debug nodes. The flows are always one way.

There are some useful examples of Node-RED with Victron devices at https://github.com/victronenergy/node-red-contrib-victron/wiki/Example-Flows

My initial Node-RED ToDo List

This was my initial wish list of what I want to develop for Corinna, initially tested using the local system under Linux Mint to reduce risk before deploying on the Raspberry Pi on Corinna.

• Set up a Dashboard to display the information needed to manage Corinna's electrical system and control it.
• Make a separate diagnostic tab with options to display CPU Loads, Temperatures etc with switching to save power, if required.
• Add buttons to the diagnostic tab for Restart, Cancel and Shut Down. See above for the importance of this.
• Add button for Node-RED Restart.
• Investigate start-up and initial conditions after Deploy and RPI restarts.

These are covered in detail below.

My Node-RED implementation for Corinna.

The development has been an iterative learning process making use of both a Node-RED system running on a Linux Mint computer for a few days and then the actual Raspberry Pi 3B+ on Corinna. I however now feel it more appropriate to give a coherent description to make it easier for others to follow and to set up the system they want without the many (interesting) diversions in the path I followed. Being truthful, it was a bottom up approach without a full appreciation of what could be achieved and how it should be integrated. If you want to follow my original path or consider installing Node-RED on a Linux based laptop or desktop the original page is still available here. The write up is being tidied to only cover the latest versions namely versions 2.90 and higher when major changes were introduced.

Initiation and restarts of Node-RED and Raspberry Pi

One of my concerns from the start was that the Raspberry Pi has no tidy shut down or restart mechanisms in its hardware and depends on the Power Supply being turned on and off. I discussed the problem earlier including the terminal commands which can be used by a user logged in over SSH. Node-RED allows one to get round these problems from the dashboard and the ability to do restarts is a vital part of testing the initialisation of the system, preferably remotely to avoid going to a cold boat in the middle of winter! This was therefore to be one of my first developments for the real system on the Raspberry Pi.

Node-RED provides an Exec node which enables one to execute a single line of commands but the shutdown command normally has to be run using sudo or as root. The way the Victron Venus OS was set up when I started had Node-RED running as the Superuser (root) which means that shutdown was not a problem although running as the superuser does leave one open to security issues.

The latest version of the Venus OS has sudo built in but sudo normally calls for a password so some further modifications are required which are covered fully in Appendix I - Enabling the Use of sudo (Venus OS 2.90 and higher) . The bottom line is that it takes the addition of a single file with a few lines of active code. I also believe in commenting extensively.

I have added three buttons to my Diagnostic tab on the Dashboard for Restart, Cancel and Shut Down with notification before the Shutdown or Restart so that one has one minute to cancel the actions in case they have been activated by mistake. This enables me to check how initiation takes place when the machine is booted and the node-red-venus service is started. This is quite important as much is transient and not saved during a restart, the devices should not be affected but switch settings and modes you have set up are not preserved as they are in volatile memory and it is not obvious the initial state of say as switch is defined by default.

An aside on the dashboard: Up to now, I have been referring to the Dashboard as if it were a single panel but it can have any number of tabs and groups on each tab: initially everything was mixed together but at an early stage I separated the diagnostic and testing functions onto separate tab and I then added a third tab for the 12V system so the main display stays the same to avoid confusing other users ie Pauline. Each tab needs to be restricted in size to fit onto our tablets without the need for scrolling in normal use. As previously mentioned, there are powerful GUI tools to rearrange the dashboard layout which are well worth exploring as ones displays become more comprehensive.

Restarting the Node-RED Daemon as from a reboot - Daemontools

It is very important to be able to restart Node-RED whilst testing so I have added a button to the Diagnostic tab to restart the node-red daemon without a machine restart. This was tested on my trial Linux Mint machine and the restart of node-red works as expected with sudo present and I thought the job done but that was not the case for the Venus OS Large implementation on the Pi. What had seemed a trivial task turned into several days of frustration but a much better understanding of the Victron Venus OS.

I was unable to restart node-RED under the Venus OS Large in the same way as a normal linux machine. It seems that systemd is not in use in Venus. I did various searches of the Venus documentation and the impressive knowledge base in the community with little success. It rapidly became clear that the management of initiation and services was not by any of the usual sysVinit, upstart or systemd that I have grown up with. After spending a lot of time I eventually found that they use another completely different collection of tools called daemontools developed by D J Bernstein - see https://cr.yp.to/daemontools.html which is my main source for the following paragraphs. The clues to its use were the svc and svok that were in busybox and confirmed by the presence of a folder /service which contains a folder node-red-venus. Put ls /service -l | grep -e node-red in an Exec node if you do not have easy ssh access to check. Further searches including svc found found a brief references in https://github.com/victronenergy/venus/wiki/commandline-introduction to confirmed it was used in Venus OS large.

Having read some of the documentation I can certainly see the attractions of daemontools for a basic embedded or headless system and it is well matched to the use of busybox. One of its strengths is that services are easy to start and if your daemon dies it is automatically restarted with ~5 secs. Another is that when the system administrator restarts a service under daemontools, the service receives the same fresh process state that it received at boot time whilst with other systems one may have considerable extra work to clean up environment variables, resource limits, controlling ttys, etc.

Daemontools also has easy, reliable signaling. One can use svc to control your daemon. For example:

svc -u /service/yourdaemon: starts or brings a service back up
svc -d /service/yourdaemon: sends TERM, and leaves the service down
svc -t /service/yourdaemon: sends TERM, and automatically restarts the daemon after it terminates
svc -h /service/yourdaemon: sends HUP (see below)
svc -o /service/yourdaemon: runs the service once

HUP goes back to the days of modems and terminals standing for hangup and covered the actions when control over a process was lost. It is [extensively] used with daemons but its interpretation is less well defined - in most cases it is interpreted as a request for the daemon to reinitializing themselves. The default behavior for HUP is the same as TERM.

In both the cases of options -t and -h one makes use of the automatic restarting of the daemon with a fresh process state if it is ever found to not be running. I have tried both the -t and -h options to give my restart/re-initialisation of node-RED and both seem to behave the same and both give seem to give quite a slow response through the VRM portal the and it takes over 20 seconds for the dashboard to come back up. I made a final decision to use -t after use of both.

Other Daemontools command examples and information

• svstat /service/logscript
• svstat /service/* | grep yourdaemon
  

• Venus OS operational commandline manual
• https://github.com/victronenergy/venus/wiki
• https://github.com/victronenergy/venus/wiki/commandline-introduction
• Overview information on supervise which is top level of Daemontools and associated commands are at https://www.freebsd.org/cgi/man.cgi?query=supervise&sektion=8&apropos=0&manpath=FreeBSD+13.0-RELEASE+and+Ports and http://manpages.ubuntu.com/manpages/impish/man8/supervise.8.html

What mechanisms does Node-RED provide to help Initialisation.

The primary mechanism is arguably the inject node which can be programmed to inject almost any single message after 0.1 seconds which can be used to start and initialise flows and set up context variables. It is alleged that the first inject to be run is on the first tab but I have not checked.

switch-ui nodes can be controlled by Node-RED messages as well as from the dashboard. As well as the pass-through of messages messages with payloads 'false' and 'true' can be used to change the switch which provides a potential mechanism for initialisation of switches.

function nodes have a mechanism to run specific code on node start and stop and a setup which I have not yet investigated or needed.

It is not trivial but Global Context Variables can be saved and read from a file to maintain context information and thus retain the switch status through a restart situation. I think this is an essential change. See the following section for details.

Important: Use of Persistent Context Variables

Node-RED offers the ability to choose between storing context variables in memory or in a file on the SD Card but the facility is not enabled by default and it is best to enable it. Most of my Node-RED flows depend on it and I can not understand why it is not enabled by default.

There are two sorts of people: the trusting and the curious. This sections is for the trusting, the curious will find the full background and explanations in Appendix J -New Procedures for Node-RED settings including 'Persistence'

So the quick trusting answer is that one has to add a short file which contains any extra or changed settings where any changes will take priority. The file is not present on freshly installed system. If you find you have one you may have to add the magic incantations to it rather than overwrite it but it is unlikely. The file is placed in data/home/nodered/.node-red/settings-user.js and the contents are:

// Contents of /data/home/nodered/.node-red/settings-user.js
module.exports = {
// uiHost: "",
   contextStorage: {
     default: "memoryOnly",
     memoryOnly: { module: 'memory' },
     file: { module: 'localfilesystem' }
   },
}

There is other change in settings to seriously consider: Node-RED is now, by default, accessed at https://venus.local:1881 and the ui at https://venus.local:1881/ui . This is different to the Node-RED documentation but is obviously more secure. However the use of https on local networks with a personal secure key causes considerable grief with Android but a line can be added to settings-user.js to re-enable the http connection on port :1880. You will note there is a line uiHost: "", commented out above. If you un-comment that as I would recommend during development you will have both secure and less secure access access via port :1880 re-enabled on your local network.

Transferring or editing settings-user.js

I have largely given up bulk editing using vi, the built in editor on the Pis. Vi is a terminal editor which goes back to the early days of Unix and still has a following by sysops but has a steep learning curve. For the masochistic I have put a basic guide in Appendix A - Using the vi editor (and visudo) in Busybox.

Instead I use FileZilla, an excellent free, open source cross-platform FTP (file transfer program) to transfer back and forth over a local network from a laptop or desktop where I can use familiar tools to edit or create files. Other FTP programs should also be fine, for example, I use the popular AndFTP App on my Android Pad.

It is sensible to stop Node-RED before changing its settings file although I have never had any problems just uploading but I always reboot afterwards or restart node-red-venus. If you do want to stop and start Node-RED log in via an SSH login the magic incantations when using daemontools are:

svc -d /service/node-red-venus // sends TERM, and leaves the service down

svc -u /service/node-red-venus // starts or brings a service back up

Using persistence in Function Nodes

The use of persistence is not as simple as I expected but eventually I found the best way is to set and get variables in a function node is like in this example:

// Multiplication example to calculate power and save in another context variable

var num = global.get('shunt_voltage', 'file') * global.get('shunt_current', 'file');
global.set('shunt_power', num , 'file');
msg.payload = num
return msg;

Note the unexpected [to me] need for quotes. See https://discourse.nodered.org/t/a-guide-to-understanding-persistent-context/ for extra details.

Random Tip: The debug node has an option at the bottom called node status (32 characters) which displays the first 32 characters of the debug message below it as well as in the debug side pane. This is very convenient during testing so I always use it and have left many of the debug nodes in place with their normal side pane output disabled.

Conclusions - How is the Raspberry Pi system with Node-RED, working in practice?

As far as I can tell the complete Raspberry Pi system is all working as it would with a Victron Venus GX (which has no display), in fact it is better in many ways at a much reduced cost. The major cost has been the VE.Direct to USB isolated cables, which are required, and are more expensive (£27) than the VE.Direct cables to Victron's own Gateways. The Raspberry Pi 'gateway' also has Bluetooth so it can be accessed directly by the SmartConnect App for setting up, a feature which most of the Victron GX devices lack.

So we now have several routes to control and monitor the system and can do it locally and remotely. As far as I can tell there is no difference in functionality between using the SmartConnect App via Bluetooth or the VRM portal, except perhaps in smoothness as there are always some lags through the internet. I find the WiFi gives much better coverage through the boat than Bluetooth despite being provided by an elderly tethered Samsung Galaxy S3 mini (3g) which lives on a bracket in a window. So we have everything we had before with more local range even before we started to use the portal for world wide coverage. I am yet to check the data use fully but I do not need to worry too much in the UK and EU with a GiffGaff Goodybag providing 15 Gbytes of 4g data a month for £10. Initial impression is that it may be ~1Gbyte/month

The Portal is very useful (and it is frequently getting updated and improved) and it is very easy to watch the Solar and Battery status real time, daily and historic data from the comfort of the house. The views can be fully customised and the configuration is saved on the Portal. There is an App for Android devices as well as access through a browser. It does not seem to be possible to simply control my devices to say switch the inverter to and from Eco mode or configure any parameters directly from the Portal but that can all be done by the SmartConnect App and both can be open at the same time. I can see the logic for separating monitoring from configuration but a few simple switches would be convenient, in particular inverter mode.

I have checked that the data continues to be stored in the Raspberry Pi if the internet connection is broken and then uploads to the portal as soon as it is reconnected. However, if the Pi is rebooted local data not already uploaded does seem to be lost. Even if the internet connection is temporarily broken one can still access the basic dashboard via any machine on the local wifi network (or via a network cable for ultimate reliability onboard) at venus.local.

I gradually built up a comprehensive set of data stored on the VRM Portal where it is stored for at least 6 months and can also be downloaded for additional processing in addition to that gathered via Bluetooth which is limited to 30 days.

The use of the Venus OS Large with Node-RED has added an extra dimension and is my major access when onboard for remote monitoring. Node-RED allows much greater flexibility including 'intelligent' control of the devices via your own configurable dashboards. Node-RED can be accessed and programmed via the VRM portal and I have done all the Node-RED programming and 99% of testing remotely. At an early stage I implemented remote control of the inverter including an option to automatically return the Inverter to Eco (Search) mode from Continuously On after an hour to save power if one forgets. I have also added an extra 'Tab' to the dashboard to monitor the Pi CPU loads and processor temperature and shut down and restart the Pi and Node-RED itself. And that was during the early days!

Finally the latest versions from 2.90 make a quantum step forwards especially for users of the Raspberry Pi 4B (v1.4 and v1.5) which are now an obvious choice. There are major improvements for all users of the Venus OS on all platforms, particular for users of the Large version with Node-RED where the install and update proceedures are now harmonised with the normal version. Security is much improved resulting in minor changes in some Node-RED flows because of the loophole of running as root being rightly closed and by use of https: for networking.

Appendices

Appendix A - Using the vi editor (and visudo) in Busybox

One may need to edit a configuration file on the Raspberry Pi using access via SSH. The Pi uses Busybox to provide a very cut down number of Linux utilities which mostly have reduced functionality - vi is no exception. vi was THE original unix editor and almost every linux system has vi or its advanced version vim. It has a steep learning curve for those used to recent graphical editors but is very effective to make quick changes to files when you are experienced especially on a headless system such as a server or embedded system when only 'terminal' access is available. It is worth noting that in many cases the use of vi can be avoided by using SFTP and file managers such as FileZilla and editing on a different machine with more powerful editing tools before returning the file.

visudo is a special vi which Must be used to edit the /etc/sudoers file and checks the edit is valid.

General Tutorials on vi:

• https://staff.washington.edu/rells/R110 - the initial 'Basics' section may be enough.
• https://www.tutorialspoint.com/unix/unix-vi-editor.htm - a sensible alternative but you may need to read further.
• https://k.japko.eu/busybox-vi-tutorial.html - one of few dedicated tutorials to the busybox version of vi I have found and claims to cover every command as he has worked from source code. Has a list of differences from Vim at the end which will be useful to regular users of Vim. I have downloaded as a pdf in case it evaporates! This is a must read before playing with visudo as the busybox vi commands are so different from ordinary vi and visudo is not the place to expriment!

The version of vi in Busybox is cut down in some facilities but easier than I remember to use in other ways. All the basic operation seems to remain the same excepting many :set commands are missing.

Cut and paste via Ctrl C, V and X are not available in vi but in Busybox vi the cursor movement keys seem to work [almost] as normal and in Command and Insert modes. Center click and roll also seem to work if you have a mouse.

Esssential summary:

• i to enter insert mode
• Escape to return to command mode
• :write saves work
• :quit to leave after changes saved
• :quit! forces quit with unsaved changes
• yank, put and delete are the internal equivalents of copy, paste & cut and work in Command mode and are more powerful than a newcomer to vi might expect as they are leveraged by various selectors for complete lines and multiples.
• right click cut and paste works internally and to other programs which is different to what I recall of vim.

Use under Android. Escape is fundamental so you may need to use the hackers keyboard. JuicySSH seems to add some useful extras to the keyboard which may make its use possible. Also in most implementations including vi Ctrl [ is equivalent to Esc . The internal yank and put may be better than android's copy and paste. I have not tried a copy paste from another program into vi running through JuicySSH yet.

Appendix B - Working with Switch Nodes

Switches (and hence switch nodes) are going to be a fundamental building block we will use in Node-RED user interface (ui). Working with flows (messages) and the switch node is rather different to normal programming with variables and needs a little thought as to how to implement it. First lets have a look at the help information for the Switch Node.

Extracting the relevant bits of information that I use.

• The switch node only produces a message when it changes state - by default this is boolean true and false.
• The switch can be changed by clicking/sliding the switch in the dashboard or by sending the same messages to it that it sends. They will change the state displayed and pass on the message so everything stays in synchronisation.
• A topic can be specified which will be added to the output as msg.topic.
• It does not maintain the state if Node-RED (or the machine) is restarted.

So we have to work with one off messages indicating a change of state from the switch node and that means that for most purposes one has to use them to set a 'context' variable which can be used for what are thought of as normal switch functions a steady on/off state for logical decisions and a way of enabling and inhibiting a flow of information. We also need a way to ensure that we maintain the state of the switch through power cycling and restarting like a physical switch or, at a minimum, starting in a defined state.

What we end up with is that the we use the on/off messages to set a context variable, I have chosen to use a global one although that is a wider scope than I am currently using. I am also using a persistent variable (one that is saved periodically in a file so it survives power loses, restarts etc. The 'persistence' facility is not installed by default and 4 lines have to be added to system file to enable it. I have discussed hot to do that elsewhere. Once that variable is available I can be used in Function Nodes in many ways including 'gating' a flow of messages. It can also be used to restore the state of the switch after a restart.

I have used two implementations of the principle above. The first uses a Change node to take true|false outputs from the Switch Node to Set a global contact variable stored in a file and a separate Function node to carry out the gating action. The value of the context node can be Injected when Node-RED is restarted to give persistence to the Switch Node. This makes maximum use of built in facilities but risks an ill defined state the very first time it is run as the context variable is not defined until the first change of the Switch is made in the Dashboard. If that worries you an extra Inject can be used to manually input the initial status. I favour the following solution:

If we are clever a single function node can use the messages from the Switch Node to set the context variable and 'gate' all other messages depending on the switch. The value of the context node can be Injected when Node-RED is restarted to give persistence to the Switch Node. So in many cases we can do everything with three nodes: The Switch Node for the user interface on the dashboard, a simple dual purpose Function Node and an Inject node to set the initial status or restore the status after a restart. The following is the code for a Function Node which I use to control a flow of messages used by my display of the CPU loads and CPU temperature but it could be any flow. In this case the message topic being set to gate is used to distinguish between the messages from the switch from other messages. It has an advantage over the apparently simpler method above in that it predictably covers the first ever run as well as maintaining the switch setting during restarts.

The example code given below can be imported to see how it all adds together. It requires the settings.js to have been modified to allow saving context to a file and the ui nodes to have been installed using palette -> import and searching for node-red-dashboard. It gates a timestamp flow from an Inject node to a Debug node as a useful example as that timestamp flow is often used as a clock tick in a system to, for example, update displays.

I have exported the flow as a file named example_switch_flow.json which you can imported to see how it all adds together. You should download the file (probably using a right click menu) and import it to a new flow. If you just click on it the results are unpredictable in that they will depend on browser you are using!

NOTE: It requires the settings.js to have been modified to allow saving context to a file and the ui nodes to have been installed using palette -> import and searching for node-red-dashboard. It gates a timestamp flow from an Inject node to a Debug node as a example.

If you get this example working and understand what each Node does you are probably over 80% of the way to being able to make practical use of Node-RED.

If you are using a Raspberry Pi (or Victron Controller) running Venus OS large and have a Victron device the next step is to use a display to look at the output flows from the device. If you are doing your preliminary stages on a normal computer/laptop/pi running a Linux kernel we need an alternative device to look at. The Node-RED community have provided a node which produces information from a device every controller has, the processor. This is actually very useful addition to ones system in any case as it enable one to check the processor loads and CPU temperature so you can make sure you are not overloading the system. The CPU Node works on the Pi and my systems running under Linux Mint which is flavour ultimately based on Debian and is supposed to work much more widely but I can not give any guarantees.

Appendix C - Monitoring the Venus OS using Node-RED

Warning - There is a major change in Node-RED in Venus OS Large 2.90 when Node-RED will run no longer run as root but as a user called nodered. This may require the permissions to be set for some of the following fuctions.

I think that it is important to be able to monitor the system to make sure that it is running reliably and it is well within the resources available. I have several applets on my Linux Mint Systems to enable me to do that and I would like to be able to easily check the Venus OS system on the Raspberry Pi in the same way. The information I believe is useful and I keep continuously available via applets on my Linux home computers is the CPU load, the memory (RAM) usage and the CPU temperature. I also monitor Network Data and when I have a separate Graphics processor I also monitor the GPU temperature on my home machines. I can then see at a glance when I have problems such as memory leaks and it is surprising how often, for example, a browser tab starts using excessive resources to the point the machine is overloaded.

It is very different with the Venus OS. It is an embedded system essentially doing the same thing all the time so we are not concerned with rogue browser tabs but we do need to make sure that the processor temperature is sensible as the Raspberry Pi has no active cooling so we need to keep the processor loads down to save valuable power from our batteries. We need to keep an eye on memory use as memory leaks are unacceptable on a system which is remote and may not be physically accessed for weeks over the winter. We also need to be able to do remote restarts of the Venus OS and Node-RED during development and as a contingency if one identifies a problem building up.

So, right from the start, I wanted to develop ways to monitor and investigate the OS performance and behavior remotely. Node-RED has a node which executes a terminal command and there are many terminal commands available to monitor the system so the simplest way to get a quick answer is to put a terminal command into an Exec node, activate it with an Inject Node and look at the output in the Debug Sidebar with a Debug node - a minutes work! There is a slight catch and that is that a limited subset of such commands are available through busybox so you may have to check your favourite commands exist and have the options you require so I have installed busybox on my Linux boxes running Linux Mint so I can check before playing on the Venus OS remotely. So lets look at specifics:

Restarting the Raspberry Pi and Node-RED I I have already covered those and I have them at the top of the Dashboard tab with the various System monitors which can be switched off to save power if required.

CPU Load and CPU Temperature: These are some of the most important to keep an eye on and they interact. A high CPU load is bad for a real time system and can even lead to unpredictable behavior if there is the chance of race conditions. A high load leads to high power consumption and rising CPU temperatures - once the temperature reaches about 75 degrees the processor cores re slowed down and the load factor increases. Computers for embedded systems rarely have any cooling like fans so you can end up with much less margin in computer speed than you expect and then a hot day comes along! I prefer to use code I understand although there are also a number of relevant Contributed Nodes, like the one for the CPU Load I discussed above. I had been using that for a while and I had already added it to my palett, so I have compromised and used the extra node-red-contrib-cpu Node because it did just what I wanted to extract and displays not only the processor core loads but also the temperature and seems to add very little overhead. Otherwise I have gone back to basics. So what else do we need and what are the utilities we can use from a console to get it?

Random Access Memory Usage - This is very important to monitor as the Raspberry Pi 3 only has 1 Gbyte of fast RAM. A Swap capability is not active as it would cause large numbers of reads and writes on the SSD card and, in the limit, risk premature failure. The level of 'Memory Usage' is not quite as simple as it might seem as the Linux kernel does not waste any apparently free memory and it is automatically allocated to buffers and cache to speed up access and memory is also used by temporary file systems so a well set up system should rarely have much 'free' memory. What matters more is how much memory can be made available when cache and buffer storage is recovered by the system but one must remember that as memory is recovered it is at the expense of buffers and cache which will slow down the system. My monitoring on the home computers show a near real time pie chart of the different uses and it is fascinating to see how the kernel is continually optimising as different programs run. That is obviously not appropriate for an embedded system but I have set up the node-RED dashboard to have a graph of RAM memory usage as well as an instantaneous display. I use exec nodes and the free command piped through a grep followed by spliting the resulting string into an array:

free | grep Mem

myArray = msg.payload.trim().split(/\s+/);
msg.payload = myArray
return msg;

This is my earlier coding for selecting fields from a record and uses the trim to remove any leading or trailing white space then the split to split on any intermediate white space. The relevant array element is then chosen by Array element. Most of selection is done on my later coding by using awk.

Disk Space: Recall that the Venus OS has 4 partitions, one small boot partition which also handles the switching between root file systems, two root files systems of which only one is ever used, the other is a backup of the previous install and a data partition which gives persistence between updates. The main space is basically partitioned equally between two root file systems and a data partition and symbolic links bring the parts together into a more conventional layout. There should never need to be a shortage on the Raspberry Pi when running the Venus OS as microSD cards currently get cheaper as they get larger and the root file systems are very small and the data from the devices is not only small but is only buffered before uploading to the Portal long enough to cover any breaks in network coverage. The portal stores data for months. I have added a readout of disk memory use for completeness - the root file system only occupies part of the partition and is read only so is fixed at ~90% but the file system only occupies about one third of the partition. On a 32 GByte card the data filesystem usage is well under 1%. An 8 Gbyte SD card would be generous and 4 Gbytes possible. The advantage of large cards is that their life is dramatically increased if there is unused capacity thanks to the clever algorithms in the manufacturers drivers on the cards. The Exec command used is

df -mT | grep ext4 | grep 'sda2\|home\|root\|data'

The second grep contains the 4 OR options so the code can be used on the Venus OS or on my Linux Mint OS.

The resulting string is split into an array.

myArray = msg.payload.trim().split(/\s+/);
msg.payload = myArray
return msg;

The calculation of the percentage use and saving of variables in context follows.

// Force string coercion to number by *1
global.set("root_free", msg.payload[4]*1 , "file");
global.set("root_used", msg.payload[3]*1 , "file");
root_total = msg.payload[4]*1 + msg.payload[3]*1;
global.set("root_total", root_total , "file");
root_percent_used = msg.payload[3] / root_total * 100;
root_percent_used = Math.round(root_percent_used * 10) / 10 ;
global.set("root_percent_used", root_percent_used , "file");
global.set("home_free", msg.payload[11]*1 , "file");
global.set("home_used", msg.payload[10]*1 , "file");
home_total = msg.payload[11]*1 + msg.payload[10]*1
global.set("home_total", home_total , "file");
home_percent_used = msg.payload[10] / home_total * 100;
home_percent_used = Math.round(home_percent_used * 10) / 10 ;
global.set("home_percent_used", home_percent_used , "file");
return msg;

Note the simple way used for coercion of the strings to numbers - a very fast method in Javascript

Network Address - I have run into problems using my standard tethered connections from mobile phones which have made it desirable to have some basic network information saved as persistent memory for diagnostic purposes and displayed in the Dashboard. I extract the network address from the ifconfig output in an Exec Node with this command

ifconfig | grep 192.168 | awk '{print substr($2,6); }'

Signal quality and strength: The details of signal quality and strength are currently in a separate Appendix below as they are more complex.

Current Time: I used the Simpletime Node for time with the following script in a Function Node

current_date_time_str = msg.mydate + " " + msg.mytime
global.set("current_date_time", current_date_time_str , "file");
msg.payload = current_date_time_str;
return msg

Uptime: I use the Uptime Node which is part of node-red-contrib-os followed by a Function Node containing and a Set node to set a global persistent context value.

var numb = msg.payload.uptime;
msg.payload = Math.round(numb * 100 / 3600) / 100;
return msg;

Some extra logic displays and saves the last value as well

var uptime = global.get("uptime_hours" , "file") || 0;
if(msg.payload < uptime) {
global.set("last_global_uptime", uptime , "file");
}
return msg

SSID: This proved to be much more difficult than I expected. I eventually found that connmanctl services enabled me to list the services which it had available and somewhere in the small print I found "The symbols in the output above are: '*' favorite (saved) network, 'A' autoconnectable, 'O' online and 'R' ready. If no letter is shown in the O/R column, the network is not connected. In addition, temporary states include 'a' for association, 'c' configuration and 'd' disconnecting. When any of these three letters are showing, the network is in the process of connecting or disconnecting." I could therefore use grep to select the record with the *AO flags set and then awk to extract the SSID. Note the code only extracts the start of an SSID with spaces

connmanctl services | grep AO* | grep wifi | awk '{print $2}'

Version Number of OS: This was even more tricky and the only place I could find it was in the boot log where the last few lines are

Sat Feb 5 11:41:08 2022: dbus-daemon[598]: [system] Successfully activated service 'fi.w1.wpa_supplicant1'
Sat Feb 5 11:41:08 2022: Checking available software versions...
Sat Feb 5 11:41:08 2022: Active rootfs: 2
Sat Feb 5 11:41:08 2022: Active version: 20220122213600 v2.80~41-large-25
Sat Feb 5 11:41:20 2022: Backup version: 20211217171415 v2.80~33-large-24

so again a grep and awk extracts what I want, I have also used a tail to get to the end of the boot log

cat /data/log/boot | tail -n 5 | grep "Active version" | awk '{print $(NF)}'

Note: NF is the number of fields in a record

I have more recently discovered the version number is available from /opt/victronenergy/version

catroot@raspberrypi4:~# cat /opt/victronenergy/version
v3.00~4
Victron Energy
20221018161603
root@raspberrypi4:~# cat /opt/victronenergy/version | grep v
v3.00~4

This may have advantages in that /opt/victronenergy/version has read permissions for everyone so can be accessed from Node-RED running as user nodered whilst the permissions have to be set for the boot log.

Data Usage: This has again been interesting as the normal programs are not available so I had to resort to internet searches and came up with https://serverfault.com/questions/533513/how-to-get-tx-rx-bytes-without-ifconfig which had anumber of suggestions including using cat /proc/net/dev which produces a formated version of this:

Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame comp mcast|bytes packets errs drop fifo colls carrier comp
wlan0: 1491661 25728 0 0 0 0 0 0 14153120 29370 0 0 0 0 0 0
eth0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 22049815 195673 0 0 0 0 0 0 22049815 195673 0 0 0 0 0 0
ll-eth0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

cat /proc/net/dev | grep wlan0 | awk '{print "Received bytes:",$2,"Transmitted bytes:",$10}' then gives something like this:

Received bytes: 1332540 Transmitted bytes: 14124366

This looked promising and showed that more data was being transmitted than received - this must be because I testing use of MQTT. When costing data the sum is important so I changed from using awk to my alternative of spliting the incoming record into an array when determining the data rate using the following function. Note the initialization to take care of the times when context variable are not set.

myArray = msg.payload.trim().split(/\s+/);
rxtx = myArray[1] * 1 + myArray[9] * 1;
current_rxtx = rxtx ;
last_rxtx = global.get("current_rxtx", "file") || rxtx ;
global.set("last_rxtx", current_rxtx , "file");
global.set("current_rxtx", current_rxtx , "file");
delta = current_rxtx - last_rxtx;
global.set("delta_rxtx", delta , "file");
msg.payload = Math.round(delta/(1024 * 10) * 1000) / 1000 + " KB/s" ;
return msg;

The results are interesting as there are occasional big peaks but most of the time it is steady round 2 kBytes/second. I initially had a graph to explore what was giving the peaks which seem to be the Deployments in Node-RED! With peaks of 120 kBytes/sec a graph has little long term utility.

I decided it would be more useful to have some daily totals for the current day and the last two days to allow estimates of data costs so created some more context variable in a Function Node. Note the initialisation using || 0 as the variables are not set during the first few days.

rxtx_2 = global.get("rxtx_1" , "file") || 0 ;
global.set("rxtx_2", rxtx_2, "file")
rxtx_1 = global.get("rxtx_0" , "file") || 0 ;
global.set("rxtx_1", rxtx_1, "file")
rxtx_0 = global.get("current_rxtx" , "file") || 0 ;
global.set("rxtx_0", rxtx_0, "file")
return msg;

These are then used for the Text Nodes. Note the inject Node is run at an 11 second interval to keep them asynchronous to everything else using 5 or 10 second intervals. The code is common and un-required lines can be edited out

current_rxtx = global.get("current_rxtx", "file") ;
rxtx0 = global.get("rxtx_0", "file");
rxtx1 = global.get("rxtx_1", "file");
rxtx2 = global.get("rxtx_2", "file");
// daily totals
rxtx_today = current_rxtx - rxtx0;
rxtx_yesterdays = rxtx0 - rxtx1;
rxtx_day_before = rxtx1 - rxtx2
// specific to each output
msg.payload = Math.round(rxtx_today/(1024 * 1024) * 1000) / 1000 + " Mb";
return msg;

The daily data usage is looking to be about 100 Mbytes, say 3 Gbtyes a month which is well within my 15 Gbytes allowance on my £10 a month GiffGaff Golden Goody Bag. I have yet to check how much is due to the MQTT broker.

[WIP] This write up is ongoing but I show the current status of the dashboard in January 2022. It shows that the CPU loads and memory usage are pleasingly low. The Disk spaces correspond to my trial partitioning changes for a 32 Gbyte microSD. The CPU temperature is low as the outside temperature (and Corinna) are close to freezing. The system had been restarted approximately 24 hours before this screenshot was taken. I have added indicators of up-time. The qualities of the wireless link have recently been added.

The current state of my display tab for the Raspberry Pi 3B+

Appendix D Useful Code Snippets for Node-RED Function Nodes

The earlier sections discussed how to obtain various useful information but did not have details of how it was coded into the Node-RED Function Nodes and subsequently display it in the user interface (Dashboard). This covers many of the coding techniques I used. It is useful to have a basic knowledge of Linux commands for this section and how to find additional information.

1. Selecting fields and striping characters from strings using awk

Awk is a very powerful tool and whole books have been written on how to use it. This is just a very simple example of awk which uses an ability of awk to split an input into a series of fields $1 $2 etc, print them to the output and use a substring function when printing to strip characters from the start and/or finish. This is ideal for selecting information from an Exec Node for further processing, storage or display in Node-RED and is often combined with pipes and grep to select particular lines of output (although awk can also do that itself in a relatively simple manner - see above book link - but more difficult to follow).

Select field(s) from a record (aka word from strings)

$ echo "I am very undecided on how to proceed!" | awk '{print $3,$4,"so help me"}'
very undecided so help me

Note that there must be no spaces either side of the commas.

Note 2 print gives a newline by default, if you want to avoid use printf

Strip characters from start of string in specified field.

One can use substr() to do string slicing eg

$ echo "very undecided!" | awk '{print substr($2, 3); }'
decided!

Strip a character from the end of a string

awk '{if (NR>0) {print substr($2, 1, length($2)-1)}}'

NR is the Number of Records so we can check we have records present - useful if we the input is piped from a previous test

length($2) will get us the length of the second field, deducting 1 from that to strip off the last character.

Example:

$ echo "very undecided!" | awk '{if (NR > 0) {print substr($2, 1, length($2)-1)}}'
undecided

The two can be combined and it is wise to check if it also works in busybox awk if you are testing the embedded version on a different machine.

peter@gemini:~$ busybox echo very undecided! | busybox awk '{if (NR > 0) {print substr($2, 3, length($2)-3)}}'
decided

 

2. Using Javascript to select fields

We can use javascript split(/\s+/) to divide multiple lines into an array splitting on any non printing space including line feeds and then work on array elements. It is often useful to remove leading and trailing spaces with trim() first. eg:

myArray = msg.payload.trim().split(/\s+/);
// Note array indices start at 0 so for second field use Array[1]
msg.payload = MyArray[1];

If you want to just split into lines you can use split(

The function substring( start , end ) can be used to extract part of the string between indices start and end. end is not required and note the use of length, the length of the string

let text = "Hello world!";
let result = text.substring(3, text.length - 1);
// result is "lo World"

3. Using Grep in a Exec Node

Grep is very well known and our most powerful tool for searching for PATTERNs in FILEs (or stdin). Here it is used with input piped from a terminal command to select only those lines (individual records) containing a string within an Exec Node. What is less known is that one can do OR and AND equivalents. Grep is can be used on files and here we are using it with input on the stdin

grep AND

There is no actual AND in grep but a simple way is to apply it multiple times

grep charstr1 | grep charstr2

you can also use the -E, --extended-regexp option which apparently interprets PATTERNS as extended regular expressions. I would not like to explain further!

grep -E 'charstr1.*charstr2'

Both versions seem to work under Venus OS with Busybox but I have never used the second version

grep OR

Match lines containing any of a number of strings by

grep e charstr1 e charstr2 e charstr3

or

grep 'chrstr1\|chrstr2\|chrstr3'

Both versions work under Venus with Busybox

Other Grep options I often use

-i Ignore case
-v Select non-matching lines
-w Match whole words only
-r recursive (on files)

These all work in busybox

Extract a number from a string using grep (or sed)

This is a common task when one has complex output from commands such as dbus to access values on the D-Bus. Bash has some internal functions which serve to extract integers but extracting a decimal number is more difficult. sed always provides a solution but grep turns out to be the most useful useful for both tasks. Here are a couple for integers and one for decimals. In general they are best for a single number and the tests below were on the Pi 4B and Venus OS 2.92 so use the busybox version of grep.

root@raspberrypi4:~# echo "The number for this test is 123 an integer" | sed 's/[^0-9]*//g'
123
root@raspberrypi4:~# echo "The number for this test is 123 an integer" | grep -Eo '[0-9]+'
123
root@raspberrypi4:~# echo "The number for this test is 12.34 which is a decimal" | grep -Eo '[[:digit:]]+([.][[:digit:]]+)?'
12.34
root@raspberrypi4:~#
The last is a bit obscure and the explanation is -o takes care that only the parts matching the pattern are written to stdout. The pattern greedily matches one or more digits, followed by optionally a decimal point and more digits. This skips malformed "numbers" such as .56 and 14.

4. Rounding to 2 decimal places with associated text for display in Dashboard

The reduction from multiple decimal places before output is a common requirement.

//Example from a Function Node in Node-RED
var numb = msg.payload;
numb = Math.round(numb * 100) / 100;
msg.payload = "User Disk " + numb + "% full";
return msg;

5. Context variables

Note Inject, Change, Switch, and Trigger Nodes support Context Variables directly by dropdowns.

Note 2 Use of Persistent Global Context variables ie saved to file requires modifications to the settings.js file in Node-RED

Set Persistent Global Context Variable example

var cpu_switch = true ;
global.set("cpu_switch", cpu_switch , "file");

Or one can use a Change Node

'Gate' on Global Context Variable in Function Node Example

// Set gate state from global context variable or initialize if undefined
var cpu_switch = global.get("cpu_switch" , "file") || false;
if(cpu_switch === true) { return msg }
return null

Note the format of global.get() and global.set() with single or double quotes

6. Example of using Filter, Switch and Delay Nodes

One of the things I have been doing to understand the Lithium batteries has been to get time series of measurements under under consistent stable no load conditions. The best approximation is to take measurements when only the fridge is taking power and there are no solar inputs or engine charging and there are outputs to charge the battery from the engine alternator. The measurements are taken just before the fridge draws power at which point there has been 20 or more minutes to stabilise and it is also essential that measurements are inhibited for ~90 minutes after the solar has been active and ~45 minutes after the 24|12 V link has been closed. When these conditions are satisfied the Lithium battery voltages are steady within the 10 mv resolution of the voltage measurement. So we need to have a series of inhibit flags which have a delay in clearing of 45 and 90 minutes.

The delay node seems perfect for this but there is a catch in that it delays all the members of a series of messages in a queue so if the inputs are varying too fast a previous delayed message may clear a recent input. Fortunately there is way to clear the queued messages by sending msg.reset as true (actually any value of msg.reset will work but using true makes it more obvious) . So the proceedure is to only pass a message stream if it changes (Filter Node), split it into rising and falling edges (Switch Node) passing the rising edge immediately and the falling edge after a delay (Delay Node). The rising edge also clears any delayed messages in queue via a Change Node to avoid premature clearing via old queued messages. A typical implementation is below with a couple of debug nodes. The blue square with a number below the delay node gives the number in the queue and should be 0 or 1 in this implemenation.

 

Appendix F - Reducing processor load and number of elements in a Chart Node

There are two useful nodes to reduce the number of messages being sent on.

• Delay Node. This Node can be used to delay messages but has many other options including being able to limit the rate messages are sent on and discarding intermediate messages. This allows the stream to a Chart Node to be reduced to a figure that will allow a display of say a days data at once per minute instead of just a few hours worth from Victron Devices which send messages every 5 seconds.
• Filter Node otherwise known as the Report by Exception (RBE) Node - only passes on data if the payload has changed. It can also block unless, or ignore if the value changes by a specified amount (Dead- and Narrowband mode). This can again reduce the stream of messages considerably especially for logical functions. It can be combined with the Delay Node to minimise data to a Chart Node

Another way to even out the processor loads is to run the various message streams at different speeds so they are asynchronous. The Victron devices are fixed in rate but I try to use different intervals in my inject nodes, mostly prime numbers close to the rates I want. The effect is noticeable on the chart output of CPU Load with a reduction in spikes.

The last way is to 'gate' the output to the chart nodes which are only used for diagnostics. I have that built in but it is not obvious the saving is worthwhile.

Appendix F - Measuring Signal Quality on the Raspberry Pi

The definition and measurement of signal quality is always contentious so this has a separate appendix.

The usual utility for such measurements is ifconfig but that does not give any information on signal strength, quality or noise in the busybox version so we have to go to /proc/net/wireless for information.

$ cat /proc/net/wireless
Inter-| sta-|     Quality      |   Discarded packets        | Missed          | WE
face  | tus | link level noise | nwid crypt frag retry misc | beacon          | 22
wlan0: 0000   67.  -43.  -256    0    0     0    0     33941  0
So what do they mean? The important ones are under the Quality heading:

Signal Strength (quality level in /proc/net/wireless)

Basically, the higher the signal strength (level), the more reliable the connection and higher speeds are possible. The signal strength is specified in dBm (decibels related to one milliwatt). Note Decibels are a logarithmic scale.

Values between 0 and -100 are possible, with more being better. So -51 dBm is a better signal strength than -60 dBm. For the Raspberry Pi 3B+ :

-30 dBm probably means the antenna is virtually in contact with the Wifi Router!
-50 dBm is considered an excellent signal strength.
-67 dBm is the minimum signal strength for reliable and relatively fast packet delivery.
-70 dBm is the absolute minimum signal strength for reliable packet delivery.
-90 dBm is very close to the basic noise.

On Corinna the Signal Strength is -50 dBm at ~5m when tethered to my Samsung Galaxy M32 mobile phone through 4 wooden bulkheads and slightly worse at -52 dBm with my old Samsung Galaxy S3 Mini which is also very orientation sensitive.

Link Quality

A network can be received with a very good signal strength but not as good a link quality. In simple terms, it means how much of the data you send and receive will make it to the destination in good condition.

The quality indicator includes data like Bit Error Rate, i.e., the number of bit errors in received bits that have been altered due to noise, interference, distortion, or bit synchronization errors. Others are Signal-to-Noise and Distortion Ratio. It is measured in percentage or on a scale of up to 70. The figure in /cat/net/wireless is on a scale of 70 but it is important to understand the reading is somewhat subjective and depends on the hardware and the manufacturers figures and driver software. So, unlike signal strength, it is somewhat harder to say which values are still considered to be OK but may still be useful when looking for interference from, for example, a microwave cooker in a house, dimmers on lighting or motors on a boat and choosing positions for equipment. I get a typical reading of 83% on Corinna at 5m.

Node-RED Code Snippets for Quality and Signal Strength

Exec Node

cat /proc/net/wireless | grep wlan0

Function Node

myArray = msg.payload.trim().split(/\s+/);
link_level_db = myArray[3]*1;
global.set("link_level_db", link_level_db , "file");
msg.payload = link_level_db + " dBm"
return msg;

Appendix G - Use of MQTT - Draft post to Victron Community Modifications Space

I have been trying out MQTT and started to write a post looking for answers to various problems I was having. However as fast as it got written I found answers to he problems which were holding me up and further questions raised there heads. It is currently my main documentation of what I have found so far! I have added a little formatting but little else

Is the MQTT approach a useful addition. What are the advantages and disadvantages.

Having got MQTT working I now ask myself if it is useful when there are already many ways to access information from and control Victron Devices.

• MQTT gives an alternative to context for persistent storage of variables through restarts - both are clumsy to use but both allow some diagnostics if a system crashes.
• MQTT gives access to extra parameters which the Victron Nodes do not cover. The Solar Controllers store 30 days of useful daily information which is accessible through VictronConnect and has also been made via MQTT, in particular it offers 30 days back of daily totals in Bulk, Absorption and float and total yield and load powers.
• I hoped to use MQTT to access and change the device configuration data which is normally set using VictronConnect but that does not seem to be the case.
• Node-RED provides a good easy way to set up dashboards but they are difficult to access via VRM - many steps to reach them and they quickly time out which is not the case with the Victron Broker BUT you need to have a separate machine running Node-RED to get the advantages.
• It could integrate well with a home automation service.

None of these make a very convincing case other than the daily totals to me for use via the Victron Broker. It becomes more balanced if one is on the same network when I see no reason why both should be used in parallel. I have not however yet investigated the processor etc loads of using the inbuilt Mosquitto MQTT Broker in the Victron OS.

I have also looked at the data usage and that was surprising as the stead state data usage with MQTT on my system was a order of magnitude lower than using the Victron VRM although that would normally only be used for short times.

The bottom line: I am currently not using MQTT and do not plan to in the near future ie a dead end

Appendix H - The Raspberry Pi 4B v1.5 with Venus OS 2.90 and higher

Whilst I was carrying out a number of experiments on interfacing to the GPIOs on my Raspberry Pi I began to feel slightly vulnerable as I only have the one which is at the heart of the solar and power system on Corinna so I started to look for a spare for experiments. Unfortunately the Raspberry Pi 3B+ which was recommended and all the software was written for is no longer available or is on months long delivery. I however found that a couple of people were working on the software for the latest Pi 4B - the software from Victron in Venus OS v2.80 used kernel 4.19 for the Pi 4 which is now very out of date and needed to be moved to 5.10 which seems to support most of the interfaces on the Pi 4 revs 1.4 and 1.5.

I therefore decided to risk buying a Raspberry Pi 4B which was also like finding Hen's Teeth. I eventually found a 2Gbyte RAM one in a starter kit with case, microSD, Micro HDMI cable and a 3A mains power supply at Pi Hut where I bought my last Pi 3B+. It was not a ridiculous price as would have had to buy some of the items in any case. I added a Cytron Maker Hat Base GPIO extension and development board from Pi Hut which is excellent value at £8.90 and makes developing with the GPIO much easier as all the pins have LEDs and are clearly labeled as well as many other features including a small breadboard. The combination enabled me to do a lot of development and testing prior to making any changes to my 'live' Pi 3B+ on Corinna which has just passed 2000 hours without a reboot as I write this. My tethered android phone has also been up for the same period. I have been impressed with Pi Hut

I got it working with some difficulty after some complex procedures to update the kernel and some other essential changes to allow use of Node-RED. It worked well enough to be available as backup but the procedures were time consuming and involved a lot of slow cross-compiling. However by the time I was back from our long summer cruise on Corinna a new candidate version of the Victron Venus OS 2.90~22 which supported the Raspberry Pi 4B in all versions including Large had just become available on the candidate feed at http://updates.victronenergy.com/feeds/venus/candidate/images/raspberrypi4/ . This was still a beta version but it was well advanced. That made the remainder of the original write up which covered the complex and often obscure procedures to update the kernel and install the Large modifications to Victron OS 2.80 for the Raspberry Pi 4B revs 1.3 and higher redundant and irrelevant but is still preserved here .

Details of Changes required to update to Venus OS v2.90 for the Raspberry Pi 3B+ and Pi 4B

The Venus OS v2.90 brings many desirable enhancements over the 2.8x versions, not least of which is better support for the Large versions and Full support for more recent versions of the Raspberry Pi 4B out of the box. But the upgrade may need changes for some users of Node-RED. The major difference for users of Node-RED is that earlier versions of Venus ran Node-RED under the root user whilst Venus OS 2.90 and higher run Node-RED under its own user nodered. This means the node-red folder is moved from /data/home/root/.node-red to /data/home/nodered/.node-red which will affect most[all] scripts I have written and many Exec nodes. The change in permissions will also need considerable investigation where there are interactions with the operating system - shutdown and relay operation came immediately to mind but I have found some other data has become read only.

The changes were too great to initially contemplate making on my operational system on Corinna so all investigation and development has been done on my Raspberry Pi 4B rev1.5 2 Gbyte machine. This does not have any Victron Devices connected but does have a Cytron Maker Hat Base GPIO Extension and several relays connected. It was running Venus Large 2.82 and a fresh installation of Venus OS Large v2.90~22 (the latest beta development version at the time) was carried out.

The latest candidate (beta) versions were available to download at http://updates.victronenergy.com/feeds/venus/candidate/images/raspberrypi4/ and released versions at https://updates.victronenergy.com/feeds/venus/release/images/raspberrypi4/ . The install and update procedures are covered at https://github.com/victronenergy/venus/wiki/raspberrypi-install-venus-image and as far as I can tell I did not need to do anything differently to what I did in the past and have previously covered in detail.

The following were my rough plan and notes made as I progressed with initial install of v2.90~22 with any updates noted for Released Version of 2.90.

Firstly the install which will basically remain the same as earlier versions.

• Create fresh install of OS v2.90~22 for Raspberry Pi 4 using 'standard' instructions.
• Monitor and keyboard connected for testing and initial setup with temporary ethernet connection until Wifi set up.
• Set up Wifi, Bluetooth password, root as superuser with password, VRM two way comms, etc as per existing instructions plus setting current time which I do not recall having to do previously.
• Enable Node-RED in Remote Console
• Update or create settings-user.js to give persistence to global context variables and test. - Very important. Done via sftp using FileZilla from a Linux machine. Update for released v2.90: Now uses file settings-user.js which should only contains sections which are changed but note they still need to be wrapped overall in a module.exports = { .... }
• Extra nodes in Node-RED added using Palette Manager (on Main Menu).
  1. node-red-contrib-victron (installed already)
  2. node-red-dashboard (essential)
  3. node-red-contrib-ui-artless-gauge (More flexible with linear and 270 deg gauge options)
  4. node-red-contrib-simpletime
  5. node-red-contrib-stoptimer
  6. node-red-contrib-os
  7. node-red-contrib-cpu
• Export all nodes from previous system into file and import into new 2.90 OS (Export and Import on Main Menus)

And this is what I then did after the first Install

• Investigated number of exec nodes containing absolute addressing of .node-red folder (16) in old system for Pi 4
• Identified and "solved" permissions problem with GPIOs from node-RED - several solutions found and written up below. The favoured solution removes all scripts called from exec nodes.
• Identified another permissions problem with obtaining the running OS version, solved with setting permissions to allow reading of /data/log/boot
• Searched for solutions on how to run shutdown, reboot etc which require to be run from root. Easy when sudo is available as it can be set up so certain programs and users can run sudo and hence as root without passwords but Venus OS did not by default support sudo or a similar mechanism. A solution was identified using a script running as root which checks for a flag that shutdown is requested. Update the Released version of v2.90 has sudo and sudo based solutions are also covered below.

The extra stages, resulting from these investigations, needed to completing the install can be seen here.

First stage - Using GPIOs and Relays under Venus OS 2.90 Large with Node-RED.

The original methods and code developed for the earlier versions of OS 2.80 worked very well and was used for well over many months, mostly cruising, on my Raspberry Pi 3B+running on Corinna and is still in use. The new OS version is in most ways is a huge step forwards, however, some of the changes mean that my original code no longer works. The main reason is that in OS 2.80 Node-RED used to run as a system program under root which was very dangerous as it could run any code as root and could thus do unlimited damage. It now runs as a separate user (nodered) and no longer has the permissions to set up the GPIO from within Node-RED. There are several ways round this but the easiest is to carry out the setting up as part of the initialisation process running as root with suitable permissions being set to allow the actual switching to take place form an Exec Node.

These changes have now been incorperated into the main GPIO and Relay Appendix

The next step - updating shutdown, reboot and restarting from Node-RED for v2.90

The original reboot and shut down of the Pi in v2.80 used the shutdown command which has to be run as root. The Restarting of Node-RED used the svc command to restart the node-red-venus service and also needs to run as root. These could not be run by user nodered directly. Solutions were explored in the Diary part 34.

In summary the solution chosen runs a timer loop (using a sleep command) as root which checks every second for for the presence of a file containing a command to be run, runs it and immediately deletes the file to prevent it being run extra times. The file containing the command you wish to run as root is created in Node-RED in an Exec node. As the mechanism is primarily used for a tidy shutdown rather than pulling the power plug a delay of a few seconds is quite acceptable. The security risk could be mitigated by only allowing certain commands to be run. The processing overhead is small and even a one second loop is dificult to see in the processor load in my dashboard.

Stop Press: At the last minute sudo has been added to the Venus OS 2.90~28 and higher builds. This gives a further way to implement shutdown, reboot and reload Node-RED. See below. That has currently been tested but on the grounds that "if it ain't broke, don't fix it" I initially contined to use the 'timer loop and transfer file' solution for shutdown and restarts in September 2022 and then started to change to use sudo but leaving the code for both available.

Enhanced Proccedures for Updating the Venus OS - tested with 2.90~22 to 2.90~24, then to 2.90~26 etc

The update mechanism is again is a major improvement in version 2.90 as the standard update mechanism from USB stick now works perfectly for v2.90 including the Large versions with Node-RED. The update to v2.90~24 Large was very quick and painless. At the time the gpio_list had to be updated manually and the root password needed setting in the remote console as usual. My latest GPIO implementation in Node-RED for relays does not require manual updates so the only intervention next time will be to set the root password via the remote console to allow SSH and SFTP remote access. Note updates only work when updating from 2.90~12 and higher. A full install is needed for updating from earlier versions as the disk structure was changed.

I tried the online update mechanism (called manually from the Victron Remote Console) for the update to 2.90~26 and again that worked perfectly and I also tried the switching between the two using the Remote Console. The only anamoly was that the internal relay section in the Remote Console only appeared after a reboot. All the other setup in rc.local seemed to be fine and the superuser password only needed to be set the first time, not after OS switches. All further updates used the online updating mechanism.

New Proceedures for Node-RED settings including 'Persistence'

There were some quite significant last minute changes, one of which significantly improves security by use of https: but also makes use on a local area network with android devices more difficult so may need to be 'reverted' for convenience. Another affects users who make changes to the node-RED settings file eg to implement persistent global context variables and in many ways makes that task easier. The third was very positive and added sudo. So looking at the change in configuration first and quoting loosely from the @mvader post with some additional [clarifications].

We [Victron] have changed the setup of config files a bit to be better prepared for any future requirements and this is how it works in the Official v2.90 release and onwards.

[Caution:] The vast majority of systems using Node-RED will not and (should not !!) have to modify anything described here. Venus OS itself, including Venus OS large and Node-RED, is made such that its not necessary to dive into the command line. Anyone that does go in and change them needs to be prepared for things going wrong, for example, after a firmware update. The information here provided is for the benefit of the handful of users that do need (or want) to change them; and to put it bluntly: know what they are doing and are prepared to spent hours and hours fixing issues without getting any help [from Victron staff].

On startup, Node-RED on Venus now reads three settings files:

 1. /usr/lib/node_modules/node-red/venus-settings.js is read first. Never change this file as it will be overwritten by a Venus OS update. [ This is the default settings file provided by Node-RED, adjusted for
usage on Venus OS. ]

 2. /data/home/nodered/.node-red/settings-venus.js is used to override and/or extend the first file and will survive Venus OS updates. Do not edit this file as this is for Victron to use. [ It is currently not present ]

 3. /data/home/nodered/.node-red/settings-user.js is used to override and/or extend the [earlier] files and will survive Venus OS updates. You are allowed to edit this file if you please, but don't be too surprised if that causes Node-RED to break after a [major] upgrade.

[Note: Your original /data/home/nodered/.node-red/settings.js is preserved in /data/home/nodered/.node-red/settings.js.old after the first update to Venus OS v2.90]

In practice this means that just the additions or changes one has made in /data/home/nodered/.node-red/settings.js need to be added to /data/home/nodered/.node-red/settings-user.js when you first upgrade to or reinstall v2.90. Initially it just includes your credentialSecret whilst the original settings file had hundreds of lines which were mostly comments. I have found a clever trick to get rid of all the comments and reducing the original file to ~40 lines. You use cpp -undef -P your_file.js or cc -undef -E -P -xc your_file.js in a terminal to achieve that (but do not ask me how it works).

The other change to consider is that node_RED is now, by default, accessed at https://venus.local:1881 and the ui at https://venus.local:1881/ui. This is different to the Node-RED documentation but is obviously much more secure. The use of https on local networks with a personal secure key causes considerable grief with Android but @jerone commented that a line can be added to settings-user.js to re-enable the http connection on port 1880. So my settings-user.js file now looks like this:

// Contents of /data/home/nodered/.node-red/settings-user.js

module.exports = {

/* [Caution from @mvader:] The vast majority of systems using Node-RED will not and (should not !!) have to modify this file. Venus OS itself, including Venus OS large and Node-RED, is made such that its not necessary to dive into the command line. Anyone that does go in and makes changes needs to be prepared for things going wrong, for example, after a firmware update. The information here provided is for the benefit of the handful of users that do need (or want) to change them; and to put it bluntly: know what they are doing and are prepared to spent hours and hours fixing issues without getting any help [from Victron staff]. */

/* @jeroen commented · Sep 11 2022 at 2:21 PM
if you add  uiHost: "",
to /data/home/nodered/.node-red/settings-user.js
the http server will be available again on port 1880
and that will survive a firmware update. [ Usefull for Android users on Local network]
*/

  uiHost: "",

/* * Context Storage
   * The following property can be used to enable context storage. The configuration
   * provided here will enable file-based context that flushes to disk every 30 seconds.
   * Refer to the documentation for further options: https://nodered.org/docs/api/context/
*/
   //contextStorage: {
   // default: {
   //  module:"localfilesystem"
   //  },
   //}, 

// The following is better as it provides an option of memory or file and works!

   contextStorage: {
     default: "memoryOnly",
     memoryOnly: { module: 'memory' },
     file: { module: 'localfilesystem' }
   },
}

I have coped the various sections I have changed complete with comments from settings-old.js which has a backup of my original settings.js file but the important additions are in red.

I made the change from v2.90~26 directly to v2.91~1 smoothly using the online update mechanism and keeping to the Latest Update Candidate feed so I can continue to contribute to the development program.

 

Appendix I - Controlling Relays from a Raspberry Pi GPIO

Using Node-RED and Victron Venus OS versions 2.90 and higher

This Appendix was originally written for Venus OS 2.80 where Node-RED is run as root. It was used for a seasons boating and OS v2.80 is still in use on my operation system on Corinna in September 2022. Under Venus OS v2.90 NodeRED runs as a separate user called nodered rather than as root so some changes were needed to the code. The solution that follows for OS v2.90 is more compact and avoids scripts. The original Appendix can be found here.

The addition of relays is very important as some of the less smart Victron Devices have inputs for remote control switches. It always surprises me that after 50 years or more of solid state electronic relays are still in common use, in fact their use surprised me over 50 years ago when I started designing equipment for the research satellites!

There are several different ways to access GPIOs from programs, but sysfs is a simple one that is supported by the Linux kernel which makes the devices visible in the file system. Sysfs is a pseudo filesystem provided by the Linux kernel that makes information about various kernel subsystems, hardware devices, and device drivers available in user space through virtual files. GPIO devices appear as part of sysfs so one can experiment from the command line without needing to write any code. For simple applications you can use it this way, or by putting the commands in shell scripts. Before continuing, I should mention that this interface is being deprecated in favour of a new GPIO character device API. The new API addresses a number of issues with the sysfs interface. However, it can't be easily be used from the file system like sysfs, so the examples here will use sysfs, which is still going to be supported for some time. In any case, as far as I can tell, the Venus OS only uses sysfs for GPIO control. A basic introduction to controlling the gpio using sysfs is described in https://raspberry-projects.com/pi/command-line/io-pins-command-line/io-pin-control-from-the-command-line

So what is meant by a pseudo filesystem? There are many devices which are accessible via a mechanism which looks like a folder structure - I found about 60 in /sys/class/. If we looked at /sys/class/gpio we find a list like this where a number of GPIOs are already in use.

root@raspberrypi4:~# ls /sys/class/gpio
export gpio16 gpio17 gpiochip0 gpiochip504 unexport
root@raspberrypi4:~#

This corresponds to a set of folders for the gpios which are in use and we also find two other items which behave like files, export and unexport. Only a small number of the possible gpio ports are in use and to make say GPIO23 available we need to send 23 to that 'pseudo file' export. We will now find an extra folder gpio23 as below:

root@raspberrypi4:~# echo 23 > /sys/class/gpio/export
root@raspberrypi4:~# ls /sys/class/gpio/gpio23
active_low device direction edge power subsystem uevent value
root@raspberrypi4:~# echo "out" > /sys/class/gpio/gpio23/direction
root@raspberrypi4:~# chmod 777 /sys/class/gpio/gpio23/*
root@raspberrypi4:~# echo 1 > /sys/class/gpio/gpio23/value
root@raspberrypi4:~# echo 0 > /sys/class/gpio/gpio23/value

I have then sent "out" to direction to make it an output and 1 to value to turn it on and 0 to turn it off. All quite simple. The use of export and unexport gives a crude mechanism to avoid problems if several programs or users try to access the same GPIO. I have not tried all the various parameters but active_low is useful as some common relays boards are activated with a low input. The setting of permissions allows latter versions of Node-RED which run as user nodered access.

It is quite possible that your system already has some GPIOs already in use. A fresh install of Venus OS 2.90 on a Raspberry Pi 3 or 4 do not seem to have any allocated. However some are in use by Victron Controllers and if you use extensions such as that by Kevin Windrem they will have added some extras. https://github.com/kwindrem/RpiGpioSetup/blob/main/FileSets/gpio_list has a 'standard' list of current pins used in the Venus OS including proposed extension to 6 (2) relays outputs, 5 inputs and an extra graceful shutdown pin.

Following is a summary:

# Relay 1 Pin 40 / GPIO 21
# Relay 2 Pin 11 / GPIO 17
# Relay 3 Pin 13 / GPIO 27
# Relay 4 Pin 15 / GPIO 22
# Relay 5 Pin 16 / GPIO 23
# Relay 6 Pin 18 / GPIO 24
# Digital input 1 Pin 29 / GPIO 05
# Digital input 2 Pin 31 / GPIO 06
# Digital input 3 Pin 33 / GPIO 13
# Digital input 4 Pin 35 / GPIO 19
# Digital input 5 Pin 37 / GPIO 26
#### Graceful shutdown input
#### Note this input is NOT added to the available I/O used by Venus OS !!!!
# Pin 36 / GPIO 16

An essential check before you start is via gpioinfo, a command built into the kernel as part of the new GPIO character device API. The output below is from Venus OS 2.90 on my Pi 4 - note none of the GPIOs are used here apart from GPIO23 that I was testing at the time. The output has been 'filtered' to just show GPIOs

root@raspberrypi4:~# gpioinfo 0 | grep GPIO
line 4: "GPIO_GCLK" unused input active-high
line 5: "GPIO5" unused input active-high
line 6: "GPIO6" unused input active-high
line 12: "GPIO12" unused input active-high
line 13: "GPIO13" unused input active-high
line 16: "GPIO16" unused input active-high
line 17: "GPIO17" unused input active-high
line 18: "GPIO18" unused input active-high
line 19: "GPIO19" unused input active-high
line 20: "GPIO20" unused input active-high
line 21: "GPIO21" unused input active-high
line 22: "GPIO22" unused input active-high
line 23: "GPIO23" "sysfs" output active-high [used]
line 24: "GPIO24" unused input active-high
line 25: "GPIO25" unused input active-high
line 26: "GPIO26" unused input active-high
line 27: "GPIO27" unused input active-high

There is some documentation from kernel documentation at https://www.kernel.org/doc/Documentation/gpio/sysfs.txt and https://embeddedbits.org/new-linux-kernel-gpio-user-space-interface/ has an introduction to the differences between the new api and the old depreciated sysfs mechanism. Current documentation (https://github.com/victronenergy/venus/wiki/bbb-gpio) implies that io access in the Venus OS is implemented in the kernel as sysfs control: /sys/class/gpio.

There is a good interactive pinout guide for the Raspberry Pi showing the pins which are used by convention for certain purposes and those available for general purposes at https://pinout.xyz/# This shows that most GPIOs have multiple uses. Many of the optional uses are configured by Device Tree overlays. Device Trees are well beyond what I understand enough to cover here but they makes it possible to support many hardware configurations with a single kernel and without the need to explicitly load or blacklist kernel modules. On Raspberry Pi, Device Tree usage is controlled from /boot/config.txt. By default, the Raspberry Pi kernel boots with device tree enabled but only a small set of GPIOs are configured for special purposes - eg use of pins for a serial port. gpioinfo seems to be the way to find out the current status.

Once you have chosen and set up the GPIO folders as above it was trivial to control them from within Node-RED with echo commands in an Exec Node in version v2.8x. It is not quite so simple in v2.90 as nodered no longer runs as root so you have to set permisions as well during the set up of the GPIO which is why there is the mysterious chmod line in my example above as a reminder.

There is a catch to this in as much as one is making modifications in the root file system which is overwritten by every firmware update so we come to the second part - making the set up persistent through restarts and Firmware Version Updates

A very important feature of the Venus OS is the ability to run programs in the user area during the initialisation process described at https://www.victronenergy.com/live/ccgx:root_access which describe the hooks which are built in to allow a script to be run as root whilst residing on /data and hence surviving a version update. To quote

If the files /data/rcS.local or /data/rc.local exist, they will be called
early (rcS) and late (rc) during startup. These scripts will survive upgrades
and can be used by customers to start their own custom software.

This is used by several of my updates for v2.90 and should be executable. The GPIOs are now all set up in advance within /data/rc.local. I made sure all of the code could be run after every restart without problems. My rc.local file sets up a number of activities including adjusting various permisions, copying files which are overwritten during updates and implementing sudo for resticted use by Node-RED. The relevant content of /data/rc.local is:

#!/bin/sh
# Contents of /data/rc.local - needs execute permissions.
#
# Background: When the files /data/rcS.local or /data/rc.local exist,
# they will be called early (rcS) or late (rc) during startup. These scripts will
# survive upgrades and can be used by customers to start their own custom software.
#
# This sets up gpio access for an additional 4 relay outputs
# accessible by node-red in addition to those specified using gpio_list
# Node-red now runs without root access as user nodered
# requiring the setting of permissions to allow access to specific parts of sysfs
#
echo 22 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio22/direction
chmod 777 /sys/class/gpio/gpio22/*
echo 23 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio23/direction
chmod 777 /sys/class/gpio/gpio23/*
echo 24 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio24/direction
chmod 777 /sys/class/gpio/gpio24/*
echo 25 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio25/direction
chmod 777 /sys/class/gpio/gpio25/*
#
# Replace the file which provides support for Victron relays
# as it is over-written during OS updates.
cp /data/gpio_list /etc/venus/gpio_list
chmod 744 /etc/venus/gpio_list
#
# Log when rc.local actioned
echo "$(date) rc.local actioned" >> /data/rc_local_log
chmod 666 /data/rc_local_log
#
exit 0

The GPIO section of rc.local creates the required entry for each GPIO and sets the permissions so it can be switched by Node-RED by single commands in an Exec Node. eg. echo 1> /sys/class/gpio/gpio22/value will switch a relay on.

Warning - check you do not have an existing rc.local file, if you do it may be possible to add the above section or turn it into a script and call it from rc.local. The versions of Kevin Windred's SetupHelper I have looked at only use rcS.local .

As I said above the set up of GPIO ports to control Relays from Node-RED is only part of what my rc.local does and I may cover other uses including sudo with Node-RED in a future article.

Basically the same code has been controlling some of my less clever Victron devices via relays on my narrowboat for 6 months and finally logged up 3300 hours under Venus OS 2.80 without a reboot prior to my updating to version 2.90.

Relays in Victron Devices

I have also set up two relays accessible by the Victron relay node by changing /etc/venus/gpio_list to contain:

# In /data/gpio_list (then copied by re.local to /etc/venus/gpio_list)
#- in digital_input_1
#- in digital_input_2
#- in digital_input_3
#- in digital_input_4
16 out relay_1
17 out relay_2
#- out mkx_rst
#- out vebus_standby

Note: Having changed the file the relays still need to be switched to manual control in the Remote Console

Both ways work fine with Venus OS v2.90~24 Large and higher. The first needs no loads after an update whilst the second is limited in the number of relays and needs a file to be reloaded which can be automated in the rc.local script as you will see above.

I should also mention that there is a comprehensive SetupHelper by kwindrem with associated RpiGpioSetup package which should allow Node-RED to access some additional relays but I had not explored this option when first writing this section as I already had everything I needed.

Suitable Relays for the Raspberry Pi 3B+ and 4B

https://tlfong01.blog/2020/06/27/jdvcc-relay/ has wiring diagrams of many relay boards with pictures and includes the cheap ones from Amazon I have been using for tests namely https://www.amazon.co.uk/YOUMILE-Channel-optocoupler-Support-Trigger/dp/B07TTVYGC8 , now reduced to £8.99 for 5. I however advise derating the current from 10 Amps and not using them for switching 230 volt.

Appendix I - Mastering the D-bus including an example of relay control.

This investigayion came about because SetupHelper by kwindrem with associated RpiGpioSetup package can add up to 6 relays accessible from the Remote Console but only 4 can be accessed from Node-RED due to a limitation within node-red-contrib-victron. This can add direct access to the extra relay interfaces through the D-bus interface from a normal Exec node

D-Bus is an inter-process communication mechanism—a medium for local communication between processes running on the same host. D-Bus is meant to be fast and lightweight, and is designed for use as a unified layer underneath the main environments. The D originally stood for Desktop although its use is much wider now, and is especially useful in embedded systems. It is a message based system and instead of programs needing to interact with multiple other programs all communication is routed through the central D-Bus. The messages are handled by a daemon that runs the actual bus, a kind of "street" that messages are transported over, and to which any number of processes may be connected at any given time.

Multiple buses may be active simultaneously on a single system. Most environment runs two kinds of buses: a single system bus for miscellaneous system-wide communication, e.g. notifications when a new piece of hardware is hooked up; and a session bus used by a single user's session. A session bus normally carries traffic under only a single user identity. In the case of an embedded system like the Venus OS one only needs to run a system bus.

I found getting to grips with dbus for the first time to be a nightmare even for what seemed a simple task. I eventually found an old but still popular guide via google namely Ian's Dbus Tutorial - Intro and Resources which got me started enough to know what to search for. Once you have made a start you could move onto www.freedesktop.org/wiki/Software/dbus/ - The first paragraphs were paraphased from parts of that wiki.

There are at least two ways to access the dbus under the Venus OS from the Command Line. The 'standard' low level way common to most machines is via dbus-send which actually Sets and Gets values. The other more focused solution is dbus which I understand is implemented using Python but is very slow even for the simplest tasks. The invocation using dbus is however much shorter and easier to understand. dbus-spy is a useful Victron utility which enable one to find out something about the dbus configuration and current Values in a simple way from a terminal, usually via SSH.

The dbus-spy command has a simple interface which you can navigate with the up, down, left, right keys to explore the system bus Service Names (destinations) at the first level and the Object Paths complete with the real time value returned by GetValue method at the second level. These match the inputs we need for the dbus command. For example I found the Service Name for my solar charger was com.victronenergy.solarcharger.ttyUSB3 and at the second level (Object Path) I was delighted to find Load/I . Note Service Names contain . (dot) separators whilst the Object Paths contain / (slash) separators

There is also a fairly complete List of available services and their paths on the Venus OS D-Bus available which is useful if you do not have your system accessible. The D-Bus API definition is also useful reading.

Lets have a look at how both ways look in a terminal (or Node-RED Exec node) for a Relay noting the low level numbering for Relays starts at 0. These are all single lines when you do a copy/paste

First using the slow but simple call to dbus

Turn Relay 6 on

dbus -y com.victronenergy.system /Relay/5/State SetValue 1

Turn Relay 6 off

dbus -y com.victronenergy.system /Relay/5/State SetValue 0

Example of Get Value of Relay 6's State

root@raspberrypi4:~# dbus -y com.victronenergy.system /Relay/5/State GetValue
1
root@raspberrypi4:~#

Now using the faster dbus-send

Turn Relay 5 on

dbus-send --system --print-reply --dest=com.victronenergy.system /Relay/4/State com.victronenergy.BusItem.SetValue variant:int32:1

Turn Relay 5 off

dbus-send --system --print-reply --dest=com.victronenergy.system /Relay/4/State com.victronenergy.BusItem.SetValue variant:int32:0

Get Value of Relay 5's State

dbus-send --system --print-reply --dest=com.victronenergy.system /Relay/4/State com.victronenergy.BusItem.GetValue

which returns the following

method return time=1577563971.665778 sender=:1.33 -> destination=:1.912 serial=19248 reply_serial=2

variant int32 1

Example of how to get the command to only return the value of the switch by piping through awk:

root@raspberrypi4:~# dbus-send --system --print-reply --dest=com.victronenergy.system /Relay/4/State com.victronenergy.BusItem.GetValue | awk -F'int32 ' '{printf "%s",$2}END{print '\n'}'
1
root@raspberrypi4:~#

Although the use of dbus looks much shorter and easier it is noticeably very slow ~0.5 sec versus 0.03secs for dbus-send on my Raspberry Pi 4B, a relatively fast machine, it can take nearly two seconds on some machines. @Jeroen has indicated this is probably due to the use of python in the coding. If you want to check just put a time command at the very beginning of the command lines and try both on your machine. I will use dbus-send myself.

I have tested the dbus-send solution for both Setting and Getting Values in Node-RED Exec nodes. This gets round the problem of Relays 5 and 6 not being accessible from Victron Nodes when using Kevin Windrem's RpiGpioSetup package.

Another use I have made of D-Bus

Solving the missing Load Current in the Victron Node for my SmartSolar MPPT 100|20 48V

First I logged in using SSH and used the dbus-spy terminal command. This has a simple interface which you can navigate with the up, down, left, right keys to explore the system bus Service Names (destinations) at the first level and the Object Paths complete with the real time value returned by GetValue method at the second level. These match the inputs we need for the dbus command. I found the Service Names for solar charger was com.victronenergy.solarcharger.ttyUSB3 and at the second level (Object Paths) I was delighted to find Load/I So the dbus call one needs is:

dbus -y com.victronenergy.solarcharger.ttyUSB3 /Load/I GetValue

The -y option is to specify we are using the system bus.

This will go in an Exec Node and the message we get is actually a string "value = x.yyyyyyy" so this needs to be fed into a Function node to convert it to a number and round it to a sensible number of decimal:

text = msg.payload;
text = text.substring(8, text.length - 1);
num = Math.round(text * 100) / 100;
msg.payload = num;
return msg

The only obscure point is that if a text string containing a number it is automatically cast to a number in a numerical expression in JavaScript.

So the result is:

This is not an exact replacement as I have slowed it to once every 30 seconds which saves some nodes in my flows but more importantly, the dbus call is very slow, 0.85 second on the Raspberry Pi 3B+ and I want to reduce the processing load. In the longer term I will replace the dbus call by the more complicated dbus-send.

Smarter versions to get the SmartSolar Load Current

The above solution needs to be extended because the USB ports assigned to VE.Direct interfaces may change after a firmware update and on current OS Release versions after a restart so it is essential to obtain the interface after every restart. The Venus OS gets round that by use of UIDs and these are used in the Candidate versions from v3.00~8 by Victron for VE.Direct Nodes but that does not help us here when accessing the D-Bus directly.

My solution is to:

1. Use dbus -y piped into grep to obtain the interface of the charger. I only have a single examples of the device so it is easy.
2. An Exec Node has the option to append the input msg.payload to the Command so I now 'build' the parameter string using the interface obtained from dbus -y in a Function Node and then stores it in Context using a Change Node. This flow is a one off run once after every restart so it does not matter if it is slow.
3. Next one Injects the payload containing the parameter string from Context into an Exec node and appends it to a dbus-send Command. This is fast.
4. Some of the extraction of the value can be done in the Exec node but it is finally tidied up and rounded to 2 decimal places in a Function Node

So the end result is significantly more complex but is now much faster and should even survive firmware upgrades.

Most of the heavy lifting is done in the Function Node which builds the parameter string which is stored and then passed to the Exec node. Note it is two lines only so be careful copying.

msg.payload = " --system --print-reply --dest=" + msg.payload.trim() + " /Load/I com.victronenergy.BusItem.GetValue | grep variant | grep -Eo '[[:digit:]]+([.][[:digit:]]+)?'";
return msg

These solution above can be adapted to almost any parameter on a VE.Direct (USB) interface but check every stage with debug nodes. In particular check the output of dbus-send which can be quite complex. The only 'clever' bit is the second grep-Eo '[[:digit:]]+([.][[:digit:]]+)?' which extracts decimal numbers from a text string. Those aficionados of awk will be able to formulate many alternative solutions!

The following can be used if you want to continue using dbus -y but have the USB interface found automatically.

msg.payload = " msg.payload.trim() + " /Load/I GetValue | grep value | grep -Eo '[[:digit:]]+([.][[:digit:]]+)?'";
return msg

Example to display VE.Direct USB port allocations

I wrote this as a diagnostic when investigating why nodes were becoming disconnected after a reboot or firmware upgrade. The issue has been fixed for restarts but I want to keep an eye on what is going on for a bit longer.

This implementation requires Custom Names to be set for all VE.Direct (USB) devices using the Remote Console as Custom Names are the only way to distinguish between multiple devices such as my two SmartShunts

Function Nodes labeled Build n use code like this:

stringArray = msg.payload.trim().split(/\s+/);
// Note array indices start at 0 so for second field use Array[1]
msg.payload = stringArray[0] + " /CustomName GetValue ";
global.set("service0" , stringArray[0] , "file" )
return msg;

and Function Nodes labeled Display n use code like this:

text = msg.payload;
serv = global.get("service0" , "file")
serv = serv.substring(serv.length-4 , serv.length)
msg.payload = text.substring(9, text.length - 2) + " \(" + serv +"\)" ;
return msg;

You may wonder why I mostly use global persistent context and that is because I append all of it my global context variables to a file every 2 hours as diagnostic. The disadvantage is 'wear' on the MicroSD card through repeated writes and I may review oon an item by item basis in the future to see where I really need persistence over restarts or long term diagnostics

Tip: Debug nodes give a quick look at what is going on during development if you turn on the display of status (which is infortunately limited to 32 characters). You can see how useful it is in the above screenshot both for debuging and a quick look at whats going on without swapping to the ui display.

The above was a quick and dirty implementation but it shows the simplicity and speed with which a problem can be addressed in Node-RED. It is only updated every few hours as it should never change between reboots so use of dbus rather than the faster dbus-send is fine

A simple D-Bus example which anybody can try is to get the System Name

The examples above are only useful if you have Victron VE.Direct Devices and/or Relays. The following example should work on any Raspberry Pi (or Victron controller) running the Venus OS. Note the lines may be wrapped so take care cutting and pasting when you try it.

root@raspberrypi4:~# dbus -y com.victronenergy.settings /Settings/SystemSetup/SystemName GetValue
'Raspberry Pi 4B r1.5'

Reference List of Services on the Victron Venus OS D-Bus

There is a fairly complete List of available services and their paths on the Venus OS D-Bus

Appendix K - Enabling the Use of sudo (Venus OS 2.90 and higher)

Sudo was added to the final candidate (beta) versions of the Venus OS at about v2.90~28. I continued to update from the candidate series so now 2.90 has been Released I a was on 2.91~1.

The reason for adding sudo was to provide a mechanism for Node-RED to be able to make Exec calls that require root or adm privileges such as shutdown without itself running as root. Earlier versions ran as root but in 2.90 the system was changed so Node-RED ran as user nodered which broke my shutdown, reboot, restart and GPIO code as well as some of my diagnostics. I put in place various work-arounds but Victron also responded to my posts by adding sudo to their build of the Venus OS. The underlying Linux distribution is openembedded-core (currently Yocto Dunfell in v2.90) which can be built with or without sudo.

The nodered user needs to be added to the sudo and adm groups but a password is still needed when sudo is used. There are mechanisms for allowing exceptions so sudo can be used by specified users or groups for certain activities without use of a password and this is set up in the /etc/sudoers file. Any errors in this file can render a system unusable so there is a a special program to edit it called visudo which checks the edited file is valid.

One can add an existing user to multiple secondary groups in one command, use the usermod command followed by -a to specify adding to existing groups and the -G option followed by the names of the groups separated by , (commas):

usermod -a -G group1,group2 username

In this case we are root and can run usermod and we want add the sudo and adm groups. The command id shows the groups that a user belongs to.

root@raspberrypi4:~# usermod -a -G sudo,adm nodered
root@raspberrypi4:~# id nodered
uid=993(nodered) gid=993(nodered) groups=993(nodered),4(adm),20(dialout),27(sudo)
root@raspberrypi4:~#

Group adm is used for system monitoring tasks. Members of this group can, for example, read log files in /var/log.

Now we need to modify the /etc/sudoers file. This should be initially done using visudo to ensure that the edit is correct and all tests should be done that way. Ultimately it may be safe to do a search and replace in the file using, for example, sed as the change will be very simple and previously tested.

But these are dangerous activities for which I take no responsibility. It is possible to brick a Victron controller if you get it wrong by even a single character so it is intended for use on Raspberry Pis where, in the worst case, you can start afresh with a new SDcard! Even so backup everything. Remember the definition of an pessimist is an optimist with experience.

Replacing occurrences of one string with another in files in the current directory is a natural task for sed. A typical call to replace strings looks like this:

sed -i 's/foo/bar/g' *

If we look at /etc/sudoers we find this section:

## User privilege specification
##
root ALL=(ALL) ALL

## Uncomment to allow members of group wheel to execute any command
# %wheel ALL=(ALL) ALL

## Same thing without a password
# %wheel ALL=(ALL) NOPASSWD: ALL

## Uncomment to allow members of group sudo to execute any command
# %sudo ALL=(ALL) ALL

so all we need to do is to uncomment and change the second %wheel to %nodered. I first tested sed on my Linux Mint box with a copy of the sudoers file by:

peter@gemini:~$ busybox sed -i 's/# %wheel ALL=(ALL) NOPASSWD: ALL/%nodered ALL=(ALL) NOPASSWD: ALL/g' sudoers_test
The above is all one line.

What we have so far gets us back to a situation which may be ideal during development but is a highly insecure situation for long term operation so we need to reduce the scope of sudo without passwords to a few programs. The original purpose was to be able to shutdown and and reboot the OS and restart Node-RED so we can restrict to use of the commands shutdown and perhaps svc although allowing use of svc which controls services could open up some big security holes. Anyway the next problem is to find the paths to them so the NOPASSWD: ALL can be replaced by paths to a list of executables.

The command which will find the paths for us

root@raspberrypi4:~# which svc
/usr/bin/svc
root@raspberrypi4:~# which shutdown
/sbin/shutdown
root@raspberrypi4:~#
The line we want in sudoers then becomes

%nodered ALL=(ALL) NOPASSWD: /sbin/shutdown,/usr/bin/svc

These were my test and diagnostic flows so I see the responses in the debug sidebar to the various exec nodes before changing in the operational flows.

The final step in the changes will be to make it persistent through updates by adding the appropriate usermod and sed commands to my existing /data/rc.local file.

You will recall when the files /data/rcS.local or /data/rc.local exist, they will be called early (rcS) and late (rc) during startup. These scripts will survive upgrades and can be used to start custom software.

The sed code to add as a single line is:

sed -i 's/# %wheel ALL=(ALL) NOPASSWD: ALL/%nodered ALL=(ALL) NOPASSWD: \/sbin\/shutdown,\/usr\/bin\/svc/g' /etc/sudoers

The file will only be changed once as the 'search' will not succeed on subsequent restarts or a mechanism can be put in place so the relevant code in /data/rc.local is only run once. Note the escaping of / in strings by a \.

The listing of my rc.local file above already contains my code to implement sudo in OS v2.90 to avoid another large block of code.

Trivia: You may ask why /etc/sudoers has references to %wheel (group wheel): This refers to the term "Big Wheel" or "Wheel" which was a popular term in the 60s. A Big Wheel was an important person. A big wheel might be the head of a company, a political leader etc. In Unix the group wheel (or pre-unix users with a wheel bit set) often had almost or equal powers to the superuser/root which is what you get if those sections in the sudoers file are uncommented.

Appendix L - Full list of activities to set up my system under Venus OS 2.90 Large

These are the activities required after the first update to Venus OS 2.90 which has most likely been a fresh install as per the instructions above. Subsequent firmware updates as Victron call them preserve the data partition where all the files here are stored. The majority of the activities are run by the script file /data/rc.local which the Venus OS runs always as root during startup provided it exists. The various activities have been covered independently and in considerable detail in earlier appendices and many only take place if, for example, associated data files are present

The activities run within /data/rc.local (as root) during startup include:

1. Set up GPIOs (always take place and alternative allocation have to be edited into rc.local)
2. Fix permissions to allow nodered to read the boot log to determine the current OS version running - chmod 666 /data/log/boot
3. Run timer loop in /data/every-second.sh but only if the file is present
4. Replace /etc/venus/gpio_list from /data/gpio_list but only if the file is present
5. Create folder /data/home/nodered/.node-red/log and set permissions to 777 if it does not exist
6. Add code for use of sudo without a password for user nodered (uses sed) - options for commands that can be run can be edited in rc.local

So the extra actions which follow a 'standard' install are:

• sftp files listed below to /data/rc.local, /data/every-second.sh and /data/gpio_list from local machine (using FileZilla)
• Set /data/rc.local and /data/every-second.sh to be executable (Using FileZilla or SSH login)
• sftp file to /data/home/nodered/.node-red/settings_user.js edit in red sections if it exists. (Download from RPI, edit locally and re-upload using FileZilla)

After which Firmware updates should not need any additional activities over the usual re-entering of the Superuser password using the Remote Console. Note: the Raspberry Pi now also needs to also be rebooted from the remote console after the password entry before sudo is fully activated.

The following are the important files as of 3rd November 2022 which all need to be transferred to /data - their final locations are in their comments. Note: Several need to have execute permissions set.

#!/bin/sh
# Contents of /data/rc.local - needs execute permissions.
# Background: When the files /data/rcS.local or /data/rc.local exist,
# they will be called early (rcS) or late (rc) during startup. These scripts will
# survive upgrades and can be used by customers to start their own custom software.
#
# This sets up gpio access for an additional 4 relay outputs
# accessible by node-red in addition to those specified using gpio_list
# Node-red now runs without root access as user nodered
# requiring the setting of permissions to allow access to specific parts of sysfs
echo 22 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio22/direction
chmod 777 /sys/class/gpio/gpio22/*
echo 23 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio23/direction
chmod 777 /sys/class/gpio/gpio23/*
echo 24 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio24/direction
chmod 777 /sys/class/gpio/gpio24/*
echo 25 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio25/direction
chmod 777 /sys/class/gpio/gpio25/*
#
# Fix permissions to allow nodered to read the boot log to determine the current
# OS version running
chmod 666 /data/log/boot
#
# Run timer loop as independent job - an alternative approach to sudo without password
if [ -f /data/every-second.sh ]; then {
echo "$(date) Found every-second.sh - timer loop started" >> /data/rc_local_log;
sh /data/every-second.sh &
}
fi
#
# Create log folder in /data/home/nodered/.node-red/
if [ ! -d /data/home/nodered/.node-red/log ] ; then {
mkdir /data/home/nodered/.node-red/log
chown nodered: /data/home/nodered/.node-red/log
chmod 777 /data/home/nodered/.node-red/log
echo "$(date) Missing log folder created in .node-red by rc.local" >> /data/rc_local_log;
}
fi
#
# Exit if rc.local has already been called after writing log
if [ -f /etc/rc_local_called ]; then {
echo "$(date) Normal Reboot setup completed by rc.local" >> /data/rc_local_log;
exit 0
}
fi
#
# Set up to allow use of sudo without password by user nodered
# Requires sudo and adm groups to added to user nodered
usermod -a -G sudo,adm nodered
# Modify /etc/sudoers file by search and replace so only it is only changed once
# Select only one option 1st allows ALL commands to be run without a password by sudo
# sed -i 's/# %wheel ALL=(ALL) NOPASSWD: ALL/%nodered ALL=(ALL) NOPASSWD: ALL/g' /etc/sudoers
sed -i 's/# %wheel ALL=(ALL) NOPASSWD: ALL/%nodered ALL=(ALL) NOPASSWD: \/sbin\/shutdown,\/usr\/bin\/svc/g' /etc/sudoers
# sed -i 's/# %wheel ALL=(ALL) NOPASSWD: ALL/%nodered ALL=(ALL) NOPASSWD: \/sbin\/shutdown/g' /etc/sudoers
#
# Log when rc.local actioned - should only be once after an OS update overwriting /etc/rc_local_called
echo "$(date) OS version update: sudo has been set up by rc.local" >> /data/rc_local_log
chmod 666 /data/rc_local_log
echo "1" > /etc/rc_local_called
chmod 666 /etc/rc_local_called
#
# Replace the file which provides support for Victron relays
# as it is over-written during OS updates.
if [ -f /data/gpio_list ]; then {
cp /data/gpio_list /etc/venus/gpio_list
chmod 744 /etc/venus/gpio_list
echo "$(date) gpio_list found - Victron Relays set up by rc.local" >> /data/rc_local_log;
}
fi
#
exit 0

# /data/gpio_list (then copied by re.local to /etc/venus/gpio_list)
#- in digital_input_1
#- in digital_input_2
#- in digital_input_3
#- in digital_input_4
16 out relay_1
17 out relay_2
#- out mkx_rst
#- out vebus_standby#!/bin/sh

# /data/every-second.sh (run by rc.local) - needs execute permissions.
#
FILE=/data/home/nodered/.node-red/cmdfile
while true
do
if test -f "$FILE"; then
cmd=$(cat "$FILE")
echo "$(date) - Running $cmd" >> /data/root_command_log
rm "$FILE"
eval "$cmd"
fi
sleep 1
done

// Contents of /data/home/nodered/.node-red/settings-user.js

module.exports = {

/* [Caution from @mvader:] The vast majority of systems using Node-RED will not and (should not !!) have to modify this file. Venus OS itself, including Venus OS large and Node-RED, is made such that its not necessary to dive into the command line. Anyone that does go in and makes changes needs to be prepared for things going wrong, for example, after a firmware update. The information here provided is for the benefit of the handful of users that do need (or want) to change them; and to put it bluntly: know what they are doing and are prepared to spent hours and hours fixing issues without getting any help [from Victron staff]. */

/* @jeroen commented · Sep 11 2022 at 2:21 PM
if you add  uiHost: "",
to /data/home/nodered/.node-red/settings-user.js
the http server will be available again on port 1880
and that will survive a firmware update. [ Useful for Android users on Local network]
*/

  uiHost: "",

/* * Context Storage
   * The following property can be used to enable context storage. The configuration
   * provided here will enable file-based context that flushes to disk every 30 seconds.
   * Refer to the documentation for further options: https://nodered.org/docs/api/context/
*/
   //contextStorage: {
   // default: {
   //  module:"localfilesystem"
   //  },
   //}, 

// The following is better as it provides an option of memory or file and works!

   contextStorage: {
     default: "memoryOnly",
     memoryOnly: { module: 'memory' },
     file: { module: 'localfilesystem' }
   },
}

The rc.local file is mandatory as is settings-user.js if you want persistence in Context variables

Note to users of Kevin Windrem's SetupHelper and GpioSetup. If you only need 4 relays accessible from Node-RED use GpioSetup and the Victron Nodes, and comment out the GPIO section in rc.local and do not add a gpio_list file. If you need more you need to take care they do not overlap - Kevin Windrem has excellent configuration information available. Otherwise the two approaches seem compatible as his setup only uses the rcS.local file.

Conclusions

Everybody jumps to the end to see the conclusions but in this case they are at the end of the Main text not the Appendices at Conclusions - How is the overall Raspberry Pi system with Node-RED, working in practice?