Live boat instruments' data on internet via GSM - open source project

AngusMcDoon

Well-known member
Joined
20 Oct 2004
Messages
8,659
Location
Up some Hebridean loch
Visit site
My boat is an IoT thing. It's joined all the other useless things on the interweb like fridges and lightbulbs. This is yet another pointless project to keep me amused while I'm not allowed out to worry the sheep. What I've done is wired up a cheap GSM modem (SIM800L) which costs about £3.50 on a breakout board to a very cheap Blue Pill dev board which costs even less at about £2 and connected the lot to my Seatalk network to read boat data. The code on the Blue Pill reads the numbers from Seatalk and wraps them up in MQTT messages, then sends them off to a public MQTT broker. Now the data are available out there anything can just grab it and do what it wants with it, like this web page I knocked together with a bit of rough and ready javascript...

http://bluepilldemo.000webhostapp.com/myboatisathing.html

(Note: has to be http on that link, not https. It's to do with the lack of a https websocket interface on the Hive public MQTT broker)

The web page shows what's coming in over Seatalk. Clicking the button displays a map showing the lat/long data as a red square (you can click on the red square for added japes).

This is the Blue Pill board...

IMG_20200509_121127351.jpg

and this is the SIM800L modem with a cheapo disposable PAYG SIM with a fiver of credit...

IMG_20200509_121134787.jpg

The SIM800L has a particular power requirement: 4.2 Volts and up to 2 Amps momentarily, so it needs a bit of a power supply jiggery-pokery to enable that. I've used a LM217 wiring it up as to its datasheet.

The software on the Blue Pill sounds simple enough - go round in a loop reading the Seatalk data, package up some MQTT messages, send them to the modem using a TCP socket connected to the broker. However...

- Seatalk is simple enough. I've whipped that out of many other pointless projects.

- MQTT is a simple protocol, and there are libraries out there to do the packaging, but they are all too big for the limited STM32F103C8 on the Blue Pill board. I only want a QoS of 0 and the published libraries all provide the full range of QoS which bloats the code. So I got the MQTT standards out and rolled out my own code to create the messages. That's a bit of a kerfuffle but not too onerous.

- The final bit it the hard bit - talking to the modem in a reliable way. Modems use a command set called AT, and despite it being only human readable messages pinging backwards and forwards along 2-way serial lines, it's the most awkward and unpredictable protocol to implement. Writing to the modem is ok, but reading from it is problematic. The commands are irregular in format, it's highly asynchronous, and the command/response sequences can go many ways - you just don't know when you are going to get what. Two further difficulties are that the complete AT command set for a modem is huge - many thousands of pages of documents, and non-standard. Some basic commands are common across manufacturers, but most are not. As the code doesn't know what or when the responses will be or will arrive they are buffered using DMA and are processed when processing time is available. This benefits from the use of a separate thread so FreeRTOS has been used as an OS. This means that the code must be made thread safe as there may be multiple threads trying to access the modem, but there's only one modem. The internal server concept has been used with the modem running as an internal server in its own thread with queues to communicate AT commands and responses to the client threads.

The code on the Blue Pill works, but could (and will be) better. Currently it's not fault tolerant. If a connection to the broker fails it all falls over and doesn't attempt to reconnect. This is not good for a socket going over a GSM connection as unexpected disconnects are, er, expected.

This will all be open sauce once I've got things a bit more organised, and then I'll plonk it on Github with my other YAPPs. I've yet to draw up the circuit diagram schematic; currently it exists only as a tangle of wires on the breadboard and a tangle of lines in my head, so more to come later.
 
Last edited:

rogerthebodger

Well-known member
Joined
3 Nov 2001
Messages
12,544
Visit site
That's great I would be interesting in building one.

Let us know when you have published the info.

How easy would it be to convert to NMEA 0183
 

V1701

Well-known member
Joined
1 Oct 2009
Messages
4,603
Location
South Coast UK
Visit site
I got some but not all of that, very neat! If it could be made reliable and put in a fancy looking box some people would pay a small fortune for that...
 

Mistroma

Well-known member
Joined
22 Feb 2009
Messages
4,893
Location
Greece briefly then Scotland for rest of summer
www.mistroma.com
I got some but not all of that, very neat! If it could be made reliable and put in a fancy looking box some people would pay a small fortune for that...

You could look here Boat Vitals.

I don't know if Vasko is still selling units but used to be very cheap. I remember that he did develop nice Wireless networked sensors you just plugged in around the boat and the main unit picked them up automatically. Not certain if this was developed from that. I forgot to ask last time we met in Greece.
 

AngusMcDoon

Well-known member
Joined
20 Oct 2004
Messages
8,659
Location
Up some Hebridean loch
Visit site
That's great I would be interesting in building one.

Let us know when you have published the info.

How easy would it be to convert to NMEA 0183

Schematic done and the project is on github here...

miniwinwm/BluePillDemo

NMEA-0183 could be added. There's a framework for it and a few message type listeners in this project...

miniwinwm/BoatDataHub

See src/nmea.c and src/nmea.h

Here's an ebay link (while it's available) for the SIM800L board...

SIM800L GPRS GSM Module Card Board Quad-band Onboard + Antenna | eBay

Here's one a bit more with a better antenna (I use an external antenna like this, not that spring thing)...

SIM 800L GPRS GSM SIM MODULE BOARD QUAD-BAND TTL WITH ANTENNA FOR | eBay

Here's a Blue Pill board. The price seems to vary daily...

1X STM32F103C8T6 Arm STM32 Minimum System Development Board for \ | eBay

You need a ST-Link programmer to get the code on the board...

ST-Link V2 Programming Unit mini STM8 STM32 Emulator Downloader | eBay

The code is built in STM32CubeIDE from ST, which is free.
 
Last edited:

AngusMcDoon

Well-known member
Joined
20 Oct 2004
Messages
8,659
Location
Up some Hebridean loch
Visit site
Thanks John will have a look

It's likely to be updated in the near future so if you download (or do a clone) do another download (or pull) every now and then. I've just added the MQTT publish topic root as a #define in main.c instead of hardcoded.

Now the device restarts on a comms failure of any kind and picks up where it left off.
 
Last edited:

AngusMcDoon

Well-known member
Joined
20 Oct 2004
Messages
8,659
Location
Up some Hebridean loch
Visit site
@AngusMcDoon Any reasons not to use the Ardunio IDE/Libs?

Arduino and its environment is not used industrially. My day job is an embedded developer working with ARM and RX processors. I do this open source stuff as a learning exercise. The goal of the last few weeks has to get to grips with the new STM32CubeIDE and CMSIS RTOS v2 API. In the process of that I have created a whole bunch of example projects of which this is one. The others are here...

miniwinwm/BluePillDemo
 
Last edited:

AngusMcDoon

Well-known member
Joined
20 Oct 2004
Messages
8,659
Location
Up some Hebridean loch
Visit site
Anyone tried lora to upload data yet?
LoRa - Wikipedia

I've never had an application that requires the range that LoRa provides. It's either been 15m maximum, for which WiFi, Bluetooth or cheapo 2.4 GHz transceiver chips work, or else it's been to get on the internet, for which inexpensive GPRS modems do the job, like the SIM800L I've used in this projects. I've also used Quectel and uBlox LTE modems.
 

AngusMcDoon

Well-known member
Joined
20 Oct 2004
Messages
8,659
Location
Up some Hebridean loch
Visit site
How easy would it be to convert to NMEA 0183

NMEA0183 version up now...

miniwinwm/BluePillDemo

It only parses RMC and DPT messages so far, publishing depth, SOG, COG, latitude and longitude to the broker and on to the website, but the framework is there to extend it for other message types. There are 'todo' comments in the code where this needs to be done. The NMEA0183 data structures and field names can be taken from this file to speed things up...

miniwinwm/BoatDataHub

If anyone wants other message types added I can either assist or maybe do them if they are simple ones like boat speed, logs, wind etc.
 

agurney

Active member
Joined
10 Jun 2009
Messages
1,518
agurney.com
Great stuff, thanks, keep it up :)

I've been playing around with ESP32 & ESP8266 controllers using an A6 module for GSM for vaguely similar stuff.
For the SIM I'm using one of ASDA's, as their allowance doesn't expire if you don't use it, and the cost of sending over SMS is acceptable.

Lockdown's given me the opportunity to tinker .. the downside is that I can't get to the boat to test it :(

[I'm a wimp; I can get my head around the Arduino IDE, and Espruino/Arduino for an STM32_100_P, but the STM32CubeIDE defeats me]
 

AngusMcDoon

Well-known member
Joined
20 Oct 2004
Messages
8,659
Location
Up some Hebridean loch
Visit site
NMEA0183 version up now...

miniwinwm/BluePillDemo

It only parses RMC and DPT messages so far, publishing depth, SOG, COG, latitude and longitude to the broker and on to the website, but the framework is there to extend it for other message types. There are 'todo' comments in the code where this needs to be done. The NMEA0183 data structures and field names can be taken from this file to speed things up...

miniwinwm/BoatDataHub
Great stuff, thanks, keep it up :)

I've been playing around with ESP32 & ESP8266 controllers using an A6 module for GSM for vaguely similar stuff.
For the SIM I'm using one of ASDA's, as their allowance doesn't expire if you don't use it, and the cost of sending over SMS is acceptable.

Lockdown's given me the opportunity to tinker .. the downside is that I can't get to the boat to test it :(

[I'm a wimp; I can get my head around the Arduino IDE, and Espruino/Arduino for an STM32_100_P, but the STM32CubeIDE defeats me]

I use an ASDA SIM as well. The network is EE which I have found has the best coverage. There aren't many places on the west of Scotland where it doesn't work, although I've found 4G coverage is better than 2G.

Everyone struggles with STM32CubeIDE at first, and then later on as well! It's aimed at industrial and professional users rather than students and hobbyists, as the Arduino environment is, which means it's much more complex to use - but you can do much more with it. For example, you can configure timers in all their multiple modes which you can't do in Arduino. It also allows integration of FreeRTOS. I've found that my projects are of a complexity now that they benefit from an OS rather than the bare bones of my YAPP days.
 

fifer

Well-known member
Joined
12 Apr 2013
Messages
353
Visit site
My boat is an IoT thing. It's joined all the other useless things on the interweb like fridges and lightbulbs. This is yet another pointless project to keep me amused while I'm not allowed out to worry the sheep. What I've done is wired up a cheap GSM modem (SIM800L) which costs about £3.50 on a breakout board to a very cheap Blue Pill dev board which costs even less at about £2 and connected the lot to my Seatalk network to read boat data. The code on the Blue Pill reads the numbers from Seatalk and wraps them up in MQTT messages, then sends them off to a public MQTT broker. Now the data are available out there anything can just grab it and do what it wants with it, like this web page I knocked together with a bit of rough and ready javascript...

http://bluepilldemo.000webhostapp.com/myboatisathing.html

(Note: has to be http on that link, not https. It's to do with the lack of a https websocket interface on the Hive public MQTT broker)

The web page shows what's coming in over Seatalk. Clicking the button displays a map showing the lat/long data as a red square (you can click on the red square for added japes).

This is the Blue Pill board...

View attachment 89922

and this is the SIM800L modem with a cheapo disposable PAYG SIM with a fiver of credit...

View attachment 89923

The SIM800L has a particular power requirement: 4.2 Volts and up to 2 Amps momentarily, so it needs a bit of a power supply jiggery-pokery to enable that. I've used a LM217 wiring it up as to its datasheet.

The software on the Blue Pill sounds simple enough - go round in a loop reading the Seatalk data, package up some MQTT messages, send them to the modem using a TCP socket connected to the broker. However...

- Seatalk is simple enough. I've whipped that out of many other pointless projects.

- MQTT is a simple protocol, and there are libraries out there to do the packaging, but they are all too big for the limited STM32F103C8 on the Blue Pill board. I only want a QoS of 0 and the published libraries all provide the full range of QoS which bloats the code. So I got the MQTT standards out and rolled out my own code to create the messages. That's a bit of a kerfuffle but not too onerous.

- The final bit it the hard bit - talking to the modem in a reliable way. Modems use a command set called AT, and despite it being only human readable messages pinging backwards and forwards along 2-way serial lines, it's the most awkward and unpredictable protocol to implement. Writing to the modem is ok, but reading from it is problematic. The commands are irregular in format, it's highly asynchronous, and the command/response sequences can go many ways - you just don't know when you are going to get what. Two further difficulties are that the complete AT command set for a modem is huge - many thousands of pages of documents, and non-standard. Some basic commands are common across manufacturers, but most are not. As the code doesn't know what or when the responses will be or will arrive they are buffered using DMA and are processed when processing time is available. This benefits from the use of a separate thread so FreeRTOS has been used as an OS. This means that the code must be made thread safe as there may be multiple threads trying to access the modem, but there's only one modem. The internal server concept has been used with the modem running as an internal server in its own thread with queues to communicate AT commands and responses to the client threads.

The code on the Blue Pill works, but could (and will be) better. Currently it's not fault tolerant. If a connection to the broker fails it all falls over and doesn't attempt to reconnect. This is not good for a socket going over a GSM connection as unexpected disconnects are, er, expected.

This will all be open sauce once I've got things a bit more organised, and then I'll plonk it on Github with my other YAPPs. I've yet to draw up the circuit diagram schematic; currently it exists only as a tangle of wires on the breadboard and a tangle of lines in my head, so more to come later.

Really interesting project. I was hoping to do something similar (but probably at higher cost) using a RPi, openplotter and a GSM dongle or MiFi to access boat data directly from home. The principle stumbling block appears to be the lack of a static IP for the boat, which makes two way transfer between boat and home more tenuous when IP address change regularly.

Are you getting around this issue by only accessing the data indirectly through a remote server which hosts the data for you rather than talking to the boat system directly? If so could you point me in the direction of services that allow you to do this? I'm pretty unfamilar with MQTT but keen to learn. It seems like NodeRed in openplotter could be used to manage data distribution through the attached modem to an external site.
 

agurney

Active member
Joined
10 Jun 2009
Messages
1,518
agurney.com
To get around your IP problem I use this service - remote.it - remote.it - running on a raspberry pi.
The Pi runs continually, and as long as it can connect to the marina wifi I can access the Pi remotely.
It doesn't matter that the Pi doesn't have a static address, and the marina's IP address keeps changing .. though I do have a couple of hundred miles round trip when the marina changes its password!
For a similar reason I use a dynamic DNS service at home so the remote Pi can call home.
 

GHA

Well-known member
Joined
26 Jun 2013
Messages
12,257
Location
Hopefully somewhere warm
Visit site
Really interesting project. I was hoping to do something similar (but probably at higher cost) using a RPi, openplotter and a GSM dongle or MiFi to access boat data directly from home. The principle stumbling block appears to be the lack of a static IP for the boat, which makes two way transfer between boat and home more tenuous when IP address change regularly.
If you have web access then getting onto openplotter is very easy, just create an account with real vnc and you can view the Pi from anywhere. No need to bother with ip addresses. This can be useful if you don't have a separate USB WiFi dongle, openplotter can use the internal WiFi adaptor to create a local network and attach to a different WiFi network at the same time but it's not very stable with more than on machine attached to the openplotter network so if you have a Pi with no monitor viewing over the Web gets round this.
 

agurney

Active member
Joined
10 Jun 2009
Messages
1,518
agurney.com
If you have web access then getting onto openplotter is very easy, just create an account with real vnc and you can view the Pi from anywhere. No need to bother with ip addresses. This can be useful if you don't have a separate USB WiFi dongle, openplotter can use the internal WiFi adaptor to create a local network and attach to a different WiFi network at the same time but it's not very stable with more than on machine attached to the openplotter network so if you have a Pi with no monitor viewing over the Web gets round this.

RealVNC is great, but you have to know where the target is located (e.g. Marina's network), and hope that they aren't blocking the VNC TCP ports (never mind tedious stuff like port forwarding and other boats connected to the network). Outbound is trivial, but initiating a connection from outside .. how does openplotter get around that?
 
Top