IoT, budget VPS and correctly changing JAVA_OPTS in Thingsboard
Published on:
This will be a bit of a story of my IoT world exploration, encountering a bottlneck and fixing it.
For a few months now I have been working on ESP32 devices that connect to Deye solar inverters through RS485 and Modbus.
Its very refreshing for me to discover IoT hardware and make something physical that allows you to control devices remotely.
Learning protocols, cavets, wiring and all of that is fun coming from web development.
I started with ESP32 Dev kits and now using Waveshare ESP32 boards. Waveshare boards are perfect for RS485/CAN communication, small, cheap and can be mounted on DIN rails.
For controlling the boards, I chose Thingsboard platform, not sure if there are any alternatives but it is really good and very extensive.
I tried hosting it on a cheapest VPS I managed to find.
First I tried Contabo, you can get 4vCPUs, 8GB RAM and 100GB SSD for about 5 EUR. It was working fine but I was hosting otherthings except thingsboard and I run eventually run into CPU and RAM limits.
Then I found LuxVPS's "special deals", 4vCPUs, 16GB RAM and 100GB NVMe for 7 EUR. They use old Intel Xeon CPU's but honestly for a budget VPS with that much RAM its a great deal, especially during RAM shortage caused by AI.
When migrating to it I instantly knew it will much better than Contabo. Transfering files from Contabo and responsiveness of the VPS were great.
After the migration to LuxVPS I started tinkering with Thingsboard docker setup and tried to assign more memory to the JVM. AI and some articles pointed my at JAVA_OPTS env var. I added it in the docker-compose.yml and left it there for some time until I realized the avg. CPU usage was about 50%. I knew something was wrong. I figured out that it was related to JVM GC as CPU was spiking from ~20% to ~60%+ every few seconds. I tried adjusting the JAVA_OPTS more but couldn't figure it out, nothing was really making it better.
After some time, I got an email from LuxVPS that their special deals will end soon and I decided to upgrade to higher tier VPS. I went with 6vCPUs, 26GB RAM, 250GB NVMe for 12 EUR. This will let me scale thingsboard if I ever end up working with hundreds of devices.
LuxVPS doesn't help you with migration so I had to do the full migration again. After that I looked into the CPU spikes and they were still there.
It finally found a solution, it was right there on the Thingsboard documentation: https://thingsboard.io/docs/installation/ubuntu/#step-5-optional-memory-configuration. Instead of using the docker env var declaration you have to alter the JAVA_OPTS in the /etc/thingsboard/conf/thingsboard.conf file, like this:
export JAVA_OPTS="$JAVA_OPTS -Xms6G -Xmx6G -Xss512k -XX:+AlwaysPreTouch"Funny that AI and some other internet articles pointed me into a wrong direction where the real answer, as usual, is on the official documentation.