Zero Grid Export and Reading energy meter with ESP

Topics concerning storage of renewable energy e.g. home battery systems
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Zero Grid Export and Reading energy meter with ESP

Post by johu »

With rising electricity prices I am currently revisiting control of my small solar system (about 800 Wp).
The plan is to rig up some left over batteries with a cheap Chinese battery inverter to collect energy during the day and use it at night. Or more precisely prevent feeding energy to the grid (because I'm not payed for export) and to some extend, prevent using energy from the grid.

To start out we first need to know the energy flow from/to the grid. I had tried openenergymeter in the past but it was very inaccurate, I found it hard to use and also it required a Raspberry PI to run 24/7 which defeats the purpose: conserving energy.

So I researched my existing utility energy meter a bit to find out it actually outputs a serial data stream via infrared (apparently the format is called SML).

I started out with the physical layer. I mounted one of these reflective encoder heads (left over from Polo project) which essentially consists of an IR diode and a photo transistor. The diode is NOT needed! Nothing needs to be requested, the meter just dumps out the info every second.
IMG_20220324_124247.jpg
Ok, the photo transistor needs 15k pull up (47k makes it too slow, 10k gives too little amplitude). Since the waveform looks rather distorted we feed it through a logic gate, I just use two inverters 74ACT04.

Then it goes to the ESP via a 470 Ohm resistor to RX. RX is connected to 3V3 via a diode to level shift the 5V to a 3V3 signal.
IMG_20220324_124121.jpg
Serial data:

Code: Select all

/EBZ5DD3BZ06ETA_107

1-0:0.0.0*255(1EBZ0100200608)
1-0:96.1.0*255(1EBZ0100200608)
1-0:1.8.0*255(007306.09759253*kWh)
1-0:16.7.0*255(000179.47*W)
1-0:36.7.0*255(000044.82*W)
1-0:56.7.0*255(000134.65*W)
1-0:76.7.0*255(000000.00*W)
1-0:32.7.0*255(232.6*V)
1-0:52.7.0*255(233.1*V)
1-0:72.7.0*255(232.9*V)
1-0:96.5.0*255(001C0104)
0-0:96.8.0*255(064F18F0)
!
Starts with '/' ends with '!' - handy.

Next we need to write some Arduino C++. Little surprise I took the inverter sketch and modified it. Serial interface runs at 9600 7E1 (first time I see parity used).

Soon had some output
Bildschirmfoto vom 2022-03-24 07-40-46.png
Bildschirmfoto vom 2022-03-24 07-41-20.png
Now went off on a tangent and decided to record that data to a mysql database. That allows more systematic analysis.

So now have a 1-second updating web page
Bildschirmfoto vom 2022-03-24 14-28-52.png
And a daily overview
today.png
Everything below 0 means we're exporting to the grid. Some number crunching later I found I "lost" 2 kWh today.

Decided to not run 10m of wire through the public stair case, below our apartment door and onto my desk. So put everything into a DIN rail enclosure next to a DIN rail 5V supply. Pretty huh?
IMG_20220324_190744.jpg
Code: https://github.com/jsphuebner/esp-egycounter
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
Bigpie
Posts: 1585
Joined: Wed Apr 10, 2019 8:11 pm
Location: South Yorkshire, UK
Has thanked: 74 times
Been thanked: 299 times

Re: Zero Grid Export and Reading energy meter with ESP

Post by Bigpie »

I feel your pain, I'm not paid for export. My new solar exported almost 8kwh today. Does your smart meter have the P1 port?
https://github.com/zuidwijk/dsmr/
Attachments
Screenshot 2022-03-24 at 8.08.20 pm.png
VW Beetle 2003
Outlander front generator
Prius Gen 3 inverter (EVBMW logic board)
Outlander charger
3x Golf GTE batteries
Chademo Charging
Outlander water heater
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

That looks pretty.
It doesn't have any (accessible) electrical port, just IR ones
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
Bigpie
Posts: 1585
Joined: Wed Apr 10, 2019 8:11 pm
Location: South Yorkshire, UK
Has thanked: 74 times
Been thanked: 299 times

Re: Zero Grid Export and Reading energy meter with ESP

Post by Bigpie »

It's part of home assistant. I'm working on an ESP32 to take the export current and make my EVSE max amps follow it. I might have to look into your cheap Chinese inverter solution and a chademo plug to take some power from the car and reduce the blue parts of the graph.
I've got a CT clamp around the house load and the PV generation and taking one from the other to get import/export values, not the most accurate but it'll do for now.
VW Beetle 2003
Outlander front generator
Prius Gen 3 inverter (EVBMW logic board)
Outlander charger
3x Golf GTE batteries
Chademo Charging
Outlander water heater
paaa
Posts: 209
Joined: Fri Dec 06, 2019 8:59 pm
Location: Dublin & Kilkenny Ireland
Has thanked: 2 times
Been thanked: 11 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by paaa »

Im on this journey too , im using a shelly energy meter with assistant and bought an smart evse to dump the excess energy into the car. Only caveat is that min 100% solar mode is 6a but you can blend say 50% with grid by starting charger at 3a of export as its the same price as night rate for me. One crazy option is to use a transformer to reduce voltage to 100v for better resolution as car charger sports that voltage. My next development is to use an ac coupled storage inverter(sunny boy sb storage) with a ev battery then integrate it by chademo v2g mode. Currently just got a can capture of a compatible battery used with inverter to test.
Attachments
Screenshot 2022-03-27 at 21.53.32.png
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

Yes I'm planning to couple my fathers EVSE to this. He has 8kW of solar and most goes into the grid. He manually plugs in the car when he thinks it's appropriate.
Cool thing with using the meter is there's no extra cost (except the ESP module) and the thing is crazy accurate. I can even see when I start playing a video on my laptop.
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
Bigpie
Posts: 1585
Joined: Wed Apr 10, 2019 8:11 pm
Location: South Yorkshire, UK
Has thanked: 74 times
Been thanked: 299 times

Re: Zero Grid Export and Reading energy meter with ESP

Post by Bigpie »

paaa wrote: Sun Mar 27, 2022 8:57 pm My next development is to use an ac coupled storage inverter(sunny boy sb storage) with a ev battery then integrate it by chademo v2g mode.
Those Sunny Boy Storage inverters look pretty good, I want one :D Do you need other sunny boy equipment (home manager/meter)?
VW Beetle 2003
Outlander front generator
Prius Gen 3 inverter (EVBMW logic board)
Outlander charger
3x Golf GTE batteries
Chademo Charging
Outlander water heater
paaa
Posts: 209
Joined: Fri Dec 06, 2019 8:59 pm
Location: Dublin & Kilkenny Ireland
Has thanked: 2 times
Been thanked: 11 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by paaa »

Bigpie wrote: Mon Mar 28, 2022 2:09 pm
paaa wrote: Sun Mar 27, 2022 8:57 pm My next development is to use an ac coupled storage inverter(sunny boy sb storage) with a ev battery then integrate it by chademo v2g mode.
Those Sunny Boy Storage inverters look pretty good, I want one :D Do you need other sunny boy equipment (home manager/meter)?
You need a sunspec compatible meter , I bought an elgris smart lan meter and it seems to par just fine. Its not cheap at about 180e but much cheaper than any smaller ones. Later on im hoping to just use an open source design of an ease meter that supports rs485 already to replace it. but that's a long way away.
User avatar
Bigpie
Posts: 1585
Joined: Wed Apr 10, 2019 8:11 pm
Location: South Yorkshire, UK
Has thanked: 74 times
Been thanked: 299 times

Re: Zero Grid Export and Reading energy meter with ESP

Post by Bigpie »

What's the purpose of the meter? I'm assuming it's instead of CT clamps to stop export from battery and to charge with excess?

My other cheap idea was to hook a HV battery up to the solar input on a normal PV inverter, set to 0 export using the CT clamp, and hookup an outlander charger to the pack for charging.
VW Beetle 2003
Outlander front generator
Prius Gen 3 inverter (EVBMW logic board)
Outlander charger
3x Golf GTE batteries
Chademo Charging
Outlander water heater
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

Successfully putting excess solar power into some old LFP batteries. I'm using a BeagleBone (for now) and a Manson switch mode supply with USB (for now) and am running the following script.
It fluctuates a bit (+- 10W) but reacts pretty quickly. Currently wife is cooking and the hot plates switch on and off in some 30s interval. In between it ramps up the battery and then ramps it down again within a second.

Code: Select all

#!/usr/bin/python

import serial, time, requests

ser = serial.Serial('/dev/ttyUSB0', 9600, timeout=0.2)
lastSolarPower = 0

while True:
	try:
		ser.flushInput()
		ser.write("GETD\r")
		output = ser.read_until('\r')
		print (output)
		r = requests.get("http://192.168.178.54/cmd?cmd=get%20ptotal")
		ptotal = float(r.text)
		if len(output) == 10:
			voltage = float(output[:4]) / 100
			current = float(output[4:-2]) / 100
			power = voltage * current
			solarpower = power - ptotal
			print ("battery power: " + str(power) + "W grid power: " + str(ptotal) + "W solar excess: " + str(solarpower) + "W")
			lastSolarPower = (solarpower + lastSolarPower) / 2  #IIR Filter
			newcurrent = max(0,lastSolarPower / voltage)
			ser.flushInput()
			ser.write("CURR%.3d\r" % (newcurrent * 10))
			print("Setting new current setpoint to " + str(newcurrent))
			print (ser.read_until('\r'))
	except requests.exceptions.ConnectionError:
		print ("Connection error, setting current to 0")
		ser.write("CURR000\r")
		continue
Attachments
IMG_20220329_122656.jpg
Bildschirmfoto vom 2022-03-29 12-33-33.png
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
tom91
Posts: 1273
Joined: Fri Mar 01, 2019 9:15 pm
Location: Bristol
Has thanked: 97 times
Been thanked: 202 times

Re: Zero Grid Export and Reading energy meter with ESP

Post by tom91 »

Do you have a datasheet or link for the inverter you used?
Founder Volt Influx https://www.voltinflux.com/
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

It's a Soyosource GTN1200 that I bought on aliexpress. 160€ delivered
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
jnsaff
Posts: 176
Joined: Fri Oct 18, 2019 7:42 am
Location: Tallinn, Estonia
Has thanked: 2 times
Been thanked: 8 times

Re: Zero Grid Export and Reading energy meter with ESP

Post by jnsaff »

I would also be mindful that some meters count exported energy as energy used and it will be added to your bill instead of just lost.

We have had these problems here in Estonia.
whereswally606
Posts: 43
Joined: Sun Nov 14, 2021 6:16 pm
Has thanked: 1 time
Been thanked: 2 times

Re: Zero Grid Export and Reading energy meter with ESP

Post by whereswally606 »

Got the gtn1200 24v battery type. Struggling to get their proprietary wifi dongle which looks to have an esp8266 inside to connect to my home network. Needs a 2.4ghz router and unfortunately my main router is 5 ghz and it wont pair to my BTHomeSpot powerrline ethernet 2.4ghz one.

Anyway ideally i dont want a wifi connection so i may invest in an esp with wired ethernet as i have a wired router near the battery bank.

Would be nice to have it working using their stock equipment (to test with) but their current clamp is too small and underrated for the main out of the house.

Anyway any more info on how to run the gtn 1200 using esp and how its hooked up physically will be very helpful.
Ive cheated on the current measurement and bought a shelly em and run home assistant.
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

I've done some reverse engineering of the protocol between the wifi module and the inverter: https://forum.drbacke.de/viewtopic.php?p=22798#p22798
It is in German though
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
whereswally606
Posts: 43
Joined: Sun Nov 14, 2021 6:16 pm
Has thanked: 1 time
Been thanked: 2 times

Re: Zero Grid Export and Reading energy meter with ESP

Post by whereswally606 »

Thanks johannes. Finally got the shoddy app working. Turns out that if you have a username with numbers in it the app will just throw them away and not warn you that its done it and then pretend that your username exists already but tell you your password is wrong. Anyway now i can log in to the app i have managed to add the inverter to the wifi and can program it to do constant power with safety shutoffs. This should be good enough for now to remove some 500w of base loads from the house after the sunset.

I will look into a wired esp8266 and your reversing of the protocol. Having it react fast to changes in cloud cover will aid cheaper car charging in the day too.

The other issue i had was 2.4 ghz wifi. I desynchrionized my settings for the 5 and 2.4ghz and renamed the 2.4ghz ssid. Unfortunately thus broke lots of smart things like the shelly em, all 5 tuya wifu switches and in turn lost a few settings in home assistant. That did get me a working inverter though but a painful afternoon walking round manually resetting things.

Also the soyosource app is shockingly bad and the popups are in chinese even when the lang is set to english. Thankfully i live 2 doors away from a friendly chinese family. Matched set is how the device paired is translated.

Will peruse your reversing efforts tomorrow when inhave a clearer head.
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

I was never able to install the app but got all settings done with their "cloud" website. Then removed the wifi stick.

Power setpoint can not be done via the wifi TTL port but only via the RS485 port next to it.
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

Relocated the whole system to the basement where it's cooler, more fireproof and most of all not as disturbing as when sitting in my desk. Both charger and inverter can be pretty noisy when operating at power.

Next step is replacing the lab power supply by an Eltek flatpack. I hope it goes as low as 44V.
Attachments
1660317863369.jpg
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

And today I did the Eltek replacement. Lab supply works perfectly well but I thought it is a bit over-qualified.

The Eltek uses CAN, the BeagleBone has native CAN so one less device connected over the notoriously unreliable USB.

There is one challenge though: below 48V the Eltek no longer accept the current limiting command. But has a 43.5V - 57.5V working range. So now I just do a PI loop that changes output voltage to arrive at the required output power. The measurements are updated fairly often, like 5 times per second so it seems to work reasonably well. Also the 10 mV resolution is very helpful. Mind you, the controller gains must be super low because a couple of mV make a whole lot of difference in battery current.

The driver will show up on github in a minute
Attachments
1660581470695.jpg
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
rstevens81
Posts: 349
Joined: Sun Dec 22, 2019 10:36 am
Location: Bristol, UK
Has thanked: 21 times
Been thanked: 91 times

Re: Zero Grid Export and Reading energy meter with ESP

Post by rstevens81 »

Silly question I see you have used 2 lifepo4 cells to boost the voltage was curious what you strategy was to not over charge or under discharge them?
I assume thair (ah) capacity is substantially more than the lion pack and you basically discharged both the lion and the lifepo4 cells to half charged then hooked it together, that way regardless of the the soc of the lion, the lifepo4 will never be empty or full and sit in the middle of its soc?
Rule 1 of EV Club is don't buy a rust bucket....
Which rule does everyone forget 🤪
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

rstevens81 wrote: Fri Oct 21, 2022 9:20 am Silly question I see you have used 2 lifepo4 cells to boost the voltage was curious what you strategy was to not over charge or under discharge them?
I assume thair (ah) capacity is substantially more than the lion pack and you basically discharged both the lion and the lifepo4 cells to half charged then hooked it together, that way regardless of the the soc of the lion, the lifepo4 will never be empty or full and sit in the middle of its soc?
Yes exactly. The LiIon is 40Ah and so are the LFP. Since it's 2P they well exceed the other pack despite being rather old. I just bumped them up until they were far enough from discharged and discovered they are also far enough from charged when the LiIon is full.
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

Next upgrade to this project will be tibber support or variable rate support in general.
Tibber forwards the spot market electricity prices to the consumer (I think hourly resolution). So you can move your electricity demand on low rate hours. There is no fixed pattern, in summer noon hours will be cheap because of the vast PV capacity, in winter night hours should be cheap because general demand is low and output from wind farms is higher (on average) than in summer.

Plan is to automate car charging and also to charge the ESS battery with cheap electricity and discharge it when rates are high (while still not exporting to the grid because German rules and regulations)
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

Added the Switched ADC BMS yesterday. It works a treat and now I can keep an eye on the cell voltages. That said, without any balancing after 1 year of operation the delta is 5mV.

I found when two python scripts receive CAN messages you have to configure filters. Otherwise they might "steal" each others messages.
image.png
Attachments
IMG_20230316_104223.jpg
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: Zero Grid Export and Reading energy meter with ESP

Post by johu »

I found the Flatpack consumes a hefty 200 mA from the battery once its AC is disconnected. It also keeps sending CAN messages with no AC, so apparently the battery voltage backfeeds into the control circuit. I fitted a diode to its output.
Was also having CAN bus trouble (bus off condition), maybe related to switching it off. Connecting the can bus gnd fixed the issues.

EDIT: no the CAN bus trouble wasn't fixed by adding the GND wire. As always, when something has worked for months and suddently stops working you should wonder about the latest changes.

The latest change is the addition of a relay to power down the charger when it isn't needed. It has a little boost converter to be able to close the 12V relay from a 5V supply. Now I hooked the relay up to a separate power supply et voila, everything is working. So either the extra current draw (200 mA) from the beaglebones 5V rail or the interference caused by the boost converter is disturbing communication. Will add some more capacity and see if that helps
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Post Reply