Page 1 of 2

Zero Grid Export and Reading energy meter with ESP

Posted: Thu Mar 24, 2022 8:19 pm
by johu
With rising electricity prices I am currently revisiting control of my small solar system (about 800 Wp).
The plan is to rig up some left over batteries with a cheap Chinese battery inverter to collect energy during the day and use it at night. Or more precisely prevent feeding energy to the grid (because I'm not payed for export) and to some extend, prevent using energy from the grid.

To start out we first need to know the energy flow from/to the grid. I had tried openenergymeter in the past but it was very inaccurate, I found it hard to use and also it required a Raspberry PI to run 24/7 which defeats the purpose: conserving energy.

So I researched my existing utility energy meter a bit to find out it actually outputs a serial data stream via infrared (apparently the format is called SML).

I started out with the physical layer. I mounted one of these reflective encoder heads (left over from Polo project) which essentially consists of an IR diode and a photo transistor. The diode is NOT needed! Nothing needs to be requested, the meter just dumps out the info every second.
IMG_20220324_124247.jpg
Ok, the photo transistor needs 15k pull up (47k makes it too slow, 10k gives too little amplitude). Since the waveform looks rather distorted we feed it through a logic gate, I just use two inverters 74ACT04.

Then it goes to the ESP via a 470 Ohm resistor to RX. RX is connected to 3V3 via a diode to level shift the 5V to a 3V3 signal.
IMG_20220324_124121.jpg
Serial data:

Code: Select all

/EBZ5DD3BZ06ETA_107

1-0:0.0.0*255(1EBZ0100200608)
1-0:96.1.0*255(1EBZ0100200608)
1-0:1.8.0*255(007306.09759253*kWh)
1-0:16.7.0*255(000179.47*W)
1-0:36.7.0*255(000044.82*W)
1-0:56.7.0*255(000134.65*W)
1-0:76.7.0*255(000000.00*W)
1-0:32.7.0*255(232.6*V)
1-0:52.7.0*255(233.1*V)
1-0:72.7.0*255(232.9*V)
1-0:96.5.0*255(001C0104)
0-0:96.8.0*255(064F18F0)
!
Starts with '/' ends with '!' - handy.

Next we need to write some Arduino C++. Little surprise I took the inverter sketch and modified it. Serial interface runs at 9600 7E1 (first time I see parity used).

Soon had some output
Bildschirmfoto vom 2022-03-24 07-40-46.png
Bildschirmfoto vom 2022-03-24 07-41-20.png
Now went off on a tangent and decided to record that data to a mysql database. That allows more systematic analysis.

So now have a 1-second updating web page
Bildschirmfoto vom 2022-03-24 14-28-52.png
And a daily overview
today.png
Everything below 0 means we're exporting to the grid. Some number crunching later I found I "lost" 2 kWh today.

Decided to not run 10m of wire through the public stair case, below our apartment door and onto my desk. So put everything into a DIN rail enclosure next to a DIN rail 5V supply. Pretty huh?
IMG_20220324_190744.jpg
Code: https://github.com/jsphuebner/esp-egycounter

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Thu Mar 24, 2022 8:23 pm
by Bigpie
I feel your pain, I'm not paid for export. My new solar exported almost 8kwh today. Does your smart meter have the P1 port?
https://github.com/zuidwijk/dsmr/

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Thu Mar 24, 2022 8:45 pm
by johu
That looks pretty.
It doesn't have any (accessible) electrical port, just IR ones

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Thu Mar 24, 2022 8:54 pm
by Bigpie
It's part of home assistant. I'm working on an ESP32 to take the export current and make my EVSE max amps follow it. I might have to look into your cheap Chinese inverter solution and a chademo plug to take some power from the car and reduce the blue parts of the graph.
I've got a CT clamp around the house load and the PV generation and taking one from the other to get import/export values, not the most accurate but it'll do for now.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Sun Mar 27, 2022 8:57 pm
by paaa
Im on this journey too , im using a shelly energy meter with assistant and bought an smart evse to dump the excess energy into the car. Only caveat is that min 100% solar mode is 6a but you can blend say 50% with grid by starting charger at 3a of export as its the same price as night rate for me. One crazy option is to use a transformer to reduce voltage to 100v for better resolution as car charger sports that voltage. My next development is to use an ac coupled storage inverter(sunny boy sb storage) with a ev battery then integrate it by chademo v2g mode. Currently just got a can capture of a compatible battery used with inverter to test.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Mon Mar 28, 2022 9:46 am
by johu
Yes I'm planning to couple my fathers EVSE to this. He has 8kW of solar and most goes into the grid. He manually plugs in the car when he thinks it's appropriate.
Cool thing with using the meter is there's no extra cost (except the ESP module) and the thing is crazy accurate. I can even see when I start playing a video on my laptop.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Mon Mar 28, 2022 2:09 pm
by Bigpie
paaa wrote: Sun Mar 27, 2022 8:57 pm My next development is to use an ac coupled storage inverter(sunny boy sb storage) with a ev battery then integrate it by chademo v2g mode.
Those Sunny Boy Storage inverters look pretty good, I want one :D Do you need other sunny boy equipment (home manager/meter)?

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Mon Mar 28, 2022 4:27 pm
by paaa
Bigpie wrote: Mon Mar 28, 2022 2:09 pm
paaa wrote: Sun Mar 27, 2022 8:57 pm My next development is to use an ac coupled storage inverter(sunny boy sb storage) with a ev battery then integrate it by chademo v2g mode.
Those Sunny Boy Storage inverters look pretty good, I want one :D Do you need other sunny boy equipment (home manager/meter)?
You need a sunspec compatible meter , I bought an elgris smart lan meter and it seems to par just fine. Its not cheap at about 180e but much cheaper than any smaller ones. Later on im hoping to just use an open source design of an ease meter that supports rs485 already to replace it. but that's a long way away.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Mon Mar 28, 2022 8:36 pm
by Bigpie
What's the purpose of the meter? I'm assuming it's instead of CT clamps to stop export from battery and to charge with excess?

My other cheap idea was to hook a HV battery up to the solar input on a normal PV inverter, set to 0 export using the CT clamp, and hookup an outlander charger to the pack for charging.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Tue Mar 29, 2022 11:32 am
by johu
Successfully putting excess solar power into some old LFP batteries. I'm using a BeagleBone (for now) and a Manson switch mode supply with USB (for now) and am running the following script.
It fluctuates a bit (+- 10W) but reacts pretty quickly. Currently wife is cooking and the hot plates switch on and off in some 30s interval. In between it ramps up the battery and then ramps it down again within a second.

Code: Select all

#!/usr/bin/python

import serial, time, requests

ser = serial.Serial('/dev/ttyUSB0', 9600, timeout=0.2)
lastSolarPower = 0

while True:
	try:
		ser.flushInput()
		ser.write("GETD\r")
		output = ser.read_until('\r')
		print (output)
		r = requests.get("http://192.168.178.54/cmd?cmd=get%20ptotal")
		ptotal = float(r.text)
		if len(output) == 10:
			voltage = float(output[:4]) / 100
			current = float(output[4:-2]) / 100
			power = voltage * current
			solarpower = power - ptotal
			print ("battery power: " + str(power) + "W grid power: " + str(ptotal) + "W solar excess: " + str(solarpower) + "W")
			lastSolarPower = (solarpower + lastSolarPower) / 2  #IIR Filter
			newcurrent = max(0,lastSolarPower / voltage)
			ser.flushInput()
			ser.write("CURR%.3d\r" % (newcurrent * 10))
			print("Setting new current setpoint to " + str(newcurrent))
			print (ser.read_until('\r'))
	except requests.exceptions.ConnectionError:
		print ("Connection error, setting current to 0")
		ser.write("CURR000\r")
		continue

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Sun Apr 10, 2022 6:21 am
by johu

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Tue Apr 12, 2022 8:03 am
by tom91
Do you have a datasheet or link for the inverter you used?

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Thu Apr 14, 2022 8:37 am
by johu
It's a Soyosource GTN1200 that I bought on aliexpress. 160€ delivered

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Mon Jul 25, 2022 8:18 am
by jnsaff
I would also be mindful that some meters count exported energy as energy used and it will be added to your bill instead of just lost.

We have had these problems here in Estonia.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Thu Aug 11, 2022 2:27 pm
by whereswally606
Got the gtn1200 24v battery type. Struggling to get their proprietary wifi dongle which looks to have an esp8266 inside to connect to my home network. Needs a 2.4ghz router and unfortunately my main router is 5 ghz and it wont pair to my BTHomeSpot powerrline ethernet 2.4ghz one.

Anyway ideally i dont want a wifi connection so i may invest in an esp with wired ethernet as i have a wired router near the battery bank.

Would be nice to have it working using their stock equipment (to test with) but their current clamp is too small and underrated for the main out of the house.

Anyway any more info on how to run the gtn 1200 using esp and how its hooked up physically will be very helpful.
Ive cheated on the current measurement and bought a shelly em and run home assistant.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Thu Aug 11, 2022 3:40 pm
by johu
I've done some reverse engineering of the protocol between the wifi module and the inverter: https://forum.drbacke.de/viewtopic.php?p=22798#p22798
It is in German though

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Thu Aug 11, 2022 8:56 pm
by whereswally606
Thanks johannes. Finally got the shoddy app working. Turns out that if you have a username with numbers in it the app will just throw them away and not warn you that its done it and then pretend that your username exists already but tell you your password is wrong. Anyway now i can log in to the app i have managed to add the inverter to the wifi and can program it to do constant power with safety shutoffs. This should be good enough for now to remove some 500w of base loads from the house after the sunset.

I will look into a wired esp8266 and your reversing of the protocol. Having it react fast to changes in cloud cover will aid cheaper car charging in the day too.

The other issue i had was 2.4 ghz wifi. I desynchrionized my settings for the 5 and 2.4ghz and renamed the 2.4ghz ssid. Unfortunately thus broke lots of smart things like the shelly em, all 5 tuya wifu switches and in turn lost a few settings in home assistant. That did get me a working inverter though but a painful afternoon walking round manually resetting things.

Also the soyosource app is shockingly bad and the popups are in chinese even when the lang is set to english. Thankfully i live 2 doors away from a friendly chinese family. Matched set is how the device paired is translated.

Will peruse your reversing efforts tomorrow when inhave a clearer head.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Fri Aug 12, 2022 2:54 pm
by johu
I was never able to install the app but got all settings done with their "cloud" website. Then removed the wifi stick.

Power setpoint can not be done via the wifi TTL port but only via the RS485 port next to it.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Fri Aug 12, 2022 3:29 pm
by johu
Relocated the whole system to the basement where it's cooler, more fireproof and most of all not as disturbing as when sitting in my desk. Both charger and inverter can be pretty noisy when operating at power.

Next step is replacing the lab power supply by an Eltek flatpack. I hope it goes as low as 44V.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Mon Aug 15, 2022 4:35 pm
by johu
And today I did the Eltek replacement. Lab supply works perfectly well but I thought it is a bit over-qualified.

The Eltek uses CAN, the BeagleBone has native CAN so one less device connected over the notoriously unreliable USB.

There is one challenge though: below 48V the Eltek no longer accept the current limiting command. But has a 43.5V - 57.5V working range. So now I just do a PI loop that changes output voltage to arrive at the required output power. The measurements are updated fairly often, like 5 times per second so it seems to work reasonably well. Also the 10 mV resolution is very helpful. Mind you, the controller gains must be super low because a couple of mV make a whole lot of difference in battery current.

The driver will show up on github in a minute

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Fri Oct 21, 2022 9:20 am
by rstevens81
Silly question I see you have used 2 lifepo4 cells to boost the voltage was curious what you strategy was to not over charge or under discharge them?
I assume thair (ah) capacity is substantially more than the lion pack and you basically discharged both the lion and the lifepo4 cells to half charged then hooked it together, that way regardless of the the soc of the lion, the lifepo4 will never be empty or full and sit in the middle of its soc?

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Fri Oct 21, 2022 2:21 pm
by johu
rstevens81 wrote: Fri Oct 21, 2022 9:20 am Silly question I see you have used 2 lifepo4 cells to boost the voltage was curious what you strategy was to not over charge or under discharge them?
I assume thair (ah) capacity is substantially more than the lion pack and you basically discharged both the lion and the lifepo4 cells to half charged then hooked it together, that way regardless of the the soc of the lion, the lifepo4 will never be empty or full and sit in the middle of its soc?
Yes exactly. The LiIon is 40Ah and so are the LFP. Since it's 2P they well exceed the other pack despite being rather old. I just bumped them up until they were far enough from discharged and discovered they are also far enough from charged when the LiIon is full.

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Wed Feb 01, 2023 6:58 pm
by johu
Next upgrade to this project will be tibber support or variable rate support in general.
Tibber forwards the spot market electricity prices to the consumer (I think hourly resolution). So you can move your electricity demand on low rate hours. There is no fixed pattern, in summer noon hours will be cheap because of the vast PV capacity, in winter night hours should be cheap because general demand is low and output from wind farms is higher (on average) than in summer.

Plan is to automate car charging and also to charge the ESS battery with cheap electricity and discharge it when rates are high (while still not exporting to the grid because German rules and regulations)

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Thu Mar 16, 2023 9:24 am
by johu
Added the Switched ADC BMS yesterday. It works a treat and now I can keep an eye on the cell voltages. That said, without any balancing after 1 year of operation the delta is 5mV.

I found when two python scripts receive CAN messages you have to configure filters. Otherwise they might "steal" each others messages.
image.png

Re: Zero Grid Export and Reading energy meter with ESP

Posted: Mon Mar 27, 2023 8:19 am
by johu
I found the Flatpack consumes a hefty 200 mA from the battery once its AC is disconnected. It also keeps sending CAN messages with no AC, so apparently the battery voltage backfeeds into the control circuit. I fitted a diode to its output.
Was also having CAN bus trouble (bus off condition), maybe related to switching it off. Connecting the can bus gnd fixed the issues.

EDIT: no the CAN bus trouble wasn't fixed by adding the GND wire. As always, when something has worked for months and suddently stops working you should wonder about the latest changes.

The latest change is the addition of a relay to power down the charger when it isn't needed. It has a little boost converter to be able to close the 12V relay from a 5V supply. Now I hooked the relay up to a separate power supply et voila, everything is working. So either the extra current draw (200 mA) from the beaglebones 5V rail or the interference caused by the boost converter is disturbing communication. Will add some more capacity and see if that helps