ESP32 Based Web Interface & Data Logger

Discussion about various user interfaces such as web interface, displays and apps
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

Got a large parcel from JLCPCB today, I think I ordered them just a week ago. They get faster every time!

So amongst them is the new Leaf adapter board with its integrated ESP32. So grabbed Petes ESP32 port (the other one didn't work for me at all). First thing I found it is using Serial2 for inverter comms. Bummer, I wired up Serial1.

So changed the code accordingly, removed all debug output (hmm, could have just moved it to serial2) and I'm now getting replies from the inverter but it's missing large chunks of data, especially noticeable on the json command.

Have you run into this problem? How to cure it? I found the question on the web with no success reported and also you state that the implementation doesn't use DMA.

Needless to say the little brother 8266 handled serial data just fine at 1 Mbit/s
Attachments
1667404486478.jpg
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

My code uses the espressif api directly for the inverter comms and only uses the arduino library for debug. The arduino comms library is shockingly slow. If you swap the ports you need to keep using the espressif api calls on the inverter port.

Esit - it should be possible to just swap the port definitions when you open them (fairly sure there is a #define for the inverter port) without needing any major code changes.
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

Ok, I see. So went back to your version and only changed

Code: Select all

#define DBG_OUTPUT_PORT Serial2
#define INVERTER_PORT UART_NUM_0
#define INVERTER_RX 3
#define INVERTER_TX 1
According to https://www.espressif.com/sites/default ... eet_en.pdf page 9

Now I'm one step further. I listen to the inverter return data upon reset and I see

Code: Select all

fastuart
OK
Baud rate now 921600
Then I change baud rate to 921600 on my terminal to find the ESP32 still sends its commands at 115200. I wonder if it power cycles for some reason

EDIT found that one. uart_readUntill() and uart_readStartsWith() was using UART_NUM_2 hard coded. I see the json reply from the inverter now but it doesn't end up in my browser
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

Seem to remember using a different command to change the espressif api baud rate. Its possible that I forgot to use the #define in it?

Edit - its on line 505 - looks ok so not sure what the problem is, it worked fine on uart2
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

johu wrote: Wed Nov 02, 2022 4:36 pm EDIT found that one. uart_readUntill() and uart_readStartsWith() was using UART_NUM_2 hard coded. I see the json reply from the inverter now but it doesn't end up in my browser
Oops, must have missed those, do a search on the uart functions and see if any others are still hard coded
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

All good, it was another hard coded UART_NUM_2 when reading the data. Now it works :)

On to firmware update. At least that worked when I had it running via the Serial class and it no longer works with the API calls. By not working I mean it sends corrupted or incomplete firmware.
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

Not that surprised, there were a lot of untested edits to those functions to move to the api calls :(
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

Yep, found it:

Code: Select all

-    snprintf(uartMessBuff, UART_MESSBUF_SIZE, "%d", pages);
-    uart_write_bytes(INVERTER_PORT, uartMessBuff, strnlen(uartMessBuff, UART_MESSBUF_SIZE));
+    uart_write_bytes(INVERTER_PORT, &pages, 1);
Don't format page size to a string, just send it as a byte.

I'm fairly impressed with the ESP32 and your port. It seems to react a lot faster, especially establishing the connection is quicker.

Do you want me to pull request?

EDIT: this is my 4000th post, first time I notice a round number ;)
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

johu wrote: Wed Nov 02, 2022 4:57 pm I'm fairly impressed with the ESP32 and your port. It seems to react a lot faster, especially establishing the connection is quicker.

Do you want me to pull request?
Never been too keen on the esp8266 but the esp32 does seem a much nicer device to work with. I assume its down to not having to share a slower processor with the wifi stack?

Not too sure sure what that means, is that pulling your changes to my repo or the other way round? Either way can do. Think Bigpie has made a few more changes to the webpage that aren't on github yet too.
johu wrote: Wed Nov 02, 2022 4:57 pm EDIT: this is my 4000th post, first time I notice a round number ;)
:)
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

That's what I thought but now I found I can't fork your esp32 repo because you forked it from my esp8266 repo and I can't fork back ;)
So no pull requests for now.
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

Oh one thing I found in this one https://github.com/Bedz01/esp32-web-interface-port

Code: Select all

void task_handle_client(void *){
	for(;;){
		server.handleClient();
		vTaskDelay(1);
	}
}

xTaskCreate(task_handle_client,"server_handler",10000,NULL,1,NULL);
I think only then does the web server run on its own core?
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

My understanding (disnt spend much time on it though so could well be wrong) is that as standard all user code (including the web server) runs on one core and the wifi stack on the other. You can choose to run a task on the either processor if you want (and it looks like that is what is being done there) but I think you then have to start putting mutexes around any shared resources. Since it seemed to run quite happily with everything on the one processor I left it that way.
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

Ok understand. Then lets not touch that
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

Just a thought, I did try using that port to start with but couldn't get the web server to work at all, wonder if that was why?
User avatar
mjc506
Posts: 343
Joined: Wed Sep 09, 2020 9:36 pm
Location: Wales, United Kingdom
Has thanked: 30 times
Been thanked: 28 times

Re: ESP32 Based Web Interface & Data Logger

Post by mjc506 »

Careful not to trample on the wifi stack, safest just to leave that core alone if you're relying on OTA :-)

Can do multiple xTaskCreate's on one core though, if that's any help.
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

You mean while you still went via the Serial class?
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

Hadn't made any changes at all, just couldn't get the web page to work.

Turned out that porting the 8266 code was pretty simple so went that route instead.
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

Bigpie wrote: Mon Oct 24, 2022 2:38 pm Just had 70kw :O hope the logs are working.
Is 70kW still the limit for you? If so any idea what is limiting it?

Your motor parameters seem virtually identical to the Outlander rear and when I simulate that I get almost a 100kW
OutlanderRear.png
This is with a 350V battery voltage which I think is near enough to yours?

Edit - oops, meant to post this in the simulator thread :oops:
User avatar
Bigpie
Posts: 1585
Joined: Wed Apr 10, 2019 8:11 pm
Location: South Yorkshire, UK
Has thanked: 74 times
Been thanked: 299 times

Re: ESP32 Based Web Interface & Data Logger

Post by Bigpie »

I've not had chance to mess around again yet, I'd stolen the esp32 for something else at the moment but one arrived in the post today so can start logging again.
I know the front is a lower torque to the rear I think 60kw 120nm vs 70kw 200nm based on Mitsubishi information.
I did briefly try upping the throtcur value to 4.7 (currently 4.5 I think) and that seemed to work on my show test drive but didn't have my phone on showing my shunt data and the boot was open with a desk hanging out the back

My battery is 96s so around 355 nominal
VW Beetle 2003
Outlander front generator
Prius Gen 3 inverter (EVBMW logic board)
Outlander charger
3x Golf GTE batteries
Chademo Charging
Outlander water heater
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

Bigpie wrote: Wed Nov 09, 2022 5:18 pm I've not had chance to mess around again yet, I'd stolen the esp32 for something else at the moment but one arrived in the post today so can start logging again.
I know the front is a lower torque to the rear I think 60kw 120nm vs 70kw 200nm based on Mitsubishi information.
That could explain the difference. It would suggest that two motors have significantly different flux linkage values despite the inductances matching? Next time you're logging try to include a coast so that we can work the flux linkage value out.
Bigpie wrote: Wed Nov 09, 2022 5:18 pm I did briefly try upping the throtcur value to 4.7 (currently 4.5 I think) and that seemed to work on my show test drive but didn't have my phone on showing my shunt data and the boot was open with a desk hanging out the back
:lol: Not the ideal setup for a power run!
User avatar
Bigpie
Posts: 1585
Joined: Wed Apr 10, 2019 8:11 pm
Location: South Yorkshire, UK
Has thanked: 74 times
Been thanked: 299 times

Re: ESP32 Based Web Interface & Data Logger

Post by Bigpie »

and issues... With or without the extra pullup resistors.

Code: Select all

E (20534) sdmmc_sd: sdmmc_init_sd_scr: send_scr (1) returned 0x109
E (20535) vfs_fat_sdmmc: sdmmc_card_init failed (0x109).
[ 20538][E][SD_MMC.cpp:138] begin(): Failed to initialize the card (0x109). Make sure SD card lines have pull-up resistors in place.
Couldn't start SD_MMC
I had to do the flash efuse thing with this ESP32

I'll try different cards and boards tomorrow
VW Beetle 2003
Outlander front generator
Prius Gen 3 inverter (EVBMW logic board)
Outlander charger
3x Golf GTE batteries
Chademo Charging
Outlander water heater
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

Currently in the process of generating the json from BINLOG_LIST. Now I found a somewhat redundant format:

Code: Select all

{ "01": { "name":"il1",...},
  "02": { "name":"il2",...} 
}
Would you mind if I changed that to

Code: Select all

{ "il1": { ...},
  "il2": { ...} 
}
Easier to generate and kindof like the other "json"
Also for easier generation I'd like to change "signed" and "unsigned" to "int32_t" and "uint32_t", ok?
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

Losing the 01, 02, etc will break the decoder. It started off without them but unfortunately the qt json handling functions lose the order of all the json fields, and just order them alphabetically, so it needed the the extra field to force it to retain the order. The alternative would be to rewrite the decoder in something else that retains the field order.

Prefer the signed flag as it doesn't imply an incorrect bit width but if you can't make that work then the _t types would be ok. The decoder would need changing to match.
User avatar
johu
Site Admin
Posts: 5684
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 153 times
Been thanked: 960 times
Contact:

Re: ESP32 Based Web Interface & Data Logger

Post by johu »

Ah yes, that makes sense.

How about that one:

Code: Select all

[
{ "name":"il1",...},
{ "name":"il2",...}
]
That should definitely be sorted
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: ESP32 Based Web Interface & Data Logger

Post by Pete9008 »

I tried various formats, which I think included that, but couldn't find a way to make the qt libraries retain the order without the extra number. Found a few forums suggesting various class overloads or source code changes but no real solutions.

If you want to go that route it will need a decoder writing in something else (probably not a bad thing, the qt version was just a quick bodge to test if everything else worked and could do with tidying up and command line arguments adding at the very least.
Post Reply