Gen 3 inverter converter control software

Topics concerning the Toyota and Lexus inverter drop in boards
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

Eflyers wrote: Sat Jun 20, 2020 11:04 pm I hope none of this is related to the ongoing Gen 3 inverter failure and recall problems showing-up in the US: https://www.latimes.com/local/californi ... story.html
I think the recall must be due to the boost converter. We might have to expect the cheapest inverters to have a broken one.
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

Jack Bauer wrote: Fri Jun 19, 2020 1:36 pm Wow:) I have a few boards left so if its dead just let me know.
A spare board wouldn't hurt as I try to make what I can out of these rather quickly decaying inverters of mine. :-) At this rate I'm running out of replacement components too.
User avatar
NiHaoMike
Posts: 64
Joined: Mon Dec 02, 2019 4:11 am
Location: Austin, TX
Been thanked: 1 time
Contact:

Re: Gen 3 inverter converter control software

Post by NiHaoMike »

celeron55 wrote: Sat Jun 20, 2020 8:43 pm What on earth just happened?! I was merrily precharging the AC side and then bucking some reasonable output voltages to the capacitor with nothing connected to it, which I have done before with no issues as my control code is careful enough to do that. Now in my log the last output voltage was 278 volts. After that I heard a pop and each one of the four circuit breakers along the way from the house to the MG2 terminal opened, including one phase of the main 25A breaker of the house.

Both of the buck/boost converter IGBTs are dead shorts now and by quick glance there was no other damage.

I was doing buck converting with my previous total hack of a control solution for many hours at powers from 0 to 14kW in the car with no major issues, with this exact same inverter.

I guess the IGBT voltages are rated so much higher than the capacitor that if the voltage measurement was giving somewhat erroneus values, and there was slight over voltage over the total time of the test, the capacitor should have gone first. And the Prius gen3 buck output side is known to be rated for 400V all around so there's quite some headroom.

I was not driving the low side IGBT at all, because if I was, as I've tested many times now, the inverter would have stopped allowing me to drive the high side IGBT and I wouldn't have had any output voltage.

So... what caused this?

Was this inverter after all faulty to begin with and the low side IGBT just happened to give up during that specific test? That seems very unlikely after all the heavy use I previously gave to this buck converter, but I might have triggered development of the kaboom when I was trying to get the low side IGBT to do any boosting at all on the bench. So some kind of DESAT event might have been happening due to the low side IGBT being faulty, and at the same time the IGBT was getting worse.

Does that sound like a good theory?

And does anyone happen to have more inverters lying around or do I finally have to do the Africa thing and connect the boost inductor to one of the "spare" MG half bridges?
Do you have a way to check if it's actually the IGBTs themselves that are shorted or rather the parallel diodes? The diodes are not protected by the driver circuit and it's all up to the control circuit to make sure they're operating within their ratings.

Also, did you have the low side drive signal tied to the inactive state (which I recall is 5V on the gen 3) or left it floating? EMI can cause unwanted switching on floating inputs, and (much less intuitively) large loops can also pick up stray signals.

It would also be helpful to know what's the "correct" frequency to run the converter at. Too high causes excessive switching and reverse recovery losses, too low causes excessive ripple current.
My first solar power system helped Naomi Wu, now I want to do even more with DIY solar.
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

Some modding:
20200621_143249.jpg
20200621_144055.jpg
20200621_152505.jpg
The end result looks a bit suspicious to the experienced eye:
20200621_165737.jpg
I cut off the aluminium where the inductor goes to the half bridge midpoint and re-wired it from there to the MG1 phases A and B. Then bent off the boost high side from the MG rail and wired those MG1 phases to be driven by the atmega328's outputs. (Seen also in the photos is my coupling of the current sensor inputs together so that I can read the MG2 sensors using the atmega328, as my 3 phase input cable doesn't reach MG1.)

I did some bucking using this and it appears to work. I've yet to put more power than a kW or two through this.

But it seems like doing this is perfectly fine if you don't have the big bucks to spend on an actually working inverter. If you have a motor on MG2 you just need to add an external rectifier from AC to the MG rail, which is not very expensive but needs to have its own heatsink if you intend to put 10 or 20 kilowatts through it.

I blew my 100A 450VDC fuse I have on the charger cables during the Yaris inverter mess-up and don't have a spare so I need to improvise a bit or wait for a spare to arrive. (Yes, looking back the blown fuse was why I saw 0V in the inactive charger output, so the Yaris inverter is slightly less messed up than I thought.)
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

NiHaoMike wrote: Sun Jun 21, 2020 8:19 pm Do you have a way to check if it's actually the IGBTs themselves that are shorted or rather the parallel diodes? The diodes are not protected by the driver circuit and it's all up to the control circuit to make sure they're operating within their ratings.
I don't know how to check that without destroying the otherwise perfectly working inverter, which I don't intend to destroy.
NiHaoMike wrote: Sun Jun 21, 2020 8:19 pm Also, did you have the low side drive signal tied to the inactive state (which I recall is 5V on the gen 3) or left it floating? EMI can cause unwanted switching on floating inputs, and (much less intuitively) large loops can also pick up stray signals.
Damien's board always leaves them floating when they aren't active. The output circuit is this: MCU -> 1k resistor -> MMBT3904 npn transistor -> 100 ohm resistor -> IGBT driver board.
NiHaoMike wrote: Sun Jun 21, 2020 8:19 pm It would also be helpful to know what's the "correct" frequency to run the converter at. Too high causes excessive switching and reverse recovery losses, too low causes excessive ripple current.
Sure would be. The converter is really loud at audible frequencies though, even at low powers like 1kW. It's embarrassingly loud if you want to use it on a public charge point and almost ear-breakingly loud if you use it in an indoor space at high power. I wonder if Toyota's sound insulation is heavy enough to make it unhearable from the driver's seat when driving an actual gen3 Prius (at high speed when it's actually in use), or if Toyota uses 20kHz to make it silent.
User avatar
NiHaoMike
Posts: 64
Joined: Mon Dec 02, 2019 4:11 am
Location: Austin, TX
Been thanked: 1 time
Contact:

Re: Gen 3 inverter converter control software

Post by NiHaoMike »

celeron55 wrote: Sun Jun 21, 2020 10:46 pm I don't know how to check that without destroying the otherwise perfectly working inverter, which I don't intend to destroy.
Shine a flashlight at an angle into the side with the top cover removed. You can get a partial view of the chips.
Damien's board always leaves them floating when they aren't active. The output circuit is this: MCU -> 1k resistor -> MMBT3904 npn transistor -> 100 ohm resistor -> IGBT driver board.
Perhaps pulldowns on the input side of the buffer transistors would be a good idea? The stock control board uses 10k pulldowns. (I'm pretty sure the ASICs are designed to always have the signals in a defined state, so the pulldowns are just extra protection.) Also, the stock control board appears to use signal MOSFETs (or something even more exotic like current loop line drivers?) rather than bipolar transistors. Thus the threshold voltage is likely higher than the 0.7V of a bipolar transistor. Perhaps put the pulldowns right at base to emitter to raise the voltage needed at the input to switch it on? Need to do some testing to find some good values, but I would say 1k would probably be a good start. (With the high voltage off and the microcontrollers held in reset, see what voltage on the pin transitions the output to the on state. Ideally, it should be about half the logic supply voltage.)

I'm using 7400 series chips between the FPGA and power board in my design. The (DIY) translator and control power board (mounted in place of the original control board) connects to the FPGA board (outside the inverter) via a shielded cable.
My first solar power system helped Naomi Wu, now I want to do even more with DIY solar.
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

Just got AC side boost precharge working. Now my AC contactor box looks like this, nice and minimal (and messy - you aren't allowed to be this messy when you do yours :mrgreen:):
20200622_152209.jpg
That's a 39A @ 660VAC contactor, an economizer based on a huge 47000uF capacitor and some resistors, some relays wrapped in electrical tape, a 32A 3-phase RCD and not really much else.
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

Ok the code hasn't exploded my inverter anymore and on the other hand it has charged my battery at maybe 15kW or so on a public charge point (didn't have cooling and was in a hurry so had to stop there) so I think I can release it.

I did try to clean it up a lot and I haven't tested the changes, so good luck.

I'll maybe get to drawing some wiring diagrams maybe, but if you take a good read at the comments at the beginning of the .ino file and then all the #defines afterwards, you should get an idea how to put it into use.

This, of course, is an excellent piece of software and electronics to absolutely blow up your battery and set everything near it into flames, so be careful. If not for yourself, then at least for the credibility of the EV conversion community.

https://github.com/celeron55/prius3charger_buck
doobedoobedo
Posts: 260
Joined: Sat Jan 12, 2019 12:39 am
Location: UK

Re: Gen 3 inverter converter control software

Post by doobedoobedo »

I fear your avoid_explosions function may get optimised away by the compiler.
User avatar
Jack Bauer
Posts: 3563
Joined: Wed Dec 12, 2018 5:24 pm
Location: Ireland
Has thanked: 1 time
Been thanked: 87 times
Contact:

Re: Gen 3 inverter converter control software

Post by Jack Bauer »

Much appreciated. Will be testing soon.
I'm going to need a hacksaw
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

Adding to the broken buck/boost converter workaround:

The ultimate budget version would be to use only the AC phases L1+L2 or L1+N, connect them to MG1 B and C phases and rewire the inductor from the broken converter bridge to the MG1 A phase. Then you have MG2 free for a motor, and can charge using whatever power you're comfortable with putting through the single MG1 phase.

If you get a broken inverter for almost free and pick whatever motor you can get the cheapest, your conversion's most expensive component after the battery and motor will end up being the replacement logic board. Excluding batteries, a 500€ AC motor conversion, no luck, haggling or part shortages involved, starts looking like an everyday thing.

The power factor is a no-go for a commercial conversion though. For personal tinkering, I don't think anyone cares. Not being nice to the grid is a minus, but not releasing poisonous gases to the atmosphere is a plus.
MattsAwesomeStuff
Posts: 898
Joined: Fri Apr 26, 2019 5:40 pm
Has thanked: 291 times
Been thanked: 177 times

Re: Gen 3 inverter converter control software

Post by MattsAwesomeStuff »

I understand what power factor correction is, and why it's important... but I don't really know what the typical solution to it is.

I recall once upon a time you'd just add a bunch of capacitance to the circuit to balance out inductive loads. Is that still a thing people do, or, what's the modern way of correcting PF?
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

Just to clarify for all the newbies out there:

The problem with all power supplies taking in AC is that when you rectify 1-phase AC into a capacitor, there's zero current when the instantaneous AC voltage is below the capacitor voltage, and a lot of current when the instantaneous AC voltage is at the capacitor voltage. In the ideal situation (and which would be perfect power factor) you always want your device to take in current like a resistive load would, i.e. if you take 10A at 100V, then when voltage is 50V you take 5A.

Notably 3 phase AC largely doesn't have this problem because the voltage between the highest and lowest phase doesn't fluctuate that much, and certainly never goes to 0V. Using the prius inverter in the 3 phase input configuration isn't too bad for the grid other than possibly some switching noise entering the grid. Maybe someone with some experience can figure out what kind of filtering it should ideally have.
MattsAwesomeStuff wrote: Tue Jun 23, 2020 9:33 pm I understand what power factor correction is, and why it's important... but I don't really know what the typical solution to it is.

I recall once upon a time you'd just add a bunch of capacitance to the circuit to balance out inductive loads. Is that still a thing people do, or, what's the modern way of correcting PF?
You can't offset the current with capacitance or inductance because it's not offset that much - it's mainly just way too spiky.

Commercial high power devices don't just have a rectifier, but they also have a boost circuit that synchronizes with the AC waveform and boosts the appropriate current into the capacitor when the instantaneous AC voltage is below the capacitor voltage. You can often see this stage quite clearly, as at initial glance it looks like it should be just a rectifier, but then you notice there are large inductors and transistors alongside a control chip in there before the voltage is fed to the actual switch mode converter.

EDIT: I've attached a photo of the Nissan Leaf 1 phase AC charger. AC comes in via the small copper bars to the left edge of the PCB, then there's a suspiciously long row of through hole transistors + 2 diodes mounted on the heatsink below the PCB, then there's the capacitor bank. The weird wires that attach to the PCB between the input copper bars and the transistors go to the plastic-encased inductors at the top left. That's the boost converter at the input.

To the right of the capacitors below the PCB that you can't really see are the actual transistors of the charger's flyback converter. The output of the transistors leaves the PCB via wires to a transformer and from there to another PCB to be rectified again.
Attachments
P1150317s.JPG
User avatar
NiHaoMike
Posts: 64
Joined: Mon Dec 02, 2019 4:11 am
Location: Austin, TX
Been thanked: 1 time
Contact:

Re: Gen 3 inverter converter control software

Post by NiHaoMike »

celeron55 wrote: Tue Jun 23, 2020 7:59 pm The power factor is a no-go for a commercial conversion though. For personal tinkering, I don't think anyone cares. Not being nice to the grid is a minus, but not releasing poisonous gases to the atmosphere is a plus.
Can probably still get a reasonable power factor considering the bus capacitance is quite low for the power levels being dealt with. Just needs better control over the buck/boost converter.

Another trick, if the battery voltage is above the 340V or so of rectified 240V, is to use an external bridge rectifier to the buck/boost and then just use the buck/boost as a more conventional PFC. In the US, the trick can even be done with batteries as low as 170V by using a full wave rectifier from 240V, but that would require the neutral and is therefore incompatible with J1772.
My first solar power system helped Naomi Wu, now I want to do even more with DIY solar.
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

Yeah there's room for extra trickery in controlling the converter in single phase buck application, and a future boost version for charging higher voltage packs from single phase will have better pf by default.

I'm a european 3 phase guy though. 22kW type 2 is where you get the most power for the cheapest and most ubiquitously around here.
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

I finally found and fixed a PWM control bug that was messing with the boost precharge and causing large enough voltages (650-700V) to make the inverter make its characteristic "pop" sound. Now it also precharges really fast as the code doesn't need to try to work around the PWM glitch.
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

I just realized a non-isolated charger like this is actually a safety feature in a conversion: Every time you connect to an EVSE to charge, you get checked for ground leaks without having any ground leak detection circuitry in the vehicle.
User avatar
rstevens81
Posts: 352
Joined: Sun Dec 22, 2019 10:36 am
Location: Bristol, UK
Has thanked: 23 times
Been thanked: 92 times

Re: Gen 3 inverter converter control software

Post by rstevens81 »

Now we wait for winter when the wet stuff arrives to test the rcd's in the Evse's if not we could Gert some arcing around the tyres :)
btw thanks for your effort on this it is most appreciated by everyone here, even if some poor inverters had to die in the process.
Rule 1 of EV Club is don't buy a rust bucket....
Which rule does everyone forget 🤪
MattsAwesomeStuff
Posts: 898
Joined: Fri Apr 26, 2019 5:40 pm
Has thanked: 291 times
Been thanked: 177 times

Re: Gen 3 inverter converter control software

Post by MattsAwesomeStuff »

celeron55 wrote: Tue Jun 23, 2020 10:03 pmYou can't offset the current with capacitance or inductance because it's not offset that much - it's mainly just way too spiky.
Ohh, I had no idea that spiking was an issue. I thought power factor specifically meant the waveform leading or lagging from inductive/capacitive loads. Wouldn't almost every household electronic device (anything other than motors and things that heat) cause this same issue? They all have an FWB and capacitor, don't they? I guess in the scale of things, most of the power used by a home is resistive heating elements, not a lot for high power electronics.

My business' power meter has both watts and volt-amps, and often 70% of my bill is demand charges. I haven't seen, but I've been told that many commercial places will have a power factor multiplier as part of their bill. Is this spiking something that would affect my power factor in that respect (something I'd get billed for?) or does it not affect it in that way? I presume I'll be charging my car mostly at the business (no garage at home).
NiHaoMike wrote:Can probably still get a reasonable power factor considering the bus capacitance is quite low for the power levels being dealt with. Just needs better control over the buck/boost converter.
Wouldn't higher capacitance help, rather than hinder how spiky the current flow is, by better ballasting the load? Or... I guess the better it ballasts the load, the more aggressively it pulls current during the peaks of the AC?
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

MattsAwesomeStuff wrote: Thu Jun 25, 2020 5:53 pm My business' power meter has both watts and volt-amps, and often 70% of my bill is demand charges. I haven't seen, but I've been told that many commercial places will have a power factor multiplier as part of their bill. Is this spiking something that would affect my power factor in that respect (something I'd get billed for?) or does it not affect it in that way? I presume I'll be charging my car mostly at the business (no garage at home).
If they bill you based on volt-amps, a bad power factor will increase your bill, quite simply. Of course the power factor will be the sum of everything that's connected to the grid, but you can be fairly certain there's nothing that would offset the spikes, unlike how a capacitive load can offset an inductive load.
MattsAwesomeStuff wrote: Thu Jun 25, 2020 5:53 pm Wouldn't higher capacitance help, rather than hinder how spiky the current flow is, by better ballasting the load? Or... I guess the better it ballasts the load, the more aggressively it pulls current during the peaks of the AC?
With small enough capacitance compared to the power being throughput, the capacitor will start following the rectified AC waveform, which extends the time when current is flowing into the capacitor. Of course with that the AC waveform at the capacitor can start transferring into pulsating current at the output depending what kind of converter is designed after it. In some uses pulsating current won't even matter, like let's say... a battery charger. :mrgreen:

When you think about it, the boost converter based PFC can be just that - a rectifier feeding very little capacitance, fed into a boost converter that is designed to be able to modulate its PWM in sync with the AC voltage waveform to create the perfect current waveform.
User avatar
rstevens81
Posts: 352
Joined: Sun Dec 22, 2019 10:36 am
Location: Bristol, UK
Has thanked: 23 times
Been thanked: 92 times

Re: Gen 3 inverter converter control software

Post by rstevens81 »

A (potential) warning to anyone who wants to use this a Yaris inverter (as stated in Celeron's GitHub) the dc capacitors are rated at 300v (low) and 650v (high) (as stamped on the capacitor casing - attached).
This does also beg the question that if the high side capacitor is 650v then it makes it highly unlikely that the inverter ever runs at 650v.
Attachments
capacitor.jpg
Rule 1 of EV Club is don't buy a rust bucket....
Which rule does everyone forget 🤪
User avatar
Jack Bauer
Posts: 3563
Joined: Wed Dec 12, 2018 5:24 pm
Location: Ireland
Has thanked: 1 time
Been thanked: 87 times
Contact:

Re: Gen 3 inverter converter control software

Post by Jack Bauer »

Good info thanks. The 650v thing is an internet myth that will probably never die. Along with the 100A "current limit" ....
I'm going to need a hacksaw
doobedoobedo
Posts: 260
Joined: Sat Jan 12, 2019 12:39 am
Location: UK

Re: Gen 3 inverter converter control software

Post by doobedoobedo »

rstevens81 wrote: Mon Jun 29, 2020 12:20 pm A (potential) warning to anyone who wants to use this a Yaris inverter (as stated in Celeron's GitHub) the dc capacitors are rated at 300v (low) and 650v (high) (as stamped on the capacitor casing - attached).
This does also beg the question that if the high side capacitor is 650v then it makes it highly unlikely that the inverter ever runs at 650v.
I spotted that when I took mine apart. Making me a bit nervous about using mg* to boost to 400V...
User avatar
celeron55
Posts: 776
Joined: Thu Jul 04, 2019 3:04 pm
Location: Finland
Has thanked: 28 times
Been thanked: 110 times
Contact:

Re: Gen 3 inverter converter control software

Post by celeron55 »

If you read any of Toyota's gen3 or newer publications, they do always say "650V maximum".

The IGBT driver board enforces that voltage limit. If you set the boost converter at a low PWM and just wait, it'll stop when the MG rail reaches about 650V. At high PWM it will overshoot and then you're watching it drain down from some semi scary voltage from 650V up to over 1000V.

Telling it's 650V could be a marketing strategy, or something lazily taken from their safety documents. I think it probably goes to 650V, but only when you absolutely floor the throttle, and their control system is accurate enough for operating right at the limit for brief periods of time.

So who's going to buy a gen3 Prius or Yaris... for science?
User avatar
rstevens81
Posts: 352
Joined: Sun Dec 22, 2019 10:36 am
Location: Bristol, UK
Has thanked: 23 times
Been thanked: 92 times

Re: Gen 3 inverter converter control software

Post by rstevens81 »

I think the thing to remember is that inspite of this, it's a high-powered Inverter and a DC-DC so the loss of budget 3 phase charging is not the end of the world as it's still insanely good value.

It is not long ago when an inverter if much lesser power would cost thousands.
I think if you want really high powered 3 phase charging then the poor/rich man's chademo would be a better start (if you have the necessary electrical understanding and experience).

EDIT: I am talking about the Yaris inverters here, [Prius inverters all good!] Sorry Celeron
Rule 1 of EV Club is don't buy a rust bucket....
Which rule does everyone forget 🤪
Post Reply