2nd LDU blown in a month - EDIT: False alarm. [SOLVED]

[mane2]

First one blew because of BMS lifting contactors when car was doing regen. That was fixed then by disabling BMS contactor control when key was turned to 2nd position. This time, I was parked and ready to go. Turning the key and smoothly pressing the accelerator, nothing happens. Power off, try again.. still nothing. Deeper power cycle - 12v power off, HV service switch off. Leave it there for minute .. put back on, still the same thing. Usually this helps if for some reason OVERCURRENT has happened. Checking web interface with laptop, lasterr shows OVERCURRENT. Even before turning the gear on, just start signal. Last time, OVERCURRENT happened only when switching from N to D.

Had to tow the car home and now very confused what could have happened. I have had OVERCURRENT happening when reversing off the yard and changing from R to D, or wise versa. I always thought that I must have done some too sudden accelerator move or something .. but it has never happened during drive. This time I wasn't even moving. Car was parked on a bus stop while I was doing some instrument cluster coding there .. then ready to go, gear to D and nothing.

I'm really confused what can cause this. One thing that I started thinking, was that maybe the precharge resistor got broken when the first LDU broke a month ago. And after (too many) startups without that precharge, something might get broken.

Any other ideas?

[johu]

That sucks.
Is there no check in place that pre-charge was actually successful? Like the inverter waits for some threshold voltage to be reached before it closes the main relay. Normally switching a relay into an uncharged capacitor will weld the contacts and subsequently there is just always voltage on the inverter which is somewhat unsafe but wouldn't hurt otherwise.

Have you done the quick check on whether a phase leg is actually blown?

[mane2]

Update - It works again! (after one hour of standing parked without power)

I might have an idea what caused it - maybe. After previous blow up, we changed the rotor seal to the one that was recommended, the SKF one but without the sleeve (which was a mistake). That leaked a lot, so we swapped the used seal from previous LDU. We made a drain plug to inverter side, to see if there was water (due DESAT error that happened because of the moist on inverter side). I was monitoring the situation every now and then. Inverter side stopped leaking. Only some moisture when opening the plug, not a drop coming out from the case but the plug was bit moist. I just went to check it, and same thing, inverter side was ok.. Checked the speed sensor/encoder, and that was wet. See picture attached. Wiped the sensor dry .. HV and 12V power back on, and it moves! So luckily, nothing seems to be blown!

I don't know all different scenarios how OVERCURRENT gets triggered, but I started thinking that maybe the moisture/coolant in the encoder sensor caused some bad data coming into inverter or OI board, and it was stopped because of that. Can that cause overcurrent error?

I definitely need to check the precharge resistor and wiring too, but those should be okay. I haven't seen any PRECHARGE errors or like in the web interface. But can some bad settings cause that, on startup? Because that's what I've been experiencing last couple weeks.

[mane2]

Weirdness continues. Car moved this morning as well.. but later today, same thing. Lasterr = OVERCURRENT. Car was just sitting parked in warm garage.

I did notice something odd about the board. WiFi kept disconnecting.. and then, parameters were gone.

Last time the desat error happened due board being moist. I think the water/moisture is doing some damage here now. Maybe some Igbt pins are wet and it incorrectly detects some bad current, and triggers overcurrent error.

[mane2]

OEM Tesla board seems to have some goo on it. Any idea what that is? Should probably do the same for OI boards.. Is is possible to put something on board? Hot glue?

[jrbe]

You can get conformal coating in a spray can. But that's more for when everything is correct and you don't need to touch the board anymore.

You need to solve the leak, whatever that gunk is. Maybe put some on a white paper towel to see what color it is. Can maybe compare smell too.

[mane2]

Thanks. Could try that.

Gunk is coolant. I swapped the blown inverter out, and in with the new. But somehow the new seems to leak .. or the leak is coming from motor side throgh temp sensor hole, as the encoder is wet. Replacing the seal this week and see what's in there.

[mane2]

Any ideas why OVERCURRENT might happen when car is parked, gear in Neutral? I'm afraid to go anywhere, if I need to stop somewhere.. It does not happen when driving.

[johu]

If any of the 6 gate drive faults go low or the over current comparator is triggered. We are talking false alarm, so maybe still moisture somewhere?

[mane2]

I guess that's the only possible explanation.

I was trying to look how OVERCURRENT gets posted and cleared in the code, but I could only find this weird tim1_brk_isr extern function that is called somewhere else, and that posts the OVERCURRENT error.

How OVERCURRENT gets posted and cleared? It's not clear.. I guess it's not the code that posts it, but the inverter itself. Any idea how it gets cleared? Because sometimes power off (12V off is enough) sometimes it needs longer power off. (And in my case, it might take 1-2 hours, probably due moisture).

[P.S.Mangelsdorf]

Here's a question; when the car has been sitting, and you've gone to drive it and it has not started, can you hear the contactors close when they are supposed to?

In my car at least, I can hear the - and precharge close when I turn the key to run, and then the main + close when I trigger start.

Also, on mine any 12V power cycle clears the error memory.

[mane2]

Here's a question; when the car has been sitting, and you've gone to drive it and it has not started, can you hear the contactors close when they are supposed to?

In my car at least, I can hear the - and precharge close when I turn the key to run, and then the main + close when I trigger start.

Also, on mine any 12V power cycle clears the error memory.

Yes I can hear the contactors make a sound when they close, when turning the key. There is not much delay between precharge and main contactors, maybe 0,5 second or so - but I guess that's normal.

I did measured precharge contactor ohms, and it was okay.

I think it's just the moisture/water going to wrong places. I have another (3rd) P85 LDU waiting to be installed end of this week. Hopefully it's water proof

btw, how long you have to keep 12V off for?

[P.S.Mangelsdorf]

Yes I can hear the contactors make a sound when they close, when turning the key. There is not much delay between precharge and main contactors, maybe 0,5 second or so - but I guess that's normal.

Huh, at least as I have it set up, I have to manually trigger main contactors by sending a start signal, but yeah I can usually do that fairly quickly. I tend to pause for about a second.

btw, how long you have to keep 12V off for?

At least from what I've seen, any time I turn off 12V power (such as: hit OC trip, pull over, key off, key on, drive home) the error message has cleared.

[mane2]

Ah I see, so it clears the error instantly. I have done it instantly few times as well, but sometimes I have had to wait like 30 seconds or so. That's why I'm confused of the process. But maybe that was water there, doing damage every time. I hope that now as I have new OI card and "new" motor, things get easier on that front too

Manual contactor controls are probably fun. It's like starting an airplane when you need to toggle some switches first before you can go

[P.S.Mangelsdorf]

Manual contactor controls are probably fun. It's like starting an airplane when you need to toggle some switches first before you can go

I mean the LDU is doing the control, I guess its just how I've wired the 12V signals. But yes, pretty much everything else (coolant pumps, DC-DC, etc) is manually switched, so it's definitely got that vintage/race car feel starting it up. The "bomber" style gauges I have help with that too. Getting back in my very modern daily driver I've found myself reaching for switches that aren't there out of habit.

[mane2]

I was trying to look how OVERCURRENT gets posted and cleared in the code, but I could only find this weird tim1_brk_isr extern function that is called somewhere else, and that posts the OVERCURRENT error.

I'm trying to understand the fault that my 3rd LDU has. It's similar that my 2nd LDU had. Summing up the issues, it was like this

1) First LDU broke because contactor was opened during regen (bad settings, my bad)

2) Second LDU leaked from both sides. Water in rotor/stator side and water got into inverter side as well. Caused rg. DESAT error which should not be possible. After that, randomly OVERCURRENT errors when turning the power on.

3) Third LDU (the one that is in right now) is very latest revision number. They seem to be replacing P100D rear motors with that this year. So this is probably the best LDU so far. But after driving that a day, it also started to throw OVERCURRENT when starting up and latest in mid drive and that hasn't resetted after that. Impedance readings show that phases should be ok. din_ocur shows = Ok at the same time when Lasterr = OVERCURRENT.

So, my best guess so far is that the board took some water damage during the second LDU, as the random OVERCURRENT fails started at that point and continued to way newer LDU as well. But also bit skeptical that if something in my parameters for example can cause this or something else. I did drive about 300-400km with first motor with these settings, so that's not the first on my list..

Another thing I did after first blown LDU was to add a relay between BMS main contactor control. This was to prevent the BMS to open main contactor during drive. So when ignition is not in drive position, BMS can control main contactor. So it can for example disconnect battery from charger if some fault happens. When key is in drive position, the relay control is "taken over" from BMS and forced to be "on", even if BMS would want to open main contactor. But what I noticed here is that when relay control is switched to "power on" -mode, relay switches and there is a very short 12V drop. Like maybe 10 milliseconds (coil moves inside the relay). I started wondering could that short 12V drop cause the contactor to open and close and maybe cause a current spike to motor?

Measuring the phases with multimeter in impedance mode, I get 0,85 from + and - terminals. So that at least seems good. First motor did show 0,35 when at least one stage was blown.

I'm trying to understand the code that posts OVERCURRENT. This is the code that sets the din_ocur (which does not happen in my case). That seems clear. Some digital pin from motor connector gets signal and stops the motor.

Code: Select all

   
   if (hwRev != HW_REV1 && hwRev != HW_BLUEPILL)
   {
      Param::SetInt(Param::din_ocur, DigIo::ocur_in.Get());
      Param::SetInt(Param::din_desat, DigIo::desat_in.Get());
   }

This second is mystery to me:

Code: Select all

extern "C" void tim1_brk_isr(void)
{
   if (!DigIo::desat_in.Get() && hwRev != HW_REV1 && hwRev != HW_BLUEPILL)
      ErrorMessage::Post(ERR_DESAT);
   else if (!DigIo::emcystop_in.Get() && hwRev != HW_REV3)
      ErrorMessage::Post(ERR_EMCYSTOP);
   else if (!DigIo::mprot_in.Get() && hwRev != HW_BLUEPILL)
      ErrorMessage::Post(ERR_MPROT);
   else //if (ocur || hwRev == HW_REV1)
      ErrorMessage::Post(ERR_OVERCURRENT);

   timer_disable_irq(PWM_TIMER, TIM_DIER_BIE);
   Param::SetInt(Param::opmode, MOD_OFF);
   DigIo::err_out.Set();
   tripped = true;
}

So I guess here the software receives some signal from CPU and as there are no known cases what causes it, it just default to OVERCURRENT. Could someone explain this logic to me?

[jrbe]

I'm not very helpful in the coding side but have some ideas for you on the mechanical & electrical sides.

Are you using a xxdt (a normally open and normally closed position) relay? Sounds like it based on your description.

If so, during the relay switching time it can end up getting a voltage spike in regen if the shifter moves, someone bumps it, if wiring isn't 100%, or if there's an issue with the relay.
Theres a slim chance that this quick power cycle might also be scrambling some logic process that's hard to see in our observable time.

You could use diodes to the relay coil wiring so you don't back feed the board. But any failure here in regen could also blow the power section.

If your contactor doesn't have a built in economizer circuit it could be overheating and possibly dropping out with some thermal protection mechanism.

Any chance you have a grounding issue for the motor or motor controls to openinverter board? Almost sounds like a floating / weak ground or some resistance in an unexpected place tripping desat.

Have you cleaned the board and dried it in a low temp oven after all the wet episodes?
Did you dry out all the connectors & harnesses?

Also, .5 second precharge time sounds too short to me. Have you scoped the capacitor voltage while doing a precharge to main cycle? Your main contactor may be taking a huge hit because the capacitor hasn't charged up enough before the main contactor closes. Wondering if your main contactor may have suffered some damage / accumulated damage from this.

[Boxster EV]

Please can you post your parameters and firmware version. It makes no sense that you're munching through drive units like this.

Have you double checked that the OI logic board connections are perfectly clean after getting wet with coolant?

[mane2]

Please can you post your parameters and firmware version. It makes no sense that you're munching through drive units like this.

Have you double checked that the OI logic board connections are perfectly clean after getting wet with coolant?

Yeah, board was cleaned and let to dry for multiple days and using compressed air to clean as good. Did not put it to oven though.

Params here:

Code: Select all

{
  "boost": 1400,
  "fweak": 220,
  "fweakstrt": 290,
  "fconst": 300,
  "udcnom": 0,
  "fslipmin": 0.93,
  "fslipmax": 3.18,
  "fslipconstmax": 3.18,
  "polepairs": 2,
  "respolepairs": 1,
  "sincosofs": 2048,
  "encmode": 1,
  "fmax": 500,
  "numimp": 36,
  "dirchrpm": 100,
  "dirmode": 1,
  "snsm": 12,
  "pwmfrq": 1,
  "pwmpol": 0,
  "deadtime": 63,
  "ocurlim": -2100,
  "il1gain": -1,
  "il2gain": -1,
  "udcgain": 7.59,
  "udcofs": 0,
  "udclim": 320,
  "snshs": 0,
  "potmin": 960,
  "potmax": 4095,
  "pot2min": 4095,
  "pot2max": 4095,
  "potmode": 0,
  "throtramp": 55,
  "throtramprpm": 20000,
  "ampmin": 18,
  "slipstart": 50,
  "sinecurve": 0,
  "throtfilter": 4,
  "brakeregen": -85,
  "regenramp": 1,
  "regentravel": 15,
  "offthrotregen": -70,
  "cruiseregen": -30,
  "regenrampstr": 18,
  "brklightout": -10,
  "bmslimhigh": 80,
  "bmslimlow": -20,
  "udcmin": 180,
  "udcmax": 350,
  "idcmax": 935,
  "idcmin": -500,
  "idckp": 2,
  "idcflt": 9,
  "tmphsmax": 85,
  "tmpmmax": 300,
  "throtmax": 100,
  "throtmin": -100,
  "accelmax": 1000,
  "accelflt": 3,
  "iacmax": 2000,
  "ifltrise": 10,
  "ifltfall": 3,
  "chargemode": 0,
  "chargecur": 0,
  "chargekp": 80,
  "chargeki": 10,
  "chargeflt": 8,
  "chargepwmin": 0,
  "chargepwmax": 50,
  "idlespeed": -100,
  "idlethrotlim": 50,
  "idlemode": 3,
  "holdkp": -0.25,
  "speedkp": 1,
  "speedflt": 5,
  "cruisemode": 0,
  "cruisethrotlim": 50,
  "udcsw": 240,
  "udcswbuck": 540,
  "tripmode": 1,
  "bootprec": 0,
  "pwmfunc": 0,
  "pwmgain": 100,
  "pwmofs": 0,
  "canspeed": 2,
  "canperiod": 0,
  "nodeid": 1,
  "controlid": 63,
  "controlcheck": 1,
  "manualstart": 0,
  "fslipspnt": 1.12,
  "ampnom": 18
}

[Boxster EV]

Boost very low, fweak very low, fweakstart low, throtramp very high, ampmin high.

Regen setting also very aggressive.

UDCmax @ 350!

These are my settings running LDU base, 360v nominal battery (400v). Firmware v5.20r.

Here they are:

params.json

(1.57 KiB) Downloaded 53 times

[mane2]

Boost very low, fweak very low, fweakstart low, throtramp very high, ampmin high.

Regen setting also very aggressive.

UDCmax @ 350!

These are my settings running LDU base, 360v nominal battery (400v). Firmware v5.20r.

Here they are: params.json

Thanks for the settings! Will try your values once I get working motor/board

Regen is perfect for my taste, but for other settings - I'been trying to adjust those for my voltage. It's low, 230V - 300V. (12 Tesla Modules) That's why UDCMax is 350V. I think voltage also affects regen values, thus values are not comparable with your setup. I guess none of my values would cause OVERCURRENT.

btw, firmware version is the latest v5.32.R

[mane2]

Are you using a xxdt (a normally open and normally closed position) relay? Sounds like it based on your description.

If so, during the relay switching time it can end up getting a voltage spike in regen if the shifter moves, someone bumps it, if wiring isn't 100%, or if there's an issue with the relay.
Theres a slim chance that this quick power cycle might also be scrambling some logic process that's hard to see in our observable time.

You could use diodes to the relay coil wiring so you don't back feed the board. But any failure here in regen could also blow the power section.

After reading your post I went to look for the relay model. Pulling it up, One relay connector wires dropped itself! I realized that hey, this probably causes main contactor not to close at all! Or do it randomly. Which felt like something that has been happening. (Though I don't understand how that only happened when car was parked). I quickly removed the connector that had bent tab and made a new one. Put it back and I could hear the precharge and main connector close after about one second. I was sure the car would move now and that I've found the issue .. but no, still the same issue. Putting the car to D or R, OVERCURRENT trips immediately Somehow I remember that OVERCURRENT might get triggered if main contactor is not closed and motor trying to pull power through precharge restistor?

Now I started wondering, if the bad relay contact has been doing this kind quick switching during drive.. It's always closed, except when the control changes from ignition (key) to BMS. So it might have been clicking that relay during drive, causing those very short cuts (maybe 10ms) to main contactor power. Though I haven't gone through the wiring diagram, which connector it was and how it affects the main contactor. It might have opened the contactor during drive .. which is the worst case I think. Though, when it happened, I was pressing accelerator very lightly .. but I guess that could still break it.

I'm keeping the car without power now for a bit, and testing soon if it would move again. Damn I hope it just wants deeper power cycle. Need to go through that wiring diagram next.

[mane2]

It moves!! Holy shit and thank god. What a weird combination of issues, and solution was very easy and simple. Fingers crossed that it’s not just anything else.

Lesson to be learned here: OVERCURRENT can be caused also when main contactor does not close.

[mane2]

And thanks @jrbe for making me go check that relay type

Need to tweak those settings now that @boxster_ev sent

[mane2]

Boost very low, fweak very low, fweakstart low, throtramp very high, ampmin high.

Now checking your parameters.

boost: is only 1400 vs your 2100. I thought it's only related to startup? Does it make difference on higher speeds, more power?

fweak: yours is 238 mine 220. Not that big difference. I'll try that value.

fweakstrt: You have 320 vs my 290. Pretty close

throtramp: I'm not sure which way this parameter goes. I have 55 and you have 5.. Does yours react to throttle faster or slower?

ampmin: you have 1, I have 18. I think this is related to voltage or boost.. maybe both. Your's move easier maybe because higher voltage and boost. I tried to tune this so, that there is no slack in the throttle pedal. I might need to adjust this when upping the boost.

I'll have a go tomorrow and report back how it affects. Thanks again.