[DRIVING] VW Touran powered by Nissan Leaf [FINISHED]

[jrbe]

Going the air cooled, flowing through the inside of the battery box route, are you worried about humidity or corrosive spray getting into the battery box?

On another forum there was a guy who did battery testing for a living. His recommendation was potting the battery components even in sealed batteries because of all the corrosion failures he had seen.

[arber333]

Going the air cooled, flowing through the inside of the battery box route, are you worried about humidity or corrosive spray getting into the battery box?

Exactly! It is a tradeoff. On one hand you get simple airflow to cool the batteries as they drive and charge. Former is more effective so it is a good way to lower temperature before charging.

The other option is added complexity of liquid cooling. Granted it is much more efficient but the complexity and possibility of failure is so much larger...

I think in case of aircooling we would have to use a kind of hepa filter with good suction centrifugal blower. I think i just have the space in the car for that. Time will tell about corrosion... I think the problem is that modern salting dust they yare using on roads now is much more corrosive to alu parts. When alu corrodes it looses ability to dissipate heat as well.

[uhi22]

In my OEM Ioniq they go a clever way: They suck the air from the (well conditioned) cabine, which is dryed in winter and cooled in summer. Seems to work without much condension. On the outlet end of the battery box, there is a strong blower which brings the air to the outside of the vehicle.

[jrbe]

I like the water / refrigerant direction because you can cool or heat the battery as needed. If you're doing a heat pump you can be clever about potentially using the cabin heating and cooling energy for the battery and motor needs or vice versa. It's not simple though.

I wonder if the Ioniq has something like a fire door between the cabin and battery. I'd expect it does. It's efficient and cost effective but there's a lot of air and potentially contaminants going through.

A hepa filter is a good idea but a maintenance item to forget about.

A passive cooler would over cool in cold weather and make it extra tough to heat the battery.

No easy solutions.

[arber333]

I like the water / refrigerant direction because you can cool or heat the battery as needed.

That is well and good for single box. It may as well be integrated into the box itself with small 700W Volt battery coolant heater...
But i have 2 separete battery boxes and possibility for a third... this really complicates my planning...

[johu]

Yes of course we would all like Teslas Octavalve and a perfect solution that sends heat and cold wherever needed. But with the space (and fabrication) constraints that can't always be achieved. Water cooling will not fit into my box so I can't do that.
Also OEMs try to not repeat Nissans rapidgate which I agree they shouldn't. But then 1000 km road trips are not my use case. So meh...

Corrosion: the worst corrosion I had in Polo in the inside battery box! Not the poorly sealed one in the engine compartment. Now when I removed the Nissan batteries that were not at all sealed, just shielded, I didn't see any corrosion in the bus bars but some on the metal shielding of the cells (good old rust).

I suspect the venting is what keeps moisture in check and direct spray is kept away sufficiently from the covers

[Jacobsmess]

On cooling, I was thinking about having a vent opened on demand which should help reduce foreign objects getting in somewhat.

[johu]

I'm investigating a glitch now. When fixing a cell tap wire break I had to lower the front battery brick. When mounting everything again part of the BMS LV harness got caught between chassis and battery and eventually it was the BMS drive signal that got shorted to ground. So when starting the car the BMS wasn't started. But that is not the issue at hand. I had to drive several 100 km like this but besides the obvious it didn't have any adverse effect.
Back home I slightly dropped the battery brick again, freed the harness and mounted everything back up.

Now, the problem may or may not be related given the fact that I since swapped out the battery completely.

Ok, what is it??

The VCU restarts mid drive! So the ABS light comes on, I loose power (inverter CAN timeout) and after about 3s everything is back to normal (except the ABS light). I found that the ESP8266 also restarts (I see the wifi connection drop out). So at an early stage I could rule out a watchdog reset or other kind of SW error. Plus the software hadn't been changed for ages when the problem first occurred.

Sometimes I can drive for 100 km without issues. At one point the glitch even happened while stationary. So I don't really think it's vibration related.

To dig deeper I "exported" GND, 12V and 5V to the smaller engine connector that is no longer used and connected my analog discovery to those. On the first drive I caught nothing, but today already after one minute I got the error and found this:

The sampling rate is low because I wanted to real time watch the signals. So maybe 5V dipped deeper than we see. The lower voltage thereafter is because the DC/DC converter drops out.
I was expecting a dropout of a voltage, not a rise! What could cause this? Here are my candidates:
a) something with the 5V converter (can it make voltage rise?)
b) The DC/DC converter glitches - maybe the 12V feedback
c) The charge enable signal (that also feeds into the 5V converter) bring in something weird
d) It is actually a ground bounce that looks like a 12V glitch?

When I scope at higher frequency I can't see any inverter noise. So I doubt it's EMI related. Also when the input of the 5V converter rises the 5V shouldn't drop out. Maybe if it rises above the maximum of 40V? Hmm.

I then drove around with the scope at higher sample rate. And guess what: it did glitch when I pulled in at Ionity and wasn't looking!

What do you think?

[uhi22]

First, congratulations, that it is half-way reproducable, and not only once per month. This makes it easier to analyze.
Second, the spike on the 12V is perfect for triggering the scope on it, in single shot mode.
I see the 5V is perfectly stable. The "drop" for 200ms is not a real drop on the 5V, instead, it is a ground shift for 200ms. Because at the end of the "drop" of the 5V, also the 12V deviates from its normal condensator-discharge-curve.
The "drop" of the 5V has the beginning edge while the 12V has the high spike. This looks like, that during the spike time, a bigger current starts, which draws the osci ground some millivolts up (and this looks like the 5V goes down, but in fact it is stable regulated). Then the 12V raises down, like a discharge curve, and guessing it is an inductive effect, the current goes down and the ground shift disappears.
My theory: Some inductance is turned off, and feeds a current to the 12V for 200ms. The minimal buffering of the 12V line is not able to consume the current, so the voltage rises. Maybe more probable with a diode network, which leads to pumping up only one control unit, and not distributing the energy to the complete vehicle network.
The main question is: Is the observed 12V spike the root cause of the restart, or is the restart the root cause of the 12V spike. Some more osci channels would be nice, especially a debug pin of the controller where it is visible, when it does the restart.

[johu]

Hmm, the DC/DC converter shuts down when the STM resets, probably with some delay. But I have never observed a turn-off spike.

I think what I will do next is observe the 3v3 rail instead of the 5V one, trigger on the 12V spike and use a higher time resolution.

[johu]

Did another test drive today and again the error occurred while parked. Not sure if this is statistically significant but by now the error has come up 4 times after becoming stationary.

So this time we have plotted 12V and 3V3 at a higher sample rate. As you can see 3V3 doesn't change, so now I think the rise in voltage is caused by the contactor coil being unpowered shortly. Basically the contactor output becomes a high impedance input after reset, then as firmware starts up it becomes an output and is briefly pulled low until the firmware quickly realizes that the inverter is still in RUN mode and therefor turns it back on. The energy from the coil is absorbed by the 680uF input cap but of course makes voltage rise a bit.

So now I have increased the capture time before the glitch but it didn't come up again so far.

On an unrelated note I'm getting dropouts of the speedo The trip counter keeps counting but the speed display drops to 0 for a few seconds and then resumes. Strange, I thought they were using the same data source. ABS speed in VCU keeps updating though.

[uhi22]

Just some more questions: Does the controller have a reset input connected to anything more than the capacitor? (Sorry, did not look to the schematics.) Does the STM32 has some reset-reason-indication registers, where it would say what was the reason for restart? What would the STM32 do if the flash memory cells are not fully programmed and just on the border between 1 and 0, and so depending on temperature and time the content flips? Did the CAN messages change with integration of the new battery? So maybe causing a bus-load or bus-content related bug in the software? Is it sure that the ESP also resets, or does it just lost connection to the STM?

[johu]

Thanks for thinking along

Does the controller have a reset input connected to anything more than the capacitor?

It has a rather elaborate reset circuit:

But like said, the ESP on the same 3v3 rail also restarts, so it's not purely an STM32 problem

Does the STM32 has some reset-reason-indication registers, where it would say what was the reason for restart?

Indeed

Will add this to firmware.

What would the STM32 do if the flash memory cells are not fully programmed and just on the border between 1 and 0, and so depending on temperature and time the content flips?

Probably illegal instruction exception if the bit flip is in an opcode -> endless loop -> watchdog reset. Again, this wouldn't reset the ESP module though...

Did the CAN messages change with integration of the new battery?

Yes significantly. This could explain the speedo problem but not the reset as that problem existed before.

Is it sure that the ESP also resets, or does it just lost connection to the STM?

Yes I see the wifi connection drop out. Also after the incident ESP and STM are in baud rate sync. Infamously if only one of them resets they no longer communicate as one is at 921k and the other at 115k

[uhi22]

Does the ESP have a reset input? If yes, is this driven by the same reset circuit as the STM?

[johu]

The chip probably does but it is not brought out. Only 3v3, gnd, rx, tx is connected

[uhi22]

Hm, then it does not make sense that both make a reset while the 3V3 is stable.

[johu]

I assume my time window was too short. Now waiting for another one....

[johu]

Today it hit me while charging at the Tesla charger! Of course the laptop wasn't connected to the scope because I was monitoring the charging session

Charging gracefully stopped. So, as presumed before, it has nothing to do with driving or "ignition" power dropping out. During charging the VCU has two possible supply voltage sources: ignition (when on) and charge enable

[johu]

I decided to remove the scope and give it my best shot instead.
I put in a new STM32-H103. I then checked the schematic and found there's not much buffering on the 3V3 rail. So I added 100uF.
Next I found only 3.1V arrived at the ESP module. When I wiggled the pin header it went up to 3.3V. Hmm. Removed the flimsy jumper lead and soldered a wire instead. Lets see how it goes.

[johu]

It did not help Ran some software updates and cycled between various modes for testing when suddenly I got the dropout again. This time with the AC lead plugged in. I could hear a short cycle of the positive contactor. Really amazed that this short (partial) interruption hasn't done damage, like burning out the always-on precharge resistor.

Back to square one. By now we can exclude
- 12V supply (scoped)
- 5V supply (scoped)
- STM32 (exchanged)
- 3V3 regulator (exchanged)
- Software (the same version was running before and after the dropouts started)

Only two things left are the ESP8266 module and the base board, an old revision 2 inverter mainboard. Oh and the CAN transceiver.

[arber333]

Really amazed that this short (partial) interruption hasn't done damage, like burning out the always-on precharge resistor.

I recommend to you that during troubleshooting you wire your DC contactor through latching circuit.

[uhi22]

Thesis: The ESP comes into a state where it draws high current for some microseconds. This pulls the 3V3 below its own reset threshold. Also the 3V3 supervisor U2 sees this short drop, and creates a reset for the STM. The drop on 3V3 may be very short, maybe not visible with slow running scope.

I'd like to see on scope:
1. an output of the STM, which indicates the point in time when the reset happens. Can be used to trigger the scope
2. the reset input of the STM.
3. The 3V3 line.
4. The 12V, just to get the info whether the reset is before, during or after the spike on the 12V.

[johu]

I recommend to you that during troubleshooting you wire your DC contactor through latching circuit.

Yes that would be great. Could try to bodge in a self-holding relay
EDIT: now I know why nothing is damaged. The inverter times out and switches to neutral and the DC/DC converter signal drops out as well because of the restart.

The ESP comes into a state where it draws high current for some microseconds

Yes that's what I thought, too. For now I have swapped in a new ESP module. The other one has been in there for years though without issues. Maybe something degraded...

Also the 3V3 supervisor U2 sees this short drop, and creates a reset for the STM

Was going to find its datasheet but in the schematic it is labled "NA". Hmm. Well turns out there is not even a footprint for U2 on the board. U2 does not exist.

Will go back to scoping if the new ESP didn't fix it...

[uhi22]

U2 does not exist.

Okay, in this case it does not make sense to scope the reset line. Then I'd like to see what reset reason does the STM report in its reset reason registers.

[johu]

First test drive with the new ESP module and no STM restarts But sometimes the error hides for a while.

I think I will start making experiments with the old module and see if I can reproduce the spikes.