Interference: How to measure and fix it?

Nissan Leaf/e-NV200 drive stack topics
User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Interference: How to measure and fix it?

Post by janosch »

Hello,

I have seen the previous discussion here about a similar problem:(1)

Now, I have interference at 17.6kHz throughout my low voltage system - double the 8.8kHz the inverter runs at, and only when it is active - checks out.

Our setup:
- Two battery boxes, one in the front, one in the rear.
- Shielded cable in principle, yes, but that might not be connected everywhere properly, so the Faraday cage is interrupted
- No Ferrites! Yes, that was stupid, now wondering where to put it
- Our junction box isn't shielded properly, we need to rectify that

Now my questions:
1) Where should the ferrites go given that we have multiple boxes? One for each box, with both HV+/HV- through it? The Nissan one is at battery box exit with both bus bars going through a square shaped ferrite, should we have one per box? Or closer to the inverter?
2) How do I know that what I am doing is good or bad? Currently I just observe 12V pulses on the scope, but can I somehow figure out where the leak is most likely to be before I start "panic-shielding" and then don't know which of my many modifications actually made the difference.

Sorry if this was asked elsewhere/is stupid, I volunteer to collate answers into an EMF/EMI/EMC wiki page.
arber333
Posts: 3241
Joined: Mon Dec 24, 2018 1:37 pm
Location: Slovenia
Has thanked: 74 times
Been thanked: 223 times
Contact:

Re: Interference: How to measure and fix it?

Post by arber333 »

Hmm... i managed to solve my BMS system by way of routing HV cables away from 12V wiring. Even 20cm parallel will help. As you heared interference drops by the square of the distance.
User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Re: Interference: How to measure and fix it?

Post by janosch »

That is a very good shout arber, the BMS position is indeed slightly different between our two vehicles.

On a bigger scale though, how would I best go about measuring it?

Maybe it is impossible, but I was hoping I could get some sort of heatmap after dragging a sensor across the vehicle, a bit like these guys making WiFi signal strength visualisations in 2011:
https://news.ycombinator.com/item?id=2705674
User avatar
mjc506
Posts: 343
Joined: Wed Sep 09, 2020 9:36 pm
Location: Wales, United Kingdom
Has thanked: 30 times
Been thanked: 28 times

Re: Interference: How to measure and fix it?

Post by mjc506 »

I've not tried this, but my first instinct would be to use an RTL-SDR with a directional antenna and wave it around...
User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Re: Interference: How to measure and fix it?

Post by janosch »

mjc506 wrote: ↑Tue Nov 01, 2022 12:13 pm I've not tried this, but my first instinct would be to use an RTL-SDR with a directional antenna and wave it around...
Thats great!
I do have a little star-trek tricorder here that measures microTesla and shows it on a display, maybe I hack it with a colourful LED and then do the long exposure trick the guys in the video have done to create a visualisation.

Or just jot down the values I find everywhere on the vehicle on a piece of paper, make a change, jot down again and compare.

Maybe I should get the RTL-SDR & antenna!
User avatar
EV_Builder
Posts: 1199
Joined: Tue Apr 28, 2020 3:50 pm
Location: The Netherlands
Has thanked: 16 times
Been thanked: 33 times
Contact:

Re: Interference: How to measure and fix it?

Post by EV_Builder »

Where do you measure that signal? And what happens if you manipulate routeing and shielding etc. to that signal?
Are both inverters installed the same?
Converting an Porsche Panamera
see http://www.wdrautomatisering.nl for bespoke BMS modules.
User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Re: Interference: How to measure and fix it?

Post by janosch »

I measured 12V+ to my VCU and scope ground to 12V battery ground to see the 17.6kHz waveform, that is one datapoint.

The other datapoint is that the BMS does not report correct cell voltages while driving, no such problems when charging, and even when stopped at a red light it will recover and show you SOC then. CAN messages come through fine from the LeafBMS but it's content is set to MAXINT 0b11111111 for all cell values, which is the LeafBMS failure mode (as far as I understand).
arber333
Posts: 3241
Joined: Mon Dec 24, 2018 1:37 pm
Location: Slovenia
Has thanked: 74 times
Been thanked: 223 times
Contact:

Re: Interference: How to measure and fix it?

Post by arber333 »

Hm...
As far as i read about this grounding and emi solutions it allways sounds like black magic hold on to your ears whilst turning clockwise. I even noticed in ICE cars different models of the same car have different grounding locations on the chassis.

I noticed chips have gnd pins and voltage reference pin which is different pin than supply. Usually it is protected by 10R resistor. You could measure for interference transients there if circuit misbehaves...
User avatar
EV_Builder
Posts: 1199
Joined: Tue Apr 28, 2020 3:50 pm
Location: The Netherlands
Has thanked: 16 times
Been thanked: 33 times
Contact:

Re: Interference: How to measure and fix it?

Post by EV_Builder »

janosch wrote: ↑Tue Nov 01, 2022 12:29 pm I measured 12V+ to my VCU and scope ground to 12V battery ground to see the 17.6kHz waveform, that is one datapoint.

The other datapoint is that the BMS does not report correct cell voltages while driving, no such problems when charging, and even when stopped at a red light it will recover and show you SOC then. CAN messages come through fine from the LeafBMS but it's content is set to MAXINT 0b11111111 for all cell values, which is the LeafBMS failure mode (as far as I understand).
What kind of cable is used for canbus?
Converting an Porsche Panamera
see http://www.wdrautomatisering.nl for bespoke BMS modules.
User avatar
johu
Site Admin
Posts: 5685
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 154 times
Been thanked: 960 times
Contact:

Re: Interference: How to measure and fix it?

Post by johu »

I was there with Janosch in June and the problem wasn't the CAN bus. All CAN data comes through fine, e.g. the temperature values are all good. But while driving groups of cells will go into that invalid state and as a reaction SoC also goes into invalid state (102.5%).

I think the extension leads of the cell taps catch too much EMI (and add a good deal of resistance). If I remember correctly they aren't shielded.
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
User avatar
SuperV8
Posts: 148
Joined: Wed Mar 24, 2021 2:57 pm
Has thanked: 43 times
Been thanked: 10 times

Re: Interference: How to measure and fix it?

Post by SuperV8 »

What is your sensor grounding like?
I have had issues with interference in previous ICE DIY ECU projects which was caused by poor grounding.
Maybe you can use the same principles?
From here:
http://www.msextra.com/doc/pdf/MS2V357_Hardware-3.4.pdf
image.png
image.png
User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Re: Interference: How to measure and fix it?

Post by janosch »

Thank you all for chipping in, I ordered some shielding socks and tape.
Then I was forwarded this graphical guide as well about all kinds of radiation and had a good read through it:
https://www.unit3compliance.co.uk/emc-t ... d-probing/

I am absorbing as much knowledge as I can now from your resources and we will run some probes around the vehicle on Friday I hope.
I will keep this updated with what we find and how we fix it.
I will collate things into a wiki page once I feel that we are "done"/I understood the subject matter enough to write something meaningful down :)
User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Re: Interference: How to measure and fix it?

Post by janosch »

Ok, update on this:
Screenshot from 2022-11-29 14-27-26.png
PXL_20221122_150800634-min.jpg
Screenshot_20221129-141549-min.png
Got the sensing wires out of the car, shielded them with a braided sock, faffed them back in, unfortunately same behaviour as before.

Now measuring at a HV sensing wire as close to the BMS as possible (rear), as opposed to on the 12V system above:
  • noticed the noise here is 1.25kHz, which doesn't match the 17.6kHz I measured before on the 12V system! very curious
  • the noise only happens when the inverter is actually working (hard)
  • adding ferrites seems to have made no difference
  • shielding the long stretch of sensing wires seems to have made no difference (connected either end, gaps of ~20cm in front, ~10cm back)

I now wonder:
  • is the noise being picked up inside the battery boxes where HV bus and HV sensing meet?
  • why would I measure 1.25kHz noise when the inverter works hard and not 8.8kHz? Briefly even wondered if its the DC converter
  • all the way back to the initial question (although today measuring on HV side), if I had some visibility to where it is coupled in I could try more.
  • considering a load test on the sensing wires with an incandescent bulb to see if I can spot a weak one (measuring voltage drop across the wire), laborious with 48 wires though, but not impossible
Hm! The investigation continues.
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: Interference: How to measure and fix it?

Post by Pete9008 »

I'm wondering if the problem is actually due to the split pack and the voltage drop across the HV power cable running between the two. The longer cable run will cause a much higher noise voltage to appear across this cable and so across the BMS tap for the cell next to the split, and the BMS won't have been designed to cope with this. This would fit with it only occurring at higher load = higher current = higher voltage drop.

If it is this then the screening you have added wouldn't do much. Increasing the gauge of the HV cable between the two packs might help though?

On the measurement I wouldn't suggest trusting the scope automatic measurements too much, they can often be wrong unless everything is set up just right (and I can't see the 1.25kHz periodicity in that screen shot).

Edit - on measurements like this is is also worth saying what you were using as the earth reference on the scope as that can make a big difference to the measurement (the BMS and the scope could easily be seeing totally different things).

Edit2 - forgot to ask, is it always the same group of cells or is it random? The former would suggest the above especially if it is a group next to the split. The latter still could if it is always groups on the same side of the split.
User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Re: Interference: How to measure and fix it?

Post by janosch »

Great thoughts, thank you!
Pete9008 wrote: ↑Tue Nov 29, 2022 3:16 pm This would fit with it only occurring at higher load = higher current = higher voltage drop.
Thicker gauge wires is a good idea, but it doesn't appear when rapid charging at 120A. Only when the inverter is on, so Amps don't seem to be the main problem here (I think).
Pete9008 wrote: ↑Tue Nov 29, 2022 3:16 pm On the measurement I wouldn't suggest trusting the scope automatic measurements too much, they can often be wrong unless everything is set up just right (and I can't see the 1.25kHz periodicity in that screen shot).

Edit - on measurements like this is is also worth saying what you were using as the earth reference on the scope as that can make a big difference to the measurement (the BMS and the scope could easily be seeing totally different things).
On the 12V interference a few weeks ago I used 12V (-) as GND for the scope.
For the HV sensing measurement above we used the vehicle chassis (which I know is not ideal as HV (-) is not connected to chassis.
I should re-measure with HV (-) as GND!

And yes, we ought to use a master/slave BMS really, but here we are, I am band-aiding these to work as intended before moving onto a redesigned v2 or v1.1 :)
Pete9008 wrote: ↑Tue Nov 29, 2022 3:16 pm is it always the same group of cells or is it random?
Unclear, currently the BMS just gives MAXINT for all cells, so I don't know which cells specifically are giving problems at the moment, but I have previously measured MAXINT for most of the front box and okay-ish values for the rear.
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: Interference: How to measure and fix it?

Post by Pete9008 »

The fact that fast charging doesn't cause it would suggest that it could be more the volt drop due to high frequency noise currents. If this is the case then cable inductance could be just as important as resistance (not good as increased gauge cable then wouldn't help that much). One way to test it would be by trying to reduce the hf current and the only way I can think of doing this is to add capacitance. Is there any way you could temporarily add another hv capacitor module between between the inverter and the battery to see if it helps?

The scope earth reference is not an easy one, I wouldn't trust any of those for these types of measurement , as they are all likely to introduce extra noise, and so would be wary of drawing conclusions purely from scope data The important earth reference is the one the BMS is using its but not easy to access or safely use (and definitely not to be recommended doing with a mains powered scope!).

Btw - I'm assuming it is a Leaf bms is being used, is that right?
User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Re: Interference: How to measure and fix it?

Post by janosch »

Yes, LeafBMS.
"mains powered scope" <-- yeah, I wasn't too keen on connecting GND to HV (-)
Yes, capacitor can be done.
Regarding this, I have not actually measured the whole bus HV (+) to HV(-).
I have measured:
  • 17.6kHz on the 12V system
  • "1kHz" noise between cell tap and chassis GND.
measuring with scope lead on HV(+) under a bit of load I should certainly check.
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: Interference: How to measure and fix it?

Post by Pete9008 »

janosch wrote: ↑Tue Nov 29, 2022 4:18 pm "mains powered scope" <-- yeah, I wasn't too keen on connecting GND to HV (-)
:shock: literally!

Wouldn't take any risks with the scope. For this type of measurements there are so many ways for the scope readings to be invalid its just not worth the risk. For problems like this, where it is either difficult or dangerous (or both in this case!), the best approach is often to try to come up with theories for what the problem might be and then come up with a fix or test to see it helps. Even if it doesn't work it often gives you another clue to what the real problem might be.

Got to say that this is the type of problem that I always used to dread (although having said that, there was also a large sense of satisfaction when they were finally fixed). Good luck with this one!

One other thought. There must be one cell tap wire that is one one side of the split but that is actually feeding bms modules on both sides. Would it be possible to add a duplicate sense wire to avoid this?

This link might also be usefull although not sure whether its applicable to you modules? https://www.mynissanleaf.com/viewtopic.php?t=17470

Edit - My guess is that you will see the 17.6kHz just about everywhere, regardless of what ground reference you use :(
Curious what the 1.25kHz source is though?

Edit2 - see the second diagram on the link to see what I mean about the extra sense wire. If the pack split happened to coincide with the original disconnector position it would probably help.
Alibro
Posts: 829
Joined: Sun Feb 23, 2020 9:24 am
Location: Northern Ireland
Has thanked: 248 times
Been thanked: 144 times
Contact:

Re: Interference: How to measure and fix it?

Post by Alibro »

Sorry if this is stupid but I took apart a Gen1 Nissan Leaf and wondered why it has a large 12V capacitor box in the boot beside the charger.
Do you think it was to help with the same thing?
I need a bigger hammer!
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: Interference: How to measure and fix it?

Post by Pete9008 »

Alibro wrote: ↑Tue Nov 29, 2022 5:06 pm Sorry if this is stupid but I took apart a Gen1 Nissan Leaf and wondered why it has a large 12V capacitor box in the boot beside the charger.
Do you think it was to help with the same thing?
Not sure but could it be the brake backup supply??

Towards the bottom of the page https://www.marklines.com/en/report_all ... 201209
User avatar
janosch
Posts: 306
Joined: Tue Jun 30, 2020 9:23 am
Location: London, UK
Has thanked: 67 times
Been thanked: 54 times
Contact:

Re: Interference: How to measure and fix it?

Post by janosch »

yes the pack is split right through the middle where the service disconnect is.

Just reading through that thread now, one thing to note is that from the 24kWh pack to the 40kWh pack they changed the sensing chips, I was just mulling their datasheets to see if I can interrogate one of them individually, but that is getting into quite a level of detail that I would like to avoid for now: https://www.maximintegrated.com/en/prod ... 7823B.html
Attachments
MAX17823B.pdf
(1.61 MiB) Downloaded 53 times
User avatar
johu
Site Admin
Posts: 5685
Joined: Thu Nov 08, 2018 10:52 pm
Location: Kassel/Germany
Has thanked: 154 times
Been thanked: 960 times
Contact:

Re: Interference: How to measure and fix it?

Post by johu »

A man from Germany who converted his Volvo had to split his pack in 3. 1/2 in the boot like yours with the stock tap harness, 1/4 fuel tank volume, 1/4 engine compartment. Obviously for the latter two he needed to extend tap wires also. The problem does not exist for him. It is the 30 kWh pack with the "old" sense chips, may be more robust? Also the vast difference in cell voltages doesn't show.
He put the sense wires in a U-profile which provides shielding. He doesn't use a shielding sock though.
Support R/D and forum on Patreon: https://patreon.com/openinverter - Subscribe on odysee: https://odysee.com/@openinverter:9
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: Interference: How to measure and fix it?

Post by Pete9008 »

Sorry, the link BMS I posted won't be relevant to your pack.

My current theory is the following (it could be complete rubbish so details included to allow it to be shot down)

I'm still thinking that it is the voltage drop across the pack to pack cable that is causing the problem. Few sums just to see if the numbers make sense. Say switching transients with frequency content of up to 1MHz and assume 1.5m of cable with a 6mm? conductor diameter (which gives an inductance of around 2uH). That would give a cable impedance at1MHz of around 12.6R. Assuming that say 95% of the switching transients are soaked up by the inverter caps leaving 5% x say 300A?? load current = 15A noise current. 15A at 12.5R is 188V!! Obviously a bit unrealistic but there is obviously scope for significant high frequency voltages to appear across that cable. Lets say 100V

Edit - Worth mentioning the above isn't supposed to be accurate, just a very rough test to see how plausible this scenario is!

Any voltages that appear across that cable will also be present on the cell tap wires for the cells on the other side (as seen by the BMS) of that cable. This means that the BMS will have half the cell taps with the 100V of high frequency noise on them relative to the other half. Now in theory this shouldn't matter as it is a common mode signal that should cancel out when the battery voltages are measured, in reality it might well generate enough noise on the ADC conversions to cause problems.

Edit - any idea whether it's the comms that are failing or whether it is the voltage sensing?

This voltage will also appear across the comms channel between the two BMS chips on each side of the split. Again it's a diff pair and should cope but then 100V is a fair bit of noise to deal with when logic thresholds are only a volt or so so it wouldn't surprise me if the problem is there.

So if this is the problem what solutions might work?

1) Reduce the noise current at source - this is the add capacitance at the input to the inverter suggestion above. Doubling the capacitance would halve the noise current.

2) Reduce cable inductance - Increasing the gauge of the conductor would help a little but only a little. Reducing the length, if possible, would provide a much bigger benefit.

3) Add chokes - It's not possible to put chokes on the main power cable as they would just saturate. A better bet would be to get a largish toroid (would need to be a ferrite one with good hf qualities) and run all the tap cables from the second pack though it (as a common bundle), preferably with several turns. This should help reduce the common mode voltage on the sense wires but will only be effective if there is a reasonable capacitance to ground.

Edit - Worth noting that all the above are conducted noise and so screening will have very little effect on them.

It's also still definitely possible that it is radiated coupling onto the signal tap wires and if so screening should help, did the screen you added help at all?
Pete9008
Posts: 1801
Joined: Sun Apr 03, 2022 1:57 pm
Has thanked: 102 times
Been thanked: 347 times

Re: Interference: How to measure and fix it?

Post by Pete9008 »

johu wrote: ↑Tue Nov 29, 2022 6:06 pm Also the vast difference in cell voltages doesn't show.
Any more details on this?
Alibro
Posts: 829
Joined: Sun Feb 23, 2020 9:24 am
Location: Northern Ireland
Has thanked: 248 times
Been thanked: 144 times
Contact:

Re: Interference: How to measure and fix it?

Post by Alibro »

Pete9008 wrote: ↑Tue Nov 29, 2022 5:15 pm Not sure but could it be the brake backup supply??

Towards the bottom of the page https://www.marklines.com/en/report_all ... 201209
Yes that's it.
I need a bigger hammer!
Post Reply