Tesla Batteries @ 2% w/ ORION BMS

[jsimonkeller]

Hello! I know it has been a while since I have needed any assistance, but I had an emergent issue come up yesterday.

I was driving home in Nashville on a hot day and my battery was down to 10 % and I was close to home but it dropped precipitously towards zero and below 200V by the time I got into the garage and now the car will not charge and the battery voltages are all screwy and I hope I have not lost these batteries. I have the ORION 2 BMS and I am attaching screenshots. When I plug in the charger it does not engage and no AMPS show up on Orion cell data. Orion was set up with all of the discharge limits and fail safes using the ZERO EV pre-sets to prevent the batteries from dropping below certain numbers, so I am confused how this happened. I have been driving for a couple of years now with no issues, but the battery has never dropped out like this before.

I hope you have some advice or help for me. I am dreading the responses that my batteries are toast ☹

[tom91]

Uhhmm, they are dead. Look at how low those voltages have dropped. Cells below 2.8V are irreversibly damaged. You have some way lower too.

I do not know how your build has been wired up to stop driving if the pack is empty.

[johu]

Jordan contacted me by email and for completeness sake here is the relevant part of my reply which pretty much reflects Toms:

"As for your battery, I'm sorry to tell you that is toast and no longer safe to use. Discharging below 2.5V destroys the conduction foil, so even when you charge the batteries back up they will have severe rise of internal resistance and overheating risk.

Not sure how Orion acts on this but obviously it wasn't set up correctly. Even the inverter low voltage limit should have caught it but apparently isn't set up right. "

I also dug out a parameter file from August 24 with udcmin set to 270 or 3.2V per cell which would have prevented this

[jsimonkeller]

Thanks guys. Johan, thanks for pulling my old parameter file, which is still in place. I assume the car should have cut off before this number, but it did not, so I am wondering what happened here. If I am going to invest in new batteries, I want to b sure this does not ever happen again.

Also, is there any chance the ORION is giving incorrect cell voltage readings?

[jsimonkeller]

Another question . . . since my car is built to hold 14 of the Tesla Model S batteries, it would be easier to stick with those on replacement. When I look online, I see a lot of listings for "Tesla Model S/X 5.3kWh Solar S3 Battery Module (85kw Pack)" and wondered if the addition of the word "SOLAR" means the packs are different or modified from what I put into my car or if they just say "SOLAR" now to show people that these packs can double as solar batteries for those home solar setups?

Also, when I see the listings for the complete 2016 TESLA MODEL S / X LITHIUM ION HV BATTERY PACK 75kWh 350VDC COMPLETE with 14 modules, can I assume the size of the modules are the same as the 85kWh pack but just with 2 less modules? If so, this would be great for my build, since I only need 14 modules and the item is local to where I live for ease in picking it up.

I appreciate the help with these questions.

[tom91]

is there any chance the ORION is giving incorrect cell voltage readings?

Yes, but unlikely since they are all reading so low. I would suggest you measure the pack voltage another way and do simple math.

I assume the car should have cut off before this number, but it did not,

The car would have stopped you from driving, so pull the latest param file and post it here.

[jsimonkeller]

I believe Johan posted my params, but I will check the car to make sure it matches up.

Also, I took out the ORION Cell Tap Validation tool from my box of tools and checked each bank of 84 cells the voltage seems to show a different story, so now I am wondering if the ORION readings were wrong and my batteries are ok. Man this is confusing.

[jsimonkeller]

I took the multimeter to the front batter box (36 cells) and it also read out as 101.4.

[tom91]

So you need to figure out why the orion is no longer reading the right voltages then. Try plugging it all back in to see what it reads now after a power cycle.

[jsimonkeller]

When I power cycled the front battery box, the ORION readings went nutty. SOC went to 50% and then to 30% and then 5% and then settled on 10.5% and pack voltage went up to 200V and the individual cells changed again. I have not done any charging, but they went up. I am about to jettison 20k worth of batteries, so I kind of want to be sure they are screwed. Any other thoughts to bypass ORION and find out? When I left my office to head home I was at 20% and traveled only 10 miles on surface streets under 40 mph and it bottomed out before I got home. This never happened and I have never let the car get down below 10% so I have nothing to compare this to. Thanks again for any thoughts.

[tom91]

Did you power cycle the orion 12V? It seems the orion is very upset for some reason.

Who did you buy the orion from? You need some technical support from them or Ewert Energy themselves.

[jsimonkeller]

Thanks Tom. I bought it directly from EWERT and I am in an email exchange with Andrew Ewert and he believes the batteries are the problem and not his BMS. I have no way to know and he has not addressed the wacky readings. All I know is that voltage for each box (6 batteries reading 100V and 6 batteries reading 100V and 2 batteries reading 33V) makes me feel like the batteries are charged more than the ORION is telling me they are but I have no way to troubleshoot this without a spare ORION 2 box to see if the ORION is the issue. I also can't test each battery without taking the boxes out of the car and opening them up which we all know will be a PITA.

I assume turning off the ORION and plugging in the charger will do nothing.

[tom91]

So you still have not answered my first quest.

Did you fully unplug the Orion, all connectors including the 12. Then leave it for a bit, then power it all back on and see if anything changes.

As you say the celltap tester and orion do not show the same voltages.

[jsimonkeller]

Apologies for leaving that out Tom. Yes, I unplugged the entire box and left it overnight and plugged it back in this morning. The result was the same types of readings, with SOC at 230% and then 5% and then 12% and cell readings all over the place.

[tom91]

SOC is not important. it is the cell readings, none of it makes sense. So there is a major issue hardware wire as there is a discrepancy between Orion and the celltap. As you say you orion swap would be a good test.

I would ask you to put all the celltap readings into a spreadsheet so you can quickly work out the voltages. Possibly all around 2.8V and could be recoverable.

[jsimonkeller]

Now I just have to remember what is different in the ORION 1 to get that hooked back up! LOL

It has the same battery cell connectors and power, but a different thermistor plug, which I guess I do not need for this experiment, but I think it had a voltage tap piece I would need to reconnect. If that is the case, this would be a pretty big job to get that reconnected . . . . I wish I could get an ORION 2 loaner . . .

[jsimonkeller]

Hi. SO . . . I pulled the trigger on a new set of 14 modules, since I got them all together in one pack of 14 and the price was right. I was able to visually inspect them in person and they appear to be in good condition and they all have voltage at or above 30V.

When I get the new batteries into the car I will want to figure out where in my software and BMS this went wrong to avoid it ever happening again. I will also not discard the old batteries until I can some more tests on them to determin if they were, in fact, damaged. Here was the final word from Ewert:

"Best guess is the pack had a decent degree of degradation (capacity reduction) on it as I am guessing these were fairly used cells. As such, the capacity may have been quite a bit lower than what was actually programmed in to the profile which could account for the rapid drop (which, ultimately is a major reason why it is absolutely critical that the BMS be able to directly interrupt or disable the contactor or drive controls in some way if it detects that minimum voltage has been breached as the SOC potentially may not be fully accurate (say the current sensor was malfunctioning and reporting 1/2 current) and is generally never considered a "Functional Safety" function whereas more discrete metrics like voltages and temperatures most certainly are). The BMS supports SOC voltage based correction but since Tesla doesn't formally provide or publish datasheets for their packs there isn't formal or official data available to fine-tune. Technically this risk is spelled out in the setup wizard for the BMS when setting up the pack initially. I'm assuming that point is well driven home at this point so I won't continue to hark on that any further, but yes, SOC alone is not a sufficient metric to protect a pack off of and can never be relied solely on for this purpose, especially for aftermarket systems like this where the packs being installed may have significantly different metrics from a stock or "new equipment" deployment. There are ways to tune these parameters more closely of course but this can be sort of a case-by-case basis which is difficult to coordinate from an overall system level."

I will seek out the assistance of this crew to help me with this process.

Thanks again.

[muehlpower]

If I were selling Orion, I would also suspect the problem lies elsewhere. You can measure the individual cell voltages at the Orion connector with a multimeter, right? If the BMS then displays something different, the error is there, regardless of settings and SOC calculations.

[P.S.Mangelsdorf]

If I were selling Orion, I would also suspect the problem lies elsewhere. You can measure the individual cell voltages at the Orion connector with a multimeter, right? If the BMS then displays something different, the error is there, regardless of settings and SOC calculations.

Very much agree with this. Its a PITA, but using a multimeter to figure out what the real cell voltages are should tell you if the Orion is correct or not. Components fail, even trusted ones.

[jsimonkeller]

Thanks guys. This weekend I am going to take the seats out of my car, since 2 of the tesla modules are under those seats and NOT in the larger boxes that hold the other 12 batteries which will require a full tear down to access. Once I can access those 2 batteries, I have an old BatteryGO tool I used when I was storing the batteries in my basement for a couple of years (2018-2020) during the original build to make sure the voltages were holding steady, which they always were. I will let you know what it says . . . I still might try to hook up the ORION 1 and see what happens, but that has its own hassles . . . bright side is that if I have to pull the batteries, I get to refresh things that have sat for 2 years, like a new AC compressor and better wiring setup.

I do want to understand why my car did not cut out as it should have. My params are attached along with spot values as of yesterday. Ewert said that ideally the ORION BMS would control shutting down the contactor when the voltages are in danger or destroying the batteries BUT that would have to play nicely with the pre-charge contactor sequences controlled by the OI firmware.

[tom91]

Code: Select all

"udcmin": 20

this is why your drive unit kept going.

I also hope the 177V the DU reads is not real. As that would mean even worse cell voltages. Did you not leave the HV open when pulling spot values and params?

[P.S.Mangelsdorf]

I do want to understand why my car did not cut out as it should have. My params are attached along with spot values as of yesterday. Ewert said that ideally the ORION BMS would control shutting down the contactor when the voltages are in danger or destroying the batteries BUT that would have to play nicely with the pre-charge contactor sequences controlled by the OI firmware.

Well, this is going to depend on how you have the car wired. Does the Orion have control of the contactors as currently wired?

As to why the OI control board didn't shut down due to low voltage, that seems to indicate to me that you didn't actually HAVE low voltage at the drive unit. If the actual voltage reaching the drive unit was above the threshold, and the Orion is just wrong, then the OI controller wouldn't have shut down, because there was no reason to shut down.

And, as Tom pointed out while I was replying, the udcmin was extremely low.

[tom91]

The orion should pull the gear signal. Or at least that is what Zero-EV/Fellten would have recommend (I worked for them when this was decided to do)

That way you could not provide forward or reverse to the drive unit and pull power. This would be done on the discharge relay.

You may have not done this as the orion gets upset when you draw more current then it would allow.

[jsimonkeller]

Once again, thanks fellas.

As to the 170V, the voltage keeps jumping around and settled back at 200V at some point after I took these screenshots, but yes the HV was open when I took these screenshots. Again, the front box of 6 batteries is giving me 101V (now 95) and the rear box of 6 batteries is at 101V. I need to check the two under the seat, but I suspect they will get me to a combined number of at least 220, but I will see. This is why I am baffled. Also, for the last 2 years whenever I charged the car the batteries were balanced and within just .01 of each other and these numbers are so out of whack that I am trying to understand how and when it fell apart.

Below is an OLD screenshot from the ORION from October of 23.

As to the udcmin number, that is embarrasing, but what was it supposed to be? 200? If so, that's a shitty thing to have missed

Thanks for the info about the ORION pulling the gear signal. I will talk to Chris about this and see if that is something I did or did not do and if I should consider that going forward.

[jsimonkeller]

I am looking at these params and I am just flummoxed. The one that Johan sent through must have been before these were uploaded and his are how I always had it, so this one is totally whack and makes me wonder if I uploaded the wrong file at some point, but I am kicking myself pretty hard.

"udcmin": 20,
"udcmax": 520,

I would never pick these numbers on a car with a battery range of 270-350V