Tesla Model 3 Rear Drive Unit Hacking

[Jack Bauer]

Well, good news is red light stays out and pwm reaches the drivers but they are not happy. Guess I'll have to go in and see whats annoying them. In related news the variant detection is really messed up and needs to be forced to M3 but thats a small matter.

[davefiddes]

Bother. That's quite a complete failure. You'll need to trace through the initialisation code to figure out which bit it is getting upset by. An SPI capture might be necessary to verify that data is going into and then out of the SPI chain. It should look like the captures you took way back on the Tesla board.

I've got a shed that needs it's roof replaced while the weather holds. If I get some time later I'll add some better status debugging but that'll only help once the initialisation succeeds.

[Jack Bauer]

Capture attached. Funny. Am going roofing myself:) Thanks Dave.

[davefiddes]

I think I've figured out what's going on. The SPI bus is running a bit too quickly. It has to be under 5MHz.

If you modify spi_setup_teslam3() and swap the parameter passed to spi_init_master() from SPI_CR1_BAUDRATE_FPCLK_DIV_8 to SPI_CR1_BAUDRATE_FPCLK_DIV_32. I think this will make it run at 3MHz like the MG (spi_setup()) and hopefully work reliably. I don't think that anything bad will come of running the bus at a slightly slower speed.

I think it was like that because an earlier version of the SPI init was set up to work with the other SPI peripheral which is on a different clock and I forgot to change it back for v3(!) of the code.

[Jack Bauer]

Thanks Dave. Will try tomorrow.

[crasbe]

Also notable that reading out the registers after writing results in 0xAA for all of them. Perhaps it is indeed a timing thing.

One other thing that Damien wants to look into tomorrow is that the "STATUS1" is set to "UVLOH = VH undervoltage flag. It is forced high when VH is below VHoff threshold." and STATUS2 is set to "REGERRR", which can also be triggered by "– Isolated interface communication fails".

One hypothesis is that the VH is not at full level yet when /SD goes low and therefore UVLOH is latched.

[Jack Bauer]

SPI clock at divide by 32. Roughly 2MHz now. Same behaviour.

[davefiddes]

Clock looks sensible now. I'm still seeing odd values in the SPI decode. For example at 1s 854ms the driver is trying to send 0x9DD6 six times to write CFG2 register but this is decoding as 0x0DD6 and 0x1DD6 sometimes. From the behaviour this looks to be a capture artifact.

From what I can gather the initialisation is completing successfully and is reading OK status values from each of the STATUS1,2 and 3 registers (it returns 0x000C in all cases). I'm surprised it is posting GATEDRIVEFAULT or GATEDRIVEINITFAIL errors.

I have a gate driver test program for the tesla board that successfully initialises the gate drivers and runs for a few seconds before turning off the gate drive PSU and logging the power good pin and gate driver fault status. This shows the PSU fail and then a gate driver fault be reported shortly after. Perhaps doing something like this would be useful.

[uhi22]

Just to be sure: sampling clock frequency of the saleae is high enough? And SPI mode is the same for controller, gate driver and Saleae?

[crasbe]

Clock looks sensible now. I'm still seeing odd values in the SPI decode. For example at 1s 854ms the driver is trying to send 0x9DD6 six times to write CFG2 register but this is decoding as 0x0DD6 and 0x1DD6 sometimes. From the behaviour this looks to be a capture artifact.

The analyzer settings in the decoder are set to CPHA=0 whereas the bus communication is CPHA=1. Therefore the data is sometimes misinterpreted, even though it is valid.

[crasbe]

From what I can gather the initialisation is completing successfully and is reading OK status values from each of the STATUS1,2 and 3 registers (it returns 0x000C in all cases). I'm surprised it is posting GATEDRIVEFAULT or GATEDRIVEINITFAIL errors.

No, it is returning 0x08 on STATUS1, 0x04 on STATUS2 and 0xE4 on STATUS3 (but the higher bits are unspecified).
0x000C is the CRC of the NOP command that is sent on readout from the STM32.

[davefiddes]

That's correct. I have not had much sleep. I think I need a break.

Edit: One thing I remember from my earlier works with these chips is that the Don't Care bits in the status often assume random values and have to be masked off. The code deals with this but any manual analysis has to also take this into account.

[Jack Bauer]

Okay. 5 sec delay between power on and starting spi coms. No errors.
PWM reaches the drivers but is not passed through.
also set the analyser to CPHA=1.

[Jack Bauer]

Capture with pwm on.

[davefiddes]

All looks happy (really this time) assuming no fault errors being logged.

Amusingly you can see in the trace just how hard the STM32F1 has to work when generating PWM as there are some odd looking gaps between writes to the chips. This is fine by the protocol as I understand it, it's just the minimum that matters and keeping the CS asserted until everything has been clocked through all the chips.

I think the reason you are not seeing PWM on the output is because the ~SD~ pin is still de-asserted. Stick a call to TeslaM3GateDriver::Enable() in an if at the end of TeslaModel3::Initialize():

Code: Select all

void TeslaModel3::Initialize()
{
    if (!TeslaM3GateDriver::Init())
    {
        ErrorMessage::Post(ERR_GATEDRIVEINITFAIL);
    }
    else
    {
        TeslaM3GateDriver::Enable();
    }
}

It would be an idea to remove any stray calls to DigIo::gate_sd_hi.Set() or Clear(). I think it's clearer if you leave it to the gate driver code to control that.

It may start tripping faults again but you should hopefully see something on the gate driver outputs.

[Jack Bauer]

Thanks Dave will give a shot tomorrow. Yeah I recall the gaps in the spi when under pwm from when I ran the modboard. Didnt seem to cause any problems.

[crasbe]

Also the configuration is read back correctly, I just went through the logs and checked.

Does that mean we get to see some motor spinach tomorrow?

[Jack Bauer]

Sadly not. Now with SD set properly as soon as pwm hits they drivers they go into fault.

[davefiddes]

Progress.

At 3.507s in the trace it looks like the DESAT flag is triggered on the even gate drive chips which should be all the high-side drivers (U293, U292 and U291 on the Tesla board). Everything else looks OK.

You should see PWM coming out of the low-side.

I'll try and find some time to write code to get this status information out of the chips and into some spot values.

[crasbe]

Copy&Paste from Discord:

I think I might have an idea. So the DESAT circuit of the high side makes sense:
- GNDISO is at the source potential of the IGBTs through the 56R resistors,
- the drain of the IGBTs is connected to HV+,
- the DESAT circuit is now approximately as in the datasheet, with the addition of the 1k resistor, because the DESAT pin is connected to HV+ via the
VN17 diode (aka. to the drain of the IGBT),
- the JV diode is probably just protection,

The low side of the DESAT circuit is a bit odd, but we follow the same scheme:
- GNDISO is at the source potential of the IGBTs through the 56R resistors,
- GNDISO is also at HV-
- this is probably a bit redundant, because the low side IGBT source should be hard at HV- anyway,
- we don't have direct access to the Drain of the IGBTs,
- however: DESAT is determined by measuring the current, therefore it would be fine to go to the Source of the high side and run the test current through the motor windings as well
- Source of the high side = GNDISO of the high side

My hypothesis is that there is a dot missing connecting the DESAT circuitry of the low side driver to the GNDISO of the high side:

For my assumption to be valid, the low side drivers would have to have faulted out and the high side drivers should've been fine.

I found some documentation suggesting that they actually work as shift registers, BUT they are fed "backwards". The first device in the chain is the low side driver of Phase B. That means the last in the chain (and the first that is read out) is the high side driver of Phase A.
Therefore the second device that's read out is the low side driver of Phase A, which fauled out.

[crasbe]

The documentation: https://kth.diva-portal.org/smash/get/d ... TEXT01.pdf page 50

CK and CS lines are still common for all node, but now, if the master
initiates a message transfer, its data will be shifted to the first device, data of the first to the second and
so on until the circle closes and the master receives the data of the last device in the chain. Repeating
this process the number of times devices there are, the master ends up receiving the content of each slave
device’s register, meanwhile it sent one meaningful message to each one of them.

[davefiddes]

That wasn't my understanding. The way they seem to work is that the commands are processed by the first chip in the chain. If the command is valid (using the CRC I think) the first chip then passes through subsequent bits to the next chip in the chain. The process resets when the ~CS~ pin is deasserted. The result is that the MISO receives the data in the order in which it was sent to the chips (i.e. first to last).

This was verified back in the early days of the reverse engineering with SPI captures. If there is any doubt put a tap on the SDO of the first chip and add that in to the capture.

Unfortunately I don't have a working debug adapter (and other stuff to do) so can't fire up my inverter board to verify. Sounds like you and Damien have things in hand.

[crasbe]

That wasn't my understanding. The way they seem to work is that the commands are processed by the first chip in the chain. If the command is valid (using the CRC I think) the first chip then passes through subsequent bits to the next chip in the chain. The process resets when the ~CS~ pin is deasserted. The result is that the MISO receives the data in the order in which it was sent to the chips (i.e. first to last).

That is for shifting the data into the the gate drivers though. For shifting the data out of the chips, the last chip has to shift out it's own buffer first, because it couldn't have received the data from the previous chip(s) yet.

Therefore the first two bytes shifted out of the chain have to be from the last chip, which is a high side driver, which has no fault. The following two bytes are from the low side driver.

[crasbe]

Damien confirmed that Pin 16 (SENSE) and Pin 13 (GNDISO) of the High Side Drivers are connected on the original Tesla M3DU boards.
That would be supporting my hypothesis.

[Jack Bauer]