Your magnetometer/compass is not working with INAV? Is the sensor red in Configurator? You are not the only one... Not only most popular compass HMC5883l is discontinued and what you get are Chinese clones, chip labeled 5883 might not be HMC5883l but something completely different. And I2C bus is very very picky and likes to give up on you....

This is how you might fix your problems: add pullup resistors for I2C SCL and SDA lines and/or lower i2c_speed...