During my time working on Vox Machinae the single largest task I accomplished was to implement a version of PhysX to replace Unity's implementation. At that time the studio was working diligently towards the 1.0 full release of the game to get it out of early access. Part of that 1.0 milestone was to bring the game to Meta Quest devices. Oh boy, did we need to eek out every bit of performance possible.
Why?
The decision to move to our own PhysX implementation was two fold.
Firstly, Unity's PhsyX implementation will always block the main thread. Unity has multithreading built into their Rigidbody settings when it comes to how Unity components interact with PhysX. However, there is a call into PhysX with the signature (if memory serves) PxScene::fetchresults(bool). This singular function call made by Unity, and the serialization it is doing behind the scenes to make threaded data available, will cause frame drops on low end devices. For all intensive purposes, a Meta Quest device is just a glorified android phone that's running VR applications. It needed to change.
Secondly, removing Unity based physics objects allowed the game's server side code to be completely decoupled from Unity. Starting an offline game no longer adding stress to the main thread? Sign me up. Especially for ARM devices where cpu's tend to have more cores (RISC ftw). The less you run on Unity's main thread, the better. It has enough graphics rendering stuff to do on it already.
TLDR: Performance.
Scope
In Vox, all entities are built of physics bodies connected together with constraints. Physics is at the core of the game. It is the reason for the weighty feel of the grinders when they crash into each other and the terrain. The point I'm expressing here, is that it is a highly embedded system with an enormous number of moving parts.
At the time, us programmers believed that this should be a simple month long task. Just build PhysX from source and translate the Unity PhysX cruft to own own stuff. Simple right? Should just be a drop in replacement. 1 to 1 translation, right? No. Not even close. Unity's PhsyX implementation does plenty of magic under the hood when it's setting up your constraints and physics interactions. There is no 1 to 1 translation between Unity's physics components and PhysX's. Although extremely similar, enough doesn't line up to make it a simple drop in replacement.
Also, being so intensely embedded into the game, threading the entirety of the server as part of the conversion certainly added to the scope of the task.
What we thought was a 1 month task, quickly turned into a 4 month marathon with a looming release deadline on the horizon.
TLDR: Took 4x longer than we thought.
// TODO:
If reading the above doesn't deter you from implementing PhysX in Unity your-self, here's the check list of things to do.
First, contact and sign off on the Nvidia PhysX contract and compile the PhysX DLL's and libs from source. I've had to do this for both x86_64, and ARM64. I can also confirm that it does work when compiling for Playstation 4/5, but that's a whole other can o'worms.
Second, since we're using Unity, we need our own little wrapper to call into the PhysX libs. Depending on how you built the libs, you may want to have a mix of custom C and C# driver code that calls into the PhysX libs. Regardless, you'll need to wrap it all in preparation for use.
Third, despair as you realize that PhysX actors are the equivalent to rigid bodies and how much work you have to do to make code that generates complex sets of actors with attached collision shapes and joints. This is where you'll spend most of your time. In the case of Vox, we did not change the prefabs that represented our entities. Instead, I wrote code to translate Unity prefab hierarchies into sets of actors that, ideally, did the same thing. This is where no 1-1 translation exists and you need to fudge a-lot of numbers. A word of caution, write your own monobehaviors that match PhysX components exactly under the hood. Do not translate values as will make your life easier. Yes, you'll have to replace/copy-paste most values, but it will GREATLY simplify things.
Fourth, remember that you need to multithread the server and break everything while trying to do it. Ton's of Unity api calls require main thread access, so now you get to cache volatile values all over the place.
Fifth, get rid of all the stupid race conditions you left in.
Six, find out some things behave differently on android devices and spend hours figuring out why. (Check your bit alignment, trust me, one of your structs probably isn't doing what you think it is.)
Seven, tweak the existing assets endlessly and realize that it'll never be quite like it was.
And bam, you're done. A gross oversimplification, but I hope I've conveyed the feeling, there's more than you think //TODO:
TLDR: The list is always longer than you think. Account for it.
What I would change
If you find yourself in the unfortunate position where you need to replace an existing heavily embedded system, consider the following.
Does it really need to change? In our case, we might have been able to reduce physics complexity on a per platform basis. This would have eliminated the need to change the physics. Thus allowing the team to better focus on making the parts of the game that matter, better.
How will the change influence the existing systems? In our case, it was everything. Breaking that question down to each thing the physics touched would have helped. Mainly to see that there weren't 1 to 1 translations revealing the extra work required to make the change. Itemizing these might also help reveal the true time investment.
Had I asked those two questions and done my own deep investigation into how this changeover would be executed, I likely would have been able to shave a month off the total development time.
TLDR: Ask yourself, does it really need to change and how will the change influence each individual existing system it touches?