Thanks, I'll look at the box-tree crate, but I'd say using indices instead of pointers is somewhat unsatisfying. C++'s advertised property is zero-cost abstractions, and Rust is supposed to have C++'s performance without the footguns. But if you compare those vector index operatons with primitive pointer dereferences, there are a bunch of extra instructions run at each operation, including a layer of memory lookups that can cause cache pressure and misses.
I think the most practical answers I've heard so far are to 1) use Rc/Arc and weakrefs, or 2) use unsafe operations on machine pointers in a hopefully carefully contained and abstracted part of the code.
Sorry about the slow responses as I'm occupied with RL stuff most of the time for now.