I read through the logs of the IRC/Mumble meeting that happened yesterday and I saw that there was debate about how feasible full server side CPU emulation was from a performance perspective. Some people suggested that the CPU emulation should be offloaded to the client, but this creates an opportunity for cheating. My idea is a hybrid of server side/client side emulation.
Lets pretend for the sake of this example that 10 is the maximum number of CPUs that the server can emulate without taking a performance hit. While the server population is <= 10 players, the server performs full emulation of all the CPU's on its end. Now lets say that the server population is > 10. At that point it will start offloading the emulation work to the clients, HOWEVER
each client is given more than just their own CPU to emulate. Lets say that the number of CPU's to emulate given to each client is 5 for the sake of argument. This could be an adjustable variable on the client side for slower PCs with a server side variable set at an enforced minimum. If every client is emulating their own CPU plus a minimum of 4 others, then there will be overlap with multiple clients emulating the same CPU. The server waits for consensus from the client side emulation before updating the server data, however, your client doesn't get a vote on your own CPU. The purpose of emulating your own CPU is so that your client can proceed under the assumption that it's not cheating and what it's emulating is correct so you will see updates on your screen as if it was instant without any server side latency. It won't even communicate the results of your CPU emulation back to the server and as long as you are not cheating, you shouldn't notice any desync.
That works as long as a majority of people on the server are honest, but what if a large griefing team, or a single cheater with multiple clients invade a small server? The solution is to have the server continue to emulate 10 CPU's at all times even when it's offloading the emulation to the clients. The server will still wait for consensus before acting on the results of its own emulation, but if its results don't match the results of the consensus, then it will ignore the consensus and use its own results. It can then choose a course of action where it could kick/ban/temp ban/ip ban the offending clients. Perhaps there can be some threshold of number of offences in a certain amount of time before this happens. The server should randomly and rapidly rotate which 10 CPUs its checking so that it's always looking over everyone's shoulder.
Now lets say that whatever reason the emulation states of the CPUs get out of sync, be it from server latency or from cheating, how do you get it back into sync? I'm imagining a system where the emulator takes a snapshot image every X number of CPU cycles and sends that snapshot off th the server. Once the server has consensus on the snapshot, it will store the snapshots in an array of savestates. It will also send the snapshot of the CPU that belongs to a player to the client of that player at regular intervals so that it can sync up with the rest of the server. It will also do this if the server notices one of the other CPUs that you might be emulating goes out of sync. If server latency crosses a certain threshold, the server could chose to no longer ask for consensus for the 10 CPUs it happens to be emulating and it may even chose to emulate more than 10 at the cost of performance if it crosses another threshold.
This idea was inspired by the thread started by Azertyx02. viewtopic.php?f=12&t=516