The Virtual computer have a Hardware Controller. But you keep falling in the same pitfall. A device (the Hardware controller) messing with the internal registers and states of the CPU. This cann't be allowed, and not happen in real world.
In real worl examples, you have a Hardware Controller (for example in PCI bus) that negotitate wiht the devices what address will be assigned for each deviced (and other resoruces), and gets a list of each device, but to talsk with the Hardware Controller you keep using a Memory Mapped I/O (or a Port Mapped I/O).
Really, avoid the HWx thing, only make more complex the CPU replacing something like : LOAD %r0, [HWQ_ADDR] by a HWQ %r0