Wiring harnesses are not the same.
In theory, you can swap all years 4tec motors with one another. However, there are several critical things that must be observed and changed during the swap. There were a bunch of changes to the system in the 06 year like TOPS system, pick up coil position, crank position dowel, head oiling system, electronics, etc.
If you have a good complete 04 motor and putting into an 06, it is very straight forward. You will need to remove the pto housing and drill a 1/2" hole above the counter balance for the newer 06 TOPS system. This can be done without splitting the block and drilling through the hole in the CB gear but not really recommended although I have done it and it works fine. You will need to change the front oil pump assy from the 06 to the 04 for the TOPS as well. That's it.
I don't recommend swapping all electronics over. The biggest problem will be the gauge as you will need to swap the entire hood because the gauges are completely different and not interchangeable.
Overall, not a difficult swap but you need to know what you are doing. Would be much simpler if you used an 06 and newer motor where it is a simple drop in.