9 times out of 10 when people upgrade from ESX 3.0x to ESX 3.5 they do an in place upgrade and leave it at that.
Step 1 - upgrade VC
Step 2- Upgrade host 1
Step 3 - Upgrade next host
Step 4 - Rinse and Repeat

 Seems pretty straight forward right, well not exactly because you should really be considering a cold migration between your old LUNS and a new LUN created under ESX 3.5 at upgrade time for your virtual machines. This is because the VMFS file system was upgraded in the 3.5 release, and the upgrade actually made some changes to the way VMFS handles SCSI locking.

This was stolen from the VMTN forums, but rather than re-invent the wheel :

• ESX Server 3.0.2 used VMFS 3.21 and ESX Server 3.5 uses VMFS 3.31.

• The differences between VMFS 3.21 and VMFS 3.31 are mainly internal architecture, bug fixes, and small block size support (as low as 128K).

• Upgrading from VMFS 3.21 to VMFS 3.31 is not possible without reformating the volume. However, you do not need to move the VMFS volume to 3.31 to run ESX Server 3.5.

• The volumes are compatible. In other words, you can not upgrade a VMFS volume from version 3.21 to 3.31, but you can still upgrade the ESX Server to ESX Server 3.5 and continue to run on a VMFS 3.21. This means you can also run VMs from ESX Server 3.0.2 directly on VMFS 3.31.

 VMFS 3.31 in 3.5 introduced a distributed locking optimization - optimistic locking.

•     Basically, the actual acquisition of on-disk locks (involving SCSI reservations) is postponed as late as possible in the life cycle of VMFS metadata transactions.
  

• Doing so allows the number and duration of SCSI reservations to be reduced, thereby reducing their impact on VM I/O and VMFS metadata I/O originating from other hosts that share the volume.

• You may see the message - Optimistic Lock Acquired By Another Host - it means that a lock which was held optimistically (not yet acquired on-disk) during a transaction was found to have been acquired on-disk by a different host. In this case, we simply abort and retry the transaction.

 Source = http://communities.vmware.com/message/963043;jsessionid=7AC7B31D4BF678B3E0653EBF1C8667F2#963043