Here is my hardware:
3x R610 Servers with 4x onboard broadcom gigabit interfaces and 4x riser intel gigabit interfaces
2x Dell 6224 iSCSI managed L3 switches with 20Gbps stacking cables (iSCSI and Backup)
1x Dell Equalogics SAN (24X 2.5″ 600GB SAS) P4100X
1x Cisco 3560X switches (LAN)

And my situation:
I needed to get the 3 R610 servers to communicate to the SAN LUN(s), a backup network, and the user LAN via the above equipment list. And it needed to be fairly quick – iSCSI over ethernet with bonded gigabit connections quick.

My configuration per proxmox server:
prox_lan_setup.txt
I made it a text document as that’s a lot easier to see.

Unfortunately I learned that Equallogics SAN will not use Bonded connections – they want MPIO. My configuration now has MPIO for iSCSI and LACP (802.3ad) for the LAN.