CalDAV supports LOCATION property, but this must be typed every time again at VEVENT creation. Could VEVENT's or VTODO's be linked to a CardDAV contact ?
This way, you have the most recent contact info at hand (venue location, cell phone in case you run late)
The ATTENDEE property is not usefull, since a) this only displays the e-mail address, and b) you don't always want to send an invitation (imagine your hairdresser receiving invites every time his customers enter their appointments in their calendars)