For me performance is the biggest issue. I only have 3,800 members and 4,000-6,000 page views a day and the amount of memory used is huge (mostly by the 'Active Members' widget and activity stream).

Not sure why but spam has never been a problem, maybe I have just been lucky. I do use Akismet and Cloudflare so maybe that is why.