I'm not watching the Olympics except for when it's on at the YMCA while I'm working out, and the one thing that has stood out to me during whatever competitions are on (this morning was women's volleyball) - there's a hell of a lot of empty seats.
Apparently events are sold out but people just aren't going? I saw this happen at the NCAA Men's Basketball first round in Anaheim this year as well, and I think it's truly screwed up. The idea that people can afford expensive tickets - and particularly the good seats that you can see on a televised angle for a sport are good tickets - and don't bother to show up should be unacceptable to amateur sports. I suspect that if you lowered the price of the tickets to something that the average person could easily afford, you wouldn't have this problem.