I've voted 38 times today and can't up/down-vote any questions or answers, getting the Daily Vote Limit Reached message.

Does flagging a comment as Spam count against these? Or any other actions?

Edit: Just notice I downvoted twice today, might that affect the count?
Edit 2: Holocryptic has also reached the "Daily Vote Limit" message, his user page reports 33 votes.