I completely agree that there are legitimate reasons to want to undo votes on posts, such as clicking by mistake or changing your mind after some time.
As I wrote in a comment, a vote on a post cannot be canceled except for certain circumstances. The official explanation for this restriction, which I cannot find where I saw for now, is “Allowing unlimited undoing would open up the possibility of gaming the system.” Namely, if you want your answer to be voted up, you could vote down the other answers so that your answer will be shown on the top of the page (because the other answers have now negative score) and lazy users will read only your answer and upvote it. The downside of this strategy is that you lose 1 rep point every time you cast a downvote, but if you can undo the votes, you can cancel your downvotes after a week or so when no one cares what happens in a question which does not appear on the top page. Moreover, canceling the downvotes hides the evidence that you have cast many downvotes (the number of upvotes and downvotes you have cast is available on your profile page).
Many (seemingly silly) restrictions on Stack Exchange websites like this originate from the same logic of preventing gaming the system. I guess that these restrictions came from the real experience by the admins at Stack Overflow Internet Services, Inc., on Stack Overflow and other sites which have longer history than ours. I appreciate the fact that the admins put their experiences to form the whole Stack Exchange network. However, I am worried by their general tendency of overreacting to the fear that someone may game the system. I believe that different communities have different priorities and that earning the rep points is not considered to be as important in cstheory.stackexchange.com as in more popular sites such as Stack Overflow. I understand that it is sometimes difficult to prevent abusing without ruining the usability for legitimate users, but I hope that they will take the difference like this into account to design the system.