The annual Army v Navy rugby match may be forced to move to a different venue over concerns fans see it as "a drunken, al fresco fancy dress party".
After Saturday's game, residents around Twickenham Stadium were said to have been upset by fans passing out in the street and urinating in gardens.
Gareth Roberts, leader of Richmond Council, said the area was being "trashed" by those attending.
The Rugby Football Union (RFU) said it would look at the complaints.
For the third year in a row, the Army won the match, but it was marred by reports of incidents including attempted theft and a woman taken to hospital after being hit in the head with a bottle.
Mr Roberts said he wanted assurances from the RFU that residents would "not experience another day of drunken, loutish behaviour which brings the RFU and our armed services personnel into disrepute".
"Match attendees now see this event primarily as a drunken, al fresco fancy dress party with the rugby match itself being of secondary importance," he said.
Mr Roberts said those in the area around the 80,000-capacity stadium, which also hosts England international matches, were "proud to live in the home of rugby".
"They are also proud of this borough's longstanding connection to our armed forces," he continued.
"However, when they see their town being trashed by people attending the Army v Navy game they find their pride and goodwill being put to the test.
"We have long held the belief that, if it is to continue, then the fixture should be rotated among other stadia and we will be putting that case to the RFU."
You may also be interested in:
A spokeswoman for the RFU said "significant steps" were taken "to improve this event including reduced licensing hours, earlier kick-off and bar closures plus no games before the main match".
She added that "behaviour at this year's match was better than previous years".
"We will have a full debrief of this event with the police and the council as we do every year and will look at every complaint individually," she said.
The Met Police said four people were arrested for offences which included assault and public order related matters.