MILWAUKEE -- Chicago Cubs manager Craig Counsell was booed three times Monday afternoon in his return to Milwaukee after managing the Brewers for the previous nine seasons.

Counsell, 53, was booed during a short video montage, which ended with a picture of him in a Brewers uniform. It happened again after his name was announced during starting lineups and a third time during a pitching change in the eighth inning.

"The fans are here to enjoy the day, enjoy a baseball game," Counsell said after the Cubs lost to the Brewers 5-1. "They get to do what they want. Hopefully they had a good time."

Counsell surprised the baseball world -- and shocked Brewers fans -- when the Wisconsin native left the only team he has ever managed for the Cubs this past winter, signing a five-year, $40 million contract.

Monday was his first day back at American Family Field since leaving.

"The view from the dugout is certainly different," Counsell said before the game. "This is going to be an interesting day."

Counsell said he first began to have thoughts about moving on from the Brewers a couple years ago when his boss at the time, president of baseball operations David Stearns, stepped down. Stearns resurfaced with the New York Mets, where Counsell interviewed during the offseason.

"David Stearns leaving made me think about it a lot," Counsell said. "I don't know how it couldn't. It happened over time."

There were discussions about Counsell returning, but once his contract with Milwaukee ended last October, the Cubs came calling after David Ross was fired and a deal with Counsell was quickly reached.

"Life takes different turns," Counsell said. "I don't want to plan out my life forever. I want to do things that challenge me, that excite me. You have to take the ride of life and see what happens. This is not something I necessarily expected to happen, but you have to jump on the ride and go."

Counsell spent the final five seasons of his playing career in Milwaukee before eventually joining the front office and then becoming their manager in 2015. In all, he spent 18 years with the Brewers organization, taking Milwaukee to the playoffs five times as manager.

He was largely criticized in Wisconsin after accepting the Cubs' offer. Then he was booed again on Monday in his return.

"It bothers you, initially," Counsell said of the winter criticism. "But as it goes on ... you're a fan and you can feel how you want to as a fan. That's part of it. It's what makes sports fun.

"It's not my job to tell people how to feel about something or to even figure it out. Let people feel how they want to feel. I'm good with that. It doesn't have to be all positive. We're in a public job with fans. Fans can feel how they want to feel."

Right now, his successor and good friend Pat Murphy has the upper hand on Counsell as the Brewers are in first place in the National League Central while the Cubs are in second and have been scuffling of late.

"It's a team that's playing really well," Counsell said of the Brewers. "We are not playing very well right now. We're trying to get things on track."

Both teams have been hit with injuries, but the Brewers have thrived thanks to a surprisingly resourceful offense which ranks third in runs scored. Meanwhile, the Cubs are led by two top-notch starters -- ERA leader Shota Imanaga and Javier Assad.

Milwaukee began the four-game series with a 3-game lead on the Cubs in a rivalry which should grow even more now that Counsell has switched sides.

And while Monday may not be the last time he's booed at his former place of employment, Counsell won't let it change how he feels about where he grew up and worked most of his adult life.

"I live in Wisconsin," he said. "I enjoy where I live. That's all really good."