CHICAGO -- The St. Louis Cardinals and Chicago Cubs wasted no time trading blows in the first game of their crucial four-game series.

Then, the Cardinals' Matt Carpenter delivered a haymaker that put the rival Cubs on the ropes.

Carpenter homered in the 10th inning against Craig Kimbrel in the All-Star closer's return from the injured list, and St. Louis regrouped after blowing a late lead to beat the Cubs 5-4 Thursday night. The teams will meet six more times in the final 10 days of the regular season.

In the midst of a disappointing season that saw his role diminish, Carpenter emphatically pointed to no one in particular in the visitors dugout upon contact.

"I mean, it was a gut punch in that inning before," Carpenter said of a squandered 4-1 lead. "We had that game locked, then it squeaked away from us. They tied it up. Just the emotional turn to be able to get the lead again then look in there and the excitement just fired up everybody."

Carpenter homered to deep center field off Kimbrel (0-3), who was activated before the game after dealing with right elbow inflammation. Chicago dropped a game behind Milwaukee for the second NL wild-card spot and four games behind St. Louis for first place in the NL Central.

"I'm really disappointed in myself to go out there and give up a home run like that," Kimbrel said. "Season's not over. We still have a lot of games left."

Giovanny Gallegos pitched a perfect 10th for his first save in two chances. Andrew Miller (5-5) recorded the final two outs in the ninth for the victory.

The significance of an emotion-charged road victory in the series opener with emerging ace Jack Flaherty on the mound wasn't lost on Manager Mike Shildt and his team.

"Yeah, a win is a win is a win, but that's an important win for sure," Shildt said. "I mean, you start the series off and knock that first one down and look forward to tomorrow."

Cubs slugger Anthony Rizzo provided a jolt in his unexpected return from a sprained right ankle, homering in the third inning to tie the score at 1. But Flaherty was spectacular otherwise, pitching three-hit ball while giving up only the one run and striking out eight. He left after 118 pitches with a 6-2 record with a 0.93 ERA in 10 starts since Aug. 1.

The Cubs finally broke through against closer Carlos Martinez in the ninth, which Nicholas Castellanos led off with a walk then advanced to second on Kris Bryant's single. After Kyle Schwarber was called out on strikes, Ben Zobrist hit an RBI double just inside the right-field line. Willson Contreras followed with a slow roller on which Martinez threw late to the plate while Bryant scored from third.

Jason Heyward tied it with a groundout against Miller that scored pinch runner Javier Baez from third.

"I was really proud of the guys, the way they came back at the end and had a chance to win," manager Joe Maddon said.

Tommy Edman lead off the fifth with a triple against Cubs starter Kyle Hendricks for the first of his two hits. Harrison Bader followed with an RBI single to snap a 1-1 tie.

The Cardinals took a 4-1 lead with a pair of runs in the sixth. Schwarber bobbled Yadier Molina's single in left field off Hendricks for a run-scoring error. Bader hit an RBI double off reliever Brad Wieck one out later.

RIZZO'S RETURN

Rizzo left a game against Pittsburgh on Sunday because of a sprained right ankle and was expected to be in a walking boot for up to a week. Shortly after he warmed up in the outfield while team medical personnel looked on, he was inserted into the leadoff spot and tabbed to play first base about 20 minutes before first pitch.

Rizzo hit the first pitch by Flaherty in the third inning into the left-field bleachers to tie the score at 1, and fans at Wrigley Field celebrated wildly as he slowly circled the bases.

TRAINER'S ROOM

Cardinals: INF Kolten Wong left the game in the fifth inning because of a left hamstring injury. He is day to day.

Cubs: INF Addison Russell was activated after a 10-game absence while in concussion protocol. ... Baez (fractured left thumb) was cleared for baseball activities and available to pinch run with the aid of a protective mitt. ... RHP Brandon Kintzler (strained left oblique) was expected to take part in a full bullpen session in the next day or two after he reported no setbacks in a 15-pitch session Wednesday.

UP NEXT

St. Louis RHP Michael Wacha (6-7, 4.76) and Chicago RHP Alec Mills (1-0, 3.42) are the scheduled starters Friday. LHP Jose Quintana was set to start for Chicago, but his assignment was pushed back to Saturday, when LHP Cole Hamels (fatigued left shoulder) had been scheduled to pitch. Hamels was scratched but hopes to rejoin the rotation next week.