Duminy has been appointed on a permanent basis whereas Kleinveldt has come in only for the West Indies series for now.
Markram recently led Sunrisers Eastern Cape to the inaugural SA20 title and also led South Africa at the Under-19 level.
"I would like to congratulate Aiden on his appointment as captain of the Proteas T20I team," director of cricket Enoch Nkwe said in a release. "Leadership is extremely familiar to him having successfully led on so many levels. He is a player that inspires confidence and he has all the qualities to succeed in the role. We have no doubt will help take South Africa to the next level.
"At the same time, we would like to thank Temba for filling the position so aptly over the past two years. He has done a commendable job during this period and now has a new role to fulfil within the national set-up."
The games against West Indies will be Walter's first series with the South Africa team. "This is a squad that is high on confidence after beating one of the best white-ball teams in the world, England, in their most recent series," he said.
"I am also excited to have Aiden take the reigns as T20I captain as he is very familiar to the setup and works well with those around him. He is a player that leads from the front and we have no doubt he will take this team to the next level in the years ahead."
South Africa's T20I squad for West Indies series: Aiden Markram (capt), Quinton de Kock , Bjorn Fortuin, Reeza Hendricks, Marco Jansen, Heinrich Klaasen, Sisanda Magala, David Miller, Lungi Ngidi, Anrich Nortje, Wayne Parnell, Kagiso Rabada, Rilee Rossouw, Tabraiz Shamsi, Tristan Stubbs.
South Africa's squad for first two ODIs: Temba Bavuma (capt), Gerald Coetzee, Quinton de Kock, Tony de Zorzi, Bjorn Fortuin, Reeza Hendricks, Sisanda Magala, Keshav Maharaj, Wiaan Mulder, Lungi Ngidi, Ryan Rickelton, Andile Phehlukwayo, Tristan Stubbs, Lizaad Williams, Rassie van der Dussen