Ulster secured their second Pro14 win of the season but captain Iain Henderson was shown a late red card.
Henderson was penalised for a dangerous shoulder to the head of Ospreys full-back Dan Evans and is now in danger of missing some of Ireland's autumn campaign.
Tries from Jacob Stockdale and Marcell Coetzee and scrum-half John Cooney, who contributed 14 points, sealed the win.
Ospreys could only respond with four Stephen Myler penalties.
Ireland host Italy in the rearranged Six Nations game on 24 October before travelling to face France seven days later with the Ulster lock now facing the prospect of missing both games.
Henderson's rash act was in front of British and Irish Lions coach Warren Gatland who was watching less than 24-hours after being awarded a CBE in the Queens Birthday honours list.
One of the players he would have been keen to see was Wales lock Alun Wyn Jones who also received an OBE while Justin Tipuric, Rhys Webb and Stockdale are early tour contenders.
Ospreys named an unchanged side from the opening victory at Edinburgh but they could not emulate that performance with an error-strewn showing in Swansea.
Ulster scored five tries as they beat Benetton 35-24 in their first game of the season. The province suspended all training at the club on Wednesday after one senior squad member and an academy player tested positive for Covid-19.
They were cleared to play in Swansea after their latest PCR testing returned no further positive results.
It was Stockdale who powered over for the opening try, latching onto a flat Ian Madigan pass. John Cooney converted.
Tipuric then won a turnover penalty which Stephen Myler slotted over to open the Ospreys account before Cooney restored Ulster's three-point lead.
Ulster's scrum-half missed a long-range penalty before Myler slotted over his second kick to reduce the half-time deficit to 10-6.
Stockdale was causing havoc in the Ospreys defence early in the second-half with his surging bursts and clever chip kicks.
The dominance was rewarded with a well-worked move and try for impressive number eight Coetzee who had also been prominent in the build-up.
Cooney, left out of the Ireland squad this week, converted before missing a second penalty while Ulster were guilty of not capitalising on attacking dominance.
Myler slotted over two penalties to reduce the deficit to five points but the hosts' hopes of overhauling the visitors was hampered by a malfunctioning lineout.
Ulster capitalised with an excellent third try when wing Rob Lyttle went scything through the Ospreys defence before an inside pass to Cooney; who also converted.
However, Henderson's late red card soured events with Scottish referee Mike Adamson making his decision after viewing television replays of the incident at a ruck.
Ospreys head coach Toby Booth said: "It was a bit of an arm-wrestle. We won the arm wrestle last week, we lost it tonight. It's as simple as that.
"You'll hear loads of coaches talk about the ability to put people under pressure. We couldn't put Ulster under enough consistent pressure.
"They're a top side, the second-best side in the league on last season's performance, and they are going to have parts of the game where they are on top of you. If you are not resilient, you get some of what you saw tonight."
Ospreys: Dan Evans; Mat Protheroe, Owen Watkin, Kieran Williams, Luke Morgan; Stephen Myler, Rhys Webb; Nicky Smith, Sam Parry, Tom Botha, Adam Beard, Alun Wyn Jones, Olly Cracknell, Justin Tipuric (c), Morgan Morris.
Replacements: Dewi Lake, Rhodri Jones, Ma'afu Fia, Will Griffiths, Dan Lydiate, Harri Morgan, Josh Thomas, Tiaan Thomas-Wheeler.
Ulster: Jacob Stockdale; Matt Faddes, James Hume, Stewart Moore, Rob Lyttle; Ian Madigan, John Cooney; Jack McGrath, Rob Herring, Marty Moore, Alan O'Connor, Iain Henderson (capt), Matthew Rea, Marcus Rea, Marcell Coetzee.
Replacements: Adam McBurney, Eric O'Sullivan, Gareth Milasinovich, Kieran Treadwell, Greg Jones, David Shanahan, Louis Ludik, Sean Reidy.
Referee: Mike Adamson (SRU)
Assistant Referees: Dan Jones, Gwyn Morris (both WRU)
TMO: Ian Davies (WRU).