A last-gasp Christian Benteke penalty handed Tim Sherwood his first win as Aston Villa manager and lifted his side out of the Premier League relegation zone with a 2-1 home victory against West Bromwich Albion on Tuesday.
Sadio Mane struck late to rejuvenate Southampton's fading battle for Champions League qualification with a 1-0 home win against Crystal Palace to move them into fifth, while Sunderland came from behind to draw 1-1 at Hull City.
Aston Villa, who have never been relegated during the Premier League era, started the day second bottom and on a seven-game losing streak, their worst run for 52 years.
In his third game in charge, Sherwood avoided equalling Dick Taylor's 1964 record of losing his opening three league matches in dramatic circumstances.
Villa nearly took the lead when West Brom goalkeeper Ben Foster spilled a routine save but he managed to claw the ball to safety, with goalline technology proving it was a whisker away.
Gabriel Agbonlahor struck his first goal in 14 games to open the scoring after 22 minutes, latching onto Benteke's knock on to slot the ball through the legs of Foster, sparking wild celebrations from Sherwood.
Fabian Delph nearly extended their lead but he hit the post with a long-range, curling effort before the visitors equalised after 67 minutes when Saido Berahino headed in from close range. It was the striker's 12th league goal of the season.
Hapless Foster
Just as the match looked to be heading for a draw, hapless Foster fumbled the ball again and, in a desperate attempt to rectify his mistake, brought down Matthew Lowton to concede a penalty.
Belgium striker Benteke stepped up to coolly roll the ball in from the spot four minutes into added time.
"We kept at it and someone was looking down on us today. I'll try anything, we are at that stage," Sherwood told the BBC.
"We hadn't won in 12 Premier League games and had to buck that trend. I think we deserved the three points and onwards and upwards."
It was Villa's first win in 13 league games and lifted them into 17th place on 25 points, three above the relegation zone.
The two sides will meet again in the FA Cup quarter-final at Villa Park on Saturday.
Southampton, who had won once and scored just one in their last five games prior to kick off, ended a wait of over six hours for a goal at St Mary's when Mane struck in the 83rd minute.
In what had been a match low on quality, Mane produced a deft chip over Julian Speroni after the Palace keeper had parried a shot from substitute James Ward-Prowse.
Victory lifted Southampton to 49 points, one point outside the top four and a spot in European soccer's elite club competition next season.
"It was crucial game to win today, now we have 10 days until the Chelsea game (at Stamford Bridge) and it makes the situation a lot more comfortable and I hope they will get total confidence back and play like we did before," Saints boss Ronald Koeman said.
Hull City's Dame N'Doye, a deadline day signing, scored his third goal in five games to give them the lead against Sunderland with an exquisite flick after 15 minutes.
The visitors then had manager Gus Poyet sent to the stands after a heated touchline exchange with his opposite number Steve Bruce.
Jack Rodwell headed in a equaliser with 13 minutes left to keep Sunderland in 16th place on 26 points, four above the relegation places and one point below Hull.
