Synovus will keep the deposits held by the unit and resell its credit-card portfolio to Capital One Financial Corp (COF.N), the source said.
The unit, called World's Foremost Bank, was supposed to be bought by Capital One last year, but the deal wasn't able to get timely regulatory approval.
The unit, which provides branded credit cards, had nearly 2 million accounts with about $5 billion in loans and $502 million in revenue in 2015.
Cabela's in October last year agreed to be bought by rival Bass Pro Shops in a deal valued at $5.5 billion, uniting two of the country's largest hunting and fishing retailers.
Cabela's later said it would not be able to close its sale to Bass Pro in the first half of this year, as originally expected, because of approval delays.
The Federal Trade Commission, which regulates and enforces antitrust laws, had also sought more information from the companies about the deal.
Cabela's, which derives a significant amount of its business from gun sales, attributed part of its 2016 challenges to the delayed effects of a slow-down in gun sales following a shooting attack San Bernardino, California.
The hunting gun industry has been experiencing steep swings in sales amid uncertainty around gun law changes.
Cabela's stock, which has fallen more than 20 percent since the beginning of the year, were down 2.4 percent at $50.80 in extended trade on Thursday.
Columbus, Georgia-based Synovus Financial's shares closed at $39.84 on the New York Stock Exchange.
The Wall Street Journal first reported news of the agreement.
Reporting by Nikhil Subba in Bengaluru and Lauren Hirsch in New York; Editing by Maju Samuel and Shounak Dasgupta