When Cardinal Wolsey fell from power, Suffolk immediately found his influence growing on a daily basis. He was acting as High Steward at Anne Boleyn's coronation and was - against his will - send to disband Catherine of Aragon's household (something he found very uncomfortable).
But Suffolk's friendship with Henry VIII was in danger when he married Mary Tudor, Queen Dowager of France without Henry's knowledge or consent. Apparently, Charles and Mary had been in love before Mary's short marriage to Louis XII and was even courted by Francis I himself who often accused Charles of the intention to marry the widow himself. Outwardly, Henry and Francis were pleased at the union but Henry was still waiting for the gold plates and jewels that had been promised to the Queen at her wedding. King Henry decided to use Charles as the means of acquiring this and made it clear that Charles and Mary would not have his blessing before Charles had ensured the promised gold.
The couple knew that it would be difficult and Mary was afraid that she would be used for political purposes again and thus they married in secret on March 5th 1515. After the marriage, Charles informed Cardinal Wolsey of what had taken place and soon the King knew.
The only thing that saved Charles from the immense anger of Henry VIII was Cardinal Wolsey who stepped in and managed to calm the King down. Instead Charles and Mary agreed that they should pay the sum of 24.000 pounds by paying 1.000 pounds every year to the crown. Also, they had to repay Mary's dowry on the staggering sum of 200.000 pounds together with her plates and jewels. In return the couple was officially married.
Charles had already been married twice, to Margaret Neville and to Anne Browne. Anne Browne had died earlier but Margaret Neville was still alive and Charles secured a nullity of their marriage with her consent - he even had a Papal Dispensation sent from Rome to ensure the validity of his marriage to the King's sister. Mary Tudor died on June 25th 1533 and in September that same year, Charles married his young ward, Catherine Willoughby. She was only thirteen at the time and had been betrothed to another but Charles did not want to loose the large amounts of land that she brought with her and married her himself. They had two sons - Henry and Charles - but died in 1551 of the sweating sickness within an hour of each other. Charles also had two daughter from the marriage to Margaret Neville - Anne and Mary - but they were sent to live elsewhere after their parents' divorce.
When Henry VIII decided to invade France again in 1544, Charles was once again put in charge of the English troops. Charles Brandon died at Guildford on August 24th 1545 of unknown causes and his funeral was paid for by Henry VIII.