Garp adopted Ace from Gol D. Roger, who requested Ace does not deserve to be treated as a criminal just because of his family ties.
While Garp initially thought of adopting Ace as a preposterous idea, he eventually agreed to Roger’s final wish, as the latter pointed out that after multiple confrontations, he trusted the vice admiral much as a friend.
When Ace was an infant, Garp would use his vacation and often visit the village and pull funny faces to cheer him up.
Later when Ace was a troubled child that beat anyone up that insulted his father, Garp seemed amused with Ace, yet Garp never scolded Ace for his actions as a part of him felt sad that the child did not fully accept Roger as his father. After Luffy, Sabo and Ace got along well, Garp would sometimes visit them and train them by beating them up and destroying most of the forest in the process.
As well as Luffy, Garp had high expectations for Ace to be a strong Marine, but is disappointed that he chose the life of piracy, which incidentally sent him to Impel Down where they were talking about the issue.
However, Ace disagreed with Garp and remains grounded on the idea that becoming Marines would not have been possible when both of their fathers were world class criminals.
Garp, in return, showed extreme sadness about Ace’s execution. He claimed he had no pity for a pirate but believed family is different. He cares about Ace deeply, crying for his grandson as he’s about to be executed for being a pirate and desperately asking why he could not have just been a Marine. In fact, Garp wanted to kill Akainu as revenge for killing Ace.
