According to Vinsmoke Yonji in Chapter 840, Vinsmoke Judge was once part of an illegal research team along with the genius Dr. Vegapunk.
However, Chapter 995 showed another possible member of this research team, Queen the Plague. Queen recently mentioned the name Vinsmoke Judge, however, the difference here is that everyone else so far in the manga who recognized Sanji’s family through Germa 66 has referred to them as Germa 66.
Queen specifically mentions Judge’s name and judging by Queen’s genius with science, he probably knows Judge through his days with Dr. Vegapunk.
Anyways, back to Chapter 840, Yonji mentioned Vegapunks illegal research team was arrested and absorbed by the World Government. This is barring Judge who escaped and instead returned to the country of Germa and became its King.
The rest in terms of Vinsmoke Judge, is well known history. Judge and his wife Sora first had Reiju. Judge altered Reiju to have enhanced strength, speed, metallic skin, and become a superhuman. However, at this point Judge decided he wanted to take it a step farther. He wanted perfect soldiers, and in doing so next time Sora was pregnant Judge attempted to alter their bloodline elements as well not only increasing their physical stats akin to Reiju, but to also erase their emotions. So Sora took a drug in hopes to stop this; however, the drug only effected one of the quadruplets.
The surgery that caused the superhuman abilities in the Vinsmokes did not show immediate results; rather, the children slowly began to change and once this happened Judge realized Sanji was not a super being.
But why did Sanji not change? The drug made Sanji resistant to the change to his DNA, or at least stopped the changes from manifesting. There are two ways this could happen, the way I believe is that the drug gave Sanji what are in essence hormone blockers stopping the changes from manifesting. The other way is that Sanji’s body was rewritten by the drug to make him a normal human.