El final de la guerra le encontró operando en el frente oriental al mando del JG 52. A pesar de que eventualmente acabó rindiendose a los estadounidenses en Pisek, el 8 de Mayo de 1945, fue entregado a los soviéticos, y habría de pasar los proximos 4 años en el gulag.
Siegfried Knappe, quien fuera oficial de operaciones de Weidling en Berlin, le conoció en el campo de Kranogorsk. Releyendo sus memorias (Soldat: Reflexions of a German Soldier) encontré este comentario:
Courage and cowardice were very different in the prison camp from on the battlefield. In the camp, I witnessed behavior by people I would not have expected it from on the basis of the person's war record. But nobody can predict how he will react to either combat or captivity.
We had a Luftwaffe fighter pilot in the camp, Oberstleutnant Graf, who had shot down almost four hundred enemy airplanes in combat*. As one of the most successful fighter pilots in the German Air Force during the war, he had all the highest decorations for bravery. In his fighter plane, he could not have been a coward. Yet he just could not take the psychological stress of life in the prison camp, and he caved in to the Russians. He did everything the Russians asked him to do, because he was afraid they would never let him go home or would punish him in some other way if he did not. A cold and distant man who seemed to look defiant and apologetic at the same time, he was a loner with no close friends. He obviously felt he was doing what he had to do to survive and return home. He worked in the kitchen-his reward for cooperating with the Russians-so he ate better than the rest of us, although food did not seem to be his primary motivation.
He did not behave defensively, and he would rationalize his actions if provoked to do so. He could put up a good argument about not feeling loyalty to a country that no longer existed, but when he gave in to the Russians he lost his self-respect and the respect of his peers. An obviously intelligent man, he would probably have gone far in a victorious German Air Force. He never became an Activist**, but he lent
his name to articles that were published throughout the Soviet Union criticizing the German Air Force and the air forces of the western nations and praising the Russian Air Force.
(* en realidad sus victorias suman 212)
(** "anti-fascistas", aquellos prisioneros que abiertamente colaboraban con los rusos y hacían proselitismo)
En 1949 Graf fue liberado. A pesar de todo, se rehusó a formar parte de las fuerzas aéreas de la Alemania Oriental. En 1988 falleció en Engen, su pueblo natal.
