Yes, it's because he won almost all of the points in the second set. Another reason could be that when he won games he won them without you winning a point and you won them with him getting 2 or more points. Just realize that he also won 2 more games total than you did.
2010 Singles: 66-10 (no wo (4))
2010 Doubles: 21-11
2009 Singles: 105-14 (no WO (6))
2009 Doubles: 11-8
'10: W: AO, Miami, Casa, Barca, Rome, Stuttgart, Hamburg, Cincy
'09: W: 13 including Rome, Madrid, RG, Wimby, Cincy, USO