I would not think the seals would have that much friction.
Did you have the crank out of the case when you rebuilt it?
It would be possible if the seal was to wide and hitting on the bearing to cause it to bind some.
If push comes to shove and you do not see any other reason then pull the seal and measure it to see how bad the seal is rubbing on the bearing if at all.