Did Kelly Pavlik in taking the Paul Williams fight duck Arthur Abraham (the IBF Middleweight champ)?

Abraham said he is moving up to Super Middleweight because he could not get a fight with Pavlik or Felix Sturm (the WBA champ). Pavlik is putting his title on the line against Paul Williams who has never fought at 160. Does Abraham a legit Middleweight have a legit beef here.?