Based on past performance, the Carolina Panthers seem to give the Patriots the hardest time. Cam Newton is 2-0 vs the Pats. They seem to have a real hard time dealing with his ability to improvise and extend plays, more so than anyone else who plays that style of game.