It's sinful to desire a woman who isn't your wife. That's one of the ten commandments, and Jesus reiterated it during the sermon on the mount. Having sex, on the other hand, is tolerated by Paul because he doesn't expect us to be perfect, and it was obligatory in jewish tradition. In Genesis humans are commanded to be fruitful and multiply. So if anything is sinful, it's masturbation, definitely not sex. Only Gnostic Christians view sex as a real problem.