Yep, and, if you're paying attention and thoroughly exploring, you can actually go around the castle and totally avoid him. You can get incredibly strong before fighting him. It makes you feel like you're actually progressing.