When I was an undergrad CS major in the mid- to late-2000s, I was in the "Math and CS Department." So I had to take quite a few pure math courses (like numbered MA242, covering things like group theory) taught by math professors alongside math majors. CS majors in EE departments probably did a lot of engineering calculus and physics, which I didn't.
Acknowledging your point, in my department, there was an effort underway to replace those math courses with equivalent CS courses (like numbered CS235) that aimed to teach the same ideas but as they related to CS. For a while, one could take either the math or the CS course to satisfy the requirements (in practice, we just chose based on when they were offered and maybe who was teaching).
I'm sure this is still being played out at many universities as CS departments mature and become their own thing. Bureaucracies move slowly.
Edited to add: I remember abstract algebra / group theory nicely leading up to learning about public key cryptography. So it was useful. Approximation algorithms are also covered in the canonical CLRS algorithms textbook, although it's the last section in the final "Selected Topics" chapter, so an undergrad algorithms course might not get that far.