Do ppl do this? Like going to a friend's class without enrolling in it at all?

Yes. In fact, in my undergrad it was quite common for people to attend a different prof's lecture for common courses. For my 2nd calculus class I had arguably the best prof, and we had to actually switch to a larger lecture hall we had so many extra students coming. This is a bit different, as you actually are paying for the course, but just going to a different section. But if you attend another class, as long as it's large enough, nobody will notice. And at a school like U of T, where all students pay a flat full time tuition fee, it won't matter at all.

I've heard that sometimes people attend lectures of the same course with a different prof if they don't get something. I've also heard that sometimes a friend would join them for a lecture. Might want to ask the prof or someone of administration to make sure it's alright. The whole semester, though? I'd definitely ask.