It is about folders and you confirmed what I thought was the case. Except, how do you sort folders by primary name (with primary name displayed, of course)? When I go to customize view for the folders panel, the only artist choices I have are Artist and Artist (no compilations). Both use Sort Name, not Primary Name. I have lots of choice, including a number of custom fields, but I can't see one that gives me primary name.
While I was playing around here, I just discovered an exception. If I select Entire Collection for Browse By, it displays the primary name, sorted by Sort name and then Album title. But the artists are not, of course, in folders, or rather all artists are in one folder, the one called Entire Collection. I don't know whether that will satisfy Poohbear's wish or not. For me, I would still like to be able to sort by Primary Name; with that the display from Browse by Entire Collection would then meet my needs.