I feel like it might be a good idea to bake role-based access management in and then implement one-person privileges as roles (maybe even named after the person) — less technical debt incurred?

Having tick-boxes for (user * group) would be a useful display (and as groups are ticked/unticked the granular permission boxes would change accordingly) but I wouldn't think of that as the primary use case.

My use case would be:
1. Owner decides a new job exists in the community, e.g. "Archivist" who can do all the tag actions as well as edit memories.
2. Owner goes to community roles page, where they can see a list of roles, with role names, role permissions, and users authorized to act in that role. Each role has buttons to edit it and delete it, and there is a button to create a new role. There is no Archivist role already existing as far as the owner can see, so they click the New Role button.
3. Owner is presented with a new role form. There is a place to enter a display name, a place for comments, a list of privilege tick boxes and a user picker similar to the one used for building filters on individual user accounts. The owner ticks some privileges, adds some users, and clicks 'Add' to create the new role. Optionally, a check happens to verify that the role's permissions aren't identical to another role on add or save — it might be permissible but maybe the role editor could offer to merge identical roles.
4. If the owner wants to change who is an Archivist later or change Archivist powers, they can edit the role.

Any user can have any number of roles.
Editing the Administrator role is limited to adding or removing users, as they have to have all privileges.