If you don't want to bother with directories or want to use https instead of ssh, you can do remote url based dispatch in your gitconfig:

    [credential "https://github.com/org1"]
      useHttpPath = true
      helper =
      helper = /path/to/auth.sh user1
    [includeIf "hasconfig:remote.*.url:https//github.com/org1/**"]
      path = user1.gitconfig
      ; set name / email in user1.gitconfig

where auth.sh is something that can produce the right token for the given user, e.g.

    #!/bin/bash
    echo "username=$1"
    echo "password=$(gh auth token --user $1)"