Discord api join guild, unauthorized 401 error

Question

I'm trying to authenticate a user with Discord oauth2, then add this user to the guild. I'm also using Passportjs to authenticate the user, so the DiscordStrategy follows as

@Injectable()
export class DiscordStrategy extends PassportStrategy(Strategy) {
  constructor(private authService: AuthService) {
    super({
      clientID: process.env.DISCORD_CLIENT_ID,
      clientSecret: process.env.DISCORD_CLIENT_SECRET,
      callbackURL: `http://${process.env.HOST}:${process.env.PORT}/auth/discord/callback`,
      scope: ['identify', 'guilds', 'guilds.join'],
    });
  }

  async validate(accessToken: string, refreshToken: string, profile: Profile) {
    const { id } = profile;
    console.log(profile);
    const resp = await this.authService.joinGuild(accessToken, id);
    console.log(resp);
  }
}

and the authService.joinGuild

async joinGuild(accessToken: string, userId: string) {
    return this.httpService
      .put(
        `https://discord.com/api/v8/guilds/${process.env.DISCORD_GUILD_ID}/members/${userId}`,
        {
          headers: {
            Authorization: `Bot ${process.env.DISCORD_BOT_TOKEN}`,
          },
          body: {
            access_token: `${accessToken}`,
          },
        },
      )
      .pipe(
        catchError((e) => {
          throw new HttpException(e.response.data, e.response.status);
        }),
      )
      .pipe(
        map((res) => {
          console.log(res.data);
          return res.data;
        }),
      )
      .toPromise();
  }

and my response data is data: { message: '401: Unauthorized', code: 0 }

What am I doing wrong here? I tried to give my bot every permission possible as well. Thanks.