How to correctly structure a nested node-fetch request to avoid Promise<pending>?

Question

I'm attempting to

1. Fetch GET my website (with node-fetch)
2. Scrape it with Cheerio to get specific posts
3. Fetch GET from my CMS (with node-fetch) to check if there's already a post with the same time
4. If the check shows no duplicates, Fetch POST into my CMS

The problem I've run into though, is that I can console.log() the duplicate check result, but when I make a conditional with the result for the Fetch POST request, it always returns the check result as a Promise

I'm not sure how to correctly structure my async .then() calls to correctly check.

Code (edited for simplicity):

fetch('https://www.myblog.com/', { method: 'get' })
.then((res) => {
    return res.text()
  })
  .then((data) => {
    const $ = cheerio.load(data)
    const siteHeading = $('.post-item')
    siteHeading.each((index, element) => {
      const title = $(element).find('.entry-title').text()
      const desc = $(element).find('.entry-content').text()
      const exists = fetch(
        'https://mycms.com/api/articles?filters[title][$eq]=' +
          title,
        {
          method: 'GET',
        }
      )
        .then((response) => response.json())
        .then((article) => article.data[0])
        .then((exists) => {
          if (exists) {
            // ^exists always shows up as True, console.log(exists) shows Promise<pending>
            fetch('https://api.broadband.money/api/news-articles', {
              method: 'POST',
              body: JSON.stringify({
                data: {
                  title: title ? title : null,
                  desc: blurb ? blurb : null,
                },
              }),
            })
              .then((response) => response.json())
              .then((data) => console.log(data))
          }
        })
    })
  })

Answer 1

Using .then() makes code difficult to understand especially when they are nested together. You have to keep track of each of then and take care that the brackets are closed at correct place. Use async await instead. This way you can achieve your objective without so much of nesting: Example:

const data = await fetch('https://www.myblog.com/'{method:'GET'}).then(res=>res.json)

Here the program will wait for the fetch to get completed and then only it will proceed further.

Remember to use async keyword before function declaration inside which you are using fetch.

Example:

async function functionName()
{
..
const data = await fetch...............
..
}

Answer 2

First, what Marcos advises (combining async/await with .then) is an anti-pattern. Either one or the other must be used.

Second, you use extra .then where you don't need it. Here:

...
.then((exists) => {
...

Thirdly, the forEach method does not wait for the execution of asynchronous code (although in your case this does not seem to matter). Here is a small example for understanding:

function each() {
  console.log("Start");

  const arr = [0, 1, 2, 3];

  arr.forEach(async (el) => {
    await new Promise((resolve) => {
      setTimeout(() => {
        resolve(console.log("This log should be after the start", el));
      }, 1000);
    });
  });

  console.log("Finish");
}

each();

Output:

Start
Finish
This log should be after the start 0
This log should be after the start 1
This log should be after the start 2
This log should be after the start 3

Instead you should use a "for" loop:

async function loop() {
  console.log("Start");

  const arr = [0, 1, 2, 3];

  for (let i = 0; i < arr.length; i++) {
    await new Promise((resolve) => {
      setTimeout(() => {
        resolve(console.log("This log should be after the start", arr[i]));
      }, 1000);
    });
  }

  console.log("Finish");
}

loop();

Output:

Start
This log should be after the start 0
This log should be after the start 1
This log should be after the start 2
This log should be after the start 3
Finish

And finally, it seems to me that replacing this

.then((article) => article.data[0])
.then((exists) => {

with this

.then((article) => {
const exists = article.data[0];

should help. If you provide a working code, I can tell you more.