Lesson 15 of 199:37 video

JavaScript Fetch API

How a page gets data without reloading: the Fetch API and the HTTP request behind it, the promise fetch returns and why the body takes a second await, JSON with JSON.parse and JSON.stringify, sending data with POST, CORS and the same-origin policy, and why a 404 doesn't make fetch fail.

In this lesson

  1. 0:00 Intro
  2. 1:13 A request and a response
  3. 2:04 What fetch returns
  4. 2:39 Why two awaits
  5. 3:24 While it waits
  6. 4:10 JSON
  7. 5:04 Parse and stringify
  8. 5:52 Sending data
  9. 6:40 When fetch fails
  10. 7:13 Other origins and CORS
  11. 7:53 Back to the missing file
  12. 8:56 In one line
  13. 9:14 Next time

Here’s a page for a rocket called Kestrel, kestrel.html, with a heading, a Launch button, a fuel gauge and a crew list. The list is empty:

<!DOCTYPE html>
<html>
  <head>
    <title>Kestrel</title>
    <link rel="stylesheet" href="kestrel.css">
  </head>
  <body>
    <h1>Kestrel: on the pad</h1>
    <button>Launch</button>
    <progress id="fuel" value="80" max="100"></progress>
    <ul></ul>
    <script type="module" src="mission.js"></script>
  </body>
</html>

The names are in a file of their own, crew.json:

["Ada", "Lee", "Omar"]

And the page’s script, mission.js, asks the server for that file, then adds a list item for each name in it:

const list = document.querySelector("ul");

function showCrew(crew) {
  for (const name of crew) {
    const item = document.createElement("li");
    item.textContent = name;
    list.append(item);
  }
}

const response = await fetch("crew.json");
const crew = await response.json();
showCrew(crew);

Save the three files in one folder, along with the stylesheet from the last lesson, kestrel.css, whose one rule is .aboard { color: green; }. The script tag says type="module", and that’s what lets await sit outside a function. A browser loads modules only from a server, though. Open kestrel.html straight from your disk, and Chrome blocks mission.js before it runs. A server on your own machine is enough. In a terminal, in that folder, run:

python3 -m http.server

Then open http://localhost:8000/kestrel.html. The list fills in: Ada, Lee, Omar. The page didn’t reload. Two lines of JavaScript asked the server for crew.json and put the names on the page.

Now get the file’s name wrong by one letter. Change "crew.json" to "crews.json" in the fetch line, save, and reload. The list stays empty, and the console of the browser’s developer tools shows why:

GET http://localhost:8000/crews.json 404 (File not found)
Uncaught SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON

There’s no file by that name, so the server answered 404, not found. You’d expect the first line to fail. It didn’t. The error comes from the second line, and it’s a SyntaxError about a less-than sign, a character your code doesn’t contain. So why didn’t fetch fail?

By the end of this lesson you’ll know what happens on the round trip behind fetch: the request, the promise you get at once, and a response that arrives in two parts, which is why the code needs two awaits. You’ll know JSON, the text the names traveled as, and how to turn it into arrays and objects and back. And you’ll know what fetch treats as a failure. That’s where the answer is.

Put the right name back before going on. Each example below is the whole of mission.js unless it says otherwise: replace the file, save, and reload. The others go into the console, with the page open. Type their lines one at a time, unless the example says to paste it whole, and the console shows the value of each line, with strings in single quotes. Chrome also asks the server for favicon.ico, an icon for the tab, and logs a 404 when there isn’t one. That line is about the icon, not your code.

A request and a response

Start with the first line. fetch takes an address. This one is only a file name, so the browser looks for the file in the same folder as the page: http://localhost:8000/crew.json. Then it writes a request, in the format every browser and server share, HTTP. Here’s how the request for crew.json begins:

GET /crew.json HTTP/1.1
Host: localhost:8000
Connection: keep-alive
…

It opens with a method. GET means “send me this”, and the path of the file comes after it. Python’s server writes back a response. That starts with a status code, 200, for OK. Next come the headers, short lines that describe the answer, and one says the content is JSON. After a blank line comes the body. Here, that’s the file itself:

HTTP/1.0 200 OK
…
Content-type: application/json
Content-Length: 23
…

["Ada", "Lee", "Omar"]

The page reached the browser the same way: a GET for kestrel.html, and an answer with a status, headers and a body. What differs is where the answer goes. The page’s body was drawn in the window. An answer to fetch isn’t drawn as a new page. It goes to your script.

To watch it happen, reload with the Network panel of the developer tools open. It lists every request the page makes, crew.json included, with its status. fetch isn’t part of the JavaScript language, by the way. The browser provides it, as it provides setTimeout, and Node has one too. Older code uses XMLHttpRequest, and you’ll also meet libraries such as axios. Underneath, they send the same requests and get the same responses.

What fetch returns

A round trip takes time, and fetch doesn’t wait for it. It sends the request and returns straight away, and what it returns is a promise: an object that stands in for a result that isn’t ready yet. Type this into the console:

fetch("crew.json") // Promise {<pending>}

At first, the promise is pending. When the server’s answer comes in, it’s fulfilled, and the value it holds is a Response object. await holds the code on its line until then, and hands you the Response. Type these lines into the console:

const response = await fetch("crew.json");
response.status // 200
response.ok // true
response.headers.get("content-type") // 'application/json'

Header names ignore case, so "Content-Type" finds the same line. The status is on the Response, and so are the headers. The crew isn’t. The data hasn’t been read yet.

await outside a function works here because mission.js is a module. Load the same file as a plain script, without type="module", and Chrome rejects the whole file with a SyntaxError: there, await is allowed only inside an async function. Written with then, the same step is fetch("crew.json").then((response) => …). And if you leave out the first await, response is the promise itself, so the next line throws TypeError: response.json is not a function.

Why two awaits

A response arrives in two parts. The status and the headers come first, and fetch fulfills its promise once they’re in. The body may still be on its way. With three names you’d never notice. With a big file, the body can keep coming for seconds after the status is known. From a server that held the body back for two seconds, the first await was done after about 3 ms, and the second after about 2,000.

So reading the body is a step of its own, and it has a promise of its own. Call response.json() and you get that promise. It’s fulfilled when the whole body is in and has been read as JSON, and that’s what the second await is for. If you want the body as plain text, call response.text() instead. Either way, you get one read. Type these lines into the console:

const response = await fetch("crew.json");
const crew = await response.json();
crew // (3) ['Ada', 'Lee', 'Omar']
const text = await response.text(); // Uncaught TypeError: Failed to execute 'text' on 'Response': body stream already read

To read a body twice, copy the response first with response.clone(), and read each copy once. Forget the second await, and crew is a promise, not an array. Then the loop in showCrew throws TypeError: crew is not iterable.

While it waits

What is the page doing through all that waiting? Here’s mission.js with the click listener from the last lesson added, ahead of the two lines:

const heading = document.querySelector("h1");
const button = document.querySelector("button");
const list = document.querySelector("ul");

function launch() {
  heading.textContent = "Kestrel: liftoff";
}

function showCrew(crew) {
  for (const name of crew) {
    const item = document.createElement("li");
    item.textContent = name;
    list.append(item);
  }
}

button.addEventListener("click", launch);

const response = await fetch("crew.json");
const crew = await response.json();
showCrew(crew);

Say the server takes three seconds to answer. A click listener that ran for three seconds would freeze the page, since your JavaScript runs one thing at a time. A fetch that takes three seconds freezes nothing.

That’s because await pauses only the code that’s waiting. The paused code steps off the call stack, the pile of functions being run, and the stack is left empty. The event loop starts the next piece of work whenever the stack is empty, so it keeps going. Press Launch one second in, and the heading changes right then, with the list still empty. Python’s server answers in a few milliseconds, too quickly to press anything in between. With a server that held crew.json back for three seconds, the press was handled at once, and the names appeared two seconds later.

The waiting is the browser’s job. Once the response is in, the browser puts a task in the queue. That task fulfills the promise, and your code carries on from the await. (Strictly, the paused code resumes through the microtask queue, as it does after any await.) Only waiting is free, though. A long loop in your own code would still freeze the page.

JSON

Now the body. What came over the network was text: a square bracket, a double quote, a capital A, and so on. The format is JSON, short for JavaScript Object Notation, and it looks like the arrays and objects you already write. Square brackets hold a list. Curly braces hold named values. The values inside are strings, numbers, true, false and null, or more brackets and braces.

But JSON is stricter than JavaScript:

  • Every property name is in double quotes: { "name": "Ada" }, not { name: "Ada" }.
  • So is every string. "Ada" is JSON; 'Ada' isn’t.
  • No comma after the last item: ["Ada", "Lee"], not ["Ada", "Lee",].
  • No comments.

There are no functions and no undefined either. JSON carries data, never code.

Take a list of rockets, each with a name, a fuel level and a number of stages. Paste this into the console to have it there for the next section:

const fleet = [
  { name: "Kestrel", fuel: 80, stages: 2 },
  { name: "Heron",   fuel: 40, stages: 1 },
  { name: "Osprey",  fuel: 95, stages: 3 },
];

Written as JSON, in a file called fleet.json, it’s almost the same characters. The property names gain quotes, and the comma after the last rocket goes:

[
  { "name": "Kestrel", "fuel": 80, "stages": 2 },
  { "name": "Heron",   "fuel": 40, "stages": 1 },
  { "name": "Osprey",  "fuel": 95, "stages": 3 }
]

Spaces and line breaks make no difference, so the lined-up columns are fine. A JSON file holds one value. Usually that’s an array or an object, but a lone number or string counts too. And JSON isn’t only for JavaScript. Nearly every programming language can read and write it.

Parse and stringify

Still, JSON is only text. Ask a string of it for its first item, and you get a bracket. So the language has an object, also named JSON, with two methods. JSON.parse takes the text and builds the values it describes, here an array of three strings. JSON.stringify goes the other way: give it an array or an object, and it gives you the text. Type these lines into the console:

const text = '["Ada", "Lee", "Omar"]';
text[0] // '['
const crew = JSON.parse(text);
crew[0] // 'Ada'
JSON.stringify(crew) // '["Ada","Lee","Omar"]'

response.json() does that parsing for you, once the body is in. JSON.stringify(fleet) works the same way and writes the whole fleet on one line.

Some things don’t make the trip into text. A method doesn’t, and neither does a property whose value is undefined. Paste this into the console as one entry:

const rocket = {
  name: "Kestrel",
  fuel: 80,
  docked: undefined,
  launch() {
    console.log("Liftoff");
  },
};

JSON.stringify(rocket) // '{"name":"Kestrel","fuel":80}'

Each parse builds new objects, so what comes back is a copy, never the original. A Date goes in as a date and comes out as a string.

And JSON.parse is strict. One single quote, or one comma too many, and it throws a SyntaxError. Type each into the console:

JSON.parse("{'name': 'Kestrel'}") // Uncaught SyntaxError: Expected property name or '}' in JSON at position 1 (line 1 column 2)
JSON.parse('["Ada", "Lee", "Omar",]') // Uncaught SyntaxError: Unexpected token ']', ..."", "Omar",]" is not valid JSON

Once you hold a real array, the rest is the DOM, the page’s tree of elements. That’s showCrew in the first mission.js: loop over the names, and add a list item for each. With the names written into the script, the loop would start from const crew = ["Ada", "Lee", "Omar"];. The two fetch lines take that line’s place, and the loop stays as it was.

Sending data

So far the page has only asked for data. To send some, give fetch a second argument, an object of options. Set method to "POST", which tells the server: here’s something new. Put the data in body. A body has to be text, so pass the object through JSON.stringify first. Then add a header, Content-Type, that says the body is JSON:

const response = await fetch("/crew", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ name: "Zoe" }),
});
const saved = await response.json();
console.log(saved); // {id: 4, name: 'Zoe'}

This one needs a server that takes a POST at /crew, keeps the name, and answers with what it kept. Python’s server only hands out files. It answers 501, Unsupported method, with an HTML page for a body, and response.json() trips over its first character, as in the opening. With a server that accepts it, the request goes out like this:

POST /crew HTTP/1.1
Host: localhost:8000
…
Content-Type: application/json
…

{"name":"Zoe"}

The answer comes back the usual way. There’s a status, 201, for Created, and a body holding the name the server saved, now with an id: {"id":4,"name":"Zoe"}.

Both steps on the way out matter. Hand body the object itself, and the server gets fifteen characters, [object Object], since that’s what an object becomes when it’s turned into a string. Leave out the header, and Chrome labels the body text/plain. Other methods follow the same shape: PUT and PATCH change something, and DELETE removes it. Taking in a request like this and saving the name is work for code on the server. We’ll write a server when we get to Node, later in this series.

When fetch fails

There’s one case where the promise from fetch is rejected: no response came back. The server is down, or the network is, or the address leads nowhere. Then await throws on that line. What it throws is a TypeError, and in Chrome its message reads Failed to fetch. Stop the server with Ctrl+C, leave the page open, and type this into the console:

const response = await fetch("crew.json"); // Uncaught TypeError: Failed to fetch

Just above the error, Chrome logs the request that never got an answer, GET http://localhost:8000/crew.json net::ERR_CONNECTION_REFUSED. Other browsers word the message differently. The type is the same.

Inside mission.js, an error like that stops the script where it’s thrown, and the visitor is left looking at an empty list. So put both awaits in a try block. In the catch block, log the error and tell the visitor what went wrong:

const list = document.querySelector("ul");

function showCrew(crew) {
  for (const name of crew) {
    const item = document.createElement("li");
    item.textContent = name;
    list.append(item);
  }
}

try {
  const response = await fetch("crew.json");
  const crew = await response.json();
  showCrew(crew);
} catch (error) {
  console.error(error); // TypeError: Failed to fetch
  list.textContent = "Crew list unavailable";
}

Start the server again and reload, and the crew appears as before. To see the catch run, give fetch an address where nothing is listening, such as http://localhost:8001/crew.json. The console shows the error, and the list says Crew list unavailable.

Other origins and CORS

You can also get Failed to fetch while the network is working fine. Every page has an origin, made of the scheme, the host and the port at the front of its address. Kestrel’s is http://localhost:8000. Ask your own origin for a file, and you get it. Now ask another origin: a different site, or the same machine on a different port. The browser checks that server’s response for a header saying who may read it. With no such header, your script never sees the response.

Try it with a second server. Open another terminal in the same folder and run python3 -m http.server 8001. Then make mission.js ask port 8001 for the crew:

const list = document.querySelector("ul");

function showCrew(crew) {
  for (const name of crew) {
    const item = document.createElement("li");
    item.textContent = name;
    list.append(item);
  }
}

const response = await fetch("http://localhost:8001/crew.json");
// Access to fetch at 'http://localhost:8001/crew.json' from origin 'http://localhost:8000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
// GET http://localhost:8001/crew.json net::ERR_FAILED 200 (OK)
// Uncaught TypeError: Failed to fetch
const crew = await response.json();
showCrew(crew);

The request does reach the second server: its terminal logs the GET, and the console’s own line ends in 200 (OK). It’s the response your script isn’t allowed to read. That rule is called the same-origin policy. CORS, short for Cross-Origin Resource Sharing, is the system the header belongs to. A server whose responses carry Access-Control-Allow-Origin: * lets every origin read what it serves, and then the list fills as usual. Two more details: localhost and 127.0.0.1 are two origins, even on one machine, and before sending a POST with a JSON body to another origin, the browser first asks that server for permission, in a request of its own called a preflight.

The same rule is why you needed a server at the start. A page opened straight from your disk has no origin to share, so Chrome won’t load the module file beside it, and a fetch for crew.json from that page is blocked too. Serve the folder from a local server, Python’s or your editor’s live preview.

Back to the missing file

Which brings us back to the file that wasn’t there. The request went out, and the server did answer: 404. For fetch, an answer is enough. The promise is about the trip, not about the news, so it’s fulfilled and the first line runs clean. A 404 has a body as well, an error page written in HTML. Python’s begins like this:

<!DOCTYPE HTML>
<html lang="en">
…

The second line reads that page as JSON. Its first character is a less-than sign, and there’s your SyntaxError. The '<' in the message is just the first character of what came back. Most servers answer a 404 with an HTML page, so you’ll see that one often.

The fix goes between the two lines. A Response has a property named ok. It’s true for a status from 200 to 299, and false for anything else. When it’s false, throw an error of your own, with the status in it. Here it is with the misspelled name, to show it working:

const list = document.querySelector("ul");

function showCrew(crew) {
  for (const name of crew) {
    const item = document.createElement("li");
    item.textContent = name;
    list.append(item);
  }
}

try {
  const response = await fetch("crews.json");
  if (!response.ok) {
    throw new Error("Server answered " + response.status);
  }
  const crew = await response.json();
  showCrew(crew);
} catch (error) {
  console.error(error); // Error: Server answered 404
  list.textContent = "Crew list unavailable";
}

Spell the name right, and the crew appears. Point it at port 8001 with nothing running there, and the same catch logs TypeError: Failed to fetch. Point it at kestrel.css, which exists but isn’t JSON, and it logs a SyntaxError: Unexpected token '.', ".aboard { "... is not valid JSON. A missing file, a dead server and bad JSON all land in one catch. Statuses come in families, by the way: 2xx means it worked, 3xx that it moved, 4xx that the request was wrong, and 5xx that the server failed.

Summary

  • fetch sends an HTTP request and returns a promise straight away. await gives you the Response once the status and headers are in.
  • The body comes after them, so reading it takes a second promise, from response.json() or response.text(), and a second await. A body can be read only once.
  • While your code waits, the page doesn’t: await takes the code off the call stack, and the event loop carries on.
  • Data travels as JSON, text in the shape of arrays and objects, with double quotes, no trailing commas, no comments, no functions and no undefined.
  • JSON.parse turns that text into values, and JSON.stringify turns values into text. parse throws a SyntaxError on anything that isn’t valid JSON.
  • To send data, give fetch a method, a body made with JSON.stringify, and a Content-Type header that says it’s JSON.
  • The promise is rejected only when no response reaches your script: no server, no network, or another origin that didn’t allow it. Chrome calls that TypeError: Failed to fetch.
  • A 404 is still a response. Check response.ok and throw, and one try and catch handles every failure.

Next in this series: a memory for the page, with localStorage, where a script keeps data in the browser so it’s still there after a reload and after the tab is closed.

Video transcript

Intro

Here's a page for a rocket called Kestrel, with an empty crew list. Two lines of JavaScript ask the server for a file, crew dot json, and the list fills in: Ada, Lee, Omar. No reload. Now get the file's name wrong, by one letter. The server has no such file, so it answers four-oh-four: not found. You'd expect the first line to fail. It doesn't. The error comes from the second line. It's a SyntaxError, about a less-than sign that isn't anywhere in your code. So why didn't fetch fail? Today, we find out.

By the end of this video, you'll understand three things. One: the round trip behind fetch: the request it sends, the promise it hands back, and a response that arrives in two parts, which is why there are two awaits. Two: JSON, the text that data travels as, and how to turn it into arrays and objects and back. Three: what fetch counts as a failure: why a four-oh-four isn't one, and how to catch everything that can go wrong.

A request and a response

Start with the first line. fetch takes an address. This one is only a file name, so the browser looks for that file next to the page. Then it writes a request, in the format every browser and server share: HTTP. The request opens with a method, GET, which means: send me this. After it comes the file's path. The server writes back a response. That opens with a status code: two hundred, for OK. Next come the headers, short lines that describe the answer. One of them says the content is JSON. Last comes the body: here, the file itself. The browser loaded the page itself with a request like this one. The difference is where the answer goes. It isn't drawn as a new page. It goes to your script.

What fetch returns

A round trip takes time, and fetch doesn't wait for it. It sends the request and returns right away. What it returns is a promise: an object that stands for a result that isn't ready yet. At first, the promise is pending. When the server's answer comes in, it's fulfilled, and the value it holds is a Response object. await pauses the code on that line until then, and hands the Response over. On it you'll find the status: two hundred. You'll find the headers. What you won't find is the crew. The data isn't there yet.

Why two awaits

That's because a response arrives in two parts. The status and the headers come first, and fetch fulfills its promise as soon as they're in. The body may still be on its way. With three names you'd never notice. With a big file, the body can keep arriving for seconds after the status is known. So reading the body is a step of its own, with a promise of its own. Call response dot json, and you get that promise. It's fulfilled once the whole body is in and has been read as JSON. That's what the second await waits for. If you want the body as plain text instead, call response dot text. Either way you get one read. Ask for the body a second time, and you get an error.

While it waits

So what is the page doing through all that waiting? Say the server takes three seconds to answer. If a click listener ran for three seconds, the page would freeze, because your JavaScript runs one thing at a time. A fetch that takes three seconds freezes nothing. await pauses only the code that's waiting. That code steps off the call stack, the pile of functions being run, and the stack is empty. So the event loop, which starts the next piece of work whenever the stack empties, carries on. Press Launch one second in, and the heading changes right then. The waiting is the browser's job. When the response arrives, the browser puts a task in the queue. The task fulfills the promise, and your code carries on from the await.

JSON

Now the body. What came over the network is text: a bracket, a quote mark, a capital A. The format is JSON, short for JavaScript Object Notation, and it looks like the arrays and objects you already write. Square brackets hold a list. Curly braces hold named values. Inside them go strings, numbers, true, false and null. But JSON is stricter than JavaScript. Every property name sits in double quotes. Strings use double quotes, never single ones. There's no comma after the last item, and there are no comments. And there are no functions and no undefined, because JSON carries data, never code. Take a list of rockets, each with a name, a fuel level and a stage count. As JSON, it's almost the same characters. Only the property names gain quotes.

Parse and stringify

Text is all it is, though. Ask that string for its first item, and you get a bracket. So the language comes with an object, also named JSON, that has two methods. JSON dot parse takes the text and builds the real thing: an array, with three strings in it. Response dot json does that for you, once the body is in. JSON dot stringify goes the other way. Hand it an array or an object, and you get the text. A method doesn't make the trip, and neither does a property that's undefined. And parse is strict. One single quote, or one comma too many, and it throws a SyntaxError. With a real array in hand, the rest is the DOM, the page's tree of elements. Loop over the names, and add a list item for each.

Sending data

So far we've only asked for data. To send some, give fetch a second argument: an object of options. Set method to POST, which tells the server: here's something new. Put the data in body. A body has to be text, so run the object through JSON dot stringify first. Hand over the object itself, and the server receives fifteen characters: object Object, in square brackets. Then add a header, Content-Type, to say the body is JSON. The answer comes back the usual way: a status, two-oh-one, for created, and a body, holding the name the server saved, with an id. Receiving that request and saving the name takes code on the server. We'll write a server when we get to Node, later in this series.

When fetch fails

Now for failure. The promise from fetch is rejected in one situation: no response came back. The server is down, or the network is, or the address leads nowhere. Then await throws on that line. What it throws is a TypeError, and in Chrome its message is: Failed to fetch. So put both awaits in a try block. In the catch block, log the error, and tell the visitor what happened. Otherwise the code stops there, and the visitor is left looking at an empty list.

Other origins and CORS

There's one more way to get Failed to fetch, with the network working fine. Every page has an origin: the scheme, the host and the port at the front of its address. Ask your own origin for a file, and you get it. Ask another origin, a different site, or the same machine on a different port, and the browser checks the response for a header from that server that says who may read it. No header, and your script never sees the response. That rule is called the same-origin policy. CORS is the system the header belongs to. It's also why a page opened straight from your disk can't fetch the file beside it. Serve it from a local server.

Back to the missing file

Which brings us back to the file that wasn't there, at the start. The request went out, and the server did answer. Its answer was four-oh-four. For fetch, an answer is enough: the promise is about the trip, not the news. So it's fulfilled, and the first line runs clean. And a four-oh-four has a body as well: an error page, written in HTML. The second line reads that page as JSON. Its first character is a less-than sign, and that's the SyntaxError. The fix sits between the two lines. A Response has a property named ok. It's true for a status from two hundred to two ninety-nine, and false for anything else. When it's false, throw an error of your own, with the status in it. Now a missing file, a dead server and bad JSON all end up in one catch.

In one line

That's fetch in one line: it sends a request and hands back a promise; the response comes in two parts, so the body takes a second await; the data travels as JSON, which you parse into values; and only a missing response counts as failure, so check ok yourself.

Next time

Next time, we give the page a memory: localStorage, where a script keeps data in the browser, so it's still there after a reload, and after the tab is closed.

Learn more