6.3

Decoding JSON - Part 1

In the previous section, we learned how to retrieve a simple string from an HTTP server. Nowadays most client apps expect servers to send data in JSON format. In this section, we’ll change the format of our server’s response from string to JSON and use various functions in Elm to translate that JSON into Elm values.

Creating a Local JSON Server

The local HTTP server we created in the previous section is capable of sending JSON too. All we need to do is structure our data as JSON instead of a regular string like this:

But there is a better way. We can create a different server that is fine-tuned for serving JSON instead of static files using the NPM package called json-server. Go ahead and install it globally using the -g option so that it can be run from anywhere in the terminal.

$ npm install json-server -g

Now create a new file called old-school.json inside beginning-elm/server.

Add the following JSON code to the old-school.json file.

{
    "nicknames" : ["The Godfather", "The Tank", "Beanie", "Cheese"]
}

Let’s fire up a JSON server by running the following command from beginning-elm directory in terminal. You can stop the HTTP server we created in the previous section if it’s still running by pressing Ctrl + c.

$ json-server --watch server/old-school.json -p 5019

You should see an output like this:

\{^_^}/ hi!

Loading server/old-school.json
Done

Resources
http://localhost:5019/nicknames

Home
http://localhost:5019

Type s + enter at any time to create a snapshot of the database
Watching...

Like http-server, json-server requires us to specify a file from which it will serve data. In our case it’s server/old-school.json. The --watch option tells the server to watch for any changes made to old-school.json. If we don’t use this option, we’ll have to restart the server each time that file is modified. The -p option specifies the port. We are using a different port (5019) here to avoid any conflict with the server we created in the previous section in case it’s still running.

Go to http://localhost:5019/nicknames in your browser. If you see a list of nicknames as shown below then the server is working as expected.

[
  "The Godfather",
  "The Tank",
  "Beanie",
  "Cheese"
]

Notice how the URL to retrieve nicknames has changed from what it used to be in the previous section:

With the old URL we were specifying the filename because it was serving the contents of a static file. The new URL specifies which resource we want to retrieve. Rather than serving the entire content of a file, json-server allows us to define resources which are just a way to name a piece of information. By adding the following JSON to the old-school.json file, we’re assigning the name nicknames to the resource ["The Godfather", "The Tank", "Beanie", "Cheese"].

{
    "nicknames" : ["The Godfather", "The Tank", "Beanie", "Cheese"]
}

Resources can be a lot more complex than just a list of strings. Here is an example derived from json-server’s documentation page:

{
  "posts": [
    {
      "id": 1,
      "title": "json-server",
      "author": "typicode"
    },
    {
      "id": 2,
      "title": "http-server",
      "author": "indexzero"
    }
  ],
  "comments": [
    {
      "id": 1,
      "body": "some comment",
      "postId": 1
    }
  ],
  "profile": {
    "name": "typicode"
  }
}

The above JSON defines three different resources: posts, comments, and profile. A resource can be either a collection or a single entity. posts and comments are collections, whereas profile is a single entity. Each resource has a unique location from where we can access it.

This concept will become clearer if we try to retrieve these resources from a browser. Go ahead and add the above JSON to a new file called db.json in the beginning-elm/server directory. db is short for database.

Now stop the JSON server by pressing Ctrl + c and restart it so that it’ll use the db.json file instead.

$ json-server --watch server/db.json -p 5019

Notice how the output lists all available resources.

Resources
http://localhost:5019/posts
http://localhost:5019/comments
http://localhost:5019/profile

json-server has essentially created a REST API for us. Let’s retrieve the posts resource by loading http://localhost:5019/posts in browser. Your output should look like this:

[
  {
    "id": 1,
    "title": "json-server",
    "author": "typicode"
  },
  {
    "id": 2,
    "title": "http-server",
    "author": "indexzero"
  }
]

posts is also a resource itself even though it contains other resources inside it. We can retrieve an individual resource inside posts by specifying an id. If you load the URL http://localhost:5019/posts/1 in browser, you should see only one post.

{
  "id": 1,
  "title": "json-server",
  "author": "typicode"
}

Hopefully, you now understand how json-server works. We really didn’t need to learn all of this just to return some JSON from our server. The good news is all of this knowledge will come handy when we create, update, and delete resources in later sections using the POST, PATCH, and DELETE methods defined in the HTTP protocol.

Retrieving JSON from an Elm App

Let’s turn our attention back to the Elm app we wrote in the previous section. We retrieved a string and used the String.split function to extract individual nicknames from that string. We will have to rewrite some of the logic in that app to retrieve the nicknames as JSON and decode them.

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        SendHttpRequest ->
            ( model, Http.send DataReceived (Http.getString url) )

        DataReceived (Ok nicknamesStr) ->
            let
                nicknames =
                    String.split "," nicknamesStr
            in
                ( { model | nicknames = nicknames }, Cmd.none )
        .
        .

Stop the JSON server by pressing Ctrl + c if it’s running and restart it to use the old-school.json file.

$ json-server --watch server/old-school.json -p 5019

The first thing we need to change in HttpExamples.elm located inside beginning-elm/elm-examples is the URL.

url : String
url =
    "http://localhost:5019/nicknames"

The process for retrieving JSON from a server isn’t any different from retrieving a string. Although we’re retrieving JSON, the underlying data format sent by the server is still a raw string. If that’s the case, how does a client app know how to properly parse a response if it always comes down as string? It can’t use the same technique to extract nicknames from a string and also JSON.

Luckily, the server specifies which format it intends the response to be interpreted as through the use of Content-Type header. Let’s find out what value is assigned to this header when we load the nicknames JSON in Chrome.

Step #1: Load the URL http://localhost:5019/nicknames in Chrome.

Step #2: Go to the Network tab in Developer Tools window and click All. By default XHR is selected.

Step #3: Refresh the page and click the row that says nicknames.

Step #4: Look for Content-Type in the Response Headers section.

The Content-Type header suggests that the response uses UTF-8 character set and should be interpreted as JSON.

Decoding JSON

Elm provides a module called Json.Decode which defines various functions we will be using in this section to decode JSON. Let’s import it in HttpExamples.elm.

module HttpExamples exposing (..)
.
.
import Http
import Json.Decode exposing (string, list, decodeString, Decoder)

Next we need to define a decoder that knows how to translate JSON into Elm values. Add the following code right above the update function in HttpExamples.elm.

nicknamesDecoder : Decoder (List String)
nicknamesDecoder =
    list string

The expression list string creates a decoder that knows how to translate a JSON array into a list of Elm strings. That sounds confusing, doesn’t it? Let’s experiment with this expression in elm-repl to get a better understanding.

> import Json.Decode exposing (..)

> list
<function> : Json.Decode.Decoder a -> Json.Decode.Decoder (List a)

Look what we got when we entered just list. The output suggests that list is a function that takes a decoder and returns another decoder. Let’s see what we get when we enter just string.

> string
<decoder> : Json.Decode.Decoder String

We were able to type list and string in repl without having to prefix the module name because we exposed everything while importing the module Json.Decode.

String Decoder

string is a decoder that knows how to translate a JSON string into an Elm string. Here is an example:

> decodeString string "\"Beanie\""
Ok "Beanie" : Result.Result String String

We used two sets of double quotes to indicate that "\"Beanie\"" contains a JSON string within a raw string. string by itself doesn’t decode JSON. It’s like a recipe for decoding. The decodeString function is the one that does the actual decoding. It first parses the raw string into JSON and then applies the string decoder to translate that JSON into an Elm string.

The following diagram explains decodeString’s type signature in detail.

Other Primitive Decoders

Elm provides three other decoders for translating primitive JSON types into Elm values: int, float, and bool. Here are some examples showing them in action:

> decodeString int "9"
Ok 9 : Result.Result String Int

> decodeString int "2.5"
Err "Expecting an Int but instead got: 2.5" : Result.Result String Int

> decodeString float "2.5"
Ok 2.5 : Result.Result String Float

> decodeString bool "true"
Ok True : Result.Result String Bool

> decodeString bool "false"
Ok False : Result.Result String Bool

> decodeString bool "87"
Err "Expecting a Bool but instead got: 87" : Result.Result String Bool

Building Complex Decoders

JSON supports the following data types:

  • string - A string must be written in double quotes and looks very similar to an Elm string.

  • number - A number must be an integer or a float.

  • boolean - A boolean must be either true or false.

  • array - An array can contain any other JSON values including arrays themselves.

  • object - An object consists of key value pairs.

  • null - null is used to indicate an absence of a value.

We already know how to decode strings, numbers, and booleans. To decode the rest, we need to create more complex decoders using primitive decoders as building blocks. Let’s start with a JSON array.

List Decoder

Earlier we created a decoder for translating an array of JSON strings into a list of Elm strings.

nicknamesDecoder : Decoder (List String)
nicknamesDecoder =
    list string

We can decode an array of nicknames using list string like this:

> nicknameJson = "[\"The Godfather\", \"The Tank\", \"Beanie\", \"Cheese\"]"
"[\"The Godfather\", \"The Tank\", \"Beanie\", \"Cheese\"]" : String

> decodeString (list string) nicknameJson
Ok (["The Godfather","The Tank","Beanie","Cheese"])
    : Result.Result String (List String)

Similarly, we can decode an array of JSON ints (or floats) like this:

> decodeString (list int) "[1, 2, 3, 4]"
Ok [1,2,3,4] : Result.Result String (List Int)

> decodeString (list float) "[1.5, 2.5, 3.5, 4.5]"
Ok [1.5,2.5,3.5,4.5] : Result.Result String (List Float)

list itself is not a decoder. It’s a function that takes a decoder and creates a more complex decoder.

list : Decoder a -> Decoder (List a)

If we want to decode a JSON that contains an array of arrays, we can do that by using the list decoder multiple times like this:

> decodeString (list (list int)) "[[1, 2, 3], [4, 5, 6]]"
Ok [[1,2,3],[4,5,6]] : Result.Result String (List (List Int))

There is no limit to how deep this nesting can go. It’s important to remember that Elm requires all elements in a List to be of the same type. Therefore, we can’t decode a JSON array with different types.

> decodeString (list (list int)) "[[1, 2, 3], [4.5, 5.5, 6.6]]"
Err "Expecting an Int at _[1][2] but instead got: 6.6"
    : Result.Result String (List (List Int))

Replacing String.split with nicknamesDecoder

Now that we have understood how the list and string decoders work, let’s get back to finishing our app. Modify the DataReceived (Ok nicknamesStr) -> branch in the update function to replace String.split with nicknamesDecoder.

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        .
        .
        DataReceived (Ok nicknamesJson) ->
            case decodeString nicknamesDecoder nicknamesJson of
                Ok nicknames ->
                    ( { model | nicknames = nicknames }, Cmd.none )

                Err errorMessage ->
                    ( { model | errorMessage = Just errorMessage }, Cmd.none )
        .
        .

We also renamed the payload from nicknamesStr to nicknamesJson and replaced the let expression with case. Unlike String.split, decodeString returns a Result. That’s why we used a case expression to dig nicknames out of Ok. If decoding fails, we wrap the error message in Just and assign it to the errorMessage property.

Run elm-reactor from the beginning-elm directory in terminal if it’s not running already and go to this URL in your browser: http://localhost:8000/elm-examples/HttpExamples.elm. Click the Get data from server button and you should see the same list of nicknames we saw in the previous section.

Unlike http-server, json-server automatically enables Cross-Origin Resource Sharing (CORS). That’s why we didn’t get the No 'Access-Control-Allow-Origin' header is present on the requested resource. error when retrieving the nicknames.

Using Http.get Instead of Http.getString

Let’s review the process we went through to retrieve and decode JSON using Http.getString.

Step 1: Create a request using Http.getString. Specify which URL to use.

Step 2: Take the request from step 1 and create a command using Http.send.

Step 3: Ask the Elm Runtime to execute the command.

Step 4: Send DataReceived message to the update function.

  • Step 4.1: Include JSON as a payload if the request to retrieve nicknames succeeds.
  • Step 4.2: Include an error of type Http.Error as a payload if the request to retrieve nicknames fails.

Step 5: Decode JSON inside the DataReceived (Ok nicknamesJson) -> branch of update function.

  • Step 5.1: If decoding succeeds, update the nicknames property in model.
  • Step 5.2: If decoding fails, update the errorMessage property in model.

The Http module provides another function called Http.get that simplifies the process laid out above. Here’s what the process would look like if we used Http.get instead of Http.getString:

Step 1: Create a request using Http.get. Specify which URL to use. Also specify which decoder to use.

Step 2: Take the request from step 1 and create a command using Http.send.

Step 3: Ask the Elm Runtime to execute the command.

Step 4: Send DataReceived message to the update function.

  • Step 4.1: Include decoded nicknames as a payload if the request to retrieve JSON and decoding both succeed.

  • Step 4.2: Include an error of type Http.Error as a payload if either the request to retrieve JSON or decoding fails.

We don’t need step 5 at all if we use Http.get. That’s because in addition to retrieving a JSON it also decodes it. The following diagram shows how Http.get and Http.getString differ.

Let’s replace Http.getString with Http.get. The first thing we need to do is modify the SendHttpRequest branch in the update function.

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        SendHttpRequest ->
            ( model, Http.send DataReceived (Http.get url nicknamesDecoder) )
        .
        .

The code for creating a command is a bit hard to read. Let’s extract it out to a separate function called httpCommand above the update function and use the pipeline operator to make it easier on the eyes. Don’t forget to use httpCommand in SendHttpRequest -> branch inside update.

httpCommand : Cmd Msg
httpCommand =
    nicknamesDecoder
        |> Http.get "http://localhost:5019/nicknames"
        |> Http.send DataReceived


update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        SendHttpRequest ->
            ( model, httpCommand )
    .
    .

Now that we’re using the URL directly inside httpCommand, go ahead and delete the definition for url.

url : String
url =
    "http://localhost:5019/nicknames"

Next we need to modify the DataReceived (Ok nicknamesJson) -> branch in the update function.

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        .
        .
        DataReceived (Ok nicknames) ->
            ( { model | nicknames = nicknames }, Cmd.none )

        DataReceived (Err httpError) ->
            ...

Http.get really simplified that branch. We don’t need to manually decode JSON anymore. It happens automatically behind the scenes. If decoding is successful, the payload will be an Elm list which we can assign directly to the nicknames property. That means we need to modify the DataReceived message’s definition to accept List String instead of just String.

type Msg
    = SendHttpRequest
    | DataReceived (Result Http.Error (List String))

We don’t need to modify the DataReceived (Err httpError) -> branch in update at all. If decoding fails, the Http.BadPayload error will be sent which we are already handling in createErrorMessage.

Refresh the page at http://localhost:8000/elm-examples/HttpExamples.elm and click the Get data from server button. You should once again see the list of nicknames.

Summary

In this section, we learned how to create a local server that’s fine-tuned for serving JSON. We retrieved some JSON from that server and translated it into Elm values using a decoder. We also learned how to simplify the overall process of fetching and decoding JSON using the Http.get function instead of Http.getString. The diagram below shows that the overall interaction between the Elm Runtime and our code didn’t change much from the previous section.

In the next section, we will learn how to decode more complex JSON. Here is the entire code from HttpExamples.elm thus far:

module HttpExamples exposing (..)

import Html exposing (..)
import Html.Events exposing (onClick)
import Http
import Json.Decode exposing (string, list, decodeString, Decoder)


type alias Model =
    { nicknames : List String
    , errorMessage : Maybe String
    }


view : Model -> Html Msg
view model =
    div []
        [ button [ onClick SendHttpRequest ]
            [ text "Get data from server" ]
        , viewNicknamesOrError model
        ]


viewNicknamesOrError : Model -> Html Msg
viewNicknamesOrError model =
    case model.errorMessage of
        Just message ->
            viewError message

        Nothing ->
            viewNicknames model.nicknames


viewError : String -> Html Msg
viewError errorMessage =
    let
        errorHeading =
            "Couldn't fetch nicknames at this time."
    in
        div []
            [ h3 [] [ text errorHeading ]
            , text ("Error: " ++ errorMessage)
            ]


viewNicknames : List String -> Html Msg
viewNicknames nicknames =
    div []
        [ h3 [] [ text "Old School Main Characters" ]
        , ul [] (List.map viewNickname nicknames)
        ]


viewNickname : String -> Html Msg
viewNickname nickname =
    li [] [ text nickname ]


type Msg
    = SendHttpRequest
    | DataReceived (Result Http.Error (List String))


nicknamesDecoder : Decoder (List String)
nicknamesDecoder =
    list string


httpCommand : Cmd Msg
httpCommand =
    nicknamesDecoder
        |> Http.get "http://localhost:5019/nicknames"
        |> Http.send DataReceived


update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        SendHttpRequest ->
            ( model, httpCommand )

        DataReceived (Ok nicknames) ->
            ( { model | nicknames = nicknames }, Cmd.none )

        DataReceived (Err httpError) ->
            ( { model
                | errorMessage = Just (createErrorMessage httpError)
              }
            , Cmd.none
            )


createErrorMessage : Http.Error -> String
createErrorMessage httpError =
    case httpError of
        Http.BadUrl message ->
            message

        Http.Timeout ->
            "Server is taking too long to respond. Please try again later."

        Http.NetworkError ->
            "It appears you don't have an Internet connection right now."

        Http.BadStatus response ->
            response.status.message

        Http.BadPayload message response ->
            message


init : ( Model, Cmd Msg )
init =
    ( { nicknames = []
      , errorMessage = Nothing
      }
    , Cmd.none
    )


main : Program Never Model Msg
main =
    program
        { init = init
        , view = view
        , update = update
        , subscriptions = \_ -> Sub.none
        }
Back to top

New chapters are coming soon!

Sign up for the Elm Programming newsletter to get notified!

* indicates required
Close