13 Sept: Changes to the contact details API endpoint

Nathan Begbie Updated by Nathan Begbie

If you are using our Data Export API to retrieve contact details you should know about upcoming changes.

To fix a mismatch between contacts in the messages Data Export API and contacts in the contact details API we are updating the way contact details are returned on the Data Export API.

When will the change happen?

These changes will be implemented on Thursday the 22nd of Sept 2022 for those operating on whatsapp.turn.io. For those on private clouds, you can expect to have these changes on Thursday the 29th of September.

What is important to know?

There are 3 key changes you should note.

1. Some field names are being removed or changed

Given that we will only be returning the latest data for a contact, we will be removing the following fields from each JSON data object that represents a contact:

  • contact_id
  • contact_schema_id
  • generation
  • latest
2. The fields: {} key is changing to details: {}

Instead of a fields: {} key, we changed this to details: {} for the object that holds a contact's details. This brings our API more in line with our BigQuery integration.

What it use to be:

{
    "data": [
        {
            "fields": {
                ...
                "language": "eng",
                "whatsapp_id": "2712345678",
                ...
            },
        },
        ...
    ],
    ...
}

What it is changing to:

{
    "data": [
        {
            "details": {
                ...
                "language": "eng",
                "whatsapp_id": "2712345678",
                ...
            },
        },
        ...
    ],
    ...
}

Similarly, scrubbing rules previously required that one used the pattern fields.<detail_name> (e.g. "fields.surname": {"type": "hash"}) . Instead, we will use details.<detail_name> (e.g. "details.surname": {"type": "hash"} )

3. You need to paginate Contacts differently

We are updating the Contact API to only return the latest data for a contact, at the point in time when it was last updated.

The contact details data export differs in behavior from the message export endpoint in one crucial way - contacts and contacts fields will change over time where as messages are created once and are not updated. This means once a message has been logged in the database, its properties do not change. Thus, we paginate by the inserted_at field for messages.

However, contacts and their fields will change over time. For example, if we created a custom field called current_savings that we updated as a contact increased or decreased their savings. If we paginated by inserted_at, you would need to paginate over all contacts every time, in order to get updated data. Instead, we paginate by when contacts were updated. This means that the expected behavior would be to request data on a daily or weekly basis for a time period of a day or a week, respectively. This will then include all update information since you last requested the contact data.

If you have an existing cron job that ingests data, it will need to request data only for the time periods that it has not yet paginated over.

An example on how to get updated details

Imagine we have only a single user who messages us (Bob) and a custom profile field called `current_savings`. On Monday the 12th of September 2022, they message us to say that their current savings are $100.

If we run the following on Tuesday (13th) we might query this data like so:

# request a cursor for the data
$ curl -X POST "https://whatsapp.turn.io/v1/data/contacts/cursor" \
  -H "Authorization: Bearer token" \
  -H "Content-Type: application/json" \
  -H "accept: application/vnd.v1+json"
  -d '
    {
        # start of Monday the 12th
        "from": "2022-09-12T00:00:00.000Z",
        # for a period of 24 hours
        "until": "2022-09-13T00:00:00.000Z"
    }
> {
    "cursor": "an-example-cursor-1234",
    "expires_at": "2022-09-14T17:47:18.318218Z"
}
# request the data using the given cursor
$ curl -X GET "https://whatsapp.turn.io/v1/data/contacts/cursor/an-example-cursor-1234" \
    -H 'Authorization: Bearer token' \
    -H "Accept: application/vnd.v1+json" \
> {
    "data": [
        {
            "details": {
                "birthday": "1988-06-14T18:15:35.873852Z",
                "language": "eng",
                "location": null,
                "name": "bob",
                "opted_in": false,
                "opted_in_at": null,
                "surname": null,
                "whatsapp_id": "44623456700",
                "whatsapp_profile_name": "bob",
                "current_savings": 100
            },
            "id": 1,
            "inserted_at": "20222-09-12T08:30:00.000Z",
            "number_id": 1,
            "updated_at": "20222-09-12T08:30:00.000Z",
            "uuid": "f6c4e666-81ad-430e-aea5-1ba33e6f4c35"
        }
    ]
}

Great! So we have the data and can store it in our data lake, or analytics database.

Now, imagine that Bob messages again on Tuesday, to say that his savings have increased to $120. If we request the data for the same time period (Monday), on Wednesday the 14th, we will no longer receive any data for Bob:

# request a cursor for the data
$ curl -X POST "https://whatsapp.turn.io/v1/data/contacts/cursor" \
  -H "Authorization: Bearer token" \
  -H "Content-Type: application/json" \
  -H "accept: application/vnd.v1+json"
  -d '
    {
        # start of Monday the 12th
        "from": "2022-09-12T00:00:00.000Z",
        # for a period of 24 hours
        "until": "2022-09-13T00:00:00.000Z"
    }
> {
    "cursor": "another-example-cursor-5678",
    "expires_at": "2022-09-16T17:48:18.763987Z"
}
# request the data
curl -X GET "https://whatsapp.turn.io/v1/data/contacts/cursor/an-example-cursor-1234" \
    -H 'Authorization: Bearer token' \
    -H "Accept: application/vnd.v1+json" \
> {
    "data": []
}

Instead, if we want updated data, we must request the next day's data:

# request a cursor for the data
$ curl -X POST "https://whatsapp.turn.io/v1/data/contacts/cursor" \
  -H "Authorization: Bearer token" \
  -H "Content-Type: application/json" \
  -H "accept: application/vnd.v1+json"
  -d '
    {
        # start of Tuesday the 13th
        "from": "2022-09-13T00:00:00.000Z",
        # for a period of 24 hours
        "until": "2022-09-14T00:00:00.000Z"
    }
> {
    "cursor": "yet-another-example-cursor-9876",
    "expires_at": "2022-09-16T17:48:18.763987Z"
}
# request the data using the given cursor
$ curl -X GET "https://whatsapp.turn.io/v1/data/contacts/cursor/yet-another-example-cursor-9876" \
    -H 'Authorization: Bearer token' \
    -H "Accept: application/vnd.v1+json" \
> {
    "data": [
        {
            "details": {
                "birthday": "1988-06-14T18:15:35.873852Z",
                "language": "eng",
                "location": null,
                "name": "bob",
                "opted_in": false,
                "opted_in_at": null,
                "surname": null,
                "whatsapp_id": "44623456700",
                "whatsapp_profile_name": "bob",
                # note the change in value here!
                "current_savings": 120
            },
            "id": 1,
            "inserted_at": "20222-09-12T08:30:00.000Z",
            "number_id": 1,
            "updated_at": "20222-09-13T10:00:00.000Z",
            "uuid": "f6c4e666-81ad-430e-aea5-1ba33e6f4c35"
        }
    ]
}

Note: You should not need to request the data using the time-span of the number for up-to-date contact data unless you are requesting it for the first time. 

See more example calls in the API documentation.

Was this article helpful?

15 Sept 2022: Shorter messages are better

7 Sept 2022: Stack improvements

Contact