API Fundamentals
Last reviewed: 28/06/24
Barbour ABI data is about construction projects. These are described and continually updated by a team of in-house Barbour ABI researchers, who track the projects and the companies and people working on them, from initial design and planning stages through to construction completion.
Customers subscribe to Barbour ABI to gain access to a defined set of projects by specifying various criteria, such as project value, location, classification, materials, etc.
Customers access the construction projects using a range of data delivery products, including web and mobile apps, and this API.
Key Terminology
The Barbour ABI dataset is structured into the following main objects:
1. Project
2. Company
3. Person
To explain the relationships between these objects:
- A company is related to a project by the role the company plays on the project. For example the company *ABC Ltd* is the *Architect* (role) for the project *Supermarket A*.
- A company can have many people. For example, *John Black* and *Alison White* work for company *ABC Ltd*
- A specific person is responsible for a role on a project. For example, *Alison White*, who works for *ABC Ltd*, is the *Architect* working on *Supermarket A*.
Common use cases
Customers use Barbour ABI data to:
1. Find sales opportunities: search for relevant projects near me
2. Competitor analysis: identify projects that a competitor has won
3. Marketing data sets: extract a set of data for mailing
4. Analysis: compare project or company activity in different geographical areas
Data updates
The Barbour ABI research team are based in the UK, so the majority of updates occur during UK business hours. We encourage developersto build systems that continually use these APIs to ensure the latest updates are seen to users, rather than build daily batch processes. TheAPIs and repositories are hosted on a load balanced environment so resilience and uptime are high.
Some updates are done outside UK business hours, so please ensure this is considered in designing a batch processing system.
HTTP status codes
Success
All successful API calls will respond with a standard 2xx status code. These responses will not have a message body, unless a GETrequest.
Typical responses and examples for PUT/POST requests, using by advanced API consumers are:
| Status code | Description |
|---|---|
| 200 OK | Changing the property of an object, for example favouriting a project or marking a project as read. |
| 201 Created | Creating a new resource, for example creating a team, creating a user and creating a tag |
| 204 Accepted | An attempt to update or create that results in no change. For example, favouriting a project that is already favourited, or creating a team that already exists. |
Others
API calls resulting in a client or server error will respond with the appropriate status code along with a supporting message. For example:
{
"status_code" : 501,
"message" : "Support to delete teams is not available yet"
}
Rate limiting
In the future v4 will be released that includes throttling to ensure fair use and the wider user experience is not affected. When this is implemented excessive programatic API calls will result in a 429 Too Many Requests status code.
Entry point
The entry point is:
https://api.barbour-abi.com/v4
API Key and authentication
As a developer you will also need either a Developer or a Locations license. You will receive your username and will be invited to set your password in a “Welcome” email.
To get started, you will need to create an API key for your Barbour ABI subscription by using the credential from your welcome email to sign in at https://app.barbour-abi.com. Once signed in:
- Click your initials in the top right
- Click “Settings” from the drop-down menu
- Click the “Developer” tab
- Follow the instructions to create an API Key. If an API key has already been created for your account and you do not know it, then you can replace the existing API Key here.
There is a single API key for your company subscription so if you have two developers then both developers should use the same API key. You need to include this API key in an ‘x-api-key’ header of every API call you make.
Next, generate a token which is required to consume the APIs. You will need your username, password and API key for the login API as follows:
GET login
headers:
Authorization: Basic {token}
x-api-key: {api-key}
{token} is base64 encoded username:password where password is a SHA256 hash.
{api-key} is the API key provided by Barbour ABI.
An example, assuming the following:
- Username: api_user
- Password: MB_4810
1. SHA256 hash the password MB_4810 which will be 37596daf86f09dbc748480ef693fb4beb1ff5e406b21a8f671eccb06a0fd6f2d
2. Append this to the username separated by a colon to produce:
api_user:37596daf86f09dbc748480ef693fb4beb1ff5e406b21a8f671eccb06a0fd6f2d
3. Base64 encode api_user:37596daf86f09dbc748480ef693fb4beb1ff5e406b21a8f671eccb06a0fd6f2d to produce
YXBpX3VzZXI6Mzc1OTZkYWY4NmYwOWRiYzc0ODQ4MGVmNjkzZmI0YmViMWZmNWU0MDZiMjFhOGY2NzFlY2NiMDZhMGZkNmYyZA==
4. Call the login API as follows (example in Curl):
curl --location --request GET '<https://api.barbour-abi.com/v4/login'> \
--header 'Authorization: Basic YXBpX3VzZXI6Mzc1OTZkYWY4NmYwOWRiYzc0ODQ4MGVmNjkzZmI0YmViMWZmNWU0MDZiMjFhOGY2NzFlY2
--header 'x-api-key: ABI_KEY_PROVIDED_BY_BARBOUR_ABI'
This will respond with a token in the header, which must be supplied in all other API calls.
The body will contain the unique ID for the user.
{
"user_id": 1234
}
The user_id cannot be changed and should not be confused with the username which can be changed.
If a token is not used for 30 days it will become invalid and a new one must be generated as above. Also, on rare occasions tokens will be
invalidated sooner, for example with a major production release.
In your production code you should test for an invalid token and re-authenticate as necessary.
Tokens will automatically become invalid if a Barbour ABI subscription lapses.
A valid token and the API Key must be included in the header of all API calls:
headers:
Authorization: Bearer {authentication token}
x-api-key: {api-key}
Logout
It is not necessary to invalidate a token, but if required it can be explicitly destroyed as follows:
GET /logout
Provide the token in the header, like all the other API calls.
Customers API
To get details about your company:
GET customers
returns:
{
"customer_id": "ABC1245",
"company_name": "ABC Limited",
"company_website": "abclimited.co.uk",
"sector_id": 1
}
Access to the APIs is based on a subscription service with customers having access to the sub-set of Barbour ABI that matches their subscription. To see this profile, in the standard API query syntax:
GET customers/profiles
The response includes the fields where filtering is applied. For example if your subscription covers all project values, then the key project_value will be omitted.
Lookups API
New lookup ids are introduced rarely, and similarly the text associated with an id rarely changes. Therefore developers should cache local copies of the lookups, perhaps updating their cache every 24 hours or if an unrecognised id is found.
Some of the fields are based on lookups. Lookups can be numeric or strings, for example planning stages and materials respectively. To get the list of lookups:
GET lookups
The response is as follows:
{
"lookups": [{
"planning_stage": [{
"id": 2,
"description": "Outline planning"
},
{
"id": 10,
"description": "Outline approval"
}
],
"materials": [{
"id": "EC",
"description": "Envelope",
"children": [{
"id": "EC01",
"description": "Wall Cladding",
"children": [{
"id": "EC0102",
"description": "reinforced concrete"
}, {
"id": "EC0103",
"description": "precast concrete"
}]
},
{
"id": "EC02",
"description": "Wall Features"
}
]
},
{
"id": "BR",
"description": "Bridges
}
]
}]
}
For specific sets of lookups use the lookups parameter as follows:
GET lookups?lookups=planning_stage,material
For specific lookups use the query parameter. An example to get the lookups for 4 categories:
GET lookups?query={
"category":
{"operator": "=",
"value": [807, 1006, 1312, 2301]
}
}
The response will include the parent categories.
Key terminology
The Barbour ABI dataset is structured into the following main objects:
- Project
- Company
- Person
The relationships between these objects are summarised as follows:
- A company is related to a project by the role the company plays on the project. For example the company ABC Ltd is the Architect (role) for the project Supermarket A.
- A company can have many people. For example, John Black and Alison White work for company ABC Ltd
- A specific person is responsible for a role on a project. For example, Alison White, who works for ABC Ltd, is the Architect working on Supermarket A.
For a more detailed description of these relationships, see the section Project roles
Projects API
Search / project list
The most basic search (using defaults), to get some projects:
GET projects
This will respond with two components as follows:
{
"projects": [{
"project_id": 12328056,
"project_title": "Liverpool Gin Distillery, Castle Street - Refurbishment",
"project_site3": "Liverpool",
"project_value": 1000000
},
{
"project_id": 12261782,
"project_title": "Cycling Infrastructure Improvements",
"project_site3": "Warrington",
"project_value": 7700000
},
{
...
}
],
"aggregation": {
"project_count": 1030
}
}
The projects key contains a list of projects and the aggregation key contains the total number of projects available in the complete dataset. For example there could be 20 projects returned, but the count is much higher. See the section Pagination for more information on how to use aggregation to iterate using offsets.
Fields
To specify the project fields returned:
GET projects?fields=project_id,project_title
A full list of available fields can be obtained using the fields API.
Some fields are sizes or dimensions, such as project_floor_area and project_units. These fields are returned as numeric values to allow sorting and analysis. To understand the units (e.g. square metres and miles) look at the description provided in the fields API.
Wherever possible fields are coded (for example project_planning_stage and project_categories) so are returned as ids. Use the lookups API to get the display text associated with each id.
Some fields can contain multiple values so the values are returned as an array by default. For example:
"project_materials": ["BR0102","BR0105","EC0104"]
There are hierarchies for some fields which can be obtained from the fields API response. To get fields returned from the projects API in a hierarchy, use the lookup_hierarchy parameter. For example:
GET projects?fields=project_materials&lookup_hierarchy=true
This will structure the project_materials key as follows:
"project_materials": {
"BR": [{
"BR01": ["BR0102", "BR0105"]
}],
"EC": [{
"EC01": ["EC0104"]
}]
}
The naming convention for unique object indentifiers is {object_name}_id, for example project_id, company_id and person_id. The unique identifier for roles on a project is rarely required by customers but is available and named role_id. This field should not be confused with role_code which is the role (such as Architect) a company plays on a project.
The following additional fields are derived from other fields so not listed in the fields API:
- project_build_phase: The phase of the project, one “not_started”, “enabling”, “on_site” or “active_on_site”. If there is no timing information to determine the phase, the string “no_timing” is used.
- project_site_progression: A site progression indicator from 0 to 100, in increments of 5, returned only when the project is onsite.
Sort
Results can be sorted, for example to sort by project_value (descending), following by the project_id (ascending) use the following:
GET projects?sort=-project_value,project_id
The plus sign (+) can be used to specify an ascending sort, but is not necessary.
When searching on text fields, the default sort order is the best match. To override this default, specify a sort order.
A maximum of three sort orders can be provided.
Query (filtering)
Filters are applied using the query parameter. The syntax is as follows:
"query"={
"field":{
"operator": <sign>,
"value": <field value>
},
...
}
The operators are:
| Operator | Sign |
|---|---|
| Range | .. |
| Equal | = |
| Not equal | != |
| Greater than | > |
| Less than | < |
| Greater than or equal | = |
| Less than or equal | <= |
| Is empty | 0 |
| Is not empty | !0 |
If the range operator is used, then value1 and value2 keys need to be used in place of the single value key.
The query parameter needs to URL encoded. The examples in this document are not URL encoded for readability.
The use of this query language is explained next, in the form of examples.
Example 1: Projects with a value greater than £1m with between 10 and 20 units:
GET projects?query={
"project_value":{
"operator": ">",
"value": 150000
},
"project_units":{
"operator": "..",
"value1": 10,
"value2": 20
}
}
Example 2: Projects within 25 miles of me, without a contractor appointed
Assuming my location is Tower Bridge (51.5079, 0.0877) and understanding the code for contractor is 40:
GET projects?query={
"project_latitude": {
"operator": "=",
"value": 51.5079
},
"project_longitude": {
"operator": "=",
"value": 0.0877
},
"project_distance": {
"operator": "=",
"value": 25
},
"project_roles": {
"operator": "!=",
"value": 40
}
}&sort=project_distance
The units for project_distance is miles.
To get the results sorted by distance, use sort=project_distance . When sorting by distance the project_latitude and project_longitude must be included in the query.
Date filters can be applied based on specific dates or relative dates, as shown in the next two examples.
Example 3: Projects starting on/after 19th April 2019:
GET projects?query={
"project_start":{
"operator": ">=",
"value": "2019-04-19T00:00:00Z"
}
}
Example 4: Projects starting within the next 30 days:
GET projects?query={
"project_start":{
"operator": "..",
"value1": 0,
"value2": 30
}
}
Project timings are not usually precise dates so there are three date fields that can be returned using the fields parameter, the earliest possible date, the latest possible date and a displayable version of the date. An example of a project start date:
| Field | Description | Example |
|---|---|---|
| project_start_min | The earliest possible start date | 2019-05-01T00:00:00Z |
| project_start_max | The latest possible start date | 2019-05-31T00:00:00Z |
| project_start | A displayable version of the start date | May 2019 |
The API considers non-precise dates as part of the filtering and sorting so the field project_start can be used within the query parameter and sort parameter.
Example 5: Get the latitude and longitude for 3 specific projects:
GET projects?query={
"project_id":{
"operator": "=",
"value": [12388684,12429742,12421574]
}
}&fields=project_latitude,project_longitude
Example 5: A generic project text search for “northgate development”:
Support is provided for fuzzy text searches as follows:
GET projects?query={
"project_text":{
"operator": "=",
"value": "northgate development"
}
}
Searching the project_text will search the following fields for matches:
- project_id
- project_title
- project_scheme
- project_status
- project_site1
If no sort order is specified, the results will be ranked by score with exact matches ranking first. An exact match within a project_title will rank higher than an exact match within a project_scheme.
Note: project_text is not a field, so cannot be returned using the fields parameter.
Example 5: Projects in various postcodes
GET projects?query={
"project_postcode":{
"operator": "=",
"value": ["CH","LL12","LL13","CW6"]
}
}
For these filters, the postcodes can be specified as an postcode area (e.g. CH), a postcode district (e.g. CH64), a unit postcode (e.g. CH65 9HQ) or any combination of these.
All or any text search
Searching for multiple strings is achieved using the project_search parameter.
To search for projects with contain the both string “college” and “extension”:
GET projects?query={
"project_text":{
"operator": "=",
"value": ["college", "extension"]
}
}&project_search=all
To search for projects which contain either the string “University of Manchester” or “Manchester University”:
GET projects?query={
"project_text":{
"operator": "=",
"value": ["University of Manchester", "Manchester University"]
}
}&project_search=any
The project_search parameter is not available for company or people searches.
Aggregations
Group
This is used for simple analytical queries, achieved using the following three parameters:
| Field | Description |
|---|---|
| aggregate_group | The field to group by |
| aggregate_field | The field the analyze |
| aggregate_function | The function to be applied to the aggregate_field. The function is either 'sum' or 'count' |
Example 1: The total number of projects per (primary) sector:
GET projects?aggregate_group=project_primary_sector
&aggregate_field=project_id
&aggregate_function=count
This will include an additional key count_project_id in the response.
Example 2: Total value of projects at each planning stage:
GET projects?aggregate_group=project_primary_sector
&aggregate_field=project_value
&aggregate_function=sum
This will include an additional key sum_project_value in the response.
In summary, when using the aggregation parameters, the additional key in the response will always be named {aggregate_function}_{aggregate_field}
Count
This is a simplified version of the group aggregation, returning a single value.
The most common use of this is to replace the default project count with the sum of the project values as follows:
GET projects?aggregate_field=project_id
&aggregate_function=count
Implicit
An aggregated field can be listed in the fields and/or sort parameters, without explicitly using the aggregation parameters. For example a simple API call to get a list of projects with the number of different companies working on each project:
GET projects?fields=project_id,project_title,count_company_id
The implicit aggregations are particularly useful in the companies API where companies can be sorted by the projects they are working on.
Project details
To get almost all fields for a project:
GET projects/{project_id}
There are some legacy and uncommon fields which are no included in the above request.
For a limited set of fields or to get the non-default fields, use the fields parameter:
GET projects/{project_id}&fields=project_id,project_title
Project roles
A project will have one or more companies playing roles (such as architect) on projects. For each of these roles, there can be zero or more people who are the contact. Projects can have a large number of roles, so the roles are grouped for presentation purposes.
In summary the entities are organised as follows:
- Role Group (e.g. Architects)
- Role (e.g. Architect)
- Company (e.g. ABC Architecture Ltd)
- Person (e.g. John Design)
- Person (e.g. Alison Height)
- Company (e.g. Great Designs Ltd)
- Company (e.g. ABC Architecture Ltd)
- Role (e.g QS)
- Company (Super Surveying Ltd)
- Role (e.g. Architect)
- Role group (e.g Clients)
- Etc
The basic API call to get this information is:
GET projects/{project_id}/roles
This will return the set of roles and the id’s for the associated companies and people, as follows:
{
"Architects": [{
"company_id": 401983,
"role_code": 31
}, {
"company_id": 3337535,
"people": [{
"person_id": 7294575
}],
"role_code": 23
}],
"Clients": [{
"company_id": 412159,
"role_code": 11
}]
}
The ids in this response can be used by the companies API and people API to get details of the companies and people respectively.
Instead of using the companies and people APIs, additional fields (project, company or people) can be retrieved in the response using the fields parameter. This will insert additional fields into the appropriate place within JSON response. For example:
GET projects/{project_id}/roles?fields=company_name
will return exactly the same as above, but with the company_name added:
{
"Architects": [{
"company_id": 401983,
"company_name": "Arcadis LLP",
"role_code": 31
}, {
"company_id": 3337535,
"company_name": "NBBJ",
"people": [{
"person_id": 7294575
}],
"role_code": 23
}],
"Clients": [{
"company_id": 412159,
"company_name": "University of Oxford",
"role_code": 11
}]
}
Companies API
The same principles described in the projects APIs apply when searching for companies and getting the company details, so the APIs are easy to use once the projects API is understood, for example:
GET companies
GET companies?query={...}&fields=company_id,company_name
GET companies/{company_id}
The most common use of the companies API is to search for companies by company name. These searches have in-built fuzzy matching. An example:
GET companies?query={
"company_name":{
"operator": "=",
"value": "Transport for London"
}
}
If no sort order is specified, the results will be ranked by best match with the exact matches will appearing first, following by increasingly less confident matches.
Searching the company_text will search the following fields for matches
- company_id
- company_name
Apart from searching for companies, a common use of this API is to get details for a set of companies based on the payload of a project roles API call. To achieve this, use a combination of the query and fields parameters as follows:
GET companies?query={
"company_id":{
"operator": "=",
"value": [3466500,3470364,3472020]
}
}&fields=company_id,company_name,company_phone,company_latitide,company_longitude
##Aggregations
Example 1: Top architects (role 23) based on the value of projects they are working on:
GET companies?query={
"project_roles":{
"operator": "=",
"value": 23
}
}&sort=-sum_project_value&fields=company_name,sum_project_value
Note, the total value can be displayed by requesting sum_project_value using the fields parameter.
If you want to see if any of these projects have estimated values, include any_project_value_estimated in the fields parameter. This boolean key will be true if one or more of the project values are estimated.
Example 2: Top 10 contractors (role 40) currently working on the most projects:
GET companies?query={
"project_roles":{
"operator": "=",
"value": 40
},
"project_start":{
"operator": "<=",
"value": 0
},
"project_finish":{
"operator": ">=",
"value": 0
}
}&sort=-count_project_id&limit=10
In this query, “currently working” is achieved by filtering on-site projects, those which started in the past and are due to finish in the future.
Example: 3: Architects working on the most recently updated residential projects
GET companies?query={
"project_roles":{
"operator": "=",
"value": 23
},
"project_primary_sector":{
"operator": "=",
"value": 100
}
}&sort=-max_last_project_published
People API
This API follows the same principles as projects and companies.
Filtering on people can be achieved by the standard query language, for example:
GET people?query={
"person_first_name":{
"operator": "=",
"value": "John"
},
"person_first_name":{
"operator": "=",
"value": "Smith"
}
}
This will perform exact matches on these fields.
A fuzzy text search field of both person_first_name and person_last_name is also available for contact names, using the person_full_name field as follows:
GET people?query={
"person_full_name":{
"operator": "=",
"value": "John Smith"
}
}
Searching the person_text will search the following fields:
- person_id
- person_full_name
- person_email
- person_phone
Contacts can be suppressed or removed from the Barbour ABI products as part of the Barbour ABI GDPR processes. When this happens, the person and their person_id will no longer be included in any of API response. See https://barbour-abi.com/gdpr for more details.
Fields API
To get a full list of fields available with your subscription:
GET fields
This responds with a list of fields and properties of these fields. Below is a sample response with limited fields for illustration purposes:
{
"project": {
"fields": {
"project_id": {
"display_name": "Project ID",
"description": "Unique ID for project",
"can_sort": true,
"can_display": true,
"can_export": true
},
"project_planning_stage": {
"description": "Planning stage",
"lookup": "planning_stage",
"can_display": true
}
}
},
"company": {
"fields": {
"company_id": {
"description": "Unique ID for company",
"can_sort": true,
"can_display": true
}
}
}
}
All field can be searched using the query language.
The boolean can_sort indicates if the field can be sorted, using the sort parameter.
The boolean can_display indicates if the fields can be returned, using the fields parameter.
If a property is not return in the response, then it is false.
To get just the project, company or person related fields:
GET fields?object_type=project
GET fields?object_type=company
GET fields?object_type=person
Pagination
For all search calls, the top n records are returned by default, so to retrieve a larger or complete dataset a series of API calls must be made, using the limit and offset parameters.
For example, to get the first 30 records, in three batches of 10 at a time:
GET projects?limit=10&offset=0
GET projects?limit=10&offset=10
GET projects?limit=10&offset=20
If not specified, the default values for limit and offset are 50 and 0 respectively.
The maximum value for the limit parameter is 500.
The use of limit and offset allows you to iterate through the first 10000 records, but not beyond. For example a limit of 9995 with an offset of 10 will fail.
To get a larger dataset use the query parameter to reduce the total results. For example, query using the project_last_published date to get the the projects from August 2018, then iterate through these using limit and offset. Then get the the results for July 2019 and iterate through these etc.
For the best user experience, repeated API calls with limit of between 10 and 200 are recommended for performance reasons.
Merges (listing duplicates)
Duplicate projects are sometimes created, so are merged into single project when identified. When this happens all references (tags, favourites etc) are moved from the deleted project to the kept project. The following API provides the pairs of projects being merged so the necessary updates can be made to data stored outside the Barbour ABI repository.
GET projects/merges
The response will be as follows:
{
"merges": [
{
"project_id_deleted": 12483153,
"project_id_kept": 12444973,
"project_merged": "2020-03-03T17:20:08Z"
},
{ ... }
]
}
The limit and offset parameters can be used to iterate through the full data set.
The query parameter can be used to reduce the merge results, for example to find the project merges from the last 7 days:
GET projects/merges?query={
"project_merged":{
"operator": "..",
"value1": -7,
"value2": -1
}
}
To find which projects a deleted project was merged into:
GET projects/{project_id}/merges
The same principle applies to merged companies:
GET companies/merges
Experimental
Role query prioritisation
Consider the following query:
GET projects?query={
"project_geocode": {
"operator": "=",
"value": "ENCH"
},
"project_roles": {
"operator": "=",
"value": 23
},
"company_geocode": {
"operator": "=",
"value": "WS"
}}
The expected results would be projects:
- In Cheshire (geocode ENCH)
- With an Architect (role 23)
- With at least one company working in Wales (geocode WS)
However, consumers are typically interested in how the role is associated with the project, company or person. To achieve this, if the API query includes a role filter, the role filter will be associated with people fields first, followed by companies and finally projects. The resulting behaviour is therefore as follows:
| Project location | Role | Company location | Person name | Actual result |
|---|---|---|---|---|
| Cheshire | Architect | Projects in Cheshire with an architect | ||
| Cheshire | Architect | Wales | Projects in Cheshire with an architect in Wales | |
| Cheshire | Architect | Wales | Giles | Projects in Cheshire with an architect in Wales and a contact of that company working in Wales is Giles |
| Cheshire | Architect | Giles | Projects in Cheshire with a contact called Giles working as the architect | |
| Architect | Wales | Projects with an Architect in Wales |
Any questions?
For support using the Barbour ABI APIs, please sign into the Barbour ABI platform and use the “Chat with us” functionality in the bottom left of the Web platform.
