> For the complete documentation index, see [llms.txt](https://docs.brigge.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.brigge.ai/documentation/brigge-for-engineers/systems.md). # Systems ## Introduction In Brigge, a *system* is the mechanism used to connect to a third-party application or service. It contains everything required to connect and sync data. You'll find a library of standard systems in the Brigge [marketplace](/documentation/getting-started/marketplace.md), ready to install and use. Simply find the required system and install it, then [add your credentials](#credentials). Once this is done, the system becomes available for selection when configuring [*Connector* nodes](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md) in pipelines. If you can't find a system that you need in the marketplace, you can build it with Brigge's [system builder](#building-custom-systems) (or as Briggette to build it for you). {% hint style="info" %} In Brigge, you'll find all systems installed and/or built for your organisation on the [*Systems & Credentials*](#user-content-fn-1)[^1] page. {% endhint %} ## Breaking down a system Before starting work with systems, it's important to understand the moving parts and how they are related:
These parts are summarised below and then detailed in subsequent sections: {% stepper %} {% step %} **System (the external app/service)** A *System* is configured with [*Authentication flows*](#authentication-flows) and [*Endpoints*](#endpoints). An *Authentication Flow* determines how the *System* is authenticated (for example, Basic Auth, OAuth, API key), and also specific *System* behaviour when a given set of *Credentials* and *Endpoint* are selected in pipelines. {% endstep %} {% step %} **Credentials (allow access that app/service)** To authenticate a *System* for use in pipelines, you need [*Credentials*](/documentation/brigge-for-engineers/credentials.md) - a named set of authentication values. To define a set of *Credentials*, you select an *Authentication Flow* and provide the associated details (e.g. For a *Basic Auth* flow, you will likely provide a username and password). Credentials are stored securely and never exposed to users once saved. A *System* might be associated with one set of *Credentials* (for example, an API token that authorises access to Production data), or multiple sets (for example, an API token that authorises access to Production data and another API token that authorises access to Sandbox data). {% endstep %} {% step %} **Pipeline connector nodes (connect to the app/service)** To use a *System* in a pipeline, you choose a set of *Credentials* and an *Endpoint.* Keep in mind that selected *Credentials* are associated with an underlying *Authentication Flow*. So, if the selected *Endpoint* references authentication variables, these are resolved at runtime from values provided in the *Authentication Flow* setup*.* {% endstep %} {% endstepper %} ## Building custom systems With Brigge's *System Builder*, you can build any system that you need to integrate. When configuring a system, you specify its connection type (HTTP/S, FTP/S, SFTP, S3, or SQL), at least one *Authentication flow*, and required endpoints (the specific requests you'll make). The simplest way to create a system is to let our AI assistant do it for you. Simply ask Briggette to build the system and paste a link to the relevant API documentation. However, if you prefer a manual approach, you can create a system from the *Systems & Credentials* page:
From here, you build the system in three stages: [general details](#basic-details), [authentication flows](#authentication-flows), and [endpoints](#endpoints):
### General details Provide a name, logo, and general details for your new system:
General details guidance Fields and options on the *Systems* ➥ *Basic Details* page are summarised below:
Field/optionSummary
LogoEvery system requires a logo. Click the placeholder to upload an image and crop to size as needed. Supported file types are: .png, .jpg, .gif, .bmp, .svg.
NameEnter a name to identify this system throughout Brigge.
DescriptionOptionally, enter a description. This can be helpful for other users, and also for Brigge's AI tools.
ProtocolSelect the required protocol. Available options are:
HTTP/S, FTP/S, SFTP, S3, SQL. Note that the protocol cannot be changed after the system is saved. If you save the wrong protocol, you should delete the system and start again.
Rate limitThe maximum number of requests this endpoint can make within a set time period (e.g. 100 requests per minute). Requests over the limit are rejected until the period resets. Typically, the API documentation associated with this system will provide guidance on how this option should be set.
Date formatSelect the expected date format for data received from or sent to this system.
Colour (Hex)Enter a hex code directly or use the colour picker to set an accent colour for this system.
After saving these details, you can go to add [authentication flows](#authentication-flows) and [endpoints](#endpoints)
### Authentication flows An *Authentication Flow* determines: * How the *System* is authenticated (for example, Basic Auth, OAuth, API key). You choose an *Authentication* *Flow* and then, if necessary, add/update authentication variables to determine the steps a user must complete and what information they provide to authenticate. * Specific *System* behaviour in pipelines when a given set of *Credentials* and *Endpoint* are selected in a [connector node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md). For example, you can define authentication variables that are never exposed to users, but are referenced in endpoints and resolved at pipeline runtime when this authentication flow is used. #### Working with authentication flows Brigge includes a range of authentication flow templates that can be used as-is, or as a starting point for you to update. You must provide at least one authentication flow (otherwise there would be no way to authenticate the system). If needed, you can configure multiple flows - for example: * If a system supports multiple authentication types * If you need multiple 'flavours' of the same authentication type
Adding an authentication flow {% stepper %} {% step %} **Select the create authentication flow option** In the left panel, click the options menu (three dots) for `authentication`, then select `create authentication flow`:
{% endstep %} {% step %} **Enter a name and select an authentication flow template** The name specified here is displayed users when [adding credentials](/documentation/brigge-for-engineers/credentials.md) for this system:
{% endstep %} {% step %} **Select an authentication template** Brigge includes a range of authentication flow templates that can be used as-is, or as a starting point for you to [update (and optionally save as a template](#updating-an-authentication-flow)). {% hint style="info" %} For detailed information about configuring some of the most commonly used authentication templates, see [Authentication flow templates](#authentication-flow-templates). {% endhint %} Select an `authentication flow template`, then [review the configuration](#configuring-an-authentication-flow), [test the flow](#testing-an-authentication-flow), and [update settings](#updating-an-authentication-flow) as needed. {% endstep %} {% endstepper %}
Configuring an authentication flow Having [created an authentication flow](#adding-an-authentication-flow) (or selected an existing one), *General Settings* and options are displayed. For example:
Items on this page are summarised below. {% stepper %} {% step %} **General settings** The *General Settings* tab includes general information and options for the selected authentication flow:
FieldSummary
Authentication flow nameThis name is displayed to users when adding credentials for this system. You can update this if needed.
Documentation/help linkIt can be helpful to provide a link to the API documentation that you're using to build this system, for future reference.
Variables

Authentication flow variables are used to specify values that need to be passed into requests at runtime. They serve two purposes:

  • To determine fields that users must complete when adding credentials for this system. If a variable is configured with the display type set to form, users are prompted to enter (or confirm) the value when adding credentials.
  • To define values that can be added to system endpoints and resolved at runtime when the endpoint is used in conjunction with this authentication flow.

Authentication flow templates are pre-configured with known variables that the flow needs to work. These are generic - check your API documentation for the application/service you're integrating and update, add, or remove variables as needed.

{% endstep %} {% step %} **Authentication requests** An authentication flow might make one or more requests to authenticate associated [credentials](/documentation/brigge-for-engineers/credentials.md). Any requests are shown to the right of *General Settings*. For example:
If multiple requests are present, they run in sequence from left to right. If needed, you can add more requests using the `add request` button. {% hint style="info" %} Not all authentication flows need requests. For example: * *Basic auth* and API key types pass credentials into endpoint request headers on every call. In this case, you ensure that each endpoint is configured for credentials to be passed as headers. To test authentication, you test the endpoint. * OAuth 2-style flows (e.g. authorisation code, refresh token) require a preliminary call to an authentication server to obtain a token, and perhaps a second call to refresh it once expired. In this case, you would add any requests needed for authentication to happen. {% endhint %} Authentication request settings are the same as endpoint requests. For more details, see [Working with system requests](#working-with-system-requests)). {% hint style="warning" %} It can be useful to add requests here for testing purposes. That's fine, but remember to remove any unnecessary requests when you've finished testing, otherwise they will be considered part of the authentication flow. {% endhint %} {% endstep %} {% step %} **Save** If you've made any changes to the authentication flow (for example, [updating or adding variables](#working-with-variables)) they must be saved to take effect. {% endstep %} {% step %} **Test auth flow** If this authentication flow has one or more requests, use the `test auth flow` button to step through an authentication flow and check it's working correctly. If multiple authentication flows are defined for the system, you can choose which one to test - then enter (or select existing) credentials to be tested. {% hint style="warning" %} Unsaved changes are ignored during testing. {% endhint %} {% endstep %} {% endstepper %}
Working with authentication flow variables #### How to expose authentication variables for user input If a variable is configured with the `display type` set to `form`, users are prompted to enter (or confirm) a value when [adding credentials](/documentation/brigge-for-engineers/credentials.md) for this system. For example:
A basic auth flow includes two variables (username and password), both with display type set to form:
User adds credentials and selects the basic auth flow. They are prompted to authenticate with a username and password:
#### How to reference authentication variables in system endpoints You can define variables that aren't necessarily needed for authentication, but can be referenced in endpoint configurations and applied at runtime. The examples below show how this can work. **Example 1** Most REST API endpoints have the same base URL. You can define this as a variable on the authentication flow once, and then reference this in endpoints as needed. {% stepper %} {% step %} **Authentication flow** Add a `base_url` variable with a static value of `https://myexample.com/api` and the `display type` set to `none` (so it's not exposed to users when they are adding credentials). {% endstep %} {% step %} **Endpoints** Reference the `base_url` variable in endpoint URLs, for example: `GET` `{{base_url}}/contacts` `POST` `{{base_url}}/contact` {% endstep %} {% step %} **Credentials** Add credentials which use this authentication flow. {% endstep %} {% step %} **Pipeline connector nodes** Configure a pipeline [connector node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md) to use these credentials. The full URL for the selected endpoint resolves at runtime from the `base_url` on the authentication flow associated with selected credentials. {% endstep %} {% endstepper %} **Example 2** Let's take example 1 a step further. Suppose your API has *two* base URLs - one for a Sandbox and another for Production. {% stepper %} {% step %} **Authentication flows** Define two authentication flows, e.g.: *Sandbox (basic auth)* and *Production (basic auth)*. Both would have the same authentication setup, but different `base_url` values. For example: `https://sandbox.myexample.com/api` `https://production.myexample.com/api` {% endstep %} {% step %} **Endpoints** Reference the `base_url` variable in endpoint URLs, for example: `GET` `{{base_url}}/contacts` `POST` `{{base_url}}/contact`. {% endstep %} {% step %} **Credentials** Define two sets of credentials: * Sandbox credentials (using the *Sandbox (basic auth)* authentication flow. * Production credentials (using the *Production (basic auth)* authentication flow. {% endstep %} {% step %} **Pipeline connector nodes** Configure a pipeline [connector node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md) to use either *Sandbox* or *Production* credentials. The full URL for the selected endpoint resolves at runtime from the `base_url` on the authentication flow associated with the selected credentials. {% endstep %} {% endstepper %}
Duplicating an authentication flow It can sometimes be useful to make a copy of an authentication flow - for example, if you want different flavours of the same flow, or to take a backup before making changes. To do this:
Cover image
Hover your cursor over the required authentication flow and click the three-dot options menu. dupe auth flow a.png
Select the duplicate option from the options menu. dupe auth flow b.png
A new authentication flow is added to the list. It has the same name as the original flow, appended with an incremental number. You can now rename and update this flow as needed.dupe auth flow c.png
Deleting an authentication flow If an authentication flow is no longer in use, you can delete it. To do this, follow the steps below:
Cover image
Hover your cursor over the required auth flow and click the three-dot options menu. delete auth flow a.png
Select the delete option from the options menu. When prompted, confirm this action.delete auth flow b.png
### Endpoints *Endpoints* are the API requests that will be available for selection in pipelines when a [connector node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md) is configured to use the system. For example:
#### Working with endpoint groups In Brigge, endpoints are added to groups. If a system requires lots of endpoints, this is useful to organise them into logical categories. {% hint style="info" %} By default, new endpoints are ungrouped in the sidebar, which means users will see a flat list of endpoints when configuring [connector nodes](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md). If you [assign endpoints groups](#configuring-an-endpoint-general), users can navigate groups to find the required endpoint. {% endhint %}
Creating an endpoint group At least one group must exist before [endpoints are created](#creating-an-endpoint). If no groups exist, click the `create endpoint group` button in the left panel. For example:
Otherwise, follow the steps below:
Click the endpoints three-dot options menu. For example:
Then select create endpoint group. For example:
A new group is added to the end of the endpoint list. Hover your cursor over the placeholder name, then click the edit icon. For example:
Enter an confirm your preferred name:
Duplicating an endpoint group #### Introduction When an endpoint group is duplicated, the group and its associated endpoints are cloned and added to the end of the endpoints list. Existing names are appended with 1 or (if a clone already exists) the next incremental number. For example:
#### How to duplicate an endpoint group To duplicate an endpoint group, follow the steps below:
Hover your cursor over the endpoint group that you want to duplicate. A three-dot options menu is shown, for example:
Then select duplicate. For example:
A new group is added to the end of the endpoint list. Hover your cursor over the placeholder name, then click the edit icon. For example:
Enter an confirm your preferred name:
Deleting an endpoint group #### Introduction Deleting an endpoint group removes the group only - it does not remove associated endpoints. If endpoints are present, they are categorised as `ungrouped`. If needed, you can [move ungrouped endpoints to a different group](#moving-an-endpoint-to-a-different-group). #### How to delete an endpoint group To delete an endpoint group, follow the steps below:
Hover your cursor over the endpoint group that you want to delete. A three-dot options menu is shown, for example:
Then select delete. For example:
#### Working with endpoints
Creating an endpoint #### Introduction To use a system in a pipeline, it must include at least one endpoint. To create an endpoint, click the `endpoints` three-dot menu, then select `create endpoint`. For example:
A new ([ungrouped](#moving-an-endpoint-to-a-different-group)) endpoint is created, ready to configure. Configuration requirements will vary, depending on the system and the type of endpoint you're adding. #### Working with endpoints The follow sections explain how to configure a new endpoint in general terms: * [Configuring an endpoint (general)](#configuring-an-endpoint-general) * [Configuring an endpoint (request)](#configuring-an-endpoint-request) * [Configuring an endpoint (schemas)](#configuring-an-endpoint-schemas) {% hint style="info" %} If you need to create an endpoint that has similar settings to one that exists already, it is often quicker to [duplicate an existing endpoint ](#duplicating-an-endpoint)and then update the configuration. {% endhint %}
Configuring an endpoint (general settings) #### Introduction The *General settings* tab is where you define information and options for the endpoint as a whole. Some are informational, whereas others determine behaviour when this endpoint is used:
#### Understanding endpoint options Items on this page are summarised below: {% stepper %} {% step %} **Endpoint name & description** Enter a name to be displayed wherever this endpoint is used in the system. Optionally, provide a description that may prove helpful for AI queries. {% endstep %} {% step %} **Enable pagination options** If the request is fetching data and pagination is required, toggle `enable pagination` to `on`. This activates a pagination tab, where you can configure the required pagination settings. For more information, see [Endpoint pagination options](#endpoint-pagination-options). {% endstep %} {% step %} **Assign group** By default, new endpoints are ungrouped in the sidebar, which means users see a flat list of endpoints when configuring connector nodes. If you have [created groups](#creating-an-endpoint-group), you can assign this endpoint here. {% endstep %} {% step %} **Variables for this request** Endpoint variables are used to specify values that need to be passed into requests at runtime. Any variables defined here can be referenced/utilised elsewhere in the endpoint setup (e.g. in [request](#understanding-request-setup-options) configuration or [pagination](#endpoint-pagination-options) setup). Variables can be exposed to users in pipelines (so the user can enter a value) OR, they might be hidden and referenced elsewhere in the endpoint setup, to be resolved at runtime. For example, you might have a utility pipeline that's designed to be run manually, where a user can enter an `id` for the data they want to retrieve. By configuring an `id` variable with `display type` set to `form`, it is exposed for users to complete when they add this endpoint to a pipeline connector node - this is shown below:
Add an id variable to the endpoint configuration:
This allows pipeline users to enter an id value to be retrieved:
See [Working with variables](#working-with-variables) for more information about adding and managing variables. {% endstep %} {% endstepper %}
Configuring an endpoint (request settings) #### Introduction An endpoint *is* a request, so there will only ever be one request associated with any endpoint. The setup for an endpoint request is the same as that for [authentication requests](#configuring-an-authentication-flow) - for more details, see [Working with system requests](#working-with-system-requests)). For information about testing requests, see [Testing system requests](#testing-system-requests).
Configuring an endpoint (schemas) #### Introduction A schema describes the expected structure of data associated with this endpoint. It describes which fields are expected, where they are expected, and their associated data types. #### Adding schemas for an endpoint Adding an endpoint schema is optional, but advisable. It helps [pipeline](/documentation/brigge-for-transformers/pipelines.md) users navigate the the data structure for field selection [when this endpoint is used](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md) (for example, when mapping data to/from this system), and, if needed, it can also be used to validate the data structure at any point in a pipeline. For more information about working with schemas in Brigge, see [Schemas](/documentation/brigge-for-transformers/schemas.md). You can add one schema, or multiple schemas for an endpoint. {% hint style="info" %} **Why would I need more than one schema for an endpoint?** The shape of data can change as it moves through a pipeline. For example: 1. A [connector node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md) might retrieve a single payload at the start of a pipeline (let's say a `tasks` array with 20 `task` objects) and this is passed into a [split node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/split-node.md). 2. The easiest way to configure a [split node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/split-node.md) is to select a schema associated with incoming data, and navigate the data structure to choose the element where data is split. So, the first schema that we need for our endpoint is for the data in its original, unchanged state (for example: *List tasks - received*). 3. Now suppose the [split node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/split-node.md) outputs 20 payloads, with one `task` object per payload. The shape of our data is now changed, so we'd need another schema to reflect the structure of a single-item payload (for example: *List tasks - split*). 4. Finally, suppose that the 20 payloads pass into a [map node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/map-node.md), where only a subset of fields is mapped (and therefore output). Once again, the shape of our data is changed, so we'd need another schema to reflect the structure of mapped data (for example: *List tasks - mapped*). {% endhint %}
Endpoint pagination options #### Introduction Pagination lets you pull large datasets in smaller chunks instead of one large request. Most APIs cap how many records they'll return at once (typically 100 or fewer), so pagination allows you to fetch consecutive pages until there is no more to retrieve. Every API implements pagination differently. For example, some use page numbers, some use limit/skip parameters, others use cursor tokens. {% hint style="info" %} The API documentation for the system you're building will normally provide information on the expected implementation. {% endhint %} #### The Brigge approach for pagination Brigge uses a flexible, rule-based approached to configure any pagination type needed. You build logic around three things:
ItemSummary
Pagination variableA parameter that 'keeps count' (e.g. skip or page), changing with each request so the API knows where to pick up next.
New valueAn expression that updates the pagination variable after each successful request, so the next call moves forward and previous data is not repeated.
Stop condition(s)Rules that tell Brigge when to stop fetching data. Typically, this is based on the response content - for example, stopping once a page comes back with fewer records than requested.
For more information, see [Pagination options](#pagination-options). #### The pagination loop When pagination is enabled, data is retrieved in 'pages', where each page contains a limited number of items. After the initial request, Brigge checks if there is any more data to retrieve (based on [defined rules](#the-brigge-approach-for-pagination)), If yes, another (pagination) request is made for the next page of data, and so on. Irrespective of the pagination type, pagination works broadly on a loop, as can be seen in the illustrationff below:
These steps can be summarised as follows: {% stepper %} {% step %} **Initial endpoint request** Sends the first API call using parameters on the endpoint *Request* tab and receives response. {% endstep %} {% step %} **Process response** This step is only needed if the [pagination setup](#pagination-options) requires values from the response (i.e. [tagged data](#tagging-response-data-for-a-system-request)). If so, values for tagged fields are used to updates variables so they're available for pagination. {% endstep %} {% step %} **Stop condition(s)** Checks configured conditions against current variable values to determine if another request should be made for more data. If conditions are met, the loop ends and we move to the next pipeline step. If conditions are not met, the loop continues. {% endstep %} {% step %} **Recalculate** Updates the `pagination variable` using the `new value` formula for the next pagination request. {% endstep %} {% step %} **Next pagination request** Sends the next request for data using parameters on the endpoint *Pagination* tab (including the recalculated `pagination variable`, then loops back to Step 2. {% endstep %} {% endstepper %} #### Pagination options In Brigge, [enabling pagination for an endpoint](#configuring-an-endpoint-general-settings) activates a *Pagination* tab. This is where you define pagination behaviour and request settings for the required pagination type:
{% hint style="info" %} Pagination logic can (and usually does) reference other endpoint setup tabs, so there are three areas to consider: * [*General* settings tab](#configuring-an-endpoint-general-settings). Any variable created here can be referenced (by name) anywhere else in the endpoint setup, including pagination options. This is also where pagination is enabled. * [*Request* tab](#configuring-an-endpoint-request-settings). Defines the *first* call Brigge makes to the endpoint. Parameters defined here can have static values, or references to variables. A parameter is only sent if its `default` toggle is `on`. This tab also includes [tagged data](#tagging-response-data-for-a-system-request), which enables you to inject the value of a field found in response data, into a specified variable. In this way, the variable value is updated dynamically, ready for use downstream. * *Pagination* tab. When pagination is enabled, a *Pagination* tab is activated for the endpoint. This is used to define pagination behaviour, including parameters to be applied for every pagination request *after* the first endpoint request. Pagination request options take precedence for all subsequent requests until pagination stops. {% endhint %} In the steps below, we'll walk through these options and configure offset pagination for a `list contacts` endpoint, as an example. {% stepper %} {% step %} **Request URL** This is the URL to be used for pagination requests - i.e. all requests (after the initial endpoint request) to retrieve more pages until pagination stops. Typically this will be the same as the main request (on the [*Requests* tab](#configuring-an-endpoint-request-settings)) but, if needed, it can be different. For example:
{% endstep %} {% step %} **Pagination variable** This variable represents 'where we are' in the pagination sequence. Use the dropdown list to select an existing endpoint variable - all variables on the *General* tab are available for selection here. For our example, three variables have been added for this endpoint:
For the `pagination variable`, we select the `skip` variable (which we can see has a `default value` of `0`):
Also notice that we have two other variables with default values - we'll need these later: * `limit` is set to `10` * `Remaining` is set to `1` {% hint style="info" %} In the context of offset pagination, `skip` is our system's equivalent name for `offset` - i.e. it represents the number of records to be skipped (because they've already been retrieved) before retrieving the next page. {% endhint %} {% endstep %} {% step %} **How the pagination value is recalculated** Next, we define an expression that recalculates the pagination variable before each subsequent request. What this expression looks like depends entirely on the pagination type; for our `offset` pagination we are adding `skip` and `limit` variable values together:
Here, we are saying: set the `skip` variable value to the sum of `skip` (currently `0`) and `limit` (currently `10`). So, the `pagination variable` (`skip`) is set to `10`. {% endstep %} {% step %} **Rules to determine when pagination stops** Next, we define `stop conditions` to determine when Brigge stops requesting further pages. A `stop condition` is a rule which evaluates the value of a given variable against a defined value. {% hint style="info" %} Stop conditions are optional - if your pagination has some other way to determine when pagination stops, you can ignore this section. If you need multiple stop conditions, use the `add stop condition` option to add more (when multiple stop conditions are present, pagination stops if any one of them is met). {% endhint %} The variable value to be evaluated is selected from a dropdown list - again, all variables on the *General* tab are available for selection. For example:
Here, we selected the `remaining` variable. So, we are saying: check the current value of the the `remaining` variable; if it's less than or equal to `0`, stop pagination (otherwise continue). However, looking at our example, how is the `remaining` variable updated? We've already seen that it has a default value of `1`, but it will need an update after every request for this to be a valid condition. This is a scenario where [tagged data](#tagging-response-data-for-a-system-request) is useful. {% hint style="info" %} **What is tagged data?** Tagged data is a way to inject the value of a field found in response data, into a specified variable. In this way, the variable value is updated dynamically, ready for use downstream. For more information about how data is tagged, see [Tagging response data for system requests](#tagging-response-data-for-a-system-request). {% endhint %} Tagged data is added via the endpoint [*Request* tab](#configuring-an-endpoint-request-settings). For our example, the external service being integrated returns a `remaining-count` field in every response, which is the calculated number of pages left to retrieve. By tagging this field and mapping it to our `remaining` variable, we inject the `remaining-count` value, so the remaining value always reflects the current number of pages left to retrieve.
one or more checks, each comparing a variable against a value with an operator. When a condition evaluates true, Brigge stops requesting further pages. {% endstep %} {% step %} **Pagination request settings** The *Pagination* tab includes its own request `parameters`, `headers` and `body` settings. These work in exactly the same way as the [main endpoint *Requests* tab](#configuring-system-requests), but any settings defined here only apply to pagination requests (i.e. all requests after the first). This means that - if needed - the shape of pagination requests can be quite different to the main request. Any pagination settings defined here take precendence after the intial request. {% endstep %} {% endstepper %}
Duplicating an endpoint It can sometimes be useful to make a copy of an endpoint - perhaps you want to make a backup before changing settings, or maybe you have multiple endpoints that share the same (or similar) settings and want to avoid configuring each one manually.
Cover image
Hover your cursor over the required endpoint and click the three-dot options menu. dupe endpoint a1.png
Select the duplicate option from the options menu. dupe endpoint b.png
A copy of the endpoint is added to the end of the originating endpoint group. It has the same name as the original endpoint, appended with an incremental number. You can now rename and update this endpoint as needed. dupe endpoint c2.png
Moving an endpoint to a different group To move and endpoint to a different group, access its [general settings](#configuring-an-endpoint-general-settings) and select the required group from the dropdown list (all existing endpoint groups are available for selection). For example:
Deleting an endpoint If an endpoint is no longer in use, you can delete it. To do this, follow the steps below:
Cover image
Hover your cursor over the required endpoint and click the three-dot options menu. delete endpoint b.png
Select the delete option from the options menu. When prompted, confirm this action.delete endpoint b.png
{% hint style="info" %} You can't delete an endpoint if its configured in a pipeline (irrespective of whether that pipeline is active). If you attempt to do so, you'll see a warning which includes any pipelines where this endpoint is used. {% endhint %}
### System requests & variables
Configuring system requests #### Introduction When configuring [endpoints](#working-with-endpoints) and (depending on the flow type) [authentication flows](#authentication-flows), you define requests to the external application/service that you're integrating. Request settings and options are the same, irrespective of the context. Key elements are shown below:
{% hint style="info" %} An [endpoint](#working-with-endpoints) request will only ever be associated with one request, whereas an [authentication flow](#authentication-flows) can have multiple requests. {% endhint %} #### Understanding request setup options Request options shown above are summarised below: {% stepper %} {% step %} **API request URL** Enter the URL for this request. This might be a full URL, or one that includes [authentication flow variables](#working-with-authentication-flow-variables). {% hint style="info" %} You can use authentication flow variables to swap in common parts of a URL at runtime. For example, suppose 30 endpoint URLs reference an API version that's likely to change over time. You could enter the full URL for each endpoint - but when the API version changes, you'd need to make 30 updates. Alternatively, you could define a variable for the API version (e.g. `api_version` set to `1.0`) and reference this variable in endpoint URLs - so:\ \ `https://production.myexample.com/api/1.0/contacts` ...would be entered as: `https://production.myexample.com/api/{{api_version}}/contacts`\ \ At runtime, the `api_version` variable resolves to its current value (1.0). When the API version changes, you simply update the authentication flow variable. Next time endpoints are triggered in pipelines (with the associated authentication flow) the `api_version` resolves with the latest value. {% endhint %} {% endstep %} {% step %} **Request parameters** Request parameters define queries/filters, appended to the request URL at runtime. URL parameters are added as 'key pairs' - enter a `key` (a valid field name for this API) and an associated `value`. {% hint style="info" %} Check your API documentation for valid parameters. {% endhint %} Parameter values can be: * Static. You provide the required value here and this is used every time the request runs. * Dynamic. You reference an [authentication flow variable](#working-with-authentication-flow-variables); at runtime the the value resolves to whatever value is associated with the credentials used for the run. {% endstep %} {% step %} **Request headers** Request headers are added as 'key pairs' - enter a `key` (a valid field name for this API) and an associated `value`. They work in the same way as request parameters, except they are passed in the request headers. Like parameters, header values can be: * Static. You provide the required value here and this is used every time the request runs. * Dynamic. You reference an [authentication flow variable](#working-with-authentication-flow-variables); at runtime the the value resolves to whatever value is associated with the credentials used for the run. {% endstep %} {% step %} **Request body** Use `body` options to determine what (if any) content is passed in the `body` section of requests. The following options are available:
OptionSummary
NoneNo body is sent with the request. Typically used for methods that send all required information in the request URL/parameters (e.g. GET or DELETE for REST API requests).
Pipeline payloadAdds content from incoming payloads to the request body.
RawA static, raw text body - for example, JSON, XML, plain text.
Form data (URL encoded)Sends key-value pairs as application/x-www-form-urlencoded (the same format used by traditional HTML forms). Data is encoded into a single string like key1=value1&key2=value2. This type does not support file uploads.
Form data (multipart)Sends key-value pairs as multipart/form-data, where each field is a separate part of the body. This is the format required for file uploads, since it can carry binary data alongside standard fields.
GraphQLA body formatted specifically for GraphQL APIs - typically a JSON object containing a query (or mutation) string and an optional variables object, sent to a single GraphQL endpoint rather than varying by resource path.
{% endstep %} {% step %} **General request settings**
OptionSummary
Accepted status codesUse the dropdown list to select one or more response status codes which are considered as 'success' for this operation (e.g. 200 - OK). If a response status code not in this list is returned, the operation fails.
Timeout (seconds)Sets the maximum time, in seconds, that Brigge waits for a response before the request fails. If the external system doesn't respond within this window, the operation times out.
Follow redirectsDetermines whether Brigge automatically follows HTTP redirect responses (3xx status codes) from the target server. When enabled, redirects are followed to their final destination. When disabled, a redirect response is returned as-is rather than followed.
Bypass certificate verificationControls whether Brigge validates the SSL/TLS certificate of the target server. When enabled, certificate validation is skipped (this should only be used for testing or with trusted internal systems).
{% endstep %} {% step %} **Test & review the request** Once the configuration is saved, use the `test request` option to confirm it works as expected. {% hint style="info" %} If you are testing an endpoint request, you can choose to test with or without [pagination](#pagination-options), and which credentials to use. {% endhint %} When the test completes, you can inspect request and response details and, if needed, tag response fields to be passed into variables. For more information, see [Testing a request](#how-to-test-a-system-request). {% endstep %} {% step %} **Tagged data** Tagged data enables you to inject the value of a field found in response data, into a specified request variable. In this way, the variable value is updated dynamically, ready for use downstream (it can be particularly useful for pagination). For more information, see [Tagging response data for system requests](#tagging-response-data-for-system-requests). {% endstep %} {% endstepper %}
Testing system requests #### Introduction Having configured a request, use the `test request` button to run a test (with or without a payload), and then inspect full request and response details. #### How to test a system request Follow the steps below: {% stepper %} {% step %} **Test request** Make sure that settings are saved, then click the `test request` button:
{% endstep %} {% step %} **Test request** Choose whether to `enable pagination` (if defined) for this test, then `choose credentials for test` and click `next`:
{% endstep %} {% step %} **Test request** Optionally, paste a sample payload - then click the `test` button. For example:
{% hint style="info" %} Sample payload behaviour depends on [how the request body is configured](#configuring-an-endpoint-request-settings). For example, is body content is set to none, no payload is expected and so you're not prompted to enter a sample. {% endhint %} {% endstep %} {% step %} **Review request & response output** The test request completes with a success or fail outcome. In the event of a failure, it's useful to inspect the actual request that was made - for example, check if the request URL resolved as expected; review the authentication outcome, etc. For example:
For full response details, check the response panel. Here, you'll see what's returned for the test, including metadata. For example:
{% hint style="info" %} If needed, you can tag response fields to be resolved in existing variables. {% endhint %} {% endstep %} {% endstepper %}
Tagging response data for system requests #### Introduction Tagged data is a way to inject the value of a field found in response data, into a specified variable. In this way, the variable value is updated dynamically, ready for use downstream. A common use case is for [pagination](#endpoint-pagination-options), where pagination details (e.g. number of pages retrieved, number of pages remaining, total page count, etc.) are returned with each response. We can tag these fields, map them to a variable, then reference these variables as URL queries for the next pagination request. For a walkthrough of this scenario, see [Endpoint pagination options](#endpoint-pagination-options). #### How to tag response data & map it to a variable To tag a response data field and then map it to an existing endpoint variable (so this response value is injected into the variable when received), follow the steps below: {% stepper %} {% step %} **Check endpoint variable status** Access [general settings for the endpoint](#configuring-an-endpoint-general-settings) and check that the required variable exists - if it doesn't [create it](#working-with-variables) now. In the example below, we will be using a variable named `remaining`.
{% endstep %} {% step %} **Test the endpoint request** Access [request options](#understanding-request-setup-options) for the endpoint and use the [test request option](#how-to-test-a-system-request) to initiate the request. {% endstep %} {% step %} **Inspect the response** Still on the [request options](#understanding-request-setup-options) tab, expand the `response` section to view response details for the request. For example:
{% endstep %} {% step %} **Tag the required field** Click the name of the field you want to tag. For example:
{% endstep %} {% step %} **Map the tagged field to a variable** Select the variable that you want to inject with values from tagged field. For example:
Then confirm with the `create` button:
Now you'll see that that `tagged data` section is updated with the new item. For example:
Wherever this variable is referenced, the value will always be set to whatever was returned in the associated tagged field in the previous request. {% endstep %} {% endstepper %}
Working with variables #### Introduction Variables can be defined and referenced throughout the setup for a system. For example, you [define variables for an authentication flow](#how-to-expose-authentication-variables-for-user-input) to determine what fields users are prompted to enter when adding credentials for the associated system. You might also define variables for endpoints, which can be surfaced for users to populate when [configuring connector nodes](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md) to use this endpoint. #### How are variable values populated? Variable values can be derived from: * User input; users are prompted to enter (or confirm) a value when working with this system in pipelines, or adding credentials. * System input; pipeline users never see these variables, they're just used to build logic for system behaviour. * Dynamic input; values are injected from [tagged response data](#tagging-response-data-for-system-requests). #### Adding & configuring variables You can add variables via *General settings* tabs for [endpoints](#configuring-an-endpoint-general-settings) and [authentication flows](#configuring-an-authentication-flow) - just click the `create variable` option. For example:
From here, configure behaviour for this variable:
These options are summarised below, using an endpoint variable as an example: {% stepper %} {% step %} **Type** Use the `type` dropdown to select the data type associated with expected values for this variable. Choose from the following types:
TypeSummary
TextA single line of free-form text input.
Multi-line textFree-form text that spans multiple lines (for longer values like descriptions or notes).
NumberA whole number (integer), with no decimal places.
Decimal numberA number that allows decimal places, for values needing precision (e.g. prices or measurements).
Date/timeA calendar date and/or time value.
Drop down listA predefined set of options from which users pick one value.
True/false booleanA toggle between two states, true or false.
{% endstep %} {% step %} **Display type** Use the `display type` dropdown to select if/how this variable is displayed. Choose from the following options:
TypeSummary
NoneThe variable is never exposed to pipeline or system users. Typically used for building system logic (e.g. for pagination).
Form

The variable is displayed for users to complete.

For authentication flow variables, the variable is displayed to the user when adding credentials for this system. The variable value provided by the user is applied whenever this variable is referenced in the setup.

For endpoint variables, the variable is displayed to the user when this endpoint is selected in a connector node. The variable value provided by the user is applied whenever this variable is referenced in the setup.

AdvancedThe variable is displayed for users to complete, in an advanced section of the relevant form.
{% endstep %} {% step %} **Label** Enter a text label to be used throughout the system UI, to identify this variable. {% endstep %} {% step %} **Key** The programmatic identifier - a unique string that can be referenced to retrieve this variable's value. {% endstep %} {% step %} **Default value** If needed, enter a default value for the variable. For [endpoint variables](#configuring-an-endpoint-general-settings), this can be useful (for example) to start pagination counts, or to pre-populate a user-facing variable that only needs to be changed occasionally. For [authentication flow variables](#working-with-authentication-flow-variables), a default value can be useful (for example) to pass in a standard `base_url` that can be referenced in endpoints when this auth flow is applied. {% endstep %} {% step %} **Description** Optionally, provide a description which will help anyone viewing this setup in future and/or to provide more context for Brigge AI tools. {% endstep %} {% step %} **Secret** Secret values are encrypted and never exposed in the Brigge UI. {% endstep %} {% step %} **Required** If a variable is defined with a form or advanced display type, toggle this option on if you want to prevent users from saving the associated [credentials](/documentation/brigge-for-engineers/credentials.md) or [connector node](/documentation/brigge-for-transformers/pipelines/pipeline-nodes/connector-node.md) form if no value is provided. {% endstep %} {% endstepper %}
## Managing systems
Exporting a system #### Introduction When a system is exported, all of its setup is written to a JSON file. If needed, you can then import this file to create a new instance of the system. This can be useful (for example) to take a backup of a system before making significant changes, or as a starting point to create another system with very similar properties. #### How to export a system
Cover image
Select the system that you want to export from the Systems page.export system a.png
Click the three-dot options menu next to the system name.export system b.png
Select the export system option. The system is exported immediately and saved to the default downloads folder for your browser.export system c.png
Importing a system #### Introduction To import a system that was previously exported from Brigge, select the `import system` option at the top of the *Systems* page. When prompted, upload the JSON file and confirm. The new system is added and General settings are displayed, so you can change the name and make any other changes required.
Deleting a system #### Introduction If a system is no longer needed and is not configured in any pipelines (even inactive pipelines) you can delete it. #### How to delete a system
Cover image
Select the system that you want to delete from the Systems page.delete system a.png
Click the three-dot options menu next to the system name.delete system b.png
Select the delete option. If the system is found to be in use in pipelines, you'll see a warning and the delete action is not completed. Otherwise, the system is removed. delete system c.png
[^1]: **How do I get there?**\ Select *environments* from the main navigation menu:
![](https://2172380191-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8IqdqwZowcxtFpZfLa6k%2Fuploads%2Fwu1tX0PnjZ91pW6OuQIO%2Faccess%20systems.png?alt=media\&token=97feee3c-8ed0-4ae1-8b0c-505af006236a)