Pipes: Loop Through Array and Create Records for Each

I am running into a scenario where I would normally only receive 1 result from a GET request, but instead I am getting 2 results. Ideally what I would like to do is loop through an array, and create a record for each result, instead of only being able to do just one.

When I receive 2 results from my get request, it means my record is now associated with 2 objects instead of just one which I cannot really do from tadabase it seems.

FYI for anyone getting more than 1 result from a GET request. right now, the best work around is that I must specify which result I want using something like 0.name or 1.name to specify which result to use

2 Likes

Hi @danv and welcome to the community.

We are probing into how we can implement something like this in the future. It’s not on our immediate roadmap and you’ll probably need to use an external tool like Integromat if you want to loop through results/arrays.

I’d love to get a better sense of how you were hoping to use this? Are you triggering a pipe inside a Record rule? Would you mind sharing a an example of a scenario?

I have a “sync settings” where I essentially store an access token and a user ID. The next step is to go get the organization id that the user belongs to and bring it in. It’s possible for a user to belong to more than one organization, so it’s more about error handling. It’s not that I need both records, but in having them, I can easily figure out which one is the one I want to work with automatically.

I tried using the pipe as a record rule, so when the user is created, the record and their association are recorded from a GET request. After they are created, I would update those on edit in case there was a change.

Update^

We want to do an initial import. We want to take records from our system, and get them into our app. Than we want to take records from another app, and get them into our app so we can get the two to work together…If I could loop through I would have a pipe button to initiate this import, or schedule every 30 minutes to look for records that match a condition (new accounts that haven’t had their initial import yet (import count =0).

We tell the user, your account will be ready in about 30 minutes, expect an email…

Right now, I’ll need to use a workaround, which is a 3rd party tool. In our case, because of HIPAA compliance we can’t use any of the mainstream tools (Integromat, Zapier and Parabola). We are going to try and use Tray.io for this (because it’s HIPAA compliant), it can do just about anything. We will essentially do something similar, though we might use a webhook. I am still trying to wrap my mind around the best way to build this.

In my mind’s eye, I can see something along these lines…

  • Loop each for identified [array] I think this could pretty pretty straight forward… You specify the array path (or default to the first array of the response). Than configure the response keys as is normally done.

  • Pagination would be nice, but everyone does pagination a little differently, but for MVP the most common I’ve seen are the response headers containing a total pages and current page value. specify total pages path and current pages path and specify a limit parameter

I also need the ability to loop through arrays.

I see a good use case too. For example, have the user press a button and pull the latest news from News API. (I’m pasting a sample of the API output below). App should then create a record for each news story. User would discard irrelevant news (delete records) and forward to HQ with comments the relevant ones.

{
"status": "ok",
"totalResults": 100,
"articles": [
    {
        "source": {
            "id": "engadget",
            "name": "Engadget"
        },
        "author": "Kris Holt",
        "title": "Spotify's Enhance feature puts suggested songs in your playlists",
        "description": "Spotify\r\n has long had plenty of tricks up its sleeve when it comes to providing recommendations\r\n, and its latest one is about making your playlists even more banging. The Enhance feature\r\n offers personalized suggestions of tracks to add to your playlists.P\u2026",
        "url": "https:\/\/www.engadget.com\/spotify-enhance-playlist-song-suggestions-180749023.html",
        "urlToImage": "https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2021-09\/5c984660-1197-11ec-afda-6845edb39d3c",
        "publishedAt": "2021-09-09T18:07:49Z",
        "content": "Spotify\r\n has long had plenty of tricks up its sleeve when it comes to providing recommendations\r\n, and its latest one is about making your playlists even more banging. The Enhance feature\r\n offers p\u2026 [+1030 chars]"
    },
    {
        "source": {
            "id": "engadget",
            "name": "Engadget"
        },
        "author": "Igor Bonifacic",
        "title": "WhatsApp starts testing local business directories",
        "description": "WhatsApp already allows you to chat with businesses, but you may soon also have the ability to find them through the app as well. This week, the company started testing a directory feature that allows users to scan through local shops and services that have a\u2026",
        "url": "https:\/\/www.engadget.com\/whatsapp-business-directory-test-sao-paulo-161835714.html",
        "urlToImage": "https:\/\/s.yimg.com\/os\/creatr-uploaded-images\/2021-09\/e6a06010-1708-11ec-9dff-289c8ff9b4f6",
        "publishedAt": "2021-09-16T16:18:35Z",
        "content": "WhatsApp already allows you to chat with businesses, but you may soon also have the ability to find them through the app as well. This week, the company started testing a directory feature that allow\u2026 [+1348 chars]"
    },
    {
        "source": {
            "id": "the-verge",
            "name": "The Verge"
        },
        "author": "Justine Calma",
        "title": "Activists push to delay most high-profile climate summit since Paris agreement",
        "description": "High-stakes climate negotiations, called COP26, are set to take place in Glasgow in November. Activists want to postpone the conference again because delegates from the most vulnerable countries have limited access to COVID-19 vaccines and could struggle to p\u2026",
        "url": "https:\/\/www.theverge.com\/2021\/9\/7\/22660795\/activists-cop26-united-nations-climate-change-summit-glasgow",
        "urlToImage": "https:\/\/cdn.vox-cdn.com\/thumbor\/1vrToT8-loiA3L9lezjvNfLK6-Q=\/0x185:2588x1540\/fit-in\/1200x630\/cdn.vox-cdn.com\/uploads\/chorus_asset\/file\/22831946\/1234739239.jpg",
        "publishedAt": "2021-09-07T18:46:28Z",
        "content": "Many delegates still dont have access to vaccines\r\nUN Climate Change Conference or COP26 President Alok Sharma during a meeting with India Minister of Environment, Forest and Climate Change Bhupender\u2026 [+4818 chars]"
    },

(…)

Our use case for this would be project/task creation. Each new project gets assigned 60-80 tasks from a template list of 100 or so.

The workflow currently looks like this:

  1. Button triggers webhook to Integromat with project criteria
  2. Integromat runs a GET API call for all records in the task template table that fit the project criteria and returns an array (60-80 of the possible 100 depending on the project)
  3. Integromat then parses the array and creates a task record for each value in the array and connects it back to the project

It would be great to be able to do this all within Tadabase

1 Like

I need to be able to update a one-to-many connection field. I need to be able to run a pipe call that would return a response with all record ids associated with records that matched a filter. So Tadabase API “Filter by a single field” call.

Then output those records IDs into text field in CSV format. Then I would be able to run a pipe that could update a one-to-many connection since I have the records ids stored in a text field in CSV format.

Hi all,

This is very doable in Integromat - you can set up loop functions to pull down arrays and then have workflow steps that create new records for each item in the array.

We also do the reverse of this in Integromat which is to pull down a bunch of records from Tadabase and build an array in Integromat that we then put into a nightly email. For instance: a list of all the records that were added today.

Hope that helps - let me know if you need more details.

-Dan

@moe The problem with using Integromat is that in order to trigger this in a real time workflow, you have to use a webhook. This means if you are running action rules, those rules could complete before Integromat does.

This is not a good solution is you are trying to aggregate record ids, store them into a one to many connection field and then run a form or action link against that newly updated one to many connection all during the same process. This is why we need to be able to do this internally and aggregate record ids as they are needed to be joined with a connected one to many field

Having this batch nightly is just not acceptable in the real word when things need to be updated in real time.

1 Like

Hi Moe. Has a solution been introduced or is one known that is within Tadabase that will parse through an array? For example, I want to import all orders from my Shopify store and create new orders within Tadabase using a pipe. I can’t seem to do that.

1 Like