Integration

Get Involved. Join the Conversation.

Topic

    Jomar Engalan
    Aggregate data during Read Stage File in Chunks
    Topic posted August 1, 2019 by Jomar EngalanGreen Ribbon: 100+ Points, last edited August 4, 2019, tagged API, Integration, xml 
    34 Views, 3 Comments
    Title:
    Aggregate data during Read Stage File in Chunks
    Content:

    On our integration, I have an API call inside the scope of Read Stage File in Segments and we are reading the file as XML from UCM. During run, API calls are getting called per each chunk of data coming from Read File. Is it possible for Read Stage file to aggregate the data so we can send it only once to the API? 

    Thanks,

    Jomar

     

    Comment

     

    • Hemanth Lakkaraju

      Looks like your design is forced that way! Why don't you read entire file instead of segments?

      • Jomar Engalan

        Does read entire file have a limitation in file size? We did it in segments as our main concern on our integration is to handle files exceeding 10mb in size.

        • Hemanth Lakkaraju

          Yes 10MB is the limit. In that case, see if you can aggregate the data in a csv using stage write append and then map the data back from aggregated csv to actual API call outside read segments loop.