Planning

Get Involved. Join the Conversation.

Topic

    Yaniel Delgado
    What is the maximum size for a CSV file to upload from FDMEE...
    Topic posted December 7, 2018 by Yaniel DelgadoGreen Ribbon: 100+ Points, tagged Reports 
    124 Views, 1 Comment
    Title:
    What is the maximum size for a CSV file to upload from FDMEE to EPBCS in Data Load Rule?

    Comment

     

    • Peter Nitschke

      Hey Yaniel, 

      To the best of my knowledge there isn't a 'maximum' size for a CSV file as such - you're more likely to hit a performance problem in the actual loading process once the data volume gets too big. 

      For an example, we tested the following loading directly to Data Management in PBCS:

      Row Count: 1m Rows, 25 columns

      Size: 110mb

      Mapping: Mostly #SQL mapping 

      Time Taken: 4 mins to upload, 4 mins to map, 1 min to load (we had a 10 min goal)

      We did have an issue with EPMAutomate timing out and had to change to the REST API to upload the file (note this is likely fixed with a recent EPMAutomate update). The workbench performance was reasonable poor - slow to open and slow to filter (circa 20-30 secs to refresh the page) because of the volume of data, but still reasonably manageable. 

      If you're going with much greater volume than that I would recommend looking at using FDMEE on-premises and connecting directly to the source system - using SQL queries on the source side to split the loads up as needed. 

      Cheers

      Pete