Policy Automation for B2C Service

Get Involved. Join the Conversation.

Topic

    Abhishek Srivastava
    OPA Batch server not able to execute with more than one...
    Topic posted October 2, 2017 by Abhishek Srivastava, last edited October 3, 2017 
    151 Views, 5 Comments
    Title:
    OPA Batch server not able to execute with more than one subprocess
    Content:

    Hi All,

    I am using OPA Batch server to perform a task, and invoking batch processor using java project(using batch apis) . while executing batch processor with Number of processors = 1 works well, however, while executing the same with Number or processors = 2  results in following exception.

    com.oracle.determinations.batch.exceptions.BatchExecutionException: The following slave processors exited with error: [1] at 
    com.oracle.determinations.batch.local.SlaveSet.waitForAll(SlaveSet.java:60) at com.oracle.determinations.batch.BatchProcessor.execute(BatchProcessor.java:343) 
    

    I verify the logs generated by OPA and found in error log as below-

    ---------------ERROR.LOG-----------------------------
    2017-Sep-26 09:46:23,370 [Thread-14] com.oracle.determinations.batch.local.CSVWorkerWatcher
      ERROR error trying to read slave stream
    java.io.IOException: Stream closed
            at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
            at java.io.BufferedInputStream.read1(BufferedInputStream.java:272)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
            at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
            at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
            at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
            at java.io.InputStreamReader.read(InputStreamReader.java:184)
            at java.io.BufferedReader.fill(BufferedReader.java:154)
            at java.io.BufferedReader.readLine(BufferedReader.java:317)
            at java.io.BufferedReader.readLine(BufferedReader.java:382)
            at com.oracle.determinations.batch.local.CSVWorkerWatcher.run(CSVWorkerWatcher.java:23)
            at java.lang.Thread.run(Thread.java:745)

     

     

    and in my debug log I also found that -

     

    ---------------DEUBG.LOG-----------------------------

    2 - Skipping ahead: rowsToSkip=1

    2 - No more rows to read. End of data has been reached: readCount=2

    2 - No more rows to read from main table

    2 - No more cases to read

    2 - Merging summary file: filePath=/opt/DET/rulesengine/apache-tomcat-8.5.15/temp/obp-work7962966405099947650/slave-1.summary

    2 - Could not merge summary file '/opt/DET/rulesengine/apache-tomcat-8.5.15/temp/obp-work7962966405099947650/slave-1.summary': /opt/DET/rulesengine/apache-tomcat-8.5.15/temp/obp-work7962966405099947650/slave-1.summary (No such file or directory)

    2 - Deleting working directory: /opt/DET/rulesengine/apache-tomcat-8.5.15/temp/obp-work7962966405099947650

    2 - Deleting file path: /opt/DET/rulesengine/apache-tomcat-8.5.15/temp/obp-work7962966405099947650

     

    Now I am not sure why I am facing this issue and how to resolve the same.

    There is not much help documents are available over internet or OPA dumps. Appreciate any input on this.

     

     

     I am using java program to invoke the batch processor . My program outline is following.

     

    1. Create configuration

        DeterminationsBatchConfiguration configuration =configuration = new DeterminationsBatchConfiguration();

     

    2. Create Configuration Parser object

       ConfigurationParser configParser = new StringArrayConfigurationParser(configuration,new String[] { "--database", "db_url", "--userid", db_usrid", "--password", "db_pass", "--processors",2 , "--config", “rulebase.xml" }); 

       Here I am providing all batch configuration related details like database, processors and rulebase.xml file.

     

    4. BatchProcessor batchProcessor = new BatchProcessor("ServerName", configuration);

       Create batch processor instance

     

    5. batchProcessor.execute();

        execute the batch process.

     

    Quick clarification-

    Do we need to pass additional parameter for slave processes in java code( for instance memory or any other parameter )

    I have tried in my program as below

    configParser.getConfigurationOptions().addSlaveSystemParam("-Xmx128m -Xms128m ");

     

     

     

    Version:
    OPA May 2017, Oracle policy automation, OPA

    Comment

     

    • Ben O'Keeffe

      Hi Abhishek,

      There is no published API for the OPA Batch Processor as it is not intended to be used as an embedded component of another application the way your code indicates. The OPA Batch Processor is designed to be used as a stand-alone process, controlled by command-line options or an external configuration file.

      To launch the OPA Batch Processor, your application should create and monitor a separate Java process. Something like:

      String command = "java -jar <path-to-jar>/determinations-batch.jar --config ./config.xml --processors 2";
      ProcessBuilder processBuilder = new ProcessBuilder(command);
      Process process = processBuilder.start();
      

      Your application will have to manage the process to detect when it has completed, and handle any errors that are reported.

      Ben

    • Abhishek Srivastava

      Hi Ben,

        I agree that there is no published API however by utilizing jar file (batch engine) we can invoke the same task what we can do through command line. My requirement says to call it via external application and also supply dynamic configuration options.

       Considering the apporach you mentoned is also nice idea and we can give a try. However, still the question is how to achieve the same through API.

       

    • Abhishek Srivastava

      Hi Ben,

       Can you also share if there is a way to know the number of cases procesed and ignored using the comand line method while invoking the batch processor.

      Thanks,

      Abhi

    • Davin Fifield

      You should take a look at the Batch Assess REST API.

    • Ben O'Keeffe

      Hi Abhi,

      There is no API for the OPA Batch Processor. If your requirements or environment prevent you from being able to launch the OPA Batch Processor as a separate Java process, then refer to Davin's recommendation of the Batch Assess REST API. It provides similar functionality to the OPA Batch Processor as a web service.

      The OPA Batch Processor provides a summary of the number of cases processed and ignored to the standard output of the process and - if configured - to the main log file. If you are managing the OPA Batch Processor as an external process, your application can parse this information by reading the standard output or the log file.

       

      Ben