Batch Processing uses for running batch jobs. It has more capability than synchronous and you can check job monitoring. Records have more than 200 but less than 5000, so we use batch processing. It can store many records in one batch, and you can set the time interval in batch processing scheduler of Skyvva to run anytime that you want.
Priority Handling #
The priority handling is based on the interface group and not on the interface itself. When you define an interface group, e.g. IG_A you have chosen the priority e.g. High, Medium or Low.
Both Schedulers (attachment process and attachment reprocess scheduler) works with a configuration table for priority handling. The table is set the default to this value:
The configuration table for the priority handling looks like this:
The number specified in this table is the ratio between the priorities class High, Medium and Low. We use this number as a ratio because the processing of the attachment with the scheduler is done by the apex job. The maximum apex job size you can use at a given time is 5. Therefore, the number for the priority class High is set to 5. For medium, we have chosen the number 3 and for Low 1. Thus, we have a ration like 1/5 for Low, 3/5 for Medium and 5/5 for High.
The priority-handling algorithm is a weighted round robbing based on the priority ratio. To enable a weighted round robbing logic we have defined a so-called priority counter which is set dynamically by the processing of the attachment. Whenever data from a queue/interface group is processed the appropriate counter will be increased.
To understand the logic of the processing based on the priority ration let us look to this example.
The queue type is EO. The package size was not defined. Skyvva default interface group is not used. We have defined each interface as a separate interface group.
We have 3 High interface group, 2 Medium interface group, and 2 Low interface group.
Here is the initial state before the scheduler run. The counter is still at 0-0-0 meaning High = 0, Medium = 0 and Low = 0. The scheduler is scheduled to run every 2 minutes.
After the first scheduler round/run, it looks like this:
What happened now? While the scheduler run, we still have the 5-apex job available at that moment. Therefore, we can parallel the processing and use the 5-apex job at one time to process the interface group/queue.
Each apex job that is used to process the interface group e.g. to process the data package of the interface group is called a “Worker”. The scheduler is called the “Scheduler” and do not post the data. Its task is only to distribute the task to the worker. The worker will be posting the data into the application.
Whenever a worker is running, it will increase the counter of the appropriate priority class. Imagine that the first worker/apex job process the interface group “Opportunity_Balint” from line 13 in the above table. Then it increases the counter for the priority High from 0 -> 1 because the interface group/queue is of priority High.
The next worker will process the interface group/queue “Lead_Balint” and again increase the counter of the priority class High from 1 -> 2. The next worker will process the interface group/queue “Product_Balint” and increase the counter of the priority class High from 2 -> 3.
The next worker will process the interface group/queue “Case_Balint” and therefore set the priority class Medium from 0 -> 1. Note that the processing depends on the data existing in the working basket. Because there are no more data from priority class High the next priority is Medium and not Low. Therefore, Medium is processed before Low.
The next worker will process the interface group/ queue “Contact_Balint” and increase the priority class Medium from 1 -> 2. Now all 5 workers/apex job has done their work and coz. We do not have more apex jobs available than the maximum number of 5 the processing of the first round is done.
For further processing, we have to wait for the next scheduler run, which will be in the next 2 minutes. Here the result of the second scheduler run/round.
After the first scheduler run, the counter stand was 3-2-0. The second scheduler runs pick now the data of the priority class Low because in the basket there are no more data from priority High or Medium. If there were still data from priority, High or Medium than the second will first process those data before processing data from priority Low.
The interface group/queue “Account_Balint” is now processed and the counter is set for the priority Low from 0 -> 1. e.g. the total counter stand is 3-2-1.
To summarize the general logic of the priority-based round robbing scheduling we have seen is that first the data of priority High will be processed before Medium and Medium before Low.
Imagine further that we still have data to process and that at a given time we increase the counter for each priority class High, Medium and Low to its maximum number e.g. 5-3-1. This case means that data of each priority has been processed according to the ratio definition based on the ratio table above. In this situation, the counter is reset to the initial value 0-0-0 so that the processing can be rescheduling again with the initial stand. This is necessary because we want that each priority also priority Low will be processed even there are many data from class High. Therefore, this algorithm is a weighted round robbing scheduling and not an absolute round robbing algorithm.
Imagine that you have 100 % of the data in the working basket. This could be 100, 2500, 124.312, … attachments with different priority residing in the working basket. Then over time with this algorithm, we ensure that at least 10% of priority Low will also be processed and not 0 %. However, most data will be data from priority High and Medium, which will have been processed.
This is the feature of priority based handling for processing interface group/queue. With this feature, you are able to define the priority for your business data and ensure that very important data will be posted before low important data.
The priority-based handling feature only applied for the processing of the interface group/queue within an integration.
