- Enhance Agent to be an HTTP-Server
- Server Configuration on Agent UI
- Support Stored Procedure for Oracle
- Use the field "Package" on the interface level with Agent
- Use SQLite for the Agent database and Move all file base configurations to SQLite
- Automatic move integration.properties to SQLite database
- Move the Crontab file to the database
- Test Connection for every adapter from Salesforce
- An agent with offline mode and online mode
- Change Agent Scheduler to use crontab in the database
- Agent Support real bulk mode like Salesforce Data Loader
- Create New button Export and Import Properties file in Integration Properties Setup
- Streaming API runs in background service
Enhance Agent to be an HTTP-Server #
- A web server is server software or hardware dedicated to running said software, that can satisfy World Wide Web client requests. A web server can, in general, contain one or more websites. A web server processes incoming network requests over HTTP and several other related protocols.
- The primary function of a web server is to store, process and deliver web pages to clients. The communication between the client and server takes place using the Hypertext Transfer Protocol HTTP. Pages delivered are most frequently HTML documents, which may include images, style sheets, and scripts in addition to the text content.
- Multiple web servers may be used for a high-traffic website here, SKYVVA servers are installed together being used.
- Agent, commonly a web browser or web crawler, initiates communication by making a request for a specific resource using HTTP and the server responds with the content of that resource or an error message if unable to do so. The resource is typically a real file on the server’s secondary storage, but this is not necessarily the case and depends on how the web server is implemented.
- While the primary function is to serve content, full implementation of HTTP also includes ways of receiving content from clients. This feature is used for submitting web forms, including uploading files.
Server Configuration on Agent UI #
The agent is an infrastructure automation tool, Built specifically to support windows, Agent utilizes Infrastructure as Code to model infrastructure and configuration.
Agent provisions and configure servers automatically, without logging in to a command prompt.
Key areas
- The agent allows users to model the configuration of servers, roles, and environments, monitor for drift, schedule changes, and ensure consistency across servers.
- An agent can spin up cloud servers, build containers, deploy packages, patch servers, or do any other multi-server/service automation.
The agent also has batch monitoring capabilities. It can continuously monitor for server configuration batch, and can automatically repeat batch.
Key features
- An agent has a visual, web-based user interface that is designed to “create complex configurations and composition using the intuitive and then switch to-and-from query editor/text mode as needed.
- Agent aims to enable DevOps practices through its UI and shows the configuration state of an organization’s servers infrastructure (local, virtual, cloud-built). An agent has first-class Windows support.
- Agent monitors servers for configuration changes, and reports when the configuration has batch.
Support Stored Procedure for Oracle #
- Where users add support for store procedure but only for the database MySQL Server. Here user needs to add the same feature for Oracle.
- We use the store procedure to call data from the table into the store procedure and then the user can use the store procedure to push data into SKYVVA.
Use the field “Package” on the interface level with Agent #
- Since SKYVVA version 2.42 we add the feature “Package Size” on the interface level. Before we carry only package size in the integration level, where the user can set it on the integration page with the parameter name “Packet“. For the inbound interface, the is size e.g. the number of records to send with an integrate() API call. So if the user set 20 for this field and we carry 100 records for the input file the agent will commit 5 integrate () call to send all 100 records to Salesforce.
- This setting will apply to all interfaces in the period of the aforementioned integration. This was not flexible enough if different interfaces commit different package sizes. Thus we add the feature to have package size on the interface level.
- About the Agent UI screen user can see this field at the interface and the user can set it per interface. A similar field we don’t have on the salesforce side and therefore we use the existing field “Number of Records per Batch/Upload”. We use this field for both directions e.g. for inbound and outbound interface with Agent.
Indeed, the field “Number of Records per Batch/Upload” is the scope size for processing batch apex. So, it is not able to use this field corresponding. - It leads to misunderstandings and is difficult to explain to users. We need a new field that clearly corresponds to the package size as long as an interface. This field is missing in the interface object right now. For the outbound interface, we use the package size on the interface level to make an API call for reading the salesforce record in chunks or sections.
- Since we provide this feature already customers could use it. If we don’t use this field anymore we will break existing customers using this feature. To keep backward compatibility, we need to check if a value is set for the new field. Let’s call the new field, “NumberOfRecordPerAPICall_c“. If this field is set Agent will use this field and ignore the field, “Number of Records per Batch/Upload“. If this new field is not set or does not exist because of using a new Agent version with the old SKYVVA version where the new field still does not exist, then we use the field “Number of Records per Batch/Upload“.
- We should not use this field anymore for specifying the number of records per API call e.g. use as transfer package size. Instead, they should use the new field. The field “Number of Records per Batch/Upload” is exclusively reserved for batch processing purposes and really meant to be used as scope size.
Use SQLite for the Agent database and Move all file base configurations to SQLite #
SQLite is a relational database management system based on the SQL language but optimized for use in small environments such as mobile phones or small applications. SQLite is self-contained, serverless, zero-configuration, and transactional. It is very fast and lightweight, and the entire database is stored in a single file. SQLite is built for simplicity and speed compared to a hosted client-server relational database such as MySQL. It sacrifices sophistication for utility and complexity for size. Queries in SQLite are almost identical to other SQL calls.
Before starting we would highly recommend all users install DB Browser for SQLite. The browser can be downloaded from their official page easily. DB Browser for SQLite is a high-quality, visual, open-source tool to create, design, and edit database files compatible with SQLite. It will help us to see the databases being created and edited in real-time.
Contents
- CONNECTING to a Database
- CREATING a Table
- INSERTING records in a TABLE
- SELECTING records from the TABLE
- UPDATING Records in the TABLE
- DELETE Operation
- Example walk trough
Automatic move integration.properties to SQLite database #
Windows support SQLite, the SQL-based relational database management system (RDMS), for local data storage. There are a few things to note when you first work with SQLite, that may influence integration.properties in the way we develop with it:
- SQLite stores data in a simple text file. There is no granular security or user privileges for data therefore anyone with file system access to it may read its contents.
- There are only five underlying data types; NULL, INTEGER, REAL, TEXT, and BLOB. For more detail see Datatypes In SQLite.
- Binary objects (BLOBs) are stored as text representations, thus access to BLOBs is not optimal. It is recommended to store BLOBs on the file system and store the file system path in the database.
- SQLite supports concurrent read access but enforces sequential write access. This is because a file system lock is placed on the file during write operations. This is an important point to bear in mind with multi-threaded applications.
- RIGHT and FULL OUTER JOINs are not supported.
- There are limited ALTER TABLE support columns that may not be modified or deleted.
Move the Crontab file to the database #
New Agent 1.48 don’t use the crontab file in the libs folder in the Agent directory anymore, we have moved the crontab file to the database file and used it.
Test Connection for every adapter from Salesforce #
Currently, we can test the connection for an adapter using Agent with the Agent UI. For the next Agent version, we have to be capable to test connections from Salesforce as well. For all adapters, we need to provide the test functionality for a user to check if the connection works. We also need the possibility to send a test message.
An agent with offline mode and online mode #
- We are building the new Agent UI counting boost the Agent to be an HTTP listener. But this doesn’t mean that we totally remove the Agent UI. There is still some functionality where users need a local UI controlling the Agent Engine which cannot be controlled from a web front. We need to analyze and list the function which cannot be done from Salesforce UI with the new Agent Control Board. For those functions that are required but cannot be done from the Salesforce UI, we need to keep and provide or add new into the Agent UI which we currently carry right now.
- For example, if the customer refuses to open the firewall we cannot start/stop the Agent e.g. the Windows Service. So nothing will work. Then we still need to have the local Agent UI to do this. So what have right now we will keep but what for example becomes useless and obsolete is, for example, the mapping screen. We will not do mapping with the Agent UI anymore. This screen can be totally removed.
Change Agent Scheduler to use crontab in the database #
Agent 1.48 the crontab expression value is moved to store in database SQLite so for the Agent scheduler processing in code we need to change to use crontab value in database SQLite.
Agent Support real bulk mode like Salesforce Data Loader #
The Bulk API provides a programmatic option to quickly load your organization’s data into Salesforce. Using the API requires basic familiarity with software development, web services, and the Salesforce user interface.
The functionality described is available only if your org has the Bulk API feature enabled. This feature is enabled by default for Performance, Unlimited, Enterprise, and Developer Editions.
When to Use Bulk API
Bulk API is based on REST principles and is optimized for loading or deleting large sets of data. You can use it to query, queryAll, insert, update, upsert, or delete many records asynchronously by submitting batches. Salesforce processes batches in the background. SOAP API, in contrast, is optimized for real-time client applications that update a few records at a time. You can use SOAP API for processing many records, but when the data sets contain hundreds of thousands of records, SOAP API is less practical. Bulk API is designed to make it simple to process data from a few thousand to millions of records. The easiest way to use Bulk API is to enable it for processing records in Data Loader using CSV files. Using Data Loader avoids the need to write your own client application.
Create New button Export and Import Properties file in Integration Properties Setup #
1. When we go to Agent UI and then click on Integration Properties Setup it should carry a button into the Export and import properties file.
2. This button should be capable to Export and Import Properties File into Folder SKYVVA integration Agent.
Streaming API runs in background service #
After subscribing when the user closes the system the streaming API does not work. We are articulating the streaming API that provides by salesforce. For the message when the subject has to create new, update, and delete so it will push objects to clients one by one. The question is Agent can wait to get all of the streaming messages at the same time and then completely integrate outbound by interface package size?