Mass Ingestion

Back Next

Streaming Agent offline mode

You can run and monitor a

streaming ingestion

job when the CMI Streaming Agent is offline or not connected to the internet.

The Streaming Agent supports both online and offline modes of communication. In the offline mode, the

streaming ingestion

job continues to run even if the Streaming Agent does not communicate with the

Informatica Intelligent Cloud Services

for an extended period of time. The Streaming Agent continues to monitor the health and statistics of the ingestion tasks locally. When the Streaming Agent turns online and connects to the cloud services, it updates any configuration changes for the agent and tasks, as well as updates the health and statistics to the services.

To switch between the offline and online modes, you can use the command line utility provided by the

Mass Ingestion Streaming

service. Run the following command to start the command line utility:

<Informatica Secure Agent>/apps/Streaming_Ingestion_Agent/<version>/runcli.bat

The command line utility uses the command prompt

infa/stream>

and provides three groups of commands.

You can change the communication modes only through the command line utility. The Streaming Agent preserves the communication mode when the agent restarts.

The following table lists the commands of this command line utility:

Command	Description	Example
app-config	Shows the current configuration of the Streaming Agent application.	infa/stream :>app-config deploy.pull.interval : 60 health.poll.interval : 30 minifi.ingester.file.location : ./conf siagent.communication.mode : Online siagent.monitoring.persist.dir : ../data siagent.statistics.post.batchsize : 720 siagent.statistics.post.concurrency : 60 siagent.status.persist.dir : ../data statistics.poll.interval : 30
app-setconfig	Use this command to configure the following properties: siagent.communication.mode . Use to configure offline or online communication mode. siagent.statistics.post.batchsize . Use to define the number of snapshots in a batch. siagent.statistics.post.concurrency . Use to define the number of worker threads to post statistics. The --key and --value tokens are optional.	infa/stream :>app-setconfig --key siagent.statistics.post.batchsize --value 20 or infa/stream :>app-setconfig siagent.statistics.post.batchsize 20
app-status	Shows the current status of the Streaming Agent. The health status code and health error message indicates the status of the agent (service) shown on the Administrator . uptime indicates the number of seconds since the Streaming Agent application is available.	infa/stream :>app-status health error message : No errors health status code : RUNNING(0) uptime : 67828
app-statistics	Shows metadata and status of overall statistics collection in the Streaming Agent. collection interval . Interval of statistics collection, in seconds. post interval . Frequency of statistics posted or attempted post. max batch size . Maximum number of snapshots posted in a single http post. last batch size . Number of snapshots in the last http post. last time collected . Timestamp when any statistics were last collected. last time posted . Timestamp when any statistics were last posted.	infa/stream :>app-statistics collection interval : 30 last batch size : 2 last time collected : 7/3/20 10:19:03 AM IST last time posted : 7/3/20 10:18:53 AM IST max batch size : 20 pending snapshots : 3 post interval : 30
clear	Clears the screen.	-
exit, quit	Quits the application.	-
help	Shows a summary of all the commands available.	infa/stream :>help AVAILABLE COMMANDSAgent Application Commands app-config: Show agent application configuration app-setmode: Set the communication mode [Online/Offline] app-status: Show agent application statusBuilt-In Commands clear: Clear the shell screen. exit, quit: Exit the shell. help: Display help about available commands.Streaming Ingestion Task Commands task-health: Show streaming ingestion task health task-list: Show streaming ingestion task list task-metadata: Show streaming ingestion task metadata
task-list	Shows the list of streaming ingestion jobs currently deployed on the Streaming Agent.	infa/stream :>task-list 6e61e76f-2618-4292-ab3d-dd181f47ee91 ad5053c7-5ac2-493f-8cbb-a24900b61f71
task-health	Shows the health status of all streaming ingestion jobs in the Streaming Agent. Use the options --name or --id to specify a job. If none are specified, all jobs are listed.	infa/stream :>task-health --name aby_df4 processors : [{"id":"14a7a095-7fac-4fc3-ac5c-705369132516","status":"ERROR"},{"id":"821e6730-3aed-4d3f-b875-45f424b6b963","status":"RUNNING"}] status : ERROR timestamp : Sat May 09 06:04:08 IST 2020 infa/stream :>task-health 6e61e76f-2618-4292-ab3d-dd181f47ee91 processors : [{"id":"2a0b8715-aa7a-46c5-9d6a-6a356f5a0102","status":"ERROR"},{"id":"1172f3a8-35dd-41ef-be4b-bc0cf37e3794","status":"RUNNING"}] status : ERROR timestamp : Sat May 09 06:04:08 IST 2020 ad5053c7-5ac2-493f-8cbb-a24900b61f71 processors : [{"id":"14a7a095-7fac-4fc3-ac5c-705369132516","status":"ERROR"},{"id":"821e6730-3aed-4d3f-b875-45f424b6b963","status":"RUNNING"}] status : ERROR timestamp : Sat May 09 06:04:08 IST 2020
task-metadata	Shows the metadata of all streaming ingestion jobs in the Streaming Agent. Use the options --name or --id to specify a job. If none are specified, all jobs are listed.	infa/stream :>task-metadata --name aby_df4 id : ad5053c7-5ac2-493f-8cbb-a24900b61f71 name : aby_df4 runId : 9071 version : 1 infa/stream :>task-metadata 6e61e76f-2618-4292-ab3d-dd181f47ee91 id : 6e61e76f-2618-4292-ab3d-dd181f47ee91 name : aby_df2 runId : 9069 version : 8 ad5053c7-5ac2-493f-8cbb-a24900b61f71 id : ad5053c7-5ac2-493f-8cbb-a24900b61f71 name : aby_df4 runId : 9071 version : 1
task-statistics	Shows the statistics details of all streaming ingestion jobs in the Streaming Agent. Use the options --name or --id to specify a job. If none are specified, all jobs are listed.	infa/stream :>task-statistics --name aby_df1 dataflow name : aby_df1 last time collected : 1590861803731 last time posted : 1590861806091 infa/stream :>task-statistics 7b7d3c09-df43-482f-b6c8-8dd80187e6d7 dataflow name : aby_df2 last time collected : 1590861770731 last time posted : 1590861741132 decfad0a-20df-4226-84f9-1ff1ab6ef96a dataflow name : aby_df1 last time collected : 1590861768730 last time posted : 1590861771054

Online mode to Offline mode

By default, the Streaming Agent is in online mode.

To change the Streaming Agent to offline mode:

Launch the command line utility using the following command:

 In Windows:
<Informatica_Secure_Agent>/apps/Streaming_Ingestion_Agent/<version>/runcli.bat

or 

In UNIX:
runcli.sh

Set the Streaming Agent to offline mode:

app-setconfig --key siagent.communication.mode --value Offline
or
app-setconfig siagent.communication.mode Offline

The Streaming Agent stops sending health updates and statistics of any

streaming ingestion

job.

Offline mode to Online mode

To change the Streaming Agent to online mode:

Launch the command line utility using the following command:

 In Windows:
<Informatica_Secure_Agent>/apps/Streaming_Ingestion_Agent/<version>/runcli.bat

or 

In UNIX:
runcli.sh

Set the Streaming Agent to online mode:

app-setconfig --key siagent.communication.mode --value Online
or
app-setconfig siagent.communication.mode Online

The Streaming Agent starts sending health updates of all the

streaming ingestion

jobs and the updates appear in the

Monitoring

page. It starts sending statistics of all the

streaming ingestion

jobs including the statistics backlog collected while it was offline to the service. It also synchronizes the updates to the

streaming ingestion

jobs or adds the new

streaming ingestion

job deployed while it was offline to the service.

CMI Streaming Agent

Download Guide

Watch

Comments

Communities

Knowledge Base

Success Portal

0 COMMENTS

We’d like to hear from you! Log in to comment.

Rename Saved Search

Table of Contents

Mass Ingestion

Mass Ingestion

Streaming Agent offline mode

Streaming Agent offline mode

Online mode to Offline mode

Offline mode to Online mode