Secure Cloud Access uses data ingestion agents (sometimes also called log agents) to collect and parse data for unsanctioned cloud applications in your organization. Secure Cloud Access uses logs that you upload to organize and display the data used for unsanctioned cloud discovery and risk analysis. This process is called log ingestion.

You can also use data ingestion agents to automatically update the data that Secure Cloud Access uses for Exact Data Matching (EDM). This process is called EDM source ingestion. For more information about EDM, see the section Exact Data Match in the New data types section.

For log ingestion, you can create one or more data ingestion agents to forward your data using syslog UDP, syslog TCP, or to collect logs gathered from a spooling directory. A data ingestion agent can consume files in Common Event Format (CEF) or index formats. An EDM data ingestion agent can consume files in CSV format.

Secure Cloud Access supports the following agents for log ingestion:

• BlueCoat
• McAfee
• Squid
• Juniper
• Palo Alto Networks
• Cisco Meraki

• CheckPoint

  You can set up log agents for one-time or continuous log ingestion.

• One-time log ingestion enables immediate data collection using proxy logs you upload. One-time log ingestion enables you to obtain a snapshot in time (for example, within the last day) of how unsanctioned cloud applications are being used in your organization. From the cloud discovery dashboard, you can apply filters and view data as needed, and generate a risk report.
• Continuous log ingestion enables ongoing data collection using proxy logs you upload. Continuous log ingestion enables you to collect data from logs on a regular basis. It uses a log agent to push data from logs stored at an on-premise location to Secure Cloud Access. From the cloud discovery dashboard, you can filter and view the data as needed, and generate a risk report. You can stop, pause, and restart continuous log ingestion as needed.

• For EDM source ingestion, you can only set up a continuous ingestion through this functionality. To perform one-time ingestion of data for EDM, follow the steps outlined in the section New data types and select the Exact Data Match option.

  Use the Data Ingestion Agent page to define a source type, the location of your log files, and the format in which your files are stored. You can create multiple log agent sources with different configuration settings and install those log agents on various machines to fit the needs of your organization.

  Depending on the consumption type—Syslog TCP, Syslog UDP, or Spooling Directory - you must either configure your network to send files to the intended machines or install the log agents on the machine that stores the log files you are processing.

  For successful processing of log files, the logs you upload must include these fields:

• Date time / date, time
• Client IP
• URL
• Bytes in