Skip to main content

Data Lake Quickstart

GitHub page TypeScript JavaScript

The following guide will walk you through installing @paloaltonetworks/pan-cortex-data-lake, a powerful package capable of supporting your next Cortex™ app, integration or automation project.

Installing with npm#

You can add the @paloaltonetworks/pan-cortex-data-lake dependency in your project directly from the NPMJS repository

npm i @paloaltonetworks/pan-cortex-data-lake

Installing from source#

In case you need to play with a pre-release version of the package then you can add dependencies from the GitHub public repo.

npm i git://

Source code is written in TypeScript and the build process produces type definition files which means you can leverage strongly type and code auto-complete features.

import * as cortex from "@paloaltonetworks/pan-cortex-data-lake";

Cortex Data Lake API Authorization#

The classes in the package @paloaltonetworks/pan-cortex-data-lake require an object that implements the Credentials Interface.

A collection of objects implementing the interface is available in the package @paloaltonetworks/pan-cortex-hub See Hub Quickstart

Getting started with a Developer Token#

Maybe the easiest way to get started is by leveraging a Developer Token provided by the API Explorer's Token Redemption Service. Just define the needed environmental variables ...

export PAN_DEVELOPER_TOKEN=<your_developer_token>export PAN_DEVELOPER_TOKEN_PROVIDER=

...and then instantiate an object of the DevTokenCredentials class.

const hub = require("@paloaltonetworks/pan-cortex-hub");const cred = hub.DevTokenCredentials.factory();

If you want to verify the object is working as expected then just call its getToken() method with the true value and expect it to return a valid OAuth2 access token.


Basic usage#

The examples below assume the existence of a constant named cred containing an object implementing the Credentials Interface.

Querying Logging Service#

  1. Begin by importing the package @paloaltonetworks/pan-cortex-data-lake:
const dl = require("@paloaltonetworks/pan-cortex-data-lake");
  1. Next, let's construct a QueryServiceClient instance:
const qsc = dl.QueryServiceClient.factory({ cortexDefCredentials: cred });
  1. Now, let's define the SQL sentence we want to execute:
const sqlCmd =  "SELECT source_ip, dest_ip from `<tenant_id>.firewall.traffic` LIMIT 5";
  1. Pass the SQL sentence to the QueryServiceClient object to receive an iterator object:
const iter = qsc.iterator(sqlCmd);
  1. Now, let's print the execution results (notice the ES2018 for await syntax executed inside an async lambda expression)
(async () => {  for await (const page of iter2) console.log(page);})();

Example output:

2/25/2020, 13:17:28 CORTEX_SDK initial autorization header for default data lake2/25/2020, 13:17:29 CORTEX_SDK Created new HTTP2 session to[  {    source_ip: {      value: '',      hex: '00000000000000000000ffffc0a86e83'    },    dest_ip: { value: '', hex: '00000000000000000000ffff4630018b' }  },  {    source_ip: { value: '', hex: '00000000000000000000ffff0a9a0105' },    dest_ip: { value: '', hex: '00000000000000000000ffff7c2b912d' }  },  {    source_ip: {      value: '',      hex: '00000000000000000000ffff0a9af6a7'    },    dest_ip: { value: '', hex: '00000000000000000000ffff453fb0bc' }  },  {    source_ip: { value: '', hex: '00000000000000000000ffff0a9a0928' },    dest_ip: {      value: '',      hex: '00000000000000000000ffff7bc11b76'    }  },  {    source_ip: {      value: '',      hex: '00000000000000000000ffff0a9ac4a9'    },    dest_ip: {      value: '',      hex: '00000000000000000000ffff79f3e08e'    }  }]

Code reference#

Previous example code in just one block (the cred variable is supposed to exist)

const dl = require("@paloaltonetworks/pan-cortex-data-lake");const sqlCmd =  "SELECT source_ip, dest_ip from `<tenant_id>.firewall.traffic` LIMIT 5";
async function worker(iter) {  for await (const page of iter) console.log(page);}
const qsc = dl.QueryServiceClient.factory({ cortexDefCredentials: cred });worker(qsc.iterator(sqlCmd));