In Part 1, we discussed why you would want to integrate AD with Netskope, the AD integration tools Netskope offers, and briefly touched on the Netskope’s REST API. In part 2 we are going to dive deeper into using the REST API to add additional attributes and provide a sample Powershell script that provides additional automation capabilities.
Keep in mind that this document is not intended to replace the existing Netskope documentation, and will not cover the implementation details for the Netskope tools, as that is already well defined.
Netskope REST API and Directory Importer
As was mentioned in part 1, not only does the Netskope REST API offer a variety of options to query data from Netskope, but it also provides the ability upload custom user attributes from a CSV or TXT file as long as it is in a specific format (Netskope documentation on file format and limitations can be found at Administrators > Administration > REST API > Add Custom User Attributes). While this sounds eerily similar to what the Directory Importer is capable of (with a little additional configuration), using the REST API provides a few distinct advantages.
Let’s take a real world example. Your web proxy uses an employee’s email address as a user key in order to correlate traffic ‘events’ to a user, and you have configured the On Premises Log Parser (OPLP) to upload those events to Netskope. To add some additional user information to Netskope, you configure directory importer to use the mail (email) field to correlate when uploading all of your additional user attributes. But what happens when you have data from a different source, such as Forward or Reverse Proxy data, and the application is not using the email field as the user identity? (Perhaps it uses the employee ID from your HR system.)
In this scenario, the events generated from that source are not going to contain any of the additional user information that was pulled from AD via the directory importer, because the user keys do not correlate with each other. That’s where the REST API comes in. As we mentioned in part 1, you can have duplicate rows in the CSV file, one for each possible key that can be matched against, which allows the uploaded attribute data to correlate all events, even when they are using different keys. Furthermore, with the REST API based solution, you can pull user information from sources besides AD (e.g. an HR system) and include it in your file to be uploaded.
Additional User Attribute Script
Netskope provides a bash script that can be used to upload a CSV or TXT file to your tenant. The script works as designed, but when trying to implement this as part of a solution a few issues need to be considered.
It is likely that you will have already provisioned a Windows instance for the Directory importer. But since the Netskope provided script is a bash shell script, you need to provision a Linux instance in order to run the script. This is extra effort and extra cost.
You will probably need to work with your Active Directory team to obtain an export from AD, format it, FTP it over to the Linux instance, and then run the shell script to upload it to Netskope. Parts of this can be automated, but as a solution it is cumbersome and potentially fragile.
In an effort to simplify the solution, we have created a script (attached below) that performs the same functionality as Netskope’s bash script but performs it in Windows PowerShell. This means that you can run it on the same Windows instance as the Directory Importer, where it will extract the data from AD, build a correctly formatted CSV file, and upload it to your Netskope Tenant. Let’s go through the various parts.
exportADAttributesToCSV
This function is responsible for querying AD and creating the corresponding CSV file in the appropriate format in preparation for its upload to the Netskope tenant. There are lines in here that you must configure and they are as follows:
Properties to pull from AD: change the values in the parentheses to your choosing. You should limit this to 15 attributes as that is Netskope’s current limit for custom attributes
Names of the custom attributes to be uploaded: The first sw.WriteLine seen below is where the header row is written into the CSV file. These will be the names displayed within SkopeIt. Also, keep in mind the first column will be used as the key to correlate on (i.e. – mail).
ManagerDisplayName function: This block will populate manager display name with an empty string if empty or null and if not, will either query AD for each user to get their displayName (heavy performance impact), or attempt to parse the manager’s name out of the canonical name. This can be removed if managerName is not one of the attributes pulled
Adding records to the CSV: This section is where the actual records are added to the CSV file under the header row. This will need to be altered if you change the attributes in the header to match in order and count. If you have applications that use different keys, this is the place to add additional $sw.Writeline statements to add more than one record per user.
jsonValue
This function is responsible for parsing the responses from Netskope, trimming additional whitespace and returning the value at a given spot in the JSON body.
getToken
GetToken is responsible for using the NSUI and NSRestToken variables and calling the Netskope tenant to get a token for subsequent REST requests. This token will be used when uploading the custom attributes to the tenant.
uploadFile
This appropriately named function is responsible for the chunking (breaking into smaller pieces to upload) the CSV file and uploading it to the Netskope tenant. This is primarily a port of Netskope’s script over to PowerShell.
getManagerName
GetManagerName is responsible for attempting to parse the manager’s full name out of the manager’s canonical name and will require some alteration depending on the way the canonical name is formatted within AD. Currently, it is set for ‘CN=LastName\, FirstName MiddleName,OU=global,DC=mydomain,DC=com’. This function is an effort to reduce the number of requests made to AD and increase the scripts performance as it will not have to wait for multiple responses from AD. If managerName is not an attribute you intend to upload, or you elect to go the route of querying AD for the manager’s displayName, you can do that within the exportADAttributesToCSV function.
Main Block
The main block is responsible for allowing insecure HTTPS connections (the Netskope script also does this, but PowerShell has an issue with it), some timing logic for the logs and calling the various functions appropriately.
Configuring Windows for Additional Attribute PS Script
The script is a modified version of Netskope’s additional attribute shell script that is designed to run in Microsoft PowerShell. It requires the use of PowerShell v4.0 or higher, enabling the AD module for PowerShell in Roles and Features, and installing curl as PowerShell does not have native functionality to make multipart/form-data REST requests.
Installing AD Module for PowerShell
To install the AD Module for PowerShell, you must log on as an admin and follow these steps:
Open Server Manager and select ‘Manage’ in the upper right corner and then ‘Add Roles and Features’ from the drop-down
Select ‘Role-based or feature-based installation’ and hit next
Select local server and hit next
Jump to ‘Features’ in the left pane and scroll down to ‘Remote Server Administration Tools’ and expand
Expand ‘Role Administration Tools’ and ‘AD DS and AD LDS Tools’ Select ‘Active Directory module for Windows PowerShell’ and hit next Confirm the installation and complete
Installing Curl
To install curl, you must log on as an admin and follow these steps:
Download the latest curl installer for your windows environment from https://curl.haxx.se/download.html
Install curl and accept all defaults
Locate where curl executable is installed (likely C:\Program Files\cURL\bin if all defaults selected) and save this for the script variable configuration
Creating Encrypted Password File
To keep the AD user password secure, the following is a process to create a secure string and save it in an encrypted file. Since this process uses the Windows Data Protection API (DPAPI), you must do the following as the user that will be used to run the script. Failure to do so will result in the inability of the script to decrypt the password.
Log into server as user that will be running the script
Open powershell window and type “(Get-Credential).Password | ConvertFrom-SecureString | Out-File “ where is where you would like the encrypted password file to be stored. e.g. – (Get-Credential).Password | ConvertFrom-SecureString | Out-File “C:\PSScripts\ExportADUsers\SecurePW.txt”
Save for script configuration
Configuring Script Variables
The following variables should be configured in the script to match your environment:
DEBUG – toggle to increase logging when script issues are encountered.
NSUI – domain name of Netskope tenant where attributes are to be uploaded
NSRestToken – token used to make REST calls to Netskope tenant. If this value is changed on the tenant, then it must be updated in the script.
maxChunkSize – if the AD export file is larger than this value, the file will be divided into chunks of this size and one chunk smaller or equally sized. The chunks will then be uploaded to the Netskope tenant in multiple parts. Recommended size is 5MB.
path – path where scripts and files exist. Must end in /*.*
csvFile – path and name of csv file (e.g. – “$path\ALLADUsers_$logDate.csv”)
logFile – path and name of log file (e.g. – “$path\UploadCSVtoNS.log”)
ADServer – AD server name (e.g. – ”mydomain.com”)
searchBase – lowest level in AD and where queries are performed (e.g. – “OU=Global,DC=mydomain,DC=com”)
user – user used to query AD. Must have read access to AD and be the same user that created the encrypted password file
pwFile – location of the AD user encrypted password file. from the previous section
Scheduling Script via Task Scheduler
To keep up with an ever-changing global directory, this script should be automated to run to keep the user data on the Netskope tenant up to date. It is recommend to run daily using Windows Task Scheduler. If this needs to be changed:
Log into the server as an admin
Open Task Scheduler under Administrative Tools
Browse under Task Scheduler Library – Netskope to find ‘Run AD Upload Script’
Right-click and select properties
Under the tabs, these options should be set
General
When running this task, use the account that has access to AD and that you used to create the password file.
Run whether user is logged on or not – enabled
Run with highest privileges – enabled
Triggers – One trigger should exist to start the script at a given time and it should be scheduled to terminate it if it goes over an allotted amount of time.
Begin the task – On a schedule
Set timing to preferred period (e.g. – Daily @ 0800 GMT)
Stop task if runs longer than – should be set to less than the frequency your script is set to run.
Enabled – enabled
Actions – Should be set to start a program
Program/script – path to PowerShell executable (e.g. – C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe)
Add arguments – need to add script to execute once PowerShell is opened using ‘File’ argument and absolute path to script (e.g. -File “C:\PSScripts\ExportADUsers\NS_UserAttr_Upload_v1_0.ps1”)
Conditions – None
Settings
Allow task to be run on demand – enabled
Stop the task if runs longer than – enabled
If the running task does not end when requested, force it to stop – enabled
If task is already running, then the following rule applies – Do not start a new instance
Maintenance
Manual periodic cleanup of log files and csv files will need to be performed in the script directory so the server does not run out of storage space. Our recommendation is to clean files up at least once a month. Alternatively, you could implement those capabilities in your version of the script, or have a separately scheduled Powershell script to perform cleanup. To access the script, please click here.
Paul Ilechko | Senior Security Architect
Andrew Hejnas | Cloud Security Specialist & Solutions Architect