Scripts / Legacy Data Import

Scripts is SKOOR’s comprehensive solution for creating, editing, and running scripts with loads of options to customize them.

Scripts are divided into 3 main types: Python, Talend, and custom.

Scripts Page

The page is divided into three main sections:

Settings

This section includes the general settings of a script, such as:

Name and Description.
Default Timeout (in seconds)
Copy Webhook Token or Reset it (Only available for ADMINS)
Schedule Configurations: Allows multiple schedules with dedicated run configurations.
Run Configurations: Allows customization of Timeout and Environment Variables.

Files

This section offers a complete file management solution for the script, including a state-of-the-art editor for text-based data.

A download link will be displayed if the data is binary (Not editable in text form).

Run

This section is responsible for Running and monitoring the Runs of a script. It offers a small drop-zone to upload source files if required.

Creating a Script

Scripts can either be created manually, or uploaded via Zip-files as shown below:

Cornerstones of a Script

Run-script

The main file includes the logic of the script. This decides the type of the script:

For Python Script: it’s a main.py file.
For Talend Script: it’s a <scriptName>_run.sh file.
For Custom Script: it’s a run.sh file.

Please note that the type of script can’t be changed after creation. This can only done by creating a new script.

.env

It’s a simple text file, that includes environment variables. This way variables that are used in a script can be adjusted based on the use case.
The following .env file would yield the shown environment variables, allowing to adjust those in custom run configurations.

Pre-provided EnvironmentVariables

Some details about the run of the script are passed to the script environment as EnvironmentVariables, Those are:

UserId as SKOOR_USER_ID
Username as SKOOR_USER_USERNAME
User’s Fullname as SKOOR_USER_FULLNAME

The below screenshots show how to retrieve those in a Python example along with the output in the script:

import os

source_file = os.environ.get('SKOOR_SOURCE_FILE')
user_id = os.environ.get('SKOOR_USER_ID')
username = os.environ.get('SKOOR_USER_USERNAME')
userFullname = os.environ.get('SKOOR_USER_FULLNAME')

print("sourceFile: ", source_file)
print("User ID: ", user_id)
print("Username: ", username)
print("Fullname: ", userFullname)

Example using .env

The following script reads the variable foo in .env and prints it out to the console, below are the screenshots of that .env and the respective run configurations.

SourceFiles

Scripts can be designed to accept external files (source files) and process them as a part of the script logic.

The source files could be referenced in the script by using the environment variable SKOOR_SOURCE_FILE as seen in the following code snippet:

# IN PYTHON
import os

source_file = os.environ.get('SKOOR_SOURCE_FILE')
print(source_file)

#IN SHELL

echo $SKOOR_SOURCE_FILE

QuickRun

This option offers a QuickRun button that runs the script without a source file and with the default values of environment variables.
It allows to run a script quickly while developing or troubleshooting it and the Latest Job Overview button shows the console output.

Structure of the Zip-Script

The uploaded Zip file should have only a single folder in the root, named the same as the Zip archive file.

The run script should preferably be in the root of the script’s folder, and the .env file.

Running

Start a run

In the run section, select the desired configuration (and upload a source file if needed) then click “Run Script”

Cancel a run

Running as well as waiting runs can be canceled by clicking the cancel job icon in the History section.

Job Overview

To view the full details of a job, click on an entry in the History list as shown below:

Run History

The History section shows a list of all the script runs. Details about each run can be found by clicking on it.

A dedicated section shows the detailed history of the latest run of the script (skips waiting/canceled runs)

Using Webhook Tokens to run a converter

This option makes it possible to run the Script without the need to log in, and it’s a preferred option for technical users.
Each Script has its unique Webhook token and the URL will be copied to the clipboard by clicking Copy Webhook Token.

Example of using Webhook tokens

Copy the webhook token from the Script page by clicking Copy Webhook Token, for this example the Webhook URL looks as follows:

http://localhost:8090/scripts/8925cfd4-2063-4b07-9d56-956f2b754b47/run?token=6ae51c9f-d292-4d55-b512-4537121b37f5

Use the URL to send a POST request:

Using an API software like POSTMAN

A 200 response with the initial Job Entity indicates that the run has been triggered as shown in the screenshot below:

Using cURL in terminal

curl -X POST http://localhost:8090/scripts/8925cfd4-2063-4b07-9d56-956f2b754b47/run?token=6ae51c9f-d292-4d55-b512-4537121b37f5

The response would look like this

This run can be observed in SKOOR-Dashboard and the run shows created by Token-User in the History.

Deleting

Click on the trash bin icon to delete the script. The icon will appear when hovering over the script’s name with the mouse pointer.

Exporting Script as Zip

Click on the archive icon to export the script. The icon will appear when hovering over the script’s name with the mouse pointer.