Scripts / Legacy Data Import

Scripts is SKOOR’s comprehensive solution for creating, editing, and running scripts with loads of options to customize them.

Scripts Page

The page is divided into three main sections:

Settings

This section includes the general settings of a script, such as:

Name and Description.
Default Timeout (in seconds)
Copy Webhook Token or Reset it (Only available for ADMINS)
Schedule Configurations: Allows multiple schedules with dedicated run configurations.
Run Configurations: Allows customization of Timeout and Environment Variables.

Files

This section offers a complete file management solution for the script, including a state-of-the-art editor for text-based data.

A download link will be displayed if the data is binary (Not editable in text form).

Run

This section is responsible for Running and monitoring the Runs of a script. It offers a small drop-zone to upload source files if required.

Creating a Script

Scripts can either be created manually, or uploaded via Zip-files as shown below:

Cornerstones of a Script

Run-script

The main file includes the logic of the script. This determines the type of runner for the script:

Python Runner expects a main.py file.
Talend Runner expects a <scriptName>_run.sh file.
Shell Runner expects a run.sh file.

Place your main script one level below the root (or higher), since the breadth-first scan always uses the most shallow script it finds.

The following message is shown, when trying to run a Script without a main runner script.

.env

It’s a simple text file, that includes environment variables. This way variables that are used in a script can be adjusted based on the use case.
The following .env file would yield the shown environment variables, allowing to adjust those in custom run configurations.

Pre-provided EnvironmentVariables

Some details about the run of the script are passed to the script environment as EnvironmentVariables, Those are:

UserId as SKOOR_USER_ID
Username as SKOOR_USER_USERNAME
User’s Fullname as SKOOR_USER_FULLNAME

The below screenshots show how to retrieve those in a Python example along with the output in the script:

import os

source_file = os.environ.get('SKOOR_SOURCE_FILE')
user_id = os.environ.get('SKOOR_USER_ID')
username = os.environ.get('SKOOR_USER_USERNAME')
userFullname = os.environ.get('SKOOR_USER_FULLNAME')

print("sourceFile: ", source_file)
print("User ID: ", user_id)
print("Username: ", username)
print("Fullname: ", userFullname)

Example using .env

The following script reads the variable foo in .env and prints it out to the console, below are the screenshots of that .env and the respective run configurations.

SourceFiles

Scripts can be designed to accept external files (source files) and process them as a part of the script logic.

The source files could be referenced in the script by using the environment variable SKOOR_SOURCE_FILE as seen in the following code snippet:

# IN PYTHON
import os

source_file = os.environ.get('SKOOR_SOURCE_FILE')
print(source_file)

#IN SHELL

echo $SKOOR_SOURCE_FILE

Running sub-scripts

If sub-scripts are required, then they should be put inside a bin folder. Since those will be runnable.

Make sure to put the sub-scripts always in a bin folder.

The following shows an example in Shell:

The output of this run.sh is:

QuickRun

This option offers a QuickRun button that runs the script without a source file and with the default values of environment variables.
It allows to run a script quickly while developing or troubleshooting it and the Latest Job Overview button shows the console output.

Running

Start a run

In the run section, select the desired configuration (and upload a source file if needed) then click “Run Script”

Cancel a run

Running as well as waiting runs can be canceled by clicking the cancel job icon in the History section.

Poetry Opt-out

Poetry is a powerful tool used for dependency management and virtual environment management in python. This is all done by defining a pyproject.toml file.

Removing/renaming the pyproject.toml will automatically opt-out poetry usage, and use skoor-python instead.

Job Overview

To view the full details of a job, click on an entry in the History list as shown below:

Run History

The History section shows a list of all the script runs. Details about each run can be found by clicking on it.

A dedicated section shows the detailed history of the latest run of the script (skips waiting/canceled runs)

Using Webhook Tokens to run a converter

This option makes it possible to run the Script without the need to log in, and it’s a preferred option for technical users.
Each Script has its unique Webhook token and the URL will be copied to the clipboard by clicking Copy Webhook Token.

Example of using Webhook tokens

Copy the webhook token from the Script page by clicking Copy Webhook Token, for this example the Webhook URL looks as follows:

http://localhost:8090/scripts/8925cfd4-2063-4b07-9d56-956f2b754b47/run?token=6ae51c9f-d292-4d55-b512-4537121b37f5

Use the URL to send a POST request:

Using an API software like POSTMAN

A 200 response with the initial Job Entity indicates that the run has been triggered as shown in the screenshot below:

Using cURL in terminal

curl -X POST http://localhost:8090/scripts/8925cfd4-2063-4b07-9d56-956f2b754b47/run?token=6ae51c9f-d292-4d55-b512-4537121b37f5

The response would look like this

This run can be observed in SKOOR-Dashboard and the run shows created by Token-User in the History.

Deleting

Click on the trash bin icon to delete the script. The icon will appear when hovering over the script’s name with the mouse pointer.

Exporting Script as Zip

Click on the archive icon to export the script. The icon will appear when hovering over the script’s name with the mouse pointer.