Scripts / Legacy Data Import
Scripts is SKOOR’s comprehensive solution for creating, editing, and running scripts with loads of options to customize them.
Scripts Page
The page is divided into three main sections:
Settings
This section includes the general settings of a script, such as:
Name and Description.
Default Timeout (in seconds)
Copy Webhook Token or Reset it (Only available for ADMINS)
Schedule Configurations: Allows multiple schedules with dedicated run configurations.
Run Configurations: Allows customization of Timeout and Environment Variables.
Files
This section offers a complete file management solution for the script, including a state-of-the-art editor for text-based data.
A download link will be displayed if the data is binary (Not editable in text form).
Run
This section is responsible for Running and monitoring the Runs of a script. It offers a small drop-zone to upload source files if required.
Creating a Script
Scripts can either be created manually, or uploaded via Zip-files as shown below:
Cornerstones of a Script
Run-script
The main file includes the logic of the script. This determines the type of runner for the script:
Python Runner expects a
main.py
file.Talend Runner expects a
<scriptName>_run.sh
file.Shell Runner expects a
run.sh
file.
Place your main script one level below the root (or higher), since the breadth-first scan always uses the most shallow script it finds.
The following message is shown, when trying to run a Script without a main runner script.
.env
It’s a simple text file, that includes environment variables. This way variables that are used in a script can be adjusted based on the use case.
The following .env
file would yield the shown environment variables, allowing to adjust those in custom run configurations.
Pre-provided EnvironmentVariables
Some details about the run of the script are passed to the script environment as EnvironmentVariables, Those are:
UserId as
SKOOR_USER_ID
Username as
SKOOR_USER_USERNAME
User’s Fullname as
SKOOR_USER_FULLNAME
The below screenshots show how to retrieve those in a Python example along with the output in the script:
import os source_file = os.environ.get('SKOOR_SOURCE_FILE') user_id = os.environ.get('SKOOR_USER_ID') username = os.environ.get('SKOOR_USER_USERNAME') userFullname = os.environ.get('SKOOR_USER_FULLNAME') print("sourceFile: ", source_file) print("User ID: ", user_id) print("Username: ", username) print("Fullname: ", userFullname)
Example using .env
The following script reads the variable foo
in .env
and prints it out to the console, below are the screenshots of that .env
and the respective run configurations.
SourceFiles
Scripts can be designed to accept external files (source files) and process them as a part of the script logic.
The source files could be referenced in the script by using the environment variable SKOOR_SOURCE_FILE
as seen in the following code snippet:
# IN PYTHON import os source_file = os.environ.get('SKOOR_SOURCE_FILE') print(source_file)
#IN SHELL echo $SKOOR_SOURCE_FILE
Running sub-scripts
If sub-scripts are required, then they should be put inside a bin
folder. Since those will be runnable.
Make sure to put the sub-scripts always in a bin
folder.
The following shows an example in Shell:
The output of this run.sh
is:
The output of this run.sh
is:
QuickRun
This option offers a QuickRun
button that runs the script without a source file and with the default values of environment variables.
It allows to run a script quickly while developing or troubleshooting it and the Latest Job Overview
button shows the console output.
Running
Start a run
In the run section, select the desired configuration (and upload a source file if needed) then click “Run Script”
Cancel a run
Running as well as waiting runs can be canceled by clicking the cancel job icon in the History section.
Poetry Opt-out
Poetry is a powerful tool used for dependency management and virtual environment management in python. This is all done by defining a pyproject.toml
file.
Removing/renaming the pyproject.toml
will automatically opt-out poetry usage, and use skoor-python
instead.
Job Overview
To view the full details of a job, click on an entry in the History list as shown below:
Run History
The History section shows a list of all the script runs. Details about each run can be found by clicking on it.
A dedicated section shows the detailed history of the latest run of the script (skips waiting/canceled runs)
Using Webhook Tokens to run a converter
This option makes it possible to run the Script without the need to log in, and it’s a preferred option for technical users.
Each Script has its unique Webhook token and the URL will be copied to the clipboard by clicking Copy Webhook Token
.
Example of using Webhook tokens
Copy the webhook token from the Script page by clicking Copy Webhook Token
, for this example the Webhook URL looks as follows:
http://localhost:8090/scripts/8925cfd4-2063-4b07-9d56-956f2b754b47/run?token=6ae51c9f-d292-4d55-b512-4537121b37f5
Use the URL to send a POST
request:
Using an API software like POSTMAN
A 200
response with the initial Job Entity indicates that the run has been triggered as shown in the screenshot below:
Using cURL
in terminal
curl -X POST http://localhost:8090/scripts/8925cfd4-2063-4b07-9d56-956f2b754b47/run?token=6ae51c9f-d292-4d55-b512-4537121b37f5
The response would look like this
This run can be observed in SKOOR-Dashboard and the run shows created by Token-User in the History.
Deleting
Click on the trash bin icon to delete the script. The icon will appear when hovering over the script’s name with the mouse pointer.
Exporting Script as Zip
Click on the archive icon to export the script. The icon will appear when hovering over the script’s name with the mouse pointer.