System / Data Import List

Introduction

Clicking on the System / Data Import List menu displays the Data Import List page, which shows the data import records defined in the database. This page facilitates the creation, modification, and deletion of a data import.

Data imports can be organized by the product to import , the method of file transfer, and set up an automatic data import schedule.

A new NovaStar system will display an empty Data Import List.

Data Import List

The following image illustrates the System / Data Import List page after a data import has been selected in the list at the top of the page.

Data Import List (see full-size image)

Navigation, Adding, Deleting, Modifying, and Copying Data Imports

The Data Import List page conforms to a layout similar to many data types which includes a list of data items. See List Functionality and Application Functionality for more information detailing page navigation; adding, copying and deleting data imports; and modifying existing data imports.

Identification

This section defines the name and decription of the data import.

Data Imports Identification (see full-size image)

Data Label	Description
Name	Name must be unique for each data import and should describe the import. The name is used on pick lists. Max: 100 characters. Hovering over the Name label displays the index of the record in the database. Clicking the Name label displays the record index above the Name.
Description	Use this field to describe the action functionality. Max: 1000 characters.

Owner

The owner section defines the owner of the data import and the display scope: private or public. Private scope only displays this data import for the owner, the owner group supervisors, and administrators. Public scope display this data import for all group members and administrators.

Owner (see full-size image)

Data Label	Description
User	Owner of the data import. Click the user icon to assign the current user.
Scope	Public or private.

User Groups

The User groups table shows the user groups currently assigned to the data import. For more information, see the User Groups documentation.

User Groups Assigned (see full-size image)

Command

The Command displays the command to execute this data import. Data import commands are programs or scripts which import data from outside sources into NovaStar.

Command (see full-size image)

Data Label	Description
Command	Enter the command, including options, to be executed at the scheduled time. Commands are stored in the /usr/ns/cus and /usr/ns/bin folders. Click the View button to display a script command file. The most common command is nsdataimport -G#, where # is the data import record index (hover over name in the identification section). See the nsdataimport documentation
Type	Type of import source.
Source	Source line to use when importing data.
Fields	Fields from the custom import file. Visible for Custom data import type only. See Field Selector below.
Import file	File to parse as data import, for on-demand import. See Import File below.

Field Selector

When defining a Custom upload, use the Field selector to select fields in the uploaded file. To display the Field selector, click the down arrow button the right of the Fields text area. This expands the page to view the selector and toggles the button from a down arrow to an up arrow. To hide the Field selector, click the up arrow button.

Available fields are listed in the Available column. Field names are added to the Selected column using drag and drop functionality. To ignore a field, use the IgnoreField option, which can be used as many times as necessary. The order of column names in the Selected column are reorderable, also by selecting, dragging, and dropping the item. As items are added to the Selected column, the Fields text area is instantly updated.

Field Selector (see full-size image)

Import File

To import a file on demand, as opposed to routinely using the scheduling functionality, click the Upload button. After selecting a file on the local file system, the file name is displayed on the Import File Select Button, as seen on the image below. The Action List and Schedule List no longer display because their functionality is not utilized during this on demand import. To return to using the actions and schedule functionality, select none on the Import File Select Button. If desired, the file contents are viewed using the View button.

The data is imported using the Run button in the Execute section.

Import File Select Button (see full-size image)

Action List

The Action List displays the data import actions to retrieve the products from local or remote servers. Data import actions have the following types: None, COPY, FTP, LFTP, PFTP, SFTP, SMB, WEBPAGE.

When a data import record is first defined, no action records are assigned to the record.

Action List (see full-size image)

To remove an item from the action list, click the trash icon on the item. To change the order of actions, use the reorder icon by pressing the left mouse button on the reorder button, drag to the new order, and release the left mouse button (basic drag and drop functionality).

Clicking Add or the in the Action List displays a popup, which facilitates editing the action.

Action List (see full-size image)

Data Label	Description
Type	Select how the product is retrieved from the local or remote server: None File is left in Import File defined above. Copy The file is copied using a shared file system. The source folder and file name are required. FTP Copy the product file with file transfer protocol. The source host name, login name, password, folder and file name are required. LFTP Copy the product file with sophisticated file transfer protocol. The source host name, login name, password, folder and file name are required. PFTP Copy the product file with passive file transfer protocol. The source host name, login name, password, folder and file name are required. SFTP Copy the product file with secured file transfer protocol. The source host name, login name, password, folder and file name are required.
Host name	Enter the source host name as a URL or IP address
Login name	Enter the login name for the source host.
Password	Enter the password for the login name.
Folder	Enter the source folder for the file transfer. If blank then the home folder is used.
File name	Enter the file name for the source file.
Timeout	Select the file transfer timeout. The data import is canceled if the product cannot be transferred with in the timeout interval. Default: 60 seconds.

Schedule List

The Schedule List displays the data import schedules to deliver the products to local or remote servers. When a data import record is first defined, no schedule records will be assigned to the record. Data import schedules are stored in the cron system file:

/etc/cron.d/novastar5-dataimport

To remove an item from the schedule list, click the trash icon on the item.

Schedule List (see full-size image)

Clicking Add or the in the Schedule List displays a popup, which facilitates editing the schedule.

Schedule List (see full-size image)

Data Label	Description
Effective time	Enter the time for the next scheduled action. If a past time is entered, the scheduler will adjust the time to the next scheduled time based on the schedule Interval. The default if no value is entered for the Effective time is the current time. Click the button to assign the current time.
Interval	Enter how often the scheduled action should execute. Select from the list of intervals provided.
Day of Week	Select from the Day of Week multiple choice pick list for days of the week. If the scheduled action should execute only on Fridays, set the schedule Interval to 1 day and select only Friday from the Day of Week list. Multiple days can be selected.
Enabled	Select true to execute the data import at the scheduled time. Select false to leave the data import in the configuration list but do not execute it.

Command Execution

The final section on the Data Import List page is the command execution area.

Data Import Execute (see full-size image)

Data Label	Description
Time Limit	Selecting true displays the Starting Time and Ending Time. Selecting false indicates the import all data from 01/01/1980 current.
Starting Time	Set the starting time for data filing.
Ending Time	Set the ending date/time for data filing.
Time Offset	Select a date/time offset to apply to the timestamps in data import. After will adjust the timestamps forward in time, Before will adjust the timestamps into the past.
Timeout	Select the data import command timeout. The data import is canceled if the products cannot be made and transferred with in the timeout interval. Default: 60 seconds.
Quiet	Selecting true runs the command with no messages displayed. False displays messages.
Force	Selecting true forces replacing reports already in database. Otherwise, data are not reloaded if a matching ID and time exist.
Alarm	Selecting true disables alarm checking. Otherwise, alarms will be checked as data are filed.
Raw	Selecting true forces calibration of raw data.

Click the Run command button to execute the data import command; activity is shown in the display window.

Hovering over the Run button displays the command which will be run.

Show Command to be Run (see full-size image)

For more information about running commands in this area, see the Command functionality documentation.

Data Import Delete

To delete an import from the Data Import List, click that data import's row in the table, click Edit to enter into edit mode, and then click Delete. A confirmation dialog is shown to confirm that the data import should be deleted, detailing which additional data will also be deleted, including actions and scheduling. Clicking Delete again on the confirmation dialog deletes the data import. It is import to note that automatic execution of a data import can be suspended without deleting the record by setting the schedule Enabled state to false.

For more information, see Application Functionality.