Maintenance Data Import (MDI)

Setup

Import and process third-party data automatically in F&O

Introduction

Maintenance data import (MDI) allows you to process data generated from third-party providers. Using Azure File Storage, you can import counter-related data (used on objects) automatically to easily import and process counter data at the same time.

In this article, you'll find everything you need to know related to setup in F&O and Azure to get started using Maintenance Data Import.

Audience
Maintenance Manager

Getting Started

A few setup steps is required to get started with using the functionality.

  1. Go to System administration > Setup > License configuration > License codes and make sure that the DynMDI license is imported to your environment.
  2. Go to System administration > Setup > License configuration > Configuration keys and enable the check box for Dynaway Maintenance Data Import (DynMDI).
  3. Customize number sequences if needed by going to Enterprise asset management > Setup > Enterprise asset management parameters > Number sequences > Message.
  4. Two roles are available:
    • Maintenance data import manager: Grants users the permission to view, modify, and configure MDI.
    • Maintenance data import clerk: Grants users a view permission only to MDI setup and allows to use MDI to import and process files.

To identify the AOT elements related to this functionality, the elements are prefixed with 'DynMDI' word.

Setup in Azure File Storage

This article covers the setup in Azure File Storage and how to configure the connectivity between Azure and F&O.

Setup in Azure File Storage

Create Azure storage account

  1. Go to the Azure Portal main dashboard and select 'Create a resource'.
  2. Click 'Storage account'.
  3. In the 'Create a storage account' dialog, fill all the required information according to your preferences and select the 'Next: Advanced >' button.
  4. In the 'Advanced' tab, specify your preferred settings and select the 'Next: Networking >' button.
  5. In the 'Networking' tab, choose your preferred settings and seelct the 'Next: Data protection >' button.
  6. In the 'Data protection' tab, specify your preferred settings and select the 'Next: Tags >' button.
  7. In the 'Tags' tab, specify your preferred settings and select the 'Next: Review + create >' button.
  8. In the 'Review + create' tab, review the settings before selecting the 'Create' button to proceed.
  9. Deployment of a new storage account should start and be complete within 1 minute.

Step 2 - create account.
Step 2 - create account.
Step 3 - create an account - Basics.
Step 3 - create an account - Basics.
Step 4 - create an account - Advanced.
Step 4 - create an account - Advanced.
Step 5 - create an account - Networking.
Step 5 - create an account - Networking.
Step 6 - create an account - Data protection.
Step 6 - create an account - Data protection.
Step 7 - Create an account - Tags.
Step 7 - Create an account - Tags.

Create an Azure File Share

Once you have created an account, the last step is to create a file share.

  1. In your account, go to the 'File shares' menu in the left menu.
  2. Create a new file share by selecting the file share button.
  3. In the 'New file share' dialog, specify your preferred settings and select the 'Create' button.
  4. After a few seconds, a new Azure file share should be created in your storage account.
  5. Go to your newly created file share and click 'Add directory' in order to create two new folders for keeping files, eg. with the names:
    • CountersActive - this folder will store files that have been processed by MDI already.
    • CountersRead - this folder will keep files for MDI import.
  6. Come back to your Azure storage account and select 'Access keys', and then select 'Show keys'. In this way, you can see the data that should be entered into the 'Azure file share' and 'Connection string' fields visible from the table above.

Step 2 - create a new file share.
Step 2 - create a new file share.
Step 3 - specify settings for the new file share.
Step 3 - specify settings for the new file share.
Step 4 - new file share created.
Step 4 - new file share created.
Step 5 - create folders from the file share.
Step 5 - create folders from the file share.
Step 6 - view data to be put in the Azure file storages form.
Step 6 - view data to be put in the Azure file storages form.

Set Up Connection between Azure and F&O

To set up the connection between Azure and F&O, a configuration is needed. Go to Enterprise Asset Management > Setup > Data Impot > Azure file storages.

When a new 'Azure file storage' record is created, or 'Azure file storage' hyperlink is clicked, a form opens with the controls described in the table below.

CONTROL NAME

DESCRIPTION
Azure file share Holds a name of Azure file share that has been configured to be connected to by MDI. View the steps in this article to see how to retrieve it.
Connection string Holds a string value that is used to connect to Azure storage account, under which 'Azure file share' name has been defined. View the steps in this article to see how to retrieve it.
Validate

Once clicked, this button checks:

  • Validity of the 'Connection string' value.
  • Existence of folders specified in 'Folder name' cells.
  • The string set in 'Azure file share' field.
Folder type

Specifies a type for the given 'Folder name':

  • Read: It denotes a folder that holds the data to be imported to F&O in the form of files:
  • A file name must be suffixed with '.txt' extension name.
  • Only the first-level folder files are imported. Files from the sub-folders are ignored.
  • Archive: It denotes a folder that stores the already imported files.
Folder name Defines locations of folders in the 3rd party storage that will be used for MDI. The name can also contain '/' or '\' characters to locate a specific folder in a folder hierarchy.

Azure file storages form.
Azure file storages form.

Map Values Imported from a Third-party

Identifier mapping is an optional step that is used to make sure that data imported from third-parties is correctly processed in F&O.

To map out internal and external values, go to Enterprise asset management > Setup > Data import > Identifier mappings, and fill in information in the fields:

CONTROL NAME

DESCRIPTION

Identifier type

 Specifies the type, either Object or Counter.

Internal identifier

 Defines the value that the processed data from the third-party provider will have set in F&O.

External identifier

 Specifies which value from the third-party provider that will be replaced in the processed data:
  • A value can be mapped only once for specific identifier type.
Update internal identifiers Selecting this button automatically populates the form grid with all the possible mappings for Counters/Objects that can be modified later.

Import Message Type

When importing external data to F&O, you need to configure how the data will be imported and saved, and what type of data you want to import. This can be done by going to Enterprise asset management > Setup > Data Impot > Import message types.

Whenever a new 'Import message type' record is created or 'Message type' hyperlink is clicked, a form opens with controls explained in the table below. Refer to the screenshot below to see the form in F&O.

CONTROL NAME

DESCRIPTION

Azure file storage

It holds a value that represents the data source. See this article to create it.

Read folder

Specifies the location of files that hold the data to be imported. If only one folder is defined for the given 'Azure file storage', then this folder will be automatically set in this field.

Archive folder

Specifies the location of already imported files. If only one folder is defined for the given 'Azure file storage', then this folder will be automatically set in this field.

Import parameters > Import class*

Determines which data import mechanism will be used. The list of options is programmatically defined.

File format

Tells the import mechanism how to extract file data. See the screenshot below. If the 'File structure' configuration already exists and the value of the 'File format' field is modified, there will be a pop-up asking whether to continue the operation or not, since the configuration will be deleted once the value of this field is changed.

Process parameters > Process class*

Determines which data processing mechanism will be used. The list of options is programmatically defined.

File structure

This grid is used to specify the data structure of files.

Fixed values

This grid is used to overwrite values in files:

  • A key with a specific value can be overwritten only once per 'Key set'.
  • The value overwriting is done in the following sequence:
  1. If a ‘Key’ is overwritten in the ‘Fixed values’ grid, then this value will be used, even an incorrect one.
  1. If a ‘Key’ is not overwritten in the ‘Fixed values’ grid, then the configuration in 'Identifier mappings' form is used if any.

Key*

Informs MDI about what individual kinds of data can be extracted from a file. Note that:

  • Allowed values: ObjectID, CounterID, CounterValue, CounterDateTime, CounterDate, CounterTime, CounterTotal.
  • MDI ignores whether the cell value contains capital letters or not, that is 'ObjectID' and 'objectid' values are equivalent.
  • A given value can be set only once per 'Key set'.
  • If 'CounterDateTime' value is entered, then 'CounterDate' as well as 'CounterTime' value can't be entered for the given 'Key set' and vice-versa.
  • To decide whether all counter registrations should be aggregated into one counter registration, a 'CounterTotal' key with its 'Value' set to 'True' record must be put into the 'Fixed values' grid.

Key set

Allows to form groups of values as one line can contain multiple data records to be imported.

Order*

Defines the data value index, for example, this line “objID,5,counterID” has 3 indexes, namely “objID” value is at index 1, “5” is at index 2, etc. Note that:

  • This column is visible for delimiter based 'File format'.
  • When a new 'File structure' record is created, then this field is automatically filled out based on the previous highest 'Order' value.
  • The order sequence may contain gaps to allow undesirable values to be omitted when processing file data.

Prefix/Suffix padding characters

Enables to specify a set of characters that were used to create a uniform length of values:

  • The padding is done by filling out the remaining space with prefix/suffix characters. For example, the “#” character is a prefix padding character for the values “##123” and “#1234”, to make sure that the values are of the same character length. These values will be imported into F&O as “123” and “1234” respectively.
  • These characters can only be set when the 'Key' field is set to 'ObjectID' or 'CounterID'.

Key start/end position*

Allows to specify a range of consecutive characters that form a value within a file:

  • Positions cannot overlap.
  • These columns are visible for position based 'File format'.
  • For date related keys, positions are additionally validated against the date and time format that is defined under 'Import parameters'.

Import manually

Once this button is clicked, a form is opened that allows to upload a single file at a time to the given file storage as well as optionally processing it:

  • Only files with “.txt” extension can be selected.

The fields marked with an asterisk (*) are mandatory to fill out. Refer to the screenshot below to see an example of the form.

Note,

  • To successfully import counter registrations, an “Active” object ID must be specified, counter ID on which registration was made, date with time, and counter value.
  • Date and time set on the incoming data are always imported in legal entity time zone.

Import message type form.
Import message type form.

File Formatting

Currently, the MDI import mechanism can extract file data in a couple of ways. If the file format does not adhere to one of the ways, errors are shown.

Delimiter based

In this data format, the distinct values are separated by a 'Delimiter'. An example of the content of a file in this data format could look like the following, assuming that “,” is a delimiter:

objID1,countID1,-5,17/02/2022,08:00:17

objID1,countID1,-06,17/02/2022,08:19:18

objID1,countID2,7.1,17/02/2022,08:19:19

objID2,countID2,10,18/03/2022,15:19:00

Multiple groups of data

If there are multiple groups of data per line, a semicolon ";" is used as a delimiter, and a "-" is used to distinguish between multiple data in the same line:

objID;counterID;-50;00:00:00;2022-01-15;objID;counterID;1.1;2022-01-15 00:00:01

objID;counterID;-49;00:00:03;2022-01-15;objID2

objID;counterID;48;00:00:06;2022-01-15;objID;counterID;-046;2022-01-15 00:00:07

Position based

In this data format, the distinct values have fixed text positions. An example of the content of a file in this data format could look like the following:

objIDcounterID-05000:00:002022-01-15objIDcounterID-0012022-01-15 00:00:01

objIDcounterID-04900:00:032022-01-15objID2

objIDcounterID004800:00:062022-01-15objIDcounterID-0462022-01-15 00:00:07

Copyright Dynaway A/S

Privacy Policy